Skip to content

Constructor loses precision when parsing small numbers expressed in scientific notation ("e" format). #45

@StrangeRay

Description

@StrangeRay

The problem is perhaps best explained by example. Consider the construction of the decimal 1.23456789×10⁻²¹ via decimal("1.23456789e-21"). The result should display as
Decimal(0, 123456789, -29) but I get Decimal(0, 0, -20) instead. Observe the progressive loss of significance in the following constructions:

julia> using Decimals

julia> decimal("1.23456789") # Ok
Decimal(0, 123456789, -8)

julia> decimal("1.23456789e-11") # Ok
Decimal(0, 123456789, -19)

julia> decimal("1.23456789e-12") # Still Ok
Decimal(0, 123456789, -20)

julia> decimal("1.23456789e-13") # Not quite
Decimal(0, 12345679, -20)

julia> decimal("1.23456789e-14") # Constructor loses one significant digit per decade
Decimal(0, 1234568, -20)

julia> decimal("1.23456789e-15") # Trend continues ...
Decimal(0, 123457, -20)

julia> # ...
       
       decimal("1.23456789e-18") # ...
Decimal(0, 123, -20)

julia> decimal("1.23456789e-19") # ...
Decimal(0, 12, -20)

julia> decimal("1.23456789e-20") # Down to one significant digit
Decimal(0, 1, -20)

julia> decimal("1.23456789e-21") # All significant digits are lost
Decimal(0, 0, -20)

The problem is traceable to the implementation of function scinote. In particular, see line 39 in file decimal.jl.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions