-
Notifications
You must be signed in to change notification settings - Fork 22
Description
I am checking the code for the normalize
function at https://github.com/JuliaMath/Decimals.jl/blob/master/src/norm.jl and I suspect there are two problems in how the rounded
option is handled.
First, the code on line 12 is run if rounded
is set to true
. But clearly this line computes the output without rounding. It is only the else
branch that does something containing rouding. I suspect the logic here is inverted.
Second, on line 14, where rounding is performed, the digits
parameter is set to DIGITS
. But it is unclear to me where this DIGITS
comes from - it is not entered while calling the normalize
function. Is it intended to be a global variable? EDIT: I see, it is a const
set in the module to 20. It does not work anyway as I would anticipate. The code below:
julia> h = Decimal(0,2540,-4)
Decimal(0, 2540, -4)
julia> f = normalize(h)
Decimal(0, 254, -3)
julia> f = normalize(h,rounded=false)
Decimal(0, 254, -3)
julia> f = normalize(h,rounded=true)
Decimal(0, 254, -3)
julia> DIGITS=1
1
julia> f = normalize(h,rounded=true)
Decimal(0, 254, -3)
For comparision with a standard float:
julia> round(0.254,digits=1)
0.3
I am running this in REPL.