Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[BUG] CAST string to float for strings near the limit and with .0 at the end produce wrong results #10481

Open
revans2 opened this issue Feb 23, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@revans2
Copy link
Collaborator

revans2 commented Feb 23, 2024

Describe the bug

(1 until 38).map(i => "9"*i).toDF.repartition(1).selectExpr("CAST(value AS FLOAT) AS f")

Produces results that are more or less identical to the CPU. But

(1 until 38).map(i => "9"*i + ".0").toDF.repartition(1).selectExpr("CAST(value AS FLOAT) AS f").collect.foreach(System.out.println)

returns null at i = 231. Then returns totally wrong results that look like there was an overflow of some kind and the numbers are orders of magnitude smaller than they should be.

Ideally I want to be able to parse something crazy like 99999999999999999999999999999999999999.00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001 and get the right answer for a float or a double. I would love to go even crazier and have something like

(300 until 308).map(i => ("9"*i + ".0", i)).toDF("value","i").repartition(1).selectExpr("i", "CAST(value AS DOUBLE) AS f").
[300,1.0E300]
[301,1.0E301]
[302,1.0E302]
[303,1.0E303]
[304,1.0E304]
[305,1.0E305]
[306,1.0E306]
[307,1.0E307]

return correct values, where as today it returns

[300,1.0E31]
[301,1.0E32]
[302,1.0E33]
[303,1.0E34]
[304,1.0E35]
[305,1.0E36]
[306,1.0E37]
[307,1.0E38]
@revans2 revans2 added bug Something isn't working ? - Needs Triage Need team to review and classify labels Feb 23, 2024
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Feb 27, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants