Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Floating-point conversion error #2336

Closed
ekiwi opened this issue Nov 5, 2024 · 2 comments · Fixed by #2337
Closed

Floating-point conversion error #2336

ekiwi opened this issue Nov 5, 2024 · 2 comments · Fixed by #2337
Assignees

Comments

@ekiwi
Copy link
Contributor

ekiwi commented Nov 5, 2024

There is a bug in the tests provided by #2318.

tests/correctness/ieee754-float/parse-and-print.futil.data provides the input value 15.1239. However, at the end of the test, the value has changed to 15.12 in tests/correctness/ieee754-float/parse-and-print.expect.

This is even though a 32-bit IEEE float should be able to represent 15.1239. So there appears to be a bug, most likely in how the data is converted back after the simulation.

@rachitnigam
Copy link
Contributor

Argh, thanks for catching this! Also tagging @jiahanxie353 in case he can get to this before me.

@EclecticGriffin
Copy link
Collaborator

Probably an easy fix? Best guess is that the problem comes from this line:

return fp.to_dec(round_place=2)

ekiwi pushed a commit that referenced this issue Nov 6, 2024
Fixes #2336. Currently running into the same problem as
#2335 (comment)

@ekiwi how did you fix it on the Rust side?
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants