Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Specify WHITE_COMPONENTS to machine precision #128

Merged
merged 1 commit into from
Jan 8, 2025

Conversation

tomcur
Copy link
Member

@tomcur tomcur commented Jan 8, 2025

These two are off by 1 ULP.

Copy link
Collaborator

@waywardmonkeys waywardmonkeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we tighten the error bound in the white_components test?

@tomcur
Copy link
Member Author

tomcur commented Jan 8, 2025

We could tighten it a bit, but we cannot detect 1 ULP errors. Depending on the exact computations that are performed, the largest magnitude of values during computation can determine the largest absolute error, which could in principle propagate to all components. The test itself only knows about input and output magnitudes, and not the magnitudes during computation.

The roundtrip test has a tight bound of 16 epsilons relative to the largest input magnitude, but cheats a bit by doing error checking in the linear sRGB space, and the bound may still be too tight if color space transformations do very fancy things. (OTOH, maybe if conversions can't reach that bound, the computations should be done in f64...)

@tomcur tomcur added this pull request to the merge queue Jan 8, 2025
Merged via the queue into linebender:main with commit 7e966a1 Jan 8, 2025
16 checks passed
@tomcur tomcur deleted the rounding branch January 8, 2025 11:02
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants