ENH: add crude assert_allclose; use value testing in vecdot #323
Conversation
|
This should fail with array_api_strict until data-apis/array-api-strict#98 lands. |
ev-br
force-pushed
the
vecdot_assert_allclose
branch
from
1e58129 to
c8309c8
Compare
ev-br
force-pushed
the
vecdot_assert_allclose
branch
from
c8309c8 to
5983cac
Compare
|
The CI here passes. It might be because the CI doesn't run enough examples by default and didn't catch the error. |
|
It would be a good idea to run this with a lot of examples. My worry is that we will see the same sorts of flaky falure issues we saw with #168 due to loss of significance unless we filter out ill conditioned inputs (as in #290). Of course, we can always just merge this and revert it or fix it if issues crop up. The CI here doesn't run many examples relatively speaking, but the CI for array-api-strict and array-api-compat is where you usually find the errors, because they have such a large matrix that they rarer errors are very likely to show up (that is assuming the test in question isn't already being XFAILed for some reason). |
| @@ -599,3 +600,18 @@ def assert_array_elements( | |||
| at_expected = expected[idx] | |||
| msg = msg_template.format(sh.fmt_idx(out_repr, idx), at_out, at_expected) | |||
| assert at_out == at_expected, msg | |||
|
|
|||
|
|
|||
| def assert_allclose(actual, desired, *, atol=1e-7, rtol=1e-7, equal_nan=True, msg_extra=""): | |||
There was a problem hiding this comment.
There's also some similar helpers in test_operators_and_elementwise_functions.py (that file has some of the more complex helper functions, since most of the tests are very similar).
|
An alternative strategy, in view of #314 (comment), could be to only test "exact" things in hypothesis (rename |
|
Yes, it's a possibility. It's not something we've really done yet, but maybe we should think about it (see also #284). Non-hypothesis tests have their own issues, which is that they only test specific things. The hypothesis tests have been pretty good at checking a wide range of cases which would definitely be missed with traditional testing. I would definitely strongly encourage the majority of the test suite to continue to use hypothesis. Even something simple like a function with And I do agree we should rename functions to be more descriptive of what they are actually doing. |
No description provided.