-
Notifications
You must be signed in to change notification settings - Fork 521
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
consider table-driven tests #392
Comments
I am opposed to table-based testing, because it hides information from the person running the tests.
If Rust or Cargo supported the notion of sub-tests, I would withdraw this objection. Sub-tests don't address my second and third objections, but they do solve the first, and that's the most serious one. However, AFAIK Rust does not have this feature and it isn't on the roadmap. The Rusty way to approach this problem would be to use macros. We could, I suppose, develop and publish an official exercism Rust test macro; it's not hard to imagine what it would look like: macro_rules! test {
($test_name:ident, $property_test_func:ident, $( $param:expr ),*) => {
#[test]
fn $test_name() {
$property_test_func($( $param ),* )
}
}
}
macro_rules! tests {
($property_test_func:ident => $( $test_name:ident, $( $param:expr ),* );+ ) => {
$(
test!($test_name, $property_test_func, $( $param ),*)
)+
}
} However, that would introduce either explicit dependencies to some exercism-affiliated crate, or a copy-paste job by the exercise author. In either case, it obfuscates to a new student exactly how their code is being tested. Given that it would probably need to be implemented within the test file anyway, I'd prefer to leave such a macro to the individual exercise implementers. In short, I'm opposed to table-based testing and would be inclined to request changes from a PR which used it. Macros can give the appearance of table-based testing while generating a full real test suite behind the scenes, but I believe that the implementation of the relevant macro should be left to the exercise implementer. |
I also favor more granular test cases, a test called |
This PR is in response to exercism#392. Perfect-numbers currently uses table-based testing. For reasons outlined in exercism#392 (comment) I believe that table-based testing is not the appropriate way to do things. This is what macro-based tests might look like. Are they abstruse and opaque? Yes. However, they at least become individual tests, and I believe that this is _better than the status quo_.
This PR is in response to exercism#392. Perfect-numbers currently uses table-based testing. For reasons outlined in exercism#392 (comment) I believe that table-based testing is not the appropriate way to do things. This is what macro-based tests might look like. Are they abstruse and opaque? Yes. However, they at least become individual tests, and I believe that this is _better than the status quo_.
Ah good, that's a majority of us. This is excellent. I believe we are in a good position to say table-based tests are not for this track. To my knowledge, perfect-numbers is the only exercise that uses them, therefore this issue should be closed when something makes perfect-numbers no longer use them. This issue should be used to refer to in the future when queried about table-based tests. Considerations that can cause this to change in the future:
I find that https://users.rust-lang.org/t/table-driven-aka-data-driven-testing/3848 makes the same observation. It seems unlikely that the model of using My own MO with most exercises is simply to run
It may be possible to use the optional panic message argument to
Ironically, mashing all the tests into one currently makes life easier for those who prefer this approach, since now there's only one |
This PR is in response to exercism#392. Perfect-numbers currently uses table-based testing. For reasons outlined in exercism#392 (comment) I believe that table-based testing is not the appropriate way to do things. This is what macro-based tests might look like. Are they abstruse and opaque? Yes. However, they at least become individual tests, and I believe that this is _better than the status quo_.
This PR is in response to exercism#392. Perfect-numbers currently uses table-based testing. For reasons outlined in exercism#392 (comment) I believe that table-based testing is not the appropriate way to do things. This is what macro-based tests might look like. Are they abstruse and opaque? Yes. However, they at least become individual tests, and I believe that this is _better than the status quo_. Squashed commits: - Conditionally disable count-ignores.sh; disable for perfect-numbers - Document .meta/ignore-count-ignores - Update to use inline ignores
In many tests, we have some number of input/output pairs, and the validation we perform on the input/output pairs is the same. For the purpose of not repeating ourselves, we often extract the validation into one function.
For further deduplication, and for not adding so many test functions, we may consider the approach taken in https://github.com/exercism/rust/blob/master/exercises/perfect-numbers/tests/perfect-numbers.rs, a table-driven test.
Let us decide some set of circumstances under which table-based tests are ideally to be used (all the time, none of the time, or something in between).
The text was updated successfully, but these errors were encountered: