Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Faster local testing with reference.rs #4255

Merged
merged 5 commits into from
Nov 11, 2024

Conversation

RunDevelopment
Copy link
Contributor

This is basically a follow-up to #4242 to make it faster to run reference tests locally. The main problem is that tests take time to run and that references.rs had no progress indication and only reported failures after all tests went through.

This PR:

  1. Add progress indication by simply printing the current test file to stdout. Super simple and helps a lot.
  2. Stops after the first test failed, so you don't have to wait for the other tests to see what went wrong.
  3. Runs tests in order of modification date. So test files that were recently modified get tested first. This makes it very fast to iterate on tests, since my usual workflow is: edit one test file, run all tests, see the results.

Change 2 and 3 only take effect locally. CI workflows use the old strategy of running all tests in order of name and printing all error messages (if any) at the end.

With this, writing and running tests is now quite convenient. Before #4242, I had to wait roughly 2 minutes to see the results of a test I added. After #4242, it was down to around 20~30 seconds. With this PR, it's down to 2~4 seconds.

@RunDevelopment RunDevelopment changed the title Faster local testing in reference.rs Faster local testing with reference.rs Nov 8, 2024
Copy link
Collaborator

@daxpedda daxpedda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fun! Thank you for doing this!

@daxpedda daxpedda added the waiting for author Waiting for author to respond label Nov 9, 2024
Co-authored-by: daxpedda <daxpedda@gmail.com>
@daxpedda daxpedda merged commit 2463d0d into rustwasm:main Nov 11, 2024
41 checks passed
@RunDevelopment RunDevelopment deleted the quick-local-ref-tests branch November 11, 2024 09:48
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
waiting for author Waiting for author to respond
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants