-
Notifications
You must be signed in to change notification settings - Fork 63
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Fixed error reporting in the verifier #2031
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working on this, Christian. Looks good to me!
I can confirm this is working on my Mac machine, including when one of the dependencies is not installed.
One minor point is the time measurement doesn't seem working. The tests all take 0 time to complete.
The failure case is artificially injected. No need to worry about it. |
Heads up: this will clash with my refactoring in #2008. |
What do you mean? If it fails the whole test doesn't pass. Are you saying it should not be marked as failing? |
I meant I changed the test on my local branch to make it fail, so that I can observe whether the test harness can catch it. And it did successfully. The version on the remote is still passing. |
I think the reported times are execution times of the binary program. For the verifier tests, we only build and do not run the tests, so I guess this explains the numbers. |
This PR fixes several issues related to error reporting in the verifier.
LFGenerator
. This led to the problem thatGeneratorBase
cleared all reported errors after the verifier run (See this comment). This is fixed by moving the invocation of the verifier toGeneratorBase
.