Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add environment variable to force regression test harness to generate expected results #848

Merged
merged 3 commits into from
Mar 12, 2019

Conversation

jpwhite4
Copy link
Member

The regression test harness used to automatically generate
expected results if the tests did not match. This then marked
the test as skipped. This is only desired behaiour if the developer
wants to generate test data. In normal CI we want the test to fail if
the actual does not match the expected.

Also add an extra echo to the runtests.sh script to mitigate a bug in the shippable console folding.

Also switch the order of the tests run so the test generation will function correctly (the algorithm relies on the public user being run first).

jpwhite4 and others added 3 commits March 11, 2019 14:36
The regression test harness used to automatically generate
expected results if the tests did not match. This then marked
the test as skipped. This is only desired behaiour if the developer
wants to generate test data. In normal CI we want the test to fail if
the actual does not match the expected.
@jpwhite4 jpwhite4 changed the title Add env var to force regrn tst harnss to gen expctd Add environment variable to force regression test harness to generate expected results. Mar 12, 2019
@plessbd plessbd merged commit 7a11e21 into ubccr:xdmod8.1 Mar 12, 2019
jtpalmer pushed a commit to jtpalmer/xdmod that referenced this pull request Mar 13, 2019
… expected results. (ubccr#848)

The regression test harness used to automatically generate
expected results if the tests did not match. This then marked
the test as skipped. This is only desired behaiour if the developer
wants to generate test data. In normal CI we want the test to fail if
the actual does not match the expected.

Also add an extra echo to the runtests.sh script to mitigate a bug in the shippable console folding.

Also switch the order of the tests run so the test generation will function correctly (the algorithm relies on the public user being run first).
@plessbd plessbd added this to the 8.1.0 milestone Mar 23, 2019
@plessbd plessbd added the qa / testing Updates/additions to tests label Mar 23, 2019
@smgallo smgallo changed the title Add environment variable to force regression test harness to generate expected results. Add environment variable to force regression test harness to generate expected results Apr 10, 2019
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
qa / testing Updates/additions to tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants