You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When doing TDD and running tests regularly after making changes, I noticed that test execution allows Unity to use an older version of the tests and/or code than what is currently saved.
Expected behaviour:
When running unit tests from Rider, the results reflect the current state of the saved code and tests.
Actual behaviour:
When running unit tests from Rider, the results can temporarily reflect an older state of the code and tests, or permanently reflect an older state in the case where compilation of code or tests fails.
To reproduce:
Start with a test-suite with some passing tests in it.
Run the tests from Rider, all should pass.
Introduce a new test:
[Test]
public void FakeTest()
{
Assert.Fail();
}
Save all, and immediately run the unit tests (⌘S, ⌘;, R)
Rider immediately recognises the new test, but Unity doesn't, this presents as an inconclusive/not run test:
Wait a few seconds and without making any changes run the unit tests again. Unity has now caught up and runs the failing test:
Make the test pass by removing the Assert.Fail(); line, save, and immediately run the tests again. The test which should now pass, still fails:
Once again, wait a few seconds and without making any changes run the unit tests again. Unity has now caught up and all tests pass.
Introduce a compile error into the tests:
[Test]
public void FakeTest()
{
please fail to compile
}
Running the tests immediately has the same effect as above, they still all show passing. But now, even waiting a few seconds won't work because Unity never catches up - it continues to use the previous working version of the code/tests because it can't compile the new version.
Speculation on cause and proposed fix:
Cause: Rider triggers Unity to run unit tests without triggering a re-compile, or waiting for any ongoing re-compile to finish.
Proposed fix: Delay triggering a test run until Unity has compiled any saved changes.
Cause: Rider allows Unity to run old compiled code and tests when the current code/tests don't compile.
Proposed fix: Report back to Rider a total failure to run any tests in the case where Unity can't compile the latest saved changes.
The text was updated successfully, but these errors were encountered:
Rider Version:
2018.2.3
Plugin Build:
182.4231.496
Unity Version:
2018.2.8f1
When doing TDD and running tests regularly after making changes, I noticed that test execution allows Unity to use an older version of the tests and/or code than what is currently saved.
Expected behaviour:
When running unit tests from Rider, the results reflect the current state of the saved code and tests.
Actual behaviour:
When running unit tests from Rider, the results can temporarily reflect an older state of the code and tests, or permanently reflect an older state in the case where compilation of code or tests fails.
To reproduce:
Assert.Fail();
line, save, and immediately run the tests again. The test which should now pass, still fails:Speculation on cause and proposed fix:
Cause: Rider triggers Unity to run unit tests without triggering a re-compile, or waiting for any ongoing re-compile to finish.
Proposed fix: Delay triggering a test run until Unity has compiled any saved changes.
Cause: Rider allows Unity to run old compiled code and tests when the current code/tests don't compile.
Proposed fix: Report back to Rider a total failure to run any tests in the case where Unity can't compile the latest saved changes.
The text was updated successfully, but these errors were encountered: