Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Test case times sometimes reported as negative #2363

Closed
Fry-kun opened this issue Apr 13, 2023 · 0 comments · Fixed by #2364
Closed

Test case times sometimes reported as negative #2363

Fry-kun opened this issue Apr 13, 2023 · 0 comments · Fixed by #2364
Labels

Comments

@Fry-kun
Copy link
Collaborator

Fry-kun commented Apr 13, 2023

Describe the bug
https://firebase-community.slack.com/messages/test-lab/thread/C1MTSQ5QT-1677888583.152469

Flank's junit xml files list some test cases like so:

<testcase name="..." classname="..." time="-0.659">

To Reproduce

  1. Run a test with deflaking enabled & a test case failing (triggering deflake runs)
  2. Check JUnitReport.xml

Expected behavior
No negative test case times

Details (please complete the following information):
This was determined to be caused by overhead time calculation introduced in #684 to fix #557. While calculating overhead, all instances of test cases are added together (which is 5*T for 4 deflake reruns), and then subtracted from T of a single run (ignoring reruns), resulting in "negative overhead"

Additional context
Notably, all results from FTL are non-negative

@Fry-kun Fry-kun added the Bug label Apr 13, 2023
Fry-kun added a commit that referenced this issue Apr 13, 2023
…ocessing deflake reruns & reporting negative test case times.

Overhead (as described in #557) no longer exists in FTL, so this feature is not useful anymore.

Fixes #2363
Fry-kun added a commit that referenced this issue Apr 13, 2023
…ocessing deflake reruns & reporting negative test case times. (#2364)

Overhead (as described in #557) no longer exists in FTL, so this feature is not useful anymore.

Fixes #2363
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant