Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add flaky decorator to long_callback tests to retry up to 3 times #1741

Merged
merged 1 commit into from
Sep 2, 2021

Conversation

jonmmease
Copy link
Contributor

🩹 to retry the long_callback tests up to 3 times. A ===Flaky Test Report=== section should be printed out in the test logs that reports whether tests had to be rerun.

@jonmmease
Copy link
Contributor Author

Here's an example of the flaky test support that gets printed after the warnings (in this case nothing needed to be rerun)

===Flaky Test Report===

test_lcbc001_fast_input[diskcache] passed 1 out of the required 1 times. Success!
test_lcbc001_fast_input[celery] passed 1 out of the required 1 times. Success!
test_lcbc002_long_callback_running[diskcache] passed 1 out of the required 1 times. Success!
test_lcbc002_long_callback_running[celery] passed 1 out of the required 1 times. Success!
test_lcbc003_long_callback_running_cancel[diskcache] passed 1 out of the required 1 times. Success!
test_lcbc003_long_callback_running_cancel[celery] passed 1 out of the required 1 times. Success!
test_lcbc004_long_callback_progress[diskcache] passed 1 out of the required 1 times. Success!
test_lcbc004_long_callback_progress[celery] passed 1 out of the required 1 times. Success!
test_lcbc005_long_callback_caching[diskcache] passed 1 out of the required 1 times. Success!
test_lcbc005_long_callback_caching[celery] passed 1 out of the required 1 times. Success!
test_lcbc006_long_callback_caching_multi[diskcache] passed 1 out of the required 1 times. Success!
test_lcbc006_long_callback_caching_multi[celery] passed 1 out of the required 1 times. Success!
test_lcbc007_validation_layout[diskcache] passed 1 out of the required 1 times. Success!
test_lcbc007_validation_layout[celery] passed 1 out of the required 1 times. Success!

===End Flaky Test Report===

@jonmmease
Copy link
Contributor Author

There's an unrelated percy dash table test timeout, but everything else went through

Copy link
Collaborator

@alexcjohnson alexcjohnson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💃 works for me!

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants