Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Retry flaky UI tests in CI #1262

Merged
merged 3 commits into from
Aug 10, 2024
Merged

Conversation

mahendrapaipuri
Copy link
Collaborator

@mahendrapaipuri mahendrapaipuri commented Jul 17, 2024

See comment

Closes #1230

Copy link

Thank you for making this pull request.

Did you know? You can try it on Binder: Binder:lab or Binder:notebook.

Also, the version of Jupytext developed in this PR can be installed with pip:

HATCH_BUILD_HOOKS_ENABLE=true pip install git+https://github.com/mahendrapaipuri/jupytext.git@plywrgt_config

(this requires nodejs, see more at Developing Jupytext)

Copy link

codecov bot commented Jul 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.91%. Comparing base (6089897) to head (a34d772).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1262      +/-   ##
==========================================
- Coverage   95.93%   95.91%   -0.03%     
==========================================
  Files          29       29              
  Lines        4476     4476              
==========================================
- Hits         4294     4293       -1     
- Misses        182      183       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mwouts
Copy link
Owner

mwouts commented Jul 31, 2024

Hi @mahendrapaipuri , I am not sure this test is "flaky" - I had relaunched the publish CI multiple times and it failed on each attempt (https://github.com/mwouts/jupytext/actions/runs/9958694800/job/28138283717) - curious to learn what makes you think that retry would improve this?

@mahendrapaipuri
Copy link
Collaborator Author

Well, if you look into the CI runs of this PR, playwright actually marked that test as flaky. I guess the first run of that test is always a failure and it passes in the subsequent retries.

Anyways, that test does not add any value and hence, I removed it. I added a workflow to update snapshots directly from a PR by commenting "update snapshots". Let me know what you think!

@mwouts
Copy link
Owner

mwouts commented Aug 2, 2024

Thank you @mahendrapaipuri , this looks great! I will have to fix #1266 first but I am looking forward to taking your PR! Thanks

Signed-off-by: Mahendra Paipuri <mahendra.paipuri@gmail.com>
Signed-off-by: Mahendra Paipuri <mahendra.paipuri@gmail.com>
* Bump actions versions

Signed-off-by: Mahendra Paipuri <mahendra.paipuri@gmail.com>
@mwouts mwouts merged commit 070ba88 into mwouts:main Aug 10, 2024
16 checks passed
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add CI workflow to update playwright snapshots automatically
2 participants