Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Error: Cache upload failed because file read failed should not cause step to fail #120

Closed
3flex opened this issue Nov 14, 2021 · 3 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@3flex
Copy link
Contributor

3flex commented Nov 14, 2021

I migrated the detekt project to use gradle-build-action when v2 final was released (detekt/detekt@c6f796a).

It appears that the change in cache usage is causing flakiness. I just opened a number of PRs in quick succession, and they all started failing with errors similar to this:
https://github.com/detekt/detekt/runs/4201692342?check_suite_focus=true

I'm guessing this is because the cache step is caching several artifacts at the end of each build job, so we're now hitting rate limits. It's not typical that multiple PRs are opened so quickly, but when it does happen and cache failures occur thanks to rate limiting it shouldn't fail the job.

A workaround would be to only write to cache on the main branch, but I assume if multiple PRs are merged one after the after in quick succession, the same issue would occur.

@bigdaz
Copy link
Member

bigdaz commented Nov 15, 2021

Thanks for the report. I haven't been able to find any way to eliminate these 429 rate limiting errors, but they should only be logged as a warning and not cause the job to fail. (You can see these in your logs).

However, the Error that cause the failure is new to me: https://github.com/detekt/detekt/runs/4201692342?check_suite_focus=true#step:6:15, and for some reason it's not being caught in via regular exception handling.

It looks like there was a similar issue fixed for the actions/cache github action. We rely on the same underlying libraries as actions/cache so we will likely benefit from a similar fix. I'll try to get this into a 2.0.1 release soon.

Even with this fix, in many cases it's beneficial to use cache-read-only for all branch builds. This is because cache entries written for a branch are not visible to builds for other branches. (I'm actually thinking about making this the default behaviour). See here for more details.

@bigdaz bigdaz added the bug Something isn't working label Nov 15, 2021
@bigdaz bigdaz added this to the v2.0.1 milestone Nov 15, 2021
@bigdaz bigdaz changed the title Error: Cache service responded with 429 during upload chunk Error: Cache failures should be logged as warnings and not cause workflow to fail Nov 15, 2021
@bigdaz bigdaz changed the title Error: Cache failures should be logged as warnings and not cause workflow to fail Error: Cache upload failed because file read failed should not cause step to fail Nov 15, 2021
@bigdaz bigdaz self-assigned this Nov 15, 2021
bigdaz added a commit that referenced this issue Nov 15, 2021
Failures to store cache entries should not fail the action or the Job.
This fix attempts to catch and log any unexpected errors that occur when
saving cache entries.

Fixes: #119
Fixes: #120
@bigdaz bigdaz closed this as completed in 996094e Nov 15, 2021
@bigdaz
Copy link
Member

bigdaz commented Nov 17, 2021

v2.0.1 has been released, which contains a fix for this issue

@3flex
Copy link
Contributor Author

3flex commented Nov 17, 2021

Thank you! Besides this (now hopefully fixed) issue the new action seems to work well, and it's definitely much nicer having Gradle Inc supporting the action and proper cache setup than having to figure it out ourselves.

I'm actually thinking about making this the default behaviour

I think that's a better default. I think on balance it would be better to cache main branch by default and increase cache hits on all PRs. Otherwise if multiple PRs are cached and evict the main branch cache, no new PRs will get the benefit of cache hits.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants