[CI] if latest contexts have changed, retrieve the previous contexts' test cache and then use ./setup_test_cache u to update
#1099
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zacharyburnett
force-pushed
the
ci/update_partial_cache
branch
2 times, most recently
from
c5ed0b1 to
d7dce5b
Compare
zacharyburnett
changed the title
./setup_test_cache u to update a previous existing cache./setup_test_cache u to update
zacharyburnett
force-pushed
the
ci/update_partial_cache
branch
from
d7dce5b to
393f1cb
Compare
|
tested at https://github.com/spacetelescope/stenv/actions/runs/12017528869/job/33500260337 by manually creating a |
|
@spacetelescope/crds-maintainers does this look right? The log in the comment above (#1099 (comment)) show the latest context for both the HST and JWST commands in
However, the log says this throughout: So did it actually update the test cache? |
zacharyburnett
commented
Nov 25, 2024
Comment on lines
+55
to
+65
| - if: steps.lookup-cache.outputs.cache-hit != 'true' | ||
| id: retrieve-previous-cache | ||
| name: retrieve a previous combined CRDS cache | ||
| uses: actions/cache/restore@v4 | ||
| with: | ||
| path: | | ||
| ${{ env.CRDS_PATH }} | ||
| ${{ env.CRDS_TESTING_CACHE }} | ||
| key: ${{ steps.key.outputs.key }} | ||
| restore-keys: | | ||
| test-cache- |
There was a problem hiding this comment.
if a cache with the latest contexts does not exist, attempt to retrieve a previous cache that does exist (with the intent of running ./setup_test_cache /tmp u on it later)
There was a problem hiding this comment.
I'm not really sure what is happening there. I know there are differences between the two contexts, but I feel like the test cache had nothing to update.
zacharyburnett
commented
Nov 25, 2024
Comment on lines
-94
to
+91
| - if: steps.lookup-combined-cache.outputs.cache-hit != 'true' | ||
| run: ./setup_test_cache ${{ env.CRDS_TEST_ROOT }} ${{ (steps.retrieve-hst-cache.outputs.cache-hit == 'true' || steps.retrieve-jwst-cache.outputs.cache-hit == 'true') && 'u' || 'c' }} | ||
| - if: steps.lookup-combined-cache.outputs.cache-hit != 'true' | ||
| - if: steps.lookup-cache.outputs.cache-hit != 'true' | ||
| run: ./setup_test_cache ${{ env.CRDS_TEST_ROOT }} u |
There was a problem hiding this comment.
we don't ever need to run with c here, only u
hbrown-ST
approved these changes
Nov 26, 2024
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This should decrease the time it takes to make a new test cache for the latest contexts, when the contexts change. I need to test this against a context change to make sure that
ubehaves as expected EDIT: tested in #1099 (comment)This also removes duplication of caches by obviating the need for separate individual caches for each repository
Tasks
docs/pageno-changelog-entry-needed)changes/:echo "changed something" > changes/<PR#>.<changetype>.rst(see below for change types)news fragment change types...
changes/<PR#>.hst.rst: HST reference fileschanges/<PR#>.jwst.rst: JWST reference fileschanges/<PR#>.roman.rst: Roman reference fileschanges/<PR#>.doc.rst: documentation changechanges/<PR#>.testing.rst: change to tests or test automationchanges/<PR#>.general.rst: infrastructure or miscellaneous change