-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Cache NUMBA kernels between CI runs #279
Merged
Merged
Changes from 2 commits
Commits
Show all changes
50 commits
Select commit
Hold shift + click to select a range
eb8c568
Cache NUMBA kernels between CI runs
sjperkins dd6bbeb
Use actions/cache@v3
sjperkins d4cba34
Cache per python version
sjperkins 3ab69ad
runner.tmp -> runner.temp
sjperkins 353ae63
Debugging
sjperkins a60360d
Fix
sjperkins 707b176
Run entire test suite
sjperkins 5c3aed0
timestamp needed otherwise cache hit occurs and cache not updated
sjperkins edc58e8
Fix output
sjperkins 7ef4776
Add revert_me.txt
sjperkins 55fd7ba
Merge branch 'main' into cache-numba-kernels
JSKenyon d9f69ec
Use nearest-neighbour interpolation in regions where extrapolation is…
JSKenyon 0c58b33
Utilise environment variable when dask.address is unset. (#288)
JSKenyon 6a9fcf4
Add plotting functionality (#290)
JSKenyon 7cd19eb
Fix #293 - OOB access caused by `output.subtract_directions` (#294)
JSKenyon d0b6653
Namedbackups (#296)
landmanbester 6bed6f2
Selectively disable MAD flagging criteria (#298)
JSKenyon 7630360
Disable mad flagging on off-diagonals by default (#300)
JSKenyon eaa7515
Fix bug affecting non-standard columns in `input_ms.data_column` (#301)
JSKenyon 81a2ef8
Don't allow restore app to overwrite metadata (#307)
landmanbester 5478340
Fix for summary reporting SOURCE_ID as FIELD_ID (#309)
JSKenyon 9a3be80
Fix receptor summary (#310)
JSKenyon 84e317a
Fix xarray dims (#318)
JSKenyon e7e03f1
Fixes for changes relating to Numba error types. (#319)
JSKenyon abb60db
Move now-deprecated graph metrics function into the scheduler plugin …
JSKenyon 20939b3
Make small changes to enable 3.11 compatibilty. Requires changes in s…
JSKenyon ec4cf8b
Restringify keys in scheduler plugin. (#322)
JSKenyon 8008ab3
Merge branch 'v0.2.1-dev' into cache-numba-kernels
JSKenyon 2522829
Attempt very dodgy solution to caching problem.
JSKenyon 6e13b25
Look for code in the correct place.
JSKenyon 3eb7c5d
Update pyproject.toml. Add poetry.lock. Update docs. (#323)
JSKenyon 79158e2
Some debugging.
JSKenyon 92da770
Merge v0.2.1-dev.
JSKenyon e83fd40
Fix unsaved file.
JSKenyon 6348e00
More debugging.
JSKenyon 92326a7
Temporarily make test suite much smaller.
JSKenyon d1c67a9
Fix path.
JSKenyon 16bdee5
Actually fix path.
JSKenyon c2a3b29
Attempt at safer caching.
JSKenyon 0e97a1c
Merge in v0.2.2-dev
JSKenyon 32f5950
More fiddling with paths.
JSKenyon 3db9902
Fix bad tabbing.
JSKenyon 61f61b5
Try to find out where things are failing.
JSKenyon 4d1dca0
More fiddling.
JSKenyon 3d21087
More fiddling.
JSKenyon 6318f47
More fiddling.
JSKenyon f37fa83
Try restore time action.
JSKenyon d270cff
Tidy up caching approach. Use action. Restore matrix and test everyth…
JSKenyon 1da4879
Remove tmp file.
JSKenyon 65c5efa
Reword CI step name.
JSKenyon File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Constructing the key out of the date may be overkill. I suspect we could just use
numba-cache
and it would propagate and be updated between runs.I guess the downside is that it might accumulate a bunch of crufty old kernels. Note AFAICT there's a 10GB cache limit per repo and cache entries expire weekly so it may not be a big deal.
@bennahugo suggested we add the numba version to the cache key. I wonder if numba is clever enough to trigger recompiles on new numba versions.
The python version may also be relevant given a codex
__pycache__
dir looks as follows