Skip to content

Commit

Permalink
CI: caching: add early termination & run check on schedule (#2506)
Browse files Browse the repository at this point in the history
* CI: caching: add early termination

* CI: caching: run cache check 3 times a day

Recently there is a lot of PRs/builds from internal repo branches. Repo cache
has 10G pool limit & PR caches are not shared between PRs, but main branch cache
is.

Checking & when missing rebuilding it ensures that if internal branch PRs pushed main state
out of the pool - it gets reinstituted.

*afaik* there is no way to prioritize main branch cache better, since GitHub CI
FIFO caches out of the pool by build time.

Co-authored-by: Javier Neira <atreyu.bbb@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
  • Loading branch information
3 people authored Dec 21, 2021
1 parent 070f16f commit d1e6d3c
Showing 1 changed file with 20 additions and 3 deletions.
23 changes: 20 additions & 3 deletions .github/workflows/caching.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,18 @@ on:
push:
branches:
- master
schedule:
# Refresh snapshot every (02+8*x):25 UTC
# When cache is present it is a light check workflow with early termination.
# When primary cache is not hit - runs the cache generation.
# Why: GitHub repo has 10G pool & on overflow GitHub removes caches in FIFO manner.
# When internal branche PRs save into the same pool -
# their cache is accessible only inside of the scope of the PR.
# If main cache is forced out - there are no cache shared between PRs,
# which implies all PRs would start to create & save their cache.
# Reinstitution of the main chache puts it back into FIFO
# & so it gets shared across all PRs.
- cron: "25 2/8 * * *"

env:
cabalBuild: "v2-build all --enable-tests --enable-benchmarks"
Expand Down Expand Up @@ -130,6 +142,7 @@ jobs:
restore-keys: ${{ env.cache-name }}-

- name: Compiled deps cache
id: compiled-deps
uses: actions/cache@v2
env:
cache-name: compiled-deps
Expand All @@ -141,9 +154,12 @@ jobs:
${{ env.cache-name }}-${{ runner.os }}-${{ matrix.ghc }}-
${{ env.cache-name }}-${{ runner.os }}-
- run: cabal update
- if: (! steps.compiled-deps.outputs.cache-hit)
run: |
cabal update
- name: Download all sources
- if: (! steps.compiled-deps.outputs.cache-hit)
name: Download all sources
run: |
cabal $cabalBuild --only-download
Expand All @@ -152,7 +168,8 @@ jobs:
# but to cache what can be cached, so step is fault tolerant & would always succseed.
# 2021-12-11: NOTE: Building all targets, since
# current Cabal does not allow `all --enable-tests --enable-benchmarks --only-dependencies`
- name: Build all targets; try 3 times
- if: (! steps.compiled-deps.outputs.cache-hit)
name: Build all targets; try 3 times
continue-on-error: true
run: |
cabal $cabalBuild || cabal $cabalBuild || cabal $cabalBuild

0 comments on commit d1e6d3c

Please # to comment.