Switch GitHub Actions to install environments with mamba

[michaelosthege]

Our CI pipelines are taking quite a bit of compute time and per job already 3.5 minutes are just for creating the conda environments.

Switching to mamba could save quite a lot of compute time & energy there!

[michaelosthege]

When doing this, it'd be great to combine pre-commit.yml, arviz_compat.yml and pytest.yml into the same workflow.
Then we could fail-fast all jobs when the pre-commit doesn't pass (#5041)

[parthb83]

Hey @michaelosthege! I would like to work on this issue. Can you please guide me through the process of resolving this issue.

[michaelosthege]

Hey @michaelosthege! I would like to work on this issue. Can you please guide me through the process of resolving this issue.

Most importantly these lines will need to be changed: https://github.com/pymc-devs/pymc/blob/main/.github/workflows/pytest.yml#L114-L140

Try to google for other example where GitHub actions are used with mamba. Maybe there are some equivalents to the cache & setup actions in the aforementioned lines.

[parthb83]

Most importantly these lines will need to be changed: https://github.com/pymc-devs/pymc/blob/main/.github/workflows/pytest.yml#L114-L140
Try to google for other example where GitHub actions are used with mamba. Maybe there are some equivalents to the cache & setup actions in the aforementioned lines.

- name: Cache conda
        uses: actions/cache@v1
        env:
          # Increase this value to reset cache if environment-test-py37.yml has not changed
          CACHE_NUMBER: 0
        with:
          path: ~/conda_pkgs_dir
          key: ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{
            hashFiles('conda-envs/environment-test-py37.yml') }}
      - name: Cache multiple paths
        uses: actions/cache@v2
        env:
          # Increase this value to reset cache if requirements.txt has not changed
          CACHE_NUMBER: 0
        with:
          path: |
            ~/.cache/pip
            $RUNNER_TOOL_CACHE/Python/*
            ~\AppData\Local\pip\Cache
          key: ${{ runner.os }}-build-${{ matrix.python-version }}-${{
            hashFiles('requirements.txt') }}
      - uses: conda-incubator/setup-miniconda@v2
        with:
          mamba-version: "*"
          use-mamba: true
          activate-environment: pymc-test-py37
          channel-priority: strict
          environment-file: conda-envs/environment-test-py37.yml
          use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!

Ok so I changed the code by adding 2 lines mamba-version: "*" and use-mamba: true. Do you think this is sufficient to create conda environment with mamba? If so, please tell a way to run this on github actions to validate the solution.

[michaelosthege]

I don't know if this is sufficient, but probably not.

You should commit the changes and push them to a branch on your fork (https://github.com/parthb83/pymc) the CI actions should be triggered on your fork.
After you got the switch to mamba working on your fork the next step is to open a draft pull request here.

[parthb83]

I don't know if this is sufficient, but probably not.

You should commit the changes and push them to a branch on your fork (https://github.com/parthb83/pymc) the CI actions should be triggered on your fork. After you got the switch to mamba working on your fork the next step is to open a draft pull request here.

After pushing the changes to my forked repository, no CI action was triggered.

[michaelosthege]

Try to push it to your main branch.
Not sure what the exact rules are on when GH Actions are triggered, but it's probably somewhere in the GitHub documentation

[parthb83]

Hey @michaelosthege! Please check the test results here (https://github.com/parthb83/pymc/actions/runs/1743967089). I think the conda environment is being created by mamba. Please validate the results.

[michaelosthege]

looks good, @parthb83 !
It looks like it's at least a little faster than conda so that's good 👍

Can you switch the Windows jobs (further down in the file) over to mamba too?

And then open a PR (GitHub should give you a button when you visit the pull requests page).

[parthb83]

looks good, @parthb83 ! It looks like it's at least a little faster than conda so that's good +1

Can you switch the Windows jobs (further down in the file) over to mamba too?

And then open a PR (GitHub should give you a button when you visit the pull requests page).

Hey @michaelosthege! Thanks for the help. You can review the PR at #5387.

[ricardoV94]

Is this finished @parthb83?

[twiecki]

I think so.