Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

CI: non-remote GH Actions now use a python venv #16789

Merged
merged 3 commits into from
Apr 18, 2022

Conversation

gante
Copy link
Member

@gante gante commented Apr 14, 2022

What does this PR do?

This PR changes the cached python environment in non-remote GH actions to fresh virtual environments, that are rebuilt whenever setup.py gets updated. This ensures our test environment is closer to what a new user would see in the wild, and that we don't have package constraints due to other installed libraries (that come bundled in ubuntu-latest).

As a side effect, because the cached venv requires no installation of python packages on a cache hit, we also got slightly faster CI:

  • >3 mins faster in Add new model like template tests
  • >3 mins faster in Model templates runner
    (total of ~7 mins per full CI run when there is a cache hit)

Examples of runs:
cache miss -> create new venv: https://github.com/huggingface/transformers/runs/6027339356?check_suite_focus=true
cache hit -> load cached venv: https://github.com/huggingface/transformers/runs/6028381114?check_suite_focus=true

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Apr 14, 2022

The documentation is not available anymore as the PR was closed or merged.

@gante gante marked this pull request as ready for review April 14, 2022 17:58
@gante
Copy link
Member Author

gante commented Apr 14, 2022

(cc @ydshieh )

Copy link
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks a lot for diving into this!

Copy link
Member

@LysandreJik LysandreJik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @gante!

@gante gante merged commit 02de7a8 into huggingface:main Apr 18, 2022
@gante gante deleted the gha_with_venv branch April 18, 2022 08:47
elusenji pushed a commit to elusenji/transformers that referenced this pull request Jun 12, 2022
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants