Update fetch_dependencies #194
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aplaice
force-pushed
the
fix_fetch_dependencies
branch
from
432a67d to
2634e1f
Compare
Remove packages dataclasses and typing which are both built-in for python3.7. Pipfile.lock is regenerated from scratch, updating everything!
We need to replace --binary :all: with --binary $(pipenv requirements | sed -n 's/==.*//p' | tr '\n' ',') because otherwise pip tries to build everything from scratch (including the build system (??)). We need the new dulwich options: --config-settings "--global-option=--pure" because pip has changed the application of build options, so building pure python packages becomes harder. See dulwich issue 1093.
It seems that Anki 2.1.35 (the last version compatible with python3.7) is not actually compatible with the latest protobuf (possibly released after Anki 2.1.35). See: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates Both anki and protobuf are dev dependencies so this incompatibility only affects testing. We will be able to remove this workaround by upgrading to python3.9 (compatible with latest Anki).
aplaice
force-pushed
the
fix_fetch_dependencies
branch
from
2634e1f to
9e5fd2f
Compare
I don't fully understand why, but `pipenv run foo <(bar)` process substitution doesn't work in GitHub actions. AFAICT in a "normal" environment, pipenv execs into its subcommand process (here `foo`), and the subcommand inherits all its file descriptors (including the file descriptor for `<(bar)`, inherited from `fetch_dependencies.sh`. In GitHub actions, for unknown, to me, reasons, pipenv instead creates a separate child process for `foo` and does not share its file descriptors, so `foo` doesn't have access to `<(bar)`.
pyyaml detects the presence of the libyaml library, and builds with the libyaml extension if it's present. In principle, there's a --without-libyaml flag, to ensure building without libyaml even if the library is present, but it doesn't work. (Also, there is an issue with passing on `--without-libyaml` via pip to setup.py.) PYYAML_FORCE_LIBYAML=0 is not documented, but works. Another workaround would be to ensure that libyaml is not present, but that's brittle, since then the build system starts depending on global configuration (presence/absence) of system libraries.
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We need to replace
--binary :all:with--binary $(pipenv requirements | sed -n 's/==.*//p' | tr '\n' ',')because otherwise pip tries to build everything from scratch (including the build system (??)).We need the new dulwich options:
because pip has changed the application of build options, so building pure python packages becomes harder. See dulwich issue 1093.
Update Pipfile.lock
Remove packages dataclasses and typing which are both built-in for python3.7.
Pipfile.lock is regenerated from scratch, updating everything!
Force protobuf python implementation
It seems that Anki 2.1.35 (the last version compatible with python3.7) is not actually compatible with the latest protobuf (possibly released after Anki 2.1.35).
See:
https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
Both anki and protobuf are dev dependencies so this incompatibility only affects testing.
We will be able to remove this workaround by upgrading to python3.9 (compatible with latest Anki). (See: #183.)
Avoid pipenv process substitution in fetch_dependencies.sh
I don't fully understand why, but
pipenv run foo <(bar)processsubstitution doesn't work in GitHub actions.
AFAICT in a "normal" environment, pipenv execs into its subcommand process (here
foo), and the subcommand inherits all its file descriptors (including the file descriptor for<(bar), inherited fromfetch_dependencies.sh.In GitHub actions, for unknown, to me, reasons, pipenv instead creates a separate child process for
fooand does not share its file descriptors, sofoodoesn't have access to<(bar).