tox with dependencies from Artifact Registry (GCP)

[fedexist]

Hi,

I'm currently trying to use tox with a project which has dependencies stored on Artifact Registry, the Google Cloud Platform managed PyPi index, but I can't, due to how PIP_EXTRA_INDEX_URL works for both provisioning of tox and testenv.

This is my tox.ini

[tox]
envlist = py38, py39, flake8, docs
skipsdist = true
minversion = 3.24
isolated_build = True
requires =
    keyrings.google-artifactregistry-auth

[testenv]
usedevelop = True
description = invoke pytest to run automated tests
setenv =
    PYTHONPATH = {toxinidir}/src
    TOXINIDIR = {toxinidir}
    MYPYPATH = {toxinidir}/src
    PIP_EXTRA_INDEX_URL = https://europe-west3-python.pkg.dev/redacted-project-id/pypi/simple/
passenv =
    HOME
    GOOGLE_APPLICATION_CREDENTIALS
deps =
    -r{toxinidir}/requirements_dev.txt
commands =
    # various test commands
extras =
    testing

[testenv:{build,clean}]
description =
    build: Build the package in isolation according to PEP517, see https://github.com/pypa/build
    clean: Remove old distribution files and temporary build artifacts (./build and ./dist)
# https://setuptools.pypa.io/en/stable/build_meta.html#how-to-use-it
skip_install = True
changedir = {toxinidir}
basepython = python3.9
deps =
    setuptools
    setuptools-scm
    wheel
    build: build[virtualenv]
commands =
    clean: python -c 'import shutil; [shutil.rmtree(p, True) for p in ("build", "dist", "docs/_build")]'
    clean: python -c 'import pathlib, shutil; [shutil.rmtree(p, True) for p in pathlib.Path("src").glob("*.egg-info")]'
    build: python -m build {posargs}

[testenv:publish]
description =
    Publish the package you have been developing to a package index server.
    By default, it uses testpypi. If you really want to publish your package
    to be publicly accessible in PyPI, use the `-- --repository pypi` option.
skip_install = True
changedir = {toxinidir}
basepython = python3.9
passenv =
    PYPI_REPO
    GOOGLE_APPLICATION_CREDENTIALS
    HOME
deps =
    twine
commands =
    python3.9 -m twine check dist/*
    python3.9 -m twine upload --skip-existing --verbose --non-interactive {posargs:--repository-url {env:PYPI_REPO:http://default-pypi}} dist/*

As per the documentation provided by Google, I set the keyring as required and use passenv to make sure that the environment variable GOOGLE_APPLICATION_CREDENTIALS is propagated to the testenv.

What happens is the following:

1. Without setting PIP_EXTRA_INDEX_URL, tox -e clean,build,publish works: this means that the keyring plugin gets installed during the provisioning and is actually used for authenticating with Artifact Registry
2. Setting PIP_EXTRA_INDEX_URL, tox can't get past the provisioning phase because it tries to authenticate with the Artifact Registry in order to install keyrings.google-artifactregistry-auth: by default username and password are asked via prompt and tox gets stuck there, because obviously it can't input anything :)

So, what is clear here is that PIP_EXTRA_INDEX_URL is used for both tox provisioning and testenv package installation and I have a few questions:

• Is it even possible to set PIP_EXTRA_INDEX_URL only for the testenv?
• Is it possible for me, for example, to set GOOGLE_APPLICATION_CREDENTIALS during tox provisioning? Using passenv and setenv in that section didn't help.
• Is it possible to install keyrings.google-artifactregistry-auth as a sort of seed package, before everything else? Maybe there's a plugin for that?

I realize that I could put the package dependency in a requirements.txt file with the --extra-index-url flag (that is what's actually shown on the GCP github page linked earlier), but I was wondering if tox could manage this issue in a more elegant way. Or maybe I'm simply missing a way to already do what I want.

I see also that there's a somewhat related issue with #2428, regarding how install_command is used.

Thank you for your help.