pip install does not seem to evaluate environment markers correctly per PEP 508

[wimglenn]

Description

Note: Originally reported as an issue on uv, but upon closer consideration the behavior of uv pip install seems to be correct, and it is pip that looks bugged.

PEP 508 has this to say about environment markers:

The <marker_op> operators that are not in <version_cmp> perform the same as they do for strings in Python. The <version_cmp> operators use the PEP 440 version comparison rules when those are defined (that is when both sides have a valid version specifier). If there is no defined PEP 440 behaviour and the operator exists in Python, then the operator falls back to the Python behaviour.

Expected behavior

When an environment marker has a version comparison with a value that is not a valid PEP 440 version, the marker evaluation should fall back to the Python operation i.e. lexicographic string comparisons. pip install does not seem to do the "fall back to the Python behaviour" part.

pip version

24.0

Python version

3.12.3

OS

Linux

How to Reproduce

I use python:3.12.3-alpine image as an example because it has platform.release() value of "5.10.104-linuxkit" which can not be parsed as a PEP 440 version.

$ docker run --rm -it python:3.12.3-alpine sh
/ # mkdir p
/ # cd p
/p # vi pyproject.toml
/p # cat pyproject.toml 
[project]
name = "example"
version = "0.1"
dependencies = [
    "six==1.16.0; platform_release >= '5'",
    "six==1.15.0; platform_release < '5'",
]

/p # python3 -c 'import platform; print(platform.release())'
5.10.104-linuxkit
/p # python3 -c 'import platform; print(platform.release() >= "5")'
True
/p # pip install .
Processing /p
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: example
  Building wheel for example (pyproject.toml) ... done
  Created wheel for example: filename=example-0.1-py3-none-any.whl size=969 sha256=0065789fd42d65789bda8490f1ff93370c8883cc83e78c47818cb3ec4b5ee45f
  Stored in directory: /tmp/pip-ephem-wheel-cache-txslb76p/wheels/86/2f/28/da4d880cd8f840d16331bda7b3720553d63c39a3820e84e9db
Successfully built example
Installing collected packages: example
Successfully installed example-0.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
/p # pip list
Package    Version
---------- -------
example    0.1
pip        24.0
setuptools 70.0.0
wheel      0.43.0

In this context, the dependency six==1.16.0 should have been selected and installed.

/p # cat /usr/local/lib/python3.12/site-packages/example-0.1.dist-info/METADATA 
Metadata-Version: 2.1
Name: example
Version: 0.1
Requires-Dist: six ==1.15.0 ; platform_release < "5"
Requires-Dist: six ==1.16.0 ; platform_release >= "5"

/p # pip check
No broken requirements found.

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

Can you test on an older pip version? I suspect this is a side effect of packaging removing LegacyVersion, which provides a fallback to lexical comparison.

[wimglenn]

Hmm, seems unlikely given that pip 24.0 vendors packaging 21.3 and LegacyVersion is still there until packaging 22.0. I tried a few older pip versions and the issue persists.

[uranusjr]

Oh, I thought 24.0 already upgrades packaging; I guess that’s 24.1 instead.

[twslankard]

This issue appears to happen on other Linuxes too. On Ubuntu boxes I can't use the platform_release marker because none of the kernel versions are PEP 440 (unsurprisingly.)

This could be remediable if pip's marker evaluation used short-circuit boolean behavior. To illustrate, the following example fails with pip 24.1 on Ubuntu, even though there is no need to evaluate the remaining markers because the system isn't a macOS/Darwin system.

setup(
    name='examplepackage',
    install_requires=[
        "six; (platform_system == 'Darwin' and platform_release > '21.0.0')"
    ]
)

Output on a particular Ubuntu box:

pip._vendor.packaging.version.InvalidVersion: Invalid version: '6.5.0-26-generic'

[ichard26]

Hello. Thanks for the detailed report! You are indeed correct that pip is violating the specification here. However, marker evaluation is implemented by pypa/packaging. During the time between your original report and now, I've filed pypa/packaging#816 to get packaging in compliance with the spec.