pip not accepting locally installed dev version of dependency

[thony-murray]

Note

The rest of this issue description was copied directly from pypa/pip#10817 in response to this comment: pypa/pip#10817 (comment)

Description

I am trying to install a package with a mymodule~=0.16 dependency.

Locally, I have mymodule-0.16.0.dev1234 installed.

When I run pip install, I get the following error: ERROR: No matching distribution found for mymodule~=0.16

Expected behavior

I expect a pre-installed dev version to be accepted in place of the regular version.

From PEP 440 (https://www.python.org/dev/peps/pep-0440/#handling-of-pre-releases):

Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system, explicitly requested by the user, or if the only available version that satisfies the version specifier is a pre-release.

pip version

21.3.1

Python version

3.8.12

OS

macOS Monterey version 12.1

How to Reproduce

Unfortunately the two modules in question are proprietary so you cannot reproduce with the exact modules. But in general the steps are

1. Install a X.Y.0.devZ version of a module.
2. Install a module with a ~=X.Y requirement.

Output

$ pip install mymodule
...
...
Successfully installed mymodule-0.16.0.dev1234

$ pip install othermodule
Processing /path/to/othermodule
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
ERROR: Could not find a version that satisfies the requirement mymodule~=0.16 (from othermodule) (from versions: none)
ERROR: No matching distribution found for mymodule~=0.16

[brettcannon]

So I see we are considering your requested version as not matching:

>>> import packaging.version, packaging.specifiers
>>> version = packaging.version.Version("0.16.0.dev1234")
>>> version
<Version('0.16.0.dev1234')>
>>> specifier = packaging.specifiers.SpecifierSet("~=0.16")
>>> version in specifier
False

https://www.python.org/dev/peps/pep-0440/#developmental-releases says .dev comes before what's specified, so '0.16.0.dev1234' comes before 0.16.0. So it seems like 0.16.0.dev1234 is between 0.16.0 and 0.16.-1 (if there was such a thing). I guess that could be viewed as working with >=0.16, but I can also see it going the other way. I can't find any clear answer one way or the other in https://www.python.org/dev/peps/pep-0440/ .

[thony-murray]

Yeah you're correct that 0.16.0.dev1234 is before 0.16.0. However PEP 440 says the following (https://www.python.org/dev/peps/pep-0440/#handling-of-pre-releases):

Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system, explicitly requested by the user, or if the only available version that satisfies the version specifier is a pre-release.

Since 0.16.0.dev1234 is already on the system, it should satisfy the 0.16.0 requirement.

[uranusjr]

Since there are two different comparison logic (depending whether the package is already installed in the local environment), we may need to add an argument to SpecifierSet.contains(), or augment prereleases to take additional variations.

[brettcannon]

Since 0.16.0.dev1234 is already on the system, it should satisfy the 0.16.0 requirement.

Seems like it, but packaging doesn't know what's installed ...

Since there are two different comparison logic (depending whether the package is already installed in the local environment), we may need to add an argument to SpecifierSet.contains(), or augment prereleases to take additional variations.

... hence we may need a way to be told that a version is installed. 😄 Maybe a keyword-only installed argument to contains() to support the logic @tonyfm15 quoted?

[uranusjr]

Maybe a keyword-only installed argument to contains()

Aye that sounds reasonable to me.

[q0w]

It seems, that adding only installed argument to Specifier.contains() is not a solution, because of the mechanism of comparing 2 Version instances through self._key, which сomes from _cmkey(), where _pre becomes NegativeInfinity, if the version is a prerelease (as here). So maybe Version should get one more installed argument to be initialized, to modify _cmkey(..., installed) behavior:

---if pre is None and post is None and dev is not None:
+++if pre is None and post is None and dev is not None and not installed:
---if dev is None:
+++if installed or dev is None:

and _compare_greater_than_equal:

---return Version(prospective.public) >= Version(spec)
+++return Version(prospective.public, installed=prospective.is_installed) >= Version(spec)

and parse could accept installed arg for Version

[brettcannon]

Versions in and of themselves have no concept of installation; they are basically just a weird number. So I would rather not change the constructor of Version to start denoting whether a Version is "installed" or not. Since it is only how we choose to interpret the versions that is in question I would still rather make it work via Specifier.contains(). If that requires a bit more code changes to make it work then so be it.