wheel 0.38.0 causes circular dependency with setuptools

[JelleZijlstra]

Since wheel 0.38.0 was released just now, installing certain packages through pip is broken:

$ python3.11 -m pip install pyzmq==22.0.3 --install-option=--zmq=bundled
WARNING: Implying --no-binary=:all: due to the presence of --build-option / --global-option / --install-option. Consider using --config-settings for more flexibility.
DEPRECATION: --install-option is deprecated because it forces pip to use the 'setup.py install' command which is itself deprecated. pip 23.1 will enforce this behaviour change. A possible replacement is to use --config-settings. Discussion can be found at https://github.com/pypa/pip/issues/11358
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Collecting pyzmq==22.0.3
  Downloading pyzmq-22.0.3.tar.gz (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 10.6 MB/s eta 0:00:00
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [100 lines of output]
      DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
      Collecting setuptools>=40.8.0
        Downloading setuptools-65.5.0.tar.gz (2.6 MB)
           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 25.5 MB/s eta 0:00:00
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'error'
        error: subprocess-exited-with-error
      
        × pip subprocess to install backend dependencies did not run successfully.
        │ exit code: 2
        ╰─> [77 lines of output]
            DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
            Collecting wheel
              Downloading wheel-0.38.0.tar.gz (65 kB)
                 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.7/65.7 kB 1.3 MB/s eta 0:00:00
              Preparing metadata (setup.py): started
              Preparing metadata (setup.py): finished with status 'done'
            Collecting setuptools>=57.0.0
              Using cached setuptools-65.5.0.tar.gz (2.6 MB)
            ERROR: Exception:
            Traceback (most recent call last):
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
                status = run_func(*args)
                         ^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
                return func(self, options, args)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 400, in run
                requirement_set = resolver.resolve(
                                  ^^^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
                result = self._result = resolver.resolve(
                                        ^^^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
                state = resolution.resolve(requirements, max_rounds=max_rounds)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 373, in resolve
                failure_causes = self._attempt_to_pin_criterion(name)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
                criteria = self._get_updated_criteria(candidate)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 204, in _get_updated_criteria
                self._add_to_criteria(criteria, requirement, parent=candidate)
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
                if not criterion.candidates:
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
                return bool(self._sequence)
                       ^^^^^^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
                return any(self)
                       ^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
                return (c for c in iterator if id(c) not in self._incompatible_ids)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
                candidate = func()
                            ^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
                self._link_candidate_cache[link] = LinkCandidate(
                                                   ^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
                super().__init__(
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
                self.dist = self._prepare()
                            ^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
                dist = self._prepare_distribution()
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
                return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 491, in prepare_linked_requirement
                return self._prepare_linked_requirement(req, parallel_builds)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 577, in _prepare_linked_requirement
                dist = _get_prepared_distribution(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 68, in _get_prepared_distribution
                with build_tracker.track(req):
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 137, in __enter__
                return next(self.gen)
                       ^^^^^^^^^^^^^^
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/operations/build/build_tracker.py", line 122, in track
                self.add(req)
              File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pip/_internal/operations/build/build_tracker.py", line 92, in add
                raise LookupError(message)
            LookupError: https://files.pythonhosted.org/packages/c5/41/247814d8b7a044717164c74080725a6c8f3d2b5fc82b34bd825b617df663/setuptools-65.5.0.tar.gz (from https://pypi.org/simple/setuptools/) (requires-python:>=3.7) is already being built: setuptools>=40.8.0 from https://files.pythonhosted.org/packages/c5/41/247814d8b7a044717164c74080725a6c8f3d2b5fc82b34bd825b617df663/setuptools-65.5.0.tar.gz
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: subprocess-exited-with-error
      
      × pip subprocess to install backend dependencies did not run successfully.
      │ exit code: 2
      ╰─> See above for output.
      
      note: This error originates from a subprocess, and is likely not a problem with pip.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

The Internet tells me "is already being built" errors mean there is some sort of circular dependency. I suspect the change in https://github.com/pypa/wheel/compare/0.37.1..0.38.0#diff-fa602a8a75dc9dcc92261bac5f533c2a85e34fcceaff63b3a3a81d9acde2fc52 moving setuptools from setup_requires to install_requires is to blame.

[agronholm]

As distutils is deprecated, wheel needed to switch to setuptools, hence the added dependency. I've yanked the release now to stem further bug reports.

[JelleZijlstra]

Thanks! I found that --no-build-isolation works as a workaround (in a venv that already has wheel/setuptools installed).

[jmroot]

Cross-referencing the relevant setuptools issue here: pypa/setuptools#2986

#429 here is also relevant.

[agronholm]

I created a minimal PR to remove the setuptools run-time dependency.