Skip to content

Avoid using setup_requires parameter in setup.py. #136

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
FedericoRessi opened this issue Apr 10, 2017 · 4 comments
Closed

Avoid using setup_requires parameter in setup.py. #136

FedericoRessi opened this issue Apr 10, 2017 · 4 comments

Comments

@FedericoRessi
Copy link

Pip doesn't support installation packages using setup_requires because that option causes distutils to install packages without using PIP. Therefore any PIP configuration are ignored for packages listed in setup_requires. You should instead probably specify project requirements in a file called requirements.txt.

https://pip.readthedocs.io/en/1.4.1/cookbook.html#controlling-setup-requires

@RonnyPfannschmidt
Copy link
Member

once pip uses https://www.python.org/dev/peps/pep-0518/ (tracked in pypa/pip#3691 )
that won't change unless someone proposes a working solution that is not take out the automation

@FedericoRessi
Copy link
Author

It is nice to see there this pep 518 that would replace a de-facto unspecified standard with a documented one. Anyway using pip with internal pip mirrors is so common in corporative CI that it is a pity this package doesn't install without configuring the index for easy_install too.

To have the automation without relaying on setup_requires there could be some code in the setup.py that would try to import pip and use it to install as much of those packages as possible before passing remaining list as setup.

Example of setup.py

from setuptools import setup
import traceback

extra_params = {}
setup_requires = ['setuptools_scm']
try:
    import pip
    pip.main(['install'] + setup_requires)
    setup_requires = []
except Exception:
    # Going to use easy_install for
    traceback.print_exc()

setup(
    # [... other parameters here ...]
    setup_requires=setup_requires
    # [... other parameters here ...]
)

Using Pip instead of easy_install has a lot of advantages and having to deal with one installation tool instead of two would be nice and make CI behaviour more deterministic.

@RonnyPfannschmidt
Copy link
Member

@FedericoRessi for stable ci i install tools like pbr/setuptools_scm before installing the main requirements

having crude hack in ci pipelines is deterministic

btw, your example would mess up a downstream users system state in case they invoked a pip download and hit a sdist (too smart setup.py has been tried, it always breaks someone unexpectedly as well)

rayosborn referenced this issue in nexpy/nexpy Nov 20, 2017
That should fix the build dependency problem #60
ulfalizer added a commit to zephyrproject-rtos/west that referenced this issue Sep 19, 2018
The self-update mechanism checks the West and manifest repositories in
west/ for upstream changes and rebases them to the latest version if any
are found.

'west fetch' and 'west pull' run a self-update before fetching any
project data. It probably makes sense to always work with the latest
version of West and the manifest there. The self-update can be
suppressed with --no-update.

There's also a 'west update' command for manually updating West and/or
the manifest.

West automatically restarts itself after a self-update, before running
any commands or parsing the manifest. This makes it safe to add a new
feature to West and immediately start using it in the manifest.

The restarting is done with a plain exec(3). Getting this to work
required a few changes:

 - West now uses absolute instead of relative imports, to make it
   possible to execute src/west/main.py directly. This also makes it a
   bit simpler to work on West. Relative imports seem to be discouraged
   when googling around a bit.

   __init__.py was removed from src/west. West is more of an application
   than a package now (though it uses some packages internally).

 - The internal 'cmd' package was renamed to 'commands', to avoid
   clashing with a module in the Python standard library.

   'runner' was renamed to 'runners' as well, for consistency.

Using exec(3) also allowed the argv handling to be simplified a bit.

The tests now have to be run from the West repository root with
'PYTHONPATH=src/west pytest'. Maybe this could be improved. The West
module and packages have to get into sys.path somehow at least.

__init__.py was removed from the test directories as well. Don't think
having the tests in a package helps now.

A build fix and four unrelated changes are included as well:

 - setuptools-scm was added to test_requirements.txt to work around an
   issue on Python 3.4 (not sure why it didn't appear before):

     distutils.errors.DistutilsError: Could not find suitable
     distribution for Requirement.parse('setuptools-scm>=1.15.0')

   See pypi/legacy#322 and
   pytest-dev/pytest-xdist#136. I suspect the
   issue is related to setup_requires= using easy_install.

 - args.zephyr_base is no longer set to ZEPHYR_BASE, just the other way
   around. args.zephyr_base is currently unused except for setting
   ZEPHYR_BASE. I think it might be cleaner to add a new variable
   instead of directly modifying 'args' if the ZEPHYR_BASE value is
   needed elsewhere later.

 - WEST_TOPDIR was renamed to WEST_MARKER. The old name made me think
   that it was the path to the directory containing west/.

 - Project-related command-line arguments that are only used by a single
   command are now defined together with the command that uses them.

 - A bug was fixed in the bootstrapper's find_west_topdir(). The 'start'
   argument was being ignored.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
ulfalizer added a commit to zephyrproject-rtos/west that referenced this issue Sep 21, 2018
Add setuptools-scm to test_requirements.txt to work around an issue on
Python 3.4:

  distutils.errors.DistutilsError: Could not find suitable
  distribution for Requirement.parse('setuptools-scm>=1.15.0')

Not sure why it didn't appear before. Maybe something in CI changed.

See pypi/legacy#322 and
pytest-dev/pytest-xdist#136. I suspect the
issue is related to setup_requires= using easy_install.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
mbolivar pushed a commit to zephyrproject-rtos/west that referenced this issue Sep 21, 2018
Add setuptools-scm to test_requirements.txt to work around an issue on
Python 3.4:

  distutils.errors.DistutilsError: Could not find suitable
  distribution for Requirement.parse('setuptools-scm>=1.15.0')

Not sure why it didn't appear before. Maybe something in CI changed.

See pypi/legacy#322 and
pytest-dev/pytest-xdist#136. I suspect the
issue is related to setup_requires= using easy_install.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
maphew added a commit to maphew/setupext-janitor that referenced this issue Feb 8, 2019
@soerenwolfers
Copy link

This has been fixed in setuptools 42.0.0, which switched to using pip to resolve setup_requires (https://setuptools.readthedocs.io/en/latest/history.html#v42-0-0).

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants