Multihost tests: python-versioneer error

[ZlobinDA]

Hello!

I ran SSSD multihost tests at Fedora 39 server using manual https://sssd.io/contrib/tests/multihost-tests.html and had next error:

(.mh-venv) denis@localhost:~/sssd$ pip3 install -r src/tests/multihost/requirements.txt
Collecting git+https://github.com/next-actions/pytest-importance (from -r src/tests/multihost/requirements.txt (line 8))
  Cloning https://github.com/next-actions/pytest-importance to /tmp/pip-req-build-voi23vyh
  Running command git clone --filter=blob:none --quiet https://github.com/next-actions/pytest-importance /tmp/pip-req-build-voi23vyh
  Resolved https://github.com/next-actions/pytest-importance to commit eb3429c7d643fd784fac92aeac241128d4efc2cb
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting git+https://github.com/next-actions/pytest-ticket (from -r src/tests/multihost/requirements.txt (line 9))
  Cloning https://github.com/next-actions/pytest-ticket to /tmp/pip-req-build-zkdjvb63
  Running command git clone --filter=blob:none --quiet https://github.com/next-actions/pytest-ticket /tmp/pip-req-build-zkdjvb63
  Resolved https://github.com/next-actions/pytest-ticket to commit f0935ec0772dd2df9186e2cce4d5990cb4283b92
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting pexpect (from -r src/tests/multihost/requirements.txt (line 1))
  Using cached pexpect-4.9.0-py2.py3-none-any.whl.metadata (2.5 kB)
Collecting pytest (from -r src/tests/multihost/requirements.txt (line 2))
  Using cached pytest-8.1.0-py3-none-any.whl.metadata (7.6 kB)
Collecting pytest_multihost (from -r src/tests/multihost/requirements.txt (line 3))
  Using cached pytest-multihost-3.4.tar.gz (18 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting python-ldap (from -r src/tests/multihost/requirements.txt (line 4))
  Using cached python-ldap-3.4.4.tar.gz (377 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting PyYAML (from -r src/tests/multihost/requirements.txt (line 5))
  Using cached PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting pymmh3 (from -r src/tests/multihost/requirements.txt (line 6))
  Using cached pymmh3-0.0.5-py2.py3-none-any.whl.metadata (822 bytes)
Collecting ssh2-python (from -r src/tests/multihost/requirements.txt (line 7))
  Using cached ssh2-python-1.0.0.tar.gz (2.6 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [33 lines of output]
      /tmp/pip-install-54r_tc9a/ssh2-python_4c8ab7e7bd9c4ca690237f5e8caf4d0f/versioneer.py:421: SyntaxWarning: invalid escape sequence '\s'
        LONG_VERSION_PY['git'] = '''
      /tmp/pip-build-env-gv7zek22/overlay/lib/python3.12/site-packages/setuptools/_distutils/extension.py:134: UserWarning: Unknown Extension options: 'cython_compile_time_env', 'cython_directives'
        warnings.warn(msg)
      Traceback (most recent call last):
        File "/home/denis/sssd/.mh-venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/denis/sssd/.mh-venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/denis/sssd/.mh-venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-gv7zek22/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-gv7zek22/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-gv7zek22/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 487, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-gv7zek22/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 96, in <module>
        File "/tmp/pip-install-54r_tc9a/ssh2-python_4c8ab7e7bd9c4ca690237f5e8caf4d0f/versioneer.py", line 1480, in get_version
          return get_versions()["version"]
                 ^^^^^^^^^^^^^^
        File "/tmp/pip-install-54r_tc9a/ssh2-python_4c8ab7e7bd9c4ca690237f5e8caf4d0f/versioneer.py", line 1412, in get_versions
          cfg = get_config_from_root(root)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-install-54r_tc9a/ssh2-python_4c8ab7e7bd9c4ca690237f5e8caf4d0f/versioneer.py", line 342, in get_config_from_root
          parser = configparser.SafeConfigParser()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'configparser' has no attribute 'SafeConfigParser'. Did you mean: 'RawConfigParser'?
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Similar error describes in the thread pydata/pandas-datareader#969 (comment)

[ZlobinDA]

I try to use ssh2-python3 https://github.com/pycopia/ssh2-python3 instead, but had an error with Cython from the issue pycopia/ssh2-python3#8

[sumit-bose]

Hi,

which version of python are you using? Iirc, there are issue with python-3.12 and ssh2-python. If you are using python-3.12, please install python-3.11 as well and create the environment with something like:

$ python3.11 -m venv .venv
$ source .venv/bin/activate
(.venv) $ pip3 install -r requirements.txt

HTH

bye,
Sumit

[ZlobinDA]

@sumit-bose thank you for your answer!

Please, can you explain me, that manual I need to use for multihost testing? I found some different manuals:

• https://sssd.io/contrib/tests/multihost-tests.html
• https://github.com/SSSD/sssd/tree/master/src/tests/multihost
• https://tests.sssd.io/en/latest/running-tests.html
  Which one is correct?

[alexey-tikhonov]

@shridhargadekar, @jakub-vavra-cz, could you please help here?

[sumit-bose]

@sumit-bose thank you for your answer!

Please, can you explain me, that manual I need to use for multihost testing? I found some different manuals:

* https://sssd.io/contrib/tests/multihost-tests.html

* https://github.com/SSSD/sssd/tree/master/src/tests/multihost

Hi,

to run the tests from the ./src/tests/multihost both are correct, the readme form the source tree shows how you can install the required python dependencies system wide while the page from sssd.io installs the dependencies in a dedicated python environment. I would recommend the latter one, although it might require more disk space since some of the python packages might be installed multiple times (system-wide and the dedicated own) but it should cause less trouble with different version of python and of the dependencies.

* https://tests.sssd.io/en/latest/running-tests.html

This one describes how to run the tests from ./src/tests/system/. The test framework we used for the tests in the multihost directory didn't scale well with our requirements and we switch to a new one (https://github.com/SSSD/sssd-test-framework) and the tests in the system directory are using this new framework.

HTH

bye,
Sumit

  Which one is correct?

[pbrezina]

Hi, unfortunately ssh2-python is no longer compatible with python-3.12+ and the project is pretty much dead.

multihost tests are deprecated and we are converting them to a new pytest-mh based framework (src/tests/system folder). pytest-mh suffers from this issue as well, but we switched to pylibssh there - unfortunately, we are waiting for a new release of pylibssh that will include bits that we depend on. See: next-actions/pytest-mh#44 for more information.

To workaround this issue, please install python3.11-devel package and then create a virtual env with python-3.11 -m venv .venv where you can install the dependencies.

[ZlobinDA]

Thanks everyone for the help! Issue is closed.