maturin publish fails with pyenv error

[anentropic]

Please provide the following information:

• Your python version (python -V): 3.9.2
• Your pip version (pip -V): 20.2.2
  • If this isn't the latest pip version, have you checked using the latest pip version?
• The version of the bindings you're using, if any (e.g. pyo3, rust-cpython or cffi): pyo3
• Does cargo build work? yes
• If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

Please list the exact steps required to reproduce your error with all command output and if possible with a repository:

I installed maturin into my Python 3.9.2 virtualenv. I think this is what the readme suggests?

maturin develop works fine

But when I try to run maturin publish I get:

$ maturin publish
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
pyenv: python3.8: command not found

The `python3.8' command exists in these Python versions:
  3.8.6

Note: See 'pyenv help global' for tips on allowing both
      python2 and python3 to be found.
💥 maturin failed
  Caused by: Finding python interpreters failed
  Caused by: Trying to get metadata from the python interpreter 'python3.8' failed

I'm already in my virtualenv with Python 3.9.2, where maturin is installed. Why is it looking for Python 3.8 via pyenv?

[anentropic]

I guess this is supposed to be run in the docker container?

So with the docker container I can build wheels for 3.6, 3.7, 3.8, 3.9 on Linux

And potentially I could build Apple wheels on my macbook, but then it'd be better if I just did maturin build and published to pypi manually after collecting the wheels for each platform together?

And for other platforms the project has maturin as the build tool in pyproject.toml so if no wheel is available... pip install <mylib> will build from src as long as they have the rust toolchain installed?

Is that roughly right? What's the ideal workflow here?

[anentropic]

Ok I basically just confirmed all of the above

docker run --rm -v $(pwd):/io konstin2/maturin build works and publishes linux wheels plus gzipped source to pypi, pip install <mylib> on Apple works and builds from src as long as rustup/cargo is installed

[anentropic]

I can see cargo supports cross-compilation https://pyo3.rs/master/building_and_distribution.html#cross-compiling

so we can pass --target arg to maturin build and I guess build Apple & Windows wheels also using the manylinux docker container?

[messense]

1. For cross compiling to macOS, try https://github.com/tpoechtrager/osxcross
2. For cross compiling to Windows, try https://github.com/dockcross/dockcross

[anentropic]

so konstin2/maturin will not support cross-compiling? I should derive new docker images based on dockcross/osxcross plus the maturin stuff from konstin2/maturin and use those to build?

[messense]

so konstin2/maturin will not support cross-compiling?

Not supported currently I think.

I should derive new docker images based on dockcross/osxcross plus the maturin stuff from konstin2/maturin and use those to build?

That's right. Cross compiling in Rust currently needs the target platform's linker and maybe also some C headers/libraries which is not included in konstin2/maturin.

[konstin]

I'm already in my virtualenv with Python 3.9.2, where maturin is installed. Why is it looking for Python 3.8 via pyenv?

On linux and mac, all python versions in PATH are used. If you don't set your own interpreters with -i, a heuristic is used to search for python installations. (from here)

And potentially I could build Apple wheels on my macbook, but then it'd be better if I just did maturin build and published to pypi manually after collecting the wheels for each platform together?

Most projects have releases where each platform is uploaded a slightly different time. The pypi developers are aware that this is suboptimal and are working on draft releases.

In general, I recommend using one of the many CI service such as github actions to make releases across platforms.

so konstin2/maturin will not support cross-compiling? I should derive new docker images based on dockcross/osxcross plus the maturin stuff from konstin2/maturin and use those to build?

konstin2/maturin exists to make it easy to build for manylinux. If you actually want to cross compile, I'd recommend taking one of containers @messense linked and install maturin in them.

[hongquan]

You can run

maturin build -i $(which python3)

to limit to your Python in virtualenv.

[messense]

I think pyenv is sometimes counterintuitive, I'll thinking about how to make maturin works nicer with pyenv for example skipping unavailable Python interpreter installed by pyenv automatically.