Compilation failing on RH8 with GCC 11.3.0 #29
Comments
|
Hi @orliac My best guess is that the Python headers aren't being passed to the compilation commands: /work/backup/ska/orliac/spack_0200/lib/spack/env/gcc/g++
-Darrow_tables_EXPORTS
-I/tmp/orliac/spack-stage/spack-stage-py-arcae-main-bgre6mp2qhikmyafc2yzgfv5nab3cnja/spack-src/arcae/PkgConfig::casacore
-I/work/ska/soft/spack/blackhole/v2/opt/spack/linux-rhel8-cascadelake/gcc-11.3.0/py-pyarrow-12.0.0-qa5dzazodts7cvjtmy2cyfxmxdpzjy7t/lib/python3.10/site-packages/pyarrow/include
-I/work/ska/soft/spack/blackhole/v2/opt/spack/linux-rhel8-cascadelake/gcc-11.3.0/py-numpy-1.24.3-a24j2n7xqpnc4qlygxwkatwvmpbbbwxm/lib/python3.10/site-packages/numpy/core/include
-I/tmp/orliac/spack-stage/spack-stage-py-arcae-main-bgre6mp2qhikmyafc2yzgfv5nab3cnja/spack-src/cpp
-I/work/ska/soft/spack/blackhole/v2/opt/spack/linux-rhel8-cascadelake/gcc-11.3.0/python-3.10.4-tl3vrz4pgbao3vfzuyldkj5bn7qpic4g/include/python3.10
-I/tmp/orliac/spack-stage/spack-stage-py-arcae-main-bgre6mp2qhikmyafc2yzgfv5nab3cnja/spack-src/cpp/PkgConfig::casacore
-isystem
/tmp/orliac/spack-stage/spack-stage-py-arcae-main-bgre6mp2qhikmyafc2yzgfv5nab3cnja/spack-src/vcpkg/installed/x64-linux-dynamic-cxx17-abi0-rel/lib/pkgconfig/../../include
-O3 -DNDEBUG -std=c++17 -fPIC -D_GLIBCXX_USE_CXX11_ABI=0 -MD
-MT arcae/CMakeFiles/arrow_tables.dir/arrow_tables.cxx.o
-MF arcae/CMakeFiles/arrow_tables.dir/arrow_tables.cxx.o.d
-o arcae/CMakeFiles/arrow_tables.dir/arrow_tables.cxx.o
-c /tmp/orliac/spack-stage/spack-stage-py-arcae-main-bgre6mp2qhikmyafc2yzgfv5nab3cnja/spack-src/_skbuild/linux-x86_64-3.10/cmake-build/arcae/arrow_tables.cxxAlso, the following looks a bit strange to me. arcae is using P.S. Aiming to have wheels out on pypi soon. Would this be helpful? |
|
Hi Simon, it includes (in my env) but shouldn't it be including this one instead? |
|
The inclusion of the This SO answer implies that Python might need to be compiled in a particular way? https://stackoverflow.com/a/70911512 Also, python/cpython#96822 (comment) says the following:
It looks like PEP 0670 made it into 3.11. Perhaps this has something todo with how SPACK packages Python 3.11? Unfortunately these are just leads in the investigation, I don't have a comprehensive answer. |
|
Thanks @sjperkins. For information, I've opened an issue on pyarrow's side earlier on: apache/arrow#36438. I'm currently rebuilding my stack based on Python 3.11.2 (was on 3.10.4 before) but it will take a bit of time, so I'll post the outcome later on. |
|
Using Python 3.11.2 did not improve the situation: |
|
I think Python 3.10 would be required to work around PEP 0670 (if it is the problem). |
|
Hello @sjperkins Will try the wheels and will let you know. |
|
Using the wheel I face an ABI incompatibility: Looking at the symbols in (nothing found) whereas: So I guess it means that the wheel was built with |
Yes, currently arcae is built with |
It looks like it's trying to build from source. I don't think it's possible to build a working source build from scratch (because of the dependency on shared libraries built by vcpkg). At least this seems possible for me locally $ git clone git@github.com:ratt-ru/arcae.git
$ virtualenv -p python3.9 ~/venv/arcaesrc
$ . ~/venv/arcaesrc/bin/activate
(arcaesrc) $ cd /
(arcaesrc) $ pip install --no-binary=arcae ~/code/arcae[dev,test]
Processing ./code/arcae
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting appdirs (from arcae==0.1.0)
....
Building wheels for collected packages: arcae
Building wheel for arcae (pyproject.toml) ... done
Created wheel for arcae: filename=arcae-0.1.0-cp39-cp39-linux_x86_64.whl size=268963 sha256=65bbc6fe77142a92906d5a962753274ff2e9f12ffa790cc69bff15a02bbb1c98
Stored in directory: /tmp/pip-ephem-wheel-cache-35s3x43f/wheels/57/ca/66/ab589a3e8a6ecef38d467333fd4a0c91fd222b5fc2cbc98785
Successfully built arcae
Installing collected packages: mccabe, duckdb, docopt, appdirs, urllib3, unidecode, typing-extensions, tomlkit, tomli, tabulate, six, pygments, pyflakes, pycodestyle, pluggy, platformdirs, pathspec, packaging, numpy, mypy-extensions, mdurl, iniconfig, idna, exceptiongroup, contextlib2, colorama, click, charset-normalizer, certifi, schema, requests, python-casacore, pytest, pyarrow, markdown-it-py, flake8, cli-ui, black, tbump, rich, arcae
Successfully installed appdirs-1.4.4 arcae-0.1.0 black-22.1.0 certifi-2023.5.7 charset-normalizer-3.1.0 cli-ui-0.17.2 click-8.1.3 colorama-0.4.6 contextlib2-21.6.0 docopt-0.6.2 duckdb-0.8.1 exceptiongroup-1.1.2 flake8-4.0.1 idna-3.4 iniconfig-2.0.0 markdown-it-py-3.0.0 mccabe-0.6.1 mdurl-0.1.2 mypy-extensions-1.0.0 numpy-1.25.0 packaging-23.1 pathspec-0.11.1 platformdirs-3.8.0 pluggy-1.2.0 pyarrow-12.0.0 pycodestyle-2.8.0 pyflakes-2.4.0 pygments-2.15.1 pytest-7.4.0 python-casacore-3.5.2 requests-2.31.0 rich-13.4.2 schema-0.7.5 six-1.16.0 tabulate-0.8.10 tbump-6.10.0 tomli-2.0.1 tomlkit-0.11.8 typing-extensions-4.7.1 unidecode-1.3.6 urllib3-2.0.3This seems to correctly install the $ tree ~/venv/arcaesrc/lib/python3.9/site-packages/arcae -I __pycache__
/home/simon/venv/arcaesrc/lib/python3.9/site-packages/arcae
├── __init__.py
├── applications
│ ├── __init__.py
│ ├── entrypoint.py
│ └── ms_export.py
├── config.py
├── lib
│ ├── __init__.py
│ ├── arrow_tables.cpython-39-x86_64-linux-gnu.so
│ └── libarcae.so
└── tests
├── __init__.py
├── conftest.py
└── test_pytable.py
3 directories, 11 filesHowever, (arcaesrc) $ export LD_LIBRARY_PATH=~/code/arcae/vcpkg/installed/x64-linux-dynamic-cxx17-abi0-rel/lib
(arcaesrc) $ py.test -s -vvv --pyargs arcaeThe wheels correctly package the built casacore shared objects and dependencies via |
|
Did you have any luck here @orliac ? |
|
Hello @sjperkins |
|
I've created some tickets that describe solutions to some of the issues here, but I'm not intending to work on any of them soon. |
Hello,
I'm failing to compile arcae on a RH8 box using GCC 11.3.0 (using Spack 0.20.0). Here is the log:
Can you guess what the problem is? (my cython version in use is 0.29.33)
Thank you,
Etienne
The text was updated successfully, but these errors were encountered: