undefined symbol: _ZNK4sycl3_V16detail12buffer_plain13handleReleaseEv

[fcharras]

Trying to build dpctl==0.14.3dev1 after activating latest 2023.1.0 oneapi basekit release (using official online installer https://registrationcenter-download.intel.com/akdlm/IRC_NAS/7deeaac4-f605-4bcf-a81b-ea7531577c61 )

It builds successfully but then I can't import dpctl.tensor module:

  File "/root/sklearn-numba-dpex/benchmark/./kmeans.py", line 260, in <module>
    from sklearn_numba_dpex.kmeans.engine import KMeansEngine
  File "/root/sklearn-numba-dpex/sklearn_numba_dpex/kmeans/engine.py", line 7, in <module>
    import dpctl.tensor as dpt
  File "/opt/venv/lib/python3.9/site-packages/dpctl/tensor/__init__.py", line 24, in <module>
    from dpctl.tensor._copy_utils import asnumpy, astype, copy, from_numpy, to_numpy
  File "/opt/venv/lib/python3.9/site-packages/dpctl/tensor/_copy_utils.py", line 24, in <module>
    import dpctl.tensor._tensor_impl as ti
ImportError: /opt/venv/lib/python3.9/site-packages/dpctl/tensor/_tensor_impl.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZNK4sycl3_V16detail12buffer_plain13handleReleaseEv

Using ld on said file shows a lot of undefined symbols:

ld: warning: cannot find entry symbol _start; not setting start address
ld: _tensor_impl.cpython-39-x86_64-linux-gnu.so: undefined reference to `_Py_Dealloc'
ld: _tensor_impl.cpython-39-x86_64-linux-gnu.so: undefined reference to `PyUnicode_AsUTF8String'
ld: _tensor_impl.cpython-39-x86_64-linux-gnu.so: undefined reference to `PyEval_GetBuiltins'
ld: _tensor_impl.cpython-39-x86_64-linux-gnu.so: undefined reference to `PyExc_ValueError'
ld: _tensor_impl.cpython-39-x86_64-linux-gnu.so: undefined reference to `PyModule_Create2'
ld: _tensor_impl.cpython-39-x86_64-linux-gnu.so: undefined reference to `PyByteArray_Type'
ld: _tensor_impl.cpython-39-x86_64-linux-gnu.so: undefined reference to `PyCapsule_GetName'
ld: _tensor_impl.cpython-39-x86_64-linux-gnu.so: undefined reference to `_Py_FalseStruct'
ld: _tensor_impl.cpython-39-x86_64-linux-gnu.so: undefined reference to `PyImport_ImportModule'
ld: _tensor_impl.cpython-39-x86_64-linux-gnu.so: undefined reference to `PyExc_OverflowError'
ld: _tensor_impl.cpython-39-x86_64-linux-gnu.so: undefined reference to `PyComplex_AsCComplex'
...
... (more undefined symbols) ...
...

is there a recent change in the build recipe that could explain this issue ?

[fcharras]

The issue seems to come from a mistake in intel runtime packages that have been uploaded to Pypy.

The package intel-opencl-rt has not been properly uploaded to pypi, only a single win package is available.

As a result, on linux the pip dependency resolution falls back on 2023.0.0 versions for all the dependency tree.

@oleksandr-pavlyk where can I report issues for uploads to pypi ?

[oleksandr-pavlyk]

I have brought this issue to attention of our infrastructure team.

[fcharras]

Thanks a lot. Can you share the ETA if you're informed of one ? this issue makes nearly unusable both the 2023.0.0 and 2023.1.0 versions of dpcpp-related pypi packages :-/ I hope it can be hotfixed.

[oleksandr-pavlyk]

This error indicates that dpctl native extensions, built with oneAPI DPC++ 2023.1.0, are being run in Python environment where DPC++ RT from 2023.0.0 is used, instead of intended 2023.1.0.

The easiest way to trace what is being loaded is to use LD_DEBUG=libs and grep for libsycl.so.

For example, in my setup:

(dev_dpctl) opavlyk@opavlyk-mobl:~/repos/dpctl$ LD_DEBUG=libs python -c "import dpctl.tensor" |& grep libsycl.so
     27604:     find library=libsycl.so.6 [0]; searching
     27604:       trying file=/home/opavlyk/repos/dpctl/dpctl/libsycl.so.6
     27604:       trying file=/home/opavlyk/miniconda3/envs/dev_dpctl/bin/../lib/libsycl.so.6
     27604:       trying file=/opt/intel/oneapi/mkl/2023.1.0/lib/intel64/libsycl.so.6
     27604:       trying file=/opt/intel/oneapi/tbb/2021.9.0/env/../lib/intel64/gcc4.8/libsycl.so.6
     27604:       trying file=/opt/intel/oneapi/compiler/2023.1.0/linux/lib/libsycl.so.6
     27604:     calling init: /opt/intel/oneapi/compiler/2023.1.0/linux/lib/libsycl.so.6
     27604:     calling fini: /opt/intel/oneapi/compiler/2023.1.0/linux/lib/libsycl.so.6 [0]

[fcharras]

Yes I realized that, from the comment above, the explanation for the issue I encountered is that when installing the runtime with pip, the pip dependency solver fallbacks on 2023.0.0 because of this one single dependency that isn't uploaded correctly on pypy and makes dependency resolution on 2023.1.0 versions impossible. But I was expecting 2023.1.0 and didn't notice the fallback on 2023.0.0 at first.

[fcharras]

With the 2023.2.0 release, the issue have been fixed (see https://pypi.org/project/intel-opencl-rt/2023.2.0/#files )

(I notice only now, that apparently the relevant file had been made available in intel python package index at https://pypi.anaconda.org/intel/simple/intel-opencl-rt/ )