Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Fix docker build pipeline #109

Merged
merged 8 commits into from
Jul 21, 2023
Merged

Fix docker build pipeline #109

merged 8 commits into from
Jul 21, 2023

Conversation

fcharras
Copy link
Collaborator

Numba 0.57 compatibility issues are already reported upstream see IntelPython/numba-dpex#1000

docker/Dockerfile Outdated Show resolved Hide resolved
Copy link
Member

@jjerphan jjerphan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Like @ogrisel suggests, I think it is good to have a small comment for this version specifier.

@fcharras
Copy link
Collaborator Author

I'm still not building it locally

There's something wrong with dpnp build that requires further debug

@fcharras fcharras force-pushed the fix/fix_docker_build_pipeline branch from d817dc0 to a102937 Compare April 27, 2023 09:27
@fcharras
Copy link
Collaborator Author

Latest commits both address latest issues and bump to latest releases.

I'd like to bump aggressively to latest commits of master branch for numba_dpex so it includes cache performance fixes, will revert to a fixed tag once it's officially released.

@fcharras
Copy link
Collaborator Author

Fixes #94

@fcharras
Copy link
Collaborator Author

Not yet functional, see IntelPython/dpctl#1190

However the conda install works.

@fcharras
Copy link
Collaborator Author

Last push finally contains a Dockerfile for an image that builds and works. But it's not very useable, since it contains a broken dependency tree, pip resolver will try to fix it whevener it can (e.g when building scikit-learn...), and the automated resolution fix just breaks the runtime.

IMO the docker environment can't be updated until IntelPython/dpctl#1190 is fixed.

The solution of pinning everything to 2023.0.0 instead and using older 2023.0.0 oneapi basekit for building is also cumbersome, because some dependencies needs to be force-pinned to 2023.0.0 for this to work (if not the bug currently on main that blocks our weekly image publishing occurs), I'm not interested in doing extra maintaining for a version that is not the latest.

I've checked that the performance for KMeans remains good both locally and on edge cloud with the version combinations 2023.1.0 and that the unit tests pass, so aside from those build problems, the bumps work well.

I need a up-to-date environment without our patches on loading numba_dpex that got in the way of some debug I was doing for the KNN so I'll rebase other branches on this one waiting for things to be fixes upstream.

@fcharras
Copy link
Collaborator Author

Or if the the intel-opencl-rt issue is not fixed upstream it might be the opportunity to try the official numba_dpex docker image but I'm afraid we would loose some flexibility.

@fcharras
Copy link
Collaborator Author

Installers work again with 2023.2.0 release, but there's a regression in latest numba_dpex release that prevent some kernels to run on CPU (e.g. our lloyd fused kernel), see IntelPython/numba-dpex#1068

@fcharras
Copy link
Collaborator Author

Things work on my local environment, I'll merge so that the automated build pipeline can resume, and the conda install instruction are fixed.

@fcharras fcharras merged commit a9b8fcd into main Jul 21, 2023
@fcharras fcharras deleted the fix/fix_docker_build_pipeline branch July 21, 2023 13:05
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants