Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Use native GitHub ARM64 runners for Notebook Image Builds #216

Open
thesuperzapper opened this issue Feb 17, 2025 · 0 comments
Open

Use native GitHub ARM64 runners for Notebook Image Builds #216

thesuperzapper opened this issue Feb 17, 2025 · 0 comments

Comments

@thesuperzapper
Copy link
Member

thesuperzapper commented Feb 17, 2025

Background

Right now, we build the example-notebook-servers using QEMU emulation on GitHub actions AMD64 runners. These builds are extremely slow due to emulation, meaning we dont even run them in PR tests (where we should).

However, it seems like ARM64 runners are now available for all open-source github repos:

NOTE: this is only really needed for the example notebook images, because small go ones are very quick to build under emulation.

Proposal

There is a lot of discussion on https://github.com/orgs/community/discussions/148648 (and other places) on how to approach multi-arch builds with native runners, we should see what people come up with.

It's likely that unless something like what I proposed in https://github.com/orgs/community/discussions/148648#discussioncomment-12229847 gets adopted by GitHub, we would need to build the images on the native runners, and then export them as artifacts in seperate jobs for ARM/X86, then have a final job combine these artifacts using docker buildx imagetools create ... so they can be pushed to the registry under the same tag.

However, this is likely to be very painful for our notebook images, because they all depend on each other in a tree, so we would have a lot of artifacts to deal with.

Also, it would be complex to do the "parallelization" we do right now, where we build the base image first, then have three child jobs that build jupyter, rstudio and code-server by pulling the base (and again run multiple jobs downstream of these for jupyter-pytorch, and jupyter-tensorflow).

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
Status: Needs Triage
Development

No branches or pull requests

1 participant