Skip to content

ci: mirror buildkit image to ghcr #135624

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Merged
merged 1 commit into from
Jan 18, 2025

Conversation

marcoieni
Copy link
Member

Thanks to #135574 we successfully mirrored the ubuntu:22.04 image in ghcr to allow mingw-check-tidy working without Docker caching.
However, I found out that all CI jobs pull an additional image from the dockerhub, i.e. moby/buildkit:buildx-stable-1.

You can see this from the logs of the mingw-check-tidy job:

25-01-17T10:47:34.9901947Z Attempting with retry: docker buildx build --rm -t rust-ci -f /home/runner/work/rust/rust/src/ci/docker/host-x86_64/mingw-check-tidy/Dockerfile /home/runner/work/rust/rust/src/ci/docker --cache-from type=registry,ref=ghcr.io/rust-lang-ci/rust-ci-cache:818dbd7e9bcfe13d21021d72c9ea5568237eba7e220c64f0f20a1459383140536afa115373590ba6fa2de3836339f2f0e1e169134fe8d9246583741aefa19c77 --output=type=docker
2025-01-17T10:47:35.0373736Z #0 building with "quirky_keldysh" instance using docker-container driver
2025-01-17T10:47:35.0374273Z 
2025-01-17T10:47:35.0374662Z #1 [internal] booting buildkit
2025-01-17T10:47:35.1881201Z #1 pulling image moby/buildkit:buildx-stable-1
2025-01-17T10:47:37.1338973Z #1 pulling image moby/buildkit:buildx-stable-1 2.1s done
2025-01-17T10:47:37.2842181Z #1 creating container buildx_buildkit_quirky_keldysh0
2025-01-17T10:47:37.4417076Z #1 creating container buildx_buildkit_quirky_keldysh0 0.3s done

I saw jobs failing in #133912 because they failed to pull this image for the dockerhub rate limit.

So this PR mirrors this additional image in ghcr 👍

To use it, we need to add the flag --driver-opt image=ghcr.io/marcoieni/buildkit:buildx-stable-1 to docker buildx create --use --driver docker-container (I tested this in #133912).

If you think this bash script is becoming too complex, I can rewrite the script in python as I did in #135530

r? @Kobzol

@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 17, 2025
@Kobzol
Copy link
Contributor

Kobzol commented Jan 17, 2025

Huh. This means that if buildx unconditionally downloads a Docker image on every Docker build, then the Docker rate limit should be even less of a concern than I thought before, because we apparently download at least one image all the time on all Linux CI jobs :)

The bash script is a bit larger now, but we expected that; adding new images to the list shouldn't make it more complicated, so I think it's fine.

@bors r+ rollup

@bors
Copy link
Collaborator

bors commented Jan 17, 2025

📌 Commit 5f49515 has been approved by Kobzol

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 17, 2025
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 18, 2025
…iaskrgr

Rollup of 7 pull requests

Successful merges:

 - rust-lang#133700 (const-eval: detect more pointers as definitely not-null)
 - rust-lang#135290 (Encode constraints that hold at all points as logical edges in location-sensitive polonius)
 - rust-lang#135478 (Run clippy for rustc_codegen_gcc on CI)
 - rust-lang#135583 (Move `std::pipe::*` into `std::io`)
 - rust-lang#135612 (Include x scripts in tarballs)
 - rust-lang#135624 (ci: mirror buildkit image to ghcr)
 - rust-lang#135661 (Stabilize `float_next_up_down`)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 0553681 into rust-lang:master Jan 18, 2025
6 checks passed
@rustbot rustbot added this to the 1.86.0 milestone Jan 18, 2025
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Jan 18, 2025
Rollup merge of rust-lang#135624 - marcoieni:mirror-buildkit-image, r=Kobzol

ci: mirror buildkit image to ghcr
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants