Adding support to build images for multiple architectures #760
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
adejanovski
requested changes
May 24, 2024
There was a problem hiding this comment.
Thanks for this PR @EmanuelSanchez, we'd just need a slight change in the variable names in the Dockerfile.
k8s/Dockerfile
Outdated
| @@ -1,4 +1,6 @@ | |||
| FROM ubuntu:22.04 as base | |||
| ARG ARCH="amd64" | |||
There was a problem hiding this comment.
Todo: In order to integrate correctly with Docker buildx and cross platform builds, this variable needs to be renamed to TARGETARCH. See here for more info.
Suggested change
| ARG ARCH="amd64" | |
| ARG TARGETARCH="amd64" |
k8s/Dockerfile
Outdated
| @@ -34,7 +36,7 @@ RUN python3 -m pip install -U pip && pip3 install --ignore-installed --user poet | |||
| RUN cd /build && poetry build && poetry install | |||
|
|
|||
| # Could be python:slim, but we have a .sh entrypoint | |||
| FROM ubuntu:22.04 | |||
| FROM --platform=linux/${ARCH} ubuntu:22.04 | |||
There was a problem hiding this comment.
Todo:
Suggested change
| FROM --platform=linux/${ARCH} ubuntu:22.04 | |
| FROM --platform=linux/${TARGETARCH} ubuntu:22.04 |
k8s/Dockerfile
Outdated
| RUN GRPC_HEALTH_PROBE_VERSION=v0.4.25 && \ | ||
| wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64 && \ | ||
| chmod +x /bin/grpc_health_probe | ||
| wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-${ARCH} && \ |
There was a problem hiding this comment.
Todo:
Suggested change
| wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-${ARCH} && \ | |
| wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-${TARGETARCH} && \ |
|
I just pushed the changes @adejanovski |
Thanks @EmanuelSanchez. I can't seem to build locally on my machine. Looks like the wget command fails and only succeeds when I hardcode the architecture 🤔 What's the output of the |
|
Yes, I found an issue, I wasn't reusing the TARGETARCH arg, it's fixed now. |
adejanovski
reviewed
Jun 14, 2024
k8s/Dockerfile
Outdated
| @@ -1,4 +1,10 @@ | |||
| FROM ubuntu:22.04 as base | |||
| # Declare architecture variable globally | |||
| ARG TARGETARCH="amd64" | |||
There was a problem hiding this comment.
Issue: This will prevent any attempt at overriding this with a different architecture:
% docker buildx build --platform linux/arm64 -t testing/medusa:arm64 -f k8s/Dockerfile .
[+] Building 150.1s (25/25) FINISHED docker:desktop-linux
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 2.68kB 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:22.04 0.5s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 91B 0.0s
=> [internal] load build context 5.5s
=> => transferring context: 577.05MB 5.5s
=> CACHED [base 1/8] FROM docker.io/library/ubuntu:22.04@sha256:19478ce7fc2ffbce89df29fea5725a8d12e57de52eb9ea570890dc5852aac1ac 0.0s
=> [base 2/8] RUN mkdir /install 0.4s
=> [stage-1 2/13] RUN groupadd -r cassandra --gid=999 && useradd -r -g cassandra --uid=999 --create-home cassandra 0.6s
=> [base 3/8] WORKDIR /install 0.1s
=> [base 4/8] RUN apt-get update && apt-get install -y software-properties-common curl gnupg 63.8s
=> [stage-1 3/13] RUN apt-get update && apt-get install -y python3 python3-setuptools wget && rm -rf /var/lib/apt/lists/* 32.4s
=> [stage-1 4/13] RUN GRPC_HEALTH_PROBE_VERSION=v0.4.25 && wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64 && chmod +x /bin/grpc_health_p 1.1s
You can see that despite setting the arch to linux/arm64 it still pulls the amd64 probe.
Merging this first line into line 7 instead works fine:
% docker buildx build --platform linux/arm64 -t alexdejanovski269/medusa:arm64 -f k8s/Dockerfile .
[+] Building 63.4s (26/26) FINISHED docker:desktop-linux
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 2.67kB 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:22.04 1.2s
=> [auth] library/ubuntu:pull token for registry-1.docker.io 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 91B 0.0s
=> [internal] load build context 1.2s
=> => transferring context: 3.65MB 1.1s
=> [base 1/8] FROM docker.io/library/ubuntu:22.04@sha256:19478ce7fc2ffbce89df29fea5725a8d12e57de52eb9ea570890dc5852aac1ac 2.5s
=> => resolve docker.io/library/ubuntu:22.04@sha256:19478ce7fc2ffbce89df29fea5725a8d12e57de52eb9ea570890dc5852aac1ac 0.0s
=> => sha256:19478ce7fc2ffbce89df29fea5725a8d12e57de52eb9ea570890dc5852aac1ac 1.13kB / 1.13kB 0.0s
=> => sha256:ee2717369ac272fb83a9035762820588e77747d01a7e2fa1f36ded5a7fe672e4 424B / 424B 0.0s
=> => sha256:1442ff3b14f13ebbe3ea796b0963204140e94f9e9fc0776217cfb5a2d723fdd0 2.31kB / 2.31kB 0.0s
=> => sha256:9b10a938e28486049341cb41134e8c0c141b2e25870896c597e2a54df471acbb 27.36MB / 27.36MB 1.5s
=> => extracting sha256:9b10a938e28486049341cb41134e8c0c141b2e25870896c597e2a54df471acbb 0.8s
=> [base 2/8] RUN mkdir /install 0.3s
=> [stage-1 2/13] RUN groupadd -r cassandra --gid=999 && useradd -r -g cassandra --uid=999 --create-home cassandra 0.3s
=> [base 3/8] WORKDIR /install 0.0s
=> [stage-1 3/13] RUN apt-get update && apt-get install -y python3 python3-setuptools wget && rm -rf /var/lib/apt/lists/* 6.9s
=> [base 4/8] RUN apt-get update && apt-get install -y software-properties-common curl gnupg 16.0s
=> [stage-1 4/13] RUN GRPC_HEALTH_PROBE_VERSION=v0.4.25 && wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-arm64 && chmod +x /bin/grpc_health_p 1.3s
=> [stage-1 5/13] WORKDIR /home/cassandra 0.0s
I'll push a commit with this change.
|
adejanovski
approved these changes
Jun 14, 2024
adejanovski
approved these changes
Jun 14, 2024
|
Great! |
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adding support for creating images for multiple architectures using docker buildx, this will also fix the issue with the grpc-health-probe binary referenced here:
Fixes #759