Request: ARM container builds #3384
Assignees
Labels
arm
c-cro
Internal Customer Reference
c-dd
Internal Customer Reference
c-sn
Internal Customer Reference
release-engineering
Milestone
Comments
|
Builds do not currently work on arm64 boxes: |
|
Just received a request for us to have arm64 Teleport community edition container images available. A group is deploying Teleport community edition across ~12,000 IOx Cisco IR1101 devices, which requires arm64 container images. |
Closed
|
Getting a different build error now on arm64 with the latest |
|
These are the |
|
These changes to I believe |
This was referenced Nov 29, 2020
|
Any update on this now that #5610 is merged? |
Does gravitational have any affiliation with: https://hub.docker.com/r/draconrose/teleport/tags?page=1&ordering=last_updated There appears to be ARM64 docker images available for teleport on the above hub for 6.1.2 and other versions. I found those on my quest to find ARM support for teleport, then i came across this thread to find it's not officially support yet Edit: Let me know if you want the above link removed. |
|
@clarkycal We have no affiliation with that user or repository. As they don’t appear to have a Github link or any kind of public code which describes how their builds are created, I would be hesistant about trusting it. |
|
I'm also interested in v7 and v8 ARM docker builds. So to help things along, I did a local test of this using docker buildx and the binaries from https://goteleport.com/teleport/download. First, having a build context that looks like: With the Dockerfile updated: FROM ubuntu:20.04
+ARG TARGETPLATFORM
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get upgrade -y && \
DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y ca-certificates dumb-init && \
update-ca-certificates && \
apt-get -y clean && \
rm -rf /var/lib/apt/lists/*
-COPY teleport /usr/local/bin/teleport
+COPY ./$TARGETPLATFORM/teleport /usr/local/bin/teleport
-COPY tctl /usr/local/bin/tctl
+COPY ./$TARGETPLATFORM/tctl /usr/local/bin/tctl
-COPY tsh /usr/local/bin/tsh
+COPY ./$TARGETPLATFORM/tsh /usr/local/bin/tsh
ENTRYPOINT ["/usr/bin/dumb-init", "teleport", "start", "-c", "/etc/teleport/teleport.yaml"]Tips for preparing a local build. # If you're having issues with the qemu based builds not installing deb-packages properly through apt-get.
# Warning: uses `--privileged` to modify the host!
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
# Before you can use buildx for multi-arch.
docker buildx create --useI haven't spotted something for drone, but these Github Actions could be used as reference for the setup steps? Then to actually build (local command I used): For trying it out on actual different architecture hosts, I pushed it to a private registry: And was able to run a basic test on an arm64 (pi4): uname -p
# aarch64
docker run --rm --entrypoint="" registry.example.com/beanow/teleport:buildx teleport version
# Teleport v6.1.2 git:v6.1.2-0-g23ab88c00 go1.15.5Edit: as a bit more realistic test, I was able to start a |
|
Just bumping this - wondering if there are any plans to add arm64 containers to quay.io? |
|
@ollytheninja i just have created mine version as well |
|
Any updates on this? |
|
No arm64 support yet? Any update on this? |
|
FYI, this is my current patchset to build arm64 containers locally, it's change a bit since my original. I'm not sure if all of these modifications are 100% necessary. diff --git a/Makefile b/Makefile
index 6fa159ece..6dfd34b06 100644
--- a/Makefile
+++ b/Makefile
@@ -971,7 +971,7 @@ install: build
.PHONY: image
image: clean docker-binaries
cp ./build.assets/charts/Dockerfile $(BUILDDIR)/
- cd $(BUILDDIR) && docker build --no-cache . -t $(DOCKER_IMAGE):$(VERSION)
+ cd $(BUILDDIR) && docker buildx build --load --platform linux/arm64 --no-cache . -t $(DOCKER_IMAGE):$(VERSION)
if [ -f e/Makefile ]; then $(MAKE) -C e image; fi
.PHONY: publish
@@ -987,7 +987,7 @@ publish: image
.PHONY: image-ci
image-ci: clean docker-binaries
cp ./build.assets/charts/Dockerfile $(BUILDDIR)/
- cd $(BUILDDIR) && docker build --no-cache . -t $(DOCKER_IMAGE_CI):$(VERSION)
+ cd $(BUILDDIR) && docker buildx build --load --platform linux/arm64 --no-cache . -t $(DOCKER_IMAGE_CI):$(VERSION)
if [ -f e/Makefile ]; then $(MAKE) -C e image-ci; fi
.PHONY: publish-ci
diff --git a/build.assets/Makefile b/build.assets/Makefile
index 270a3081c..413bc6be9 100644
--- a/build.assets/Makefile
+++ b/build.assets/Makefile
@@ -82,7 +82,7 @@ export
#
.PHONY:build
build: buildbox
- docker run $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX) \
+ docker run --platform linux/arm64 $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX) \
make -C $(SRCDIR) ADDFLAGS='$(ADDFLAGS)' release
#
@@ -90,7 +90,7 @@ build: buildbox
#
.PHONY:build-binaries
build-binaries: buildbox
- docker run $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX) \
+ docker run --platform linux/arm64 $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX) \
make -C $(SRCDIR) ADDFLAGS='$(ADDFLAGS)' full
#
@@ -98,7 +98,7 @@ build-binaries: buildbox
#
.PHONY:build-enterprise-binaries
build-enterprise-binaries: buildbox
- docker run $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX) \
+ docker run --platform linux/arm64 $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX) \
make -C $(SRCDIR)/e ADDFLAGS='$(ADDFLAGS)' VERSION=$(VERSION) GITTAG=v$(VERSION) clean full
#
@@ -107,7 +107,7 @@ build-enterprise-binaries: buildbox
#
.PHONY:build-binaries-fips
build-binaries-fips: buildbox-fips
- docker run $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX_FIPS) \
+ docker run --platform linux/arm64 $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX_FIPS) \
make -C $(SRCDIR)/e ADDFLAGS='$(ADDFLAGS)' VERSION=$(VERSION) GITTAG=v$(VERSION) FIPS=yes clean full
#
@@ -119,9 +119,8 @@ build-binaries-fips: buildbox-fips
#
.PHONY:buildbox
buildbox:
- if [[ "$(BUILDBOX_NAME)" == "$(BUILDBOX)" ]]; then \
- if [[ $${DRONE} == "true" ]] && ! docker inspect --type=image $(BUILDBOX) 2>&1 >/dev/null; then docker pull $(BUILDBOX) || true; fi; \
- docker build --platform=linux/$(RUNTIME_ARCH) \
+ docker buildx build --platform=linux/$(RUNTIME_ARCH) \
+ --load \
--build-arg UID=$(UID) \
--build-arg GID=$(GID) \
--build-arg BUILDARCH=$(RUNTIME_ARCH) \
@@ -132,7 +131,6 @@ buildbox:
--build-arg LIBBPF_VERSION=$(LIBBPF_VERSION) \
--cache-from $(BUILDBOX) \
--tag $(BUILDBOX) . ; \
- fi
# Builds a Docker buildbox for FIPS
#
@@ -140,7 +138,9 @@ buildbox:
buildbox-fips:
if [[ "$(BUILDBOX_FIPS_NAME)" == "$(BUILDBOX_FIPS)" ]]; then \
if [[ $${DRONE} == "true" ]] && ! docker inspect --type=image $(BUILDBOX_FIPS) 2>&1 >/dev/null; then docker pull $(BUILDBOX_FIPS) || true; fi; \
- docker build \
+ docker builx build \
+ --load \
+ --platform linux/$(RUNTIME_ARCH) \
--build-arg UID=$(UID) \
--build-arg GID=$(GID) \
--build-arg BORINGCRYPTO_RUNTIME=$(BORINGCRYPTO_RUNTIME) \
@@ -155,7 +155,9 @@ buildbox-fips:
.PHONY:buildbox-centos7
buildbox-centos7:
@if [[ $${DRONE} == "true" ]] && ! docker inspect --type=image $(BUILDBOX_CENTOS7) 2>&1 >/dev/null; then docker pull $(BUILDBOX_CENTOS7) || true; fi;
- docker build \
+ docker builx build \
+ --load \
+ --platform linux/$(RUNTIME_ARCH) \
--build-arg UID=$(UID) \
--build-arg GID=$(GID) \
--build-arg BUILDARCH=$(RUNTIME_ARCH) \
@@ -172,7 +174,9 @@ buildbox-centos7:
.PHONY:buildbox-centos7-fips
buildbox-centos7-fips:
@if [[ $${DRONE} == "true" ]] && ! docker inspect --type=image $(BUILDBOX_CENTOS7_FIPS) 2>&1 >/dev/null; then docker pull $(BUILDBOX_CENTOS7_FIPS) || true; fi;
- docker build \
+ docker builx build \
+ --load \
+ --platform linux/$(RUNTIME_ARCH) \
--build-arg UID=$(UID) \
--build-arg GID=$(GID) \
--build-arg BORINGCRYPTO_RUNTIME=$(BORINGCRYPTO_RUNTIME) \
@@ -189,7 +193,9 @@ buildbox-centos7-fips:
.PHONY:buildbox-arm
buildbox-arm: buildbox
@if [[ $${DRONE} == "true" ]] && ! docker inspect --type=image $(BUILDBOX_ARM) 2>&1 >/dev/null; then docker pull $(BUILDBOX_ARM) || true; fi;
- docker build \
+ docker buildx build \
+ --load \
+ --platform linux/$(RUNTIME_ARCH) \
--build-arg BUILDBOX_VERSION=$(BUILDBOX_VERSION) \
--cache-from $(BUILDBOX) \
--cache-from $(BUILDBOX_ARM) \
@@ -203,7 +209,9 @@ buildbox-arm: buildbox
.PHONY:buildbox-arm-fips
buildbox-arm-fips: buildbox-fips
@if [[ $${DRONE} == "true" ]] && ! docker inspect --type=image $(BUILDBOX_ARM_FIPS) 2>&1 >/dev/null; then docker pull $(BUILDBOX_ARM_FIPS) || true; fi;
- docker build \
+ docker buildx build \
+ --load \
+ --platform linux/$(RUNTIME_ARCH) \
--build-arg BUILDBOX_VERSION=$(BUILDBOX_VERSION) \
--cache-from $(BUILDBOX_FIPS) \
--cache-from $(BUILDBOX_ARM_FIPS) \
@@ -229,6 +237,7 @@ buildbox-teleterm: buildbox
.PHONY: grpc
grpc: buildbox
docker run \
+ --platform linux/arm64 \
$(DOCKERFLAGS) -e CLANG_FORMAT=/usr/bin/clang-format-10 -t $(BUILDBOX) \
make -C /go/src/github.com/gravitational/teleport buildbox-grpc
@@ -254,6 +263,7 @@ clean:
.PHONY:test
test: buildbox
docker run \
+ --platform linux/arm64 \
--env TELEPORT_ETCD_TEST="yes" \
--env TELEPORT_XAUTH_TEST="yes" \
$(DOCKERFLAGS) $(NOROOT) -t $(BUILDBOX) \
@@ -267,6 +277,7 @@ test: buildbox
.PHONY:test-root
test-root: buildbox
docker run \
+ --platform linux/arm64 \
--env TELEPORT_ETCD_TEST="yes" \
--env TELEPORT_XAUTH_TEST="yes" \
$(DOCKERFLAGS) -t $(BUILDBOX) \
@@ -279,7 +290,7 @@ test-root: buildbox
.PHONY:test-sh
test-sh: buildbox
- docker run $(DOCKERFLAGS) $(NOROOT) -t $(BUILDBOX) \
+ docker run --platform linux/arm64 $(DOCKERFLAGS) $(NOROOT) -t $(BUILDBOX) \
/bin/bash -c "make -C $(SRCDIR) BATSFLAGS=$(BATSFLAGS) test-sh"
.PHONY:test-helm
@@ -295,6 +306,7 @@ test-helm-update-snapshots:
.PHONY:integration
integration: buildbox
docker run \
+ --platform linux/arm64 \
--env TELEPORT_ETCD_TEST="yes" \
$(DOCKERFLAGS) $(NOROOT) -t $(BUILDBOX) \
/bin/bash -c \
@@ -304,6 +316,7 @@ integration: buildbox
.PHONY:integration-root
integration-root: buildbox
docker run $(DOCKERFLAGS) -t $(BUILDBOX) \
+ --platform linux/arm64 \
/bin/bash -c "make -C $(SRCDIR) FLAGS='-cover' integration-root"
#
@@ -312,6 +325,7 @@ integration-root: buildbox
.PHONY:lint
lint: buildbox
docker run $(DOCKERFLAGS) $(NOROOT) -t $(BUILDBOX) \
+ --platform linux/arm64 \
/bin/bash -c "make -C $(SRCDIR) lint"
.PHONY:lint-helm
@@ -325,6 +339,7 @@ lint-helm: buildbox
.PHONY:enter
enter: buildbox
docker run $(DOCKERFLAGS) -ti $(NOROOT) \
+ --platform linux/arm64 \
-e HOME=$(SRCDIR)/build.assets -w $(SRCDIR) $(BUILDBOX) /bin/bash
#
@@ -349,6 +364,7 @@ enter/centos7: buildbox
.PHONY:release
release: buildbox
docker run $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX_NAME) \
+ --platform linux/arm64 \
/usr/bin/make release -e ADDFLAGS="$(ADDFLAGS)" OS=$(OS) ARCH=$(ARCH) RUNTIME=$(GOLANG_VERSION) FIDO2=$(FIDO2) REPRODUCIBLE=yes
# These are aliases used to make build commands uniform.
@@ -388,7 +404,7 @@ release-amd64-centos7-fips: buildbox-centos7-fips
.PHONY:release-fips
release-fips: buildbox-fips
@if [ -z ${VERSION} ]; then echo "VERSION is not set"; exit 1; fi
- docker run $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX_FIPS_NAME) \
+ docker run --platform linux/arm64 $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX_FIPS_NAME) \
/usr/bin/make -C e release -e ADDFLAGS="$(ADDFLAGS)" OS=$(OS) ARCH=$(ARCH) RUNTIME=$(GOLANG_VERSION) FIPS=yes VERSION=$(VERSION) GITTAG=v$(VERSION) REPRODUCIBLE=yes
#
@@ -396,7 +412,7 @@ release-fips: buildbox-fips
#
.PHONY:release-centos7
release-centos7: buildbox-centos7
- docker run $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX_CENTOS7) \
+ docker run --platform linux/arm64 $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX_CENTOS7) \
/usr/bin/scl enable devtoolset-11 'make release -e ADDFLAGS="$(ADDFLAGS)" OS=$(OS) ARCH=$(ARCH) RUNTIME=$(GOLANG_VERSION) FIDO2=$(FIDO2) REPRODUCIBLE=no'
#
@@ -405,7 +421,7 @@ release-centos7: buildbox-centos7
#
.PHONY:release-centos7-fips
release-centos7-fips:
- docker run $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX_CENTOS7_FIPS) \
+ docker run --platform linux/arm64 $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX_CENTOS7_FIPS) \
/usr/bin/scl enable devtoolset-11 '/usr/bin/make -C e release -e ADDFLAGS="$(ADDFLAGS)" OS=$(OS) ARCH=$(ARCH) RUNTIME=$(GOLANG_VERSION) FIPS=yes VERSION=$(VERSION) GITTAG=v$(VERSION) REPRODUCIBLE=no'
#
@@ -413,7 +429,7 @@ release-centos7-fips:
#
.PHONY:release-windows
release-windows: buildbox
- docker run $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX) \
+ docker run --platform linux/arm64 $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX) \
/usr/bin/make release -e ADDFLAGS="$(ADDFLAGS)" OS=windows RUNTIME=$(GOLANG_VERSION) REPRODUCIBLE=yes
#
@@ -421,7 +437,7 @@ release-windows: buildbox
#
.PHONY:release-windows-unsigned
release-windows-unsigned: buildbox
- docker run $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX) \
+ docker run --platform linux/arm64 $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX) \
/usr/bin/make release-windows-unsigned -e ADDFLAGS="$(ADDFLAGS)" OS=windows RUNTIME=$(GOLANG_VERSION) REPRODUCIBLE=yes
# |
|
Bump. I'm hitting this trying to deploy Teleport via the official helm charts on AWS with T4G workers. I'm going to have to build these myself which is fine but it would be really nice if these could be provided. Thanks! |
|
I believe the intention is for us to publish arm/arm64 container images by the end of the year. |
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
This is a request to provide ARM container builds on quay.io. Quay now supports multi-arch container manifests (anouncement here).
NOTE: This is related to my personal use of Teleport Community Edition, not my employer's use of Teleport Enterprise.
The text was updated successfully, but these errors were encountered: