Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

updating dockerfile and build workflow #206

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

updating dockerfile and build workflow #206

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0641d28
updating dockerfile and build workflow
John-Scira Feb 4, 2025
ac95281
github action workflow GAR auth updated
John-Scira Feb 4, 2025
b3ce604
update docker tagging
John-Scira Feb 4, 2025
9bcf9a7
updating dockerfile ssh config
John-Scira Feb 4, 2025
0f534da
changed target tag
John-Scira Feb 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 21 additions & 18 deletions .github/workflows/docker-BuildAndPush.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,31 +13,34 @@
steps:
- name: Checkout code
uses: actions/checkout@v4
- uses: webfactory/ssh-agent@v0.9.0

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Build and Push Docker images to Google Artifact Registry (GAR)' step
Uses Step
Loading
uses 'webfactory/ssh-agent' with ref 'v0.9.0', not a pinned commit hash
with:
ssh-private-key: ${{ secrets.UTIL_MODULE_REPO_KEY }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to GCP
uses: google-github-actions/auth@v2
with:
credentials_json: ${{ secrets.OPS_ARTIFACT_REGISTRY_SVC_ACCOUNT }}
ssh: |
default=${{ env.SSH_AUTH_SOCK }}

- name: Configure Docker to use the Google Artifact Registry
run: gcloud auth configure-docker us-central1-docker.pkg.dev

- name: GAR Auth
uses: docker/#-action@v3

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Build and Push Docker images to Google Artifact Registry (GAR)' step
Uses Step
Loading
uses 'docker/#-action' with ref 'v3', not a pinned commit hash
with:
registry: us-central1-docker.pkg.dev
username: _json_key
password: ${{ secrets.OPS_ARTIFACT_REGISTRY_SVC_ACCOUNT }}

- name: Build and Push Docker Images
run: |
docker build -t us-central1-docker.pkg.dev/$PROJECT_ID/$REPOSITORY/ops_terra_utils_slim:$GITHUB_SHA -f ./ops_docker_utils_slim/Dockerfile .
docker push us-central1-docker.pkg.dev/$PROJECT_ID/$REPOSITORY/ops_terra_utils_slim:$GITHUB_SHA
env:
PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
REPOSITORY: ${{ secrets.ARTIFACT_REPO }}
GITHUB_SHA: ${{ github.sha }}

- name: Set image tag to 'latest'
run: |
docker tag us-central1-docker.pkg.dev/$PROJECT_ID/$REPOSITORY/ops_terra_utils_slim:$GITHUB_SHA us-central1-docker.pkg.dev/$PROJECT_ID/$REPOSITORY/ops_terra_utils_slim:latest
docker push us-central1-docker.pkg.dev/$PROJECT_ID/$REPOSITORY/ops_terra_utils_slim:latest
uses: docker/build-push-action@v6

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Build and Push Docker images to Google Artifact Registry (GAR)' step
Uses Step
Loading
uses 'docker/build-push-action' with ref 'v6', not a pinned commit hash
with:
push: true
file: ./ops_docker_utils_slim/Dockerfile
tags: |
us-central1-docker.pkg.dev/$PROJECT_ID/$REPOSITORY/ops_terra_utils_slim:$GITHUB_SHA
us-central1-docker.pkg.dev/$PROJECT_ID/$REPOSITORY/ops_terra_utils_slim:Test
ssh: |
default=${{ env.SSH_AUTH_SOCK }}
env:
PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
REPOSITORY: ${{ secrets.ARTIFACT_REPO }}
Expand Down
27 changes: 11 additions & 16 deletions ops_docker_utils_slim/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.11.9
FROM python:3.11-bookworm


USER root
Expand Down Expand Up @@ -30,32 +30,27 @@ COPY general_markdown/ /etc/terra_utils/general_markdown
COPY .dockstore.yml /etc/terra_utils/


RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
RUN sudo apt-get update && sudo apt-get install google-cloud-cli -yq
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg && apt-get update -y && apt-get install google-cloud-cli -y

ENV PATH /usr/local/bin:$PATH

RUN sudo pip3 install --upgrade pip\
&& sudo pip3 install --upgrade -r /etc/terra-docker/requirements.txt

#install azcopy
#RUN curl -s -L https://azcopyvnext.azureedge.net/releases/release-10.24.0-20240326/azcopy_linux_amd64_10.24.0.tar.gz | tar xvzf - --strip-components=1 -C /usr/local/bin/ azcopy_linux_amd64_10.24.0/azcopy
#RUN chown root:root /usr/local/bin/azcopy
#RUN chmod +x /usr/local/bin/azcopy
#RUN ldd /usr/local/bin/azcopy


ENV AZCOPY_BUFFER_GB: 2
ENV AZCOPY_CONCURRENCY_VALUE: 4

RUN --mount=type=ssh,id=default pip3 install git+ssh://git@github.com/broadinstitute/ops_util_module.git@main

#install azcopy
RUN curl -sSL -O https://packages.microsoft.com/config/debian/12/packages-microsoft-prod.deb && \
dpkg -i packages-microsoft-prod.deb && \
rm packages-microsoft-prod.deb && \
apt-get update && \
apt-get install -y azcopy

ENV PATH $PATH:/usr/local/gcloud/google-cloud-sdk/bin

ENV PIP_USER=true
#RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - && apt-get update -y && apt-get install google-cloud-cli -y


#install gcloud cli
ENV GPG_TTY=$(tty)

CMD [ "/bin/sh" ]
Loading