Skip to content

Commit

Permalink
Merge pull request #15 from Environmental-DS-Book/post-print
Browse files Browse the repository at this point in the history
Post print
  • Loading branch information
acocac authored Mar 13, 2023
2 parents f9e70cf + a023542 commit bcb2e1d
Show file tree
Hide file tree
Showing 9 changed files with 200 additions and 66 deletions.
5 changes: 5 additions & 0 deletions .github/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"notebook_name": "general-exploration-landcover_io.ipynb",
"kernel": "python",
"quay_image": "edsbook/general-exploration-landcover_io"
}
30 changes: 30 additions & 0 deletions .github/workflows/binder.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Reference https://mybinder.readthedocs.io/en/latest/howto/gh-actions-badges.html
name: Test this PR on Binder Badge
on:
pull_request_target:
types: [opened]

permissions:
pull-requests:
write

jobs:
binder:
runs-on: ubuntu-latest
steps:
- name: comment on PR with Binder link
uses: actions/github-script@v3
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
var PR_HEAD_USERREPO = process.env.PR_HEAD_USERREPO;
var PR_HEAD_REF = process.env.PR_HEAD_REF;
github.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: `[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/${PR_HEAD_USERREPO}/${PR_HEAD_REF}) :point_left: Test this PR on Binder`
})
env:
PR_HEAD_REF: ${{ github.event.pull_request.head.ref }}
PR_HEAD_USERREPO: ${{ github.event.pull_request.head.repo.full_name }}
47 changes: 47 additions & 0 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: Build and push container image

on:
push:
branches:
- main

jobs:
build-and-push:
runs-on: ubuntu-latest
if: github.repository_owner == 'Environmental-DS-Book'
steps:
# For biggish images, github actions runs out of disk space.
# So we cleanup some unwanted things in the disk image, and reclaim that space for our docker use
# https://github.com/actions/virtual-environments/issues/2606#issuecomment-772683150
# and https://github.com/easimon/maximize-build-space/blob/b4d02c14493a9653fe7af06cc89ca5298071c66e/action.yml#L104
# This gives us a total of about 52G of free space, which should be enough for now
- name: cleanup disk space
run: |
sudo rm -rf /usr/local/lib/android /usr/share/dotnet /opt/ghc
df -h
- name: Checkout files in repo
uses: actions/checkout@main

# Set quay image
- name: Get properties from the config file
id: config_quay
uses: notiz-dev/github-action-json-property@release
with:
path: '.github/config.json'
prop_path: 'quay_image'

- name: Build and push the image to quay.io
uses: jupyterhub/repo2docker-action@master
with:
# Make sure username & password/token pair matches your registry credentials
DOCKER_USERNAME: ${{ secrets.QUAY_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.QUAY_PASSWORD }}
DOCKER_REGISTRY: "quay.io"

# Uncomment and modify the following line with your image name, otherwise no push will happen
IMAGE_NAME: ${{steps.config_quay.outputs.prop}}

# Lets us monitor disks getting full as images get bigger over time
- name: Show how much disk space is left
run: df -h
60 changes: 0 additions & 60 deletions .github/workflows/publish.yml

This file was deleted.

70 changes: 70 additions & 0 deletions .github/workflows/render.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
name: Render

on:
workflow_run:
workflows: ["Build and push container image"]
types:
- completed

# This job installs dependencies, build the jupyter notebook, and pushes it to `render`, a new `branch`
jobs:
build:
name: ${{ matrix.os }}
runs-on: ${{ matrix.os }}

if: github.repository_owner == 'Environmental-DS-Book'
strategy:
fail-fast: false
matrix:
os: ["ubuntu-latest"]
defaults:
run:
shell: bash -l {0}
steps:
- uses: actions/checkout@v2
# Set notebook name
- name: Get properties from the config file
id: config_file
uses: notiz-dev/github-action-json-property@release
with:
path: '.github/config.json'
prop_path: 'notebook_name'
# Set quay image
- name: Get properties from the config file
id: config_quay
uses: notiz-dev/github-action-json-property@release
with:
path: '.github/config.json'
prop_path: 'quay_image'
# Set kernel name
- name: Get properties from the config file
id: config_kernel
uses: notiz-dev/github-action-json-property@release
with:
path: '.github/config.json'
prop_path: 'kernel'
# Render the notebook
- name: Run the build process with Docker
uses: addnab/docker-run-action@v3
with:
image: quay.io/${{steps.config_quay.outputs.prop}}:latest
options: --user root -v ${{ github.workspace }}:/tmp
shell: bash
run: |
v=$(jupyter kernelspec list | awk '{print $1}' | egrep ${{steps.config_kernel.outputs.prop}})
jupyter-nbconvert --to notebook --ExecutePreprocessor.kernel_name="$v" --inplace --execute ${{steps.config_file.outputs.prop}}
cp ${{steps.config_file.outputs.prop}} /tmp
- name: Commit
run: |
ls
git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
git config --local user.name "github-actions[bot]"
git rm \* ':!${{steps.config_file.outputs.prop}}'
git add ${{steps.config_file.outputs.prop}}
git commit -m 'update'
- name: Push
uses: ad-m/github-push-action@master
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
branch: render
force: true
44 changes: 44 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Test container image build

on:
pull_request:

jobs:
test-build:
runs-on: ubuntu-latest
if: github.repository_owner == 'Environmental-DS-Book'
steps:

# For biggish images, github actions runs out of disk space.
# So we cleanup some unwanted things in the disk image, and reclaim that space for our docker use
# https://github.com/actions/virtual-environments/issues/2606#issuecomment-772683150
# and https://github.com/easimon/maximize-build-space/blob/b4d02c14493a9653fe7af06cc89ca5298071c66e/action.yml#L104
# This gives us a total of about 52G of free space, which should be enough for now
- name: cleanup disk space
run: |
sudo rm -rf /usr/local/lib/android /usr/share/dotnet /opt/ghc
df -h
- name: checkout files in repo
uses: actions/checkout@main

# Set quay image name
- name: Get properties from the config file
id: config_file
uses: notiz-dev/github-action-json-property@release
with:
path: '.github/config.json'
prop_path: 'quay_image'

- name: Build the image and push it if `NO_PUSH` disabled
uses: jupyterhub/repo2docker-action@master
with: # make sure username & password/token matches your registry
NO_PUSH: "true"
DOCKER_REGISTRY: "quay.io"

# Uncomment and modify the following line with your image name. a
IMAGE_NAME: ${{steps.config_quay.outputs.prop}}

# Lets us monitor disks getting full as images get bigger over time
- name: Show how much disk space is left
run: df -h
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2022 Environmental-DS-Book
Copyright (c) 2022 EDS book

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@
<a href="https://notebooks.gesis.org/binder/v2/gh/Environmental-DS-Book/general-exploration-landcover_io/main?labpath=general-exploration-landcover_io.ipynb">
<img alt="Binder" src="https://mybinder.org/badge_logo.svg">
</a>
<a href="https://github.com/Environmental-DS-Book/general-exploration-landcover_io/actions/workflows/publish.yml/badge.svg">
<img alt="Continuous integration badge" src="https://github.com/Environmental-DS-Book/general-exploration-landcover_io/actions/workflows/publish.yml/badge.svg">
<a href="https://github.com/Environmental-DS-Book/general-exploration-landcover_io/actions/workflows/render.yaml">
<img alt="Render" src="https://github.com/Environmental-DS-Book/general-exploration-landcover_io/actions/workflows/render.yaml/badge.svg">
</a>
<br/>
</p>
Expand Down Expand Up @@ -66,3 +66,4 @@ You may also download the notebook from GitHub to run it locally:

# Credits
The **How to run** section was adapted from the [Project Pythia Cookbook](https://cookbooks.projectpythia.org/) project.
The workflow actions were adapted from [2i2c’s hub-user-image-template](https://github.com/2i2c-org/hub-user-image-template) released under BSD-3-Clause license.
3 changes: 0 additions & 3 deletions config.json

This file was deleted.

0 comments on commit bcb2e1d

Please # to comment.