Improve: Consolidate artifact build process for all environments & Add Workflow for preparing release branch #4256
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Remove CI_BUILD Cmake variable. - Convert CI_TAG from ENV variable to CMake variable - CI_VERSION_PRE_RELEASE can be set in non CI builds
- add CI_TAG and CI_VERSION_PRE_RELEASE to build.sh - add useful ARG with default values to Dockerfile so they can be passed to the build.sh script
- remove CI_BUILD - convert DCI_TAG from ENV to CMake variable
…rkflow matrix usage
- Remove dependency on the workflow name - COnvert $GITHUB_WORKFLOW to $NETWORK - Create smaller functions with limited scope - Create similar deploy functions for docker and github container registries (hub.docker and ghcr)
- Remove old workflows - Create 1 workflow for all environments (Network Matrix) - Keep current build logic (build scripts still differ per OS)
- skip hub.docker deploys if DOCKER_PASSWORD is not provided - Create DOCKER_HUB variable which defaults to nanocurrency (backwards compatible) - Create DOCKER_USER variable which defaults to nanoreleaseteam (backwards compatible) - create S3_BUCKET_NAME variable that defaults to repo.nano.org if not provided (backwards compatible) - only use S3_BUILD_DIRECTORY if provided
Convert nanocurrency/nano-env image to self built ghcr.io/${{ github.repository }} image
- convert ref to CI_TAG - use CI_TAG in aws deploys
- add possibility to specify registry - use ghcr.io instead of variable for ghcr_image_name
- vars.S3_BUCKET_NAME - vars.S3_BUILD_DIRECTORY - vars.DOCKER_REGISTRY - vars.DOCKER_USER
The goal is to simplify tag generation process and commit the version_pre_release into CMakeLists.txt for each tag.
If a user checks out a specific tag and builds it, the version_pre_release is set correctly.
- remove workflow_dispatch inputs. It operates on the selected branch.
- The cronjob is executed on develop branch only.
- replace ci/actions/dev-build-tag-gen.sh with ci/actions/generate_next_git_tag.sh
- generate_next_git_tag.sh is branch agnostic and operates on ${{ github.ref }}
- generate_next_git_tag.sh succeeds even if no new tag is generated
- the workflow only executes the build jobs if a new tag was created (if: ${{ needs.prepare_build.outputs.TAG_CREATED == 'true' }})
- generate_next_git_tag.sh uses V${current_version_major}.${current_version_minor}${branch_name} for tags.
- for "develop" branch_name is converted to DB --> (e.g V26.0DB1)
- generate_next_git_tag.sh uses a -c flag that is responsible to update CMakeLists.txt with correct version_pre_release, create and push the tag to origin.
…txt in the tag used to build the node
- use the new prepare scripts (Linux, MacOS & Windows) - remove the need the dependency on nano-env:gcc
- Build the nano-env docker image in the current workflow - Use the locally built image.
- Remove duplicate BUILD_TYPE - move SANITIZER to ci/build-node.sh ARGs
- fixes a bug when we have 2 similar branches with tags (e.g. some-branch and some-branch-2)
Prevent modification of global git settings on a developer machine
- refactored script by making it more modular
- script expects releases to be made from a branch called `releases/v{Major}`
- add -i flag to indicate wether or not to increment the generated tag
- -i increments version_minor for release branches: tag --> V{Major}.{Minor + increment}
- -i increments version_pre_release for all other non-release branches: tag --> V{Major}.{Minor}{branch_name}{pre_release + increment}
- -o outputs either `version_pre_release` or `version_minor` depending on the branch
Prevent incrementing if no tag exists yet
- make sure V{Major}.0 is created even if the user forgets to set increment to 0
Make tag_created=true explicit
- by redefining local output=$1 it is set to 0 instead of "" and a file called 0 was created
…doesn't find anything
- increment is 1 by default - if increment is 0, an existing tag will be updated (origin push -f). fixup! Add increment input to the workflow
- the first tag of a releases/ branch creates no changes - a tag run with increment=0 creates no changes in both cases we want to create the tag (again)
- Checks out the repository based on github.ref. - Extracts and sets current major and minor versions from CMakeLists.txt of github.ref. - Fetches the default branch name and sets it as an output. - Checks for existence of a corresponding release branch. If present, the workflow aborts, if not, it continues and sets the new release branch name. - Increments the major version in CMakeLists.txt on the default branch, commits this change, and pushes it back. - Checks out a new release branch, sets the pre-release version to 0 in CMakeLists.txt, commits this change, and pushes the new branch to the repository. fix: Configure git and fix output variables Change the order of steps to make the flow more natural Fix curl DEFAULT_BRANCH fix fix
gr0vity-dev
changed the title
gr0vity-dev
force-pushed
the
prs/unified_artifacts_worflow
branch
2 times, most recently
from
3b11874
to
6877a57
Compare
- Autodetect if we need to increment the version or not on the release branch Expected behaviour : - Running the workflow multiple times without a new commit will rebuild and republish the same artifacts - Running the workflow after a new commit was made to the releases branch will increment version_minor by 1 - the version_minor is updated in CMakeLists.txt and committed to the release branch - a new tag is created and pushed with the incremented version_minor - Make logic for both release and other branches more obvious --> TODO?: instead of force pushing, we could remove -f option if branch or tag already exist
gr0vity-dev
force-pushed
the
prs/unified_artifacts_worflow
branch
from
6877a57
to
e6d115e
Compare
pwojcikdev
previously approved these changes
Jul 15, 2023
|
I only have a few cosmetic comments, feel free to ignore those. Generally the PR looks well organized and will be a significant improvement. |
Convert if/else into case..in
pwojcikdev
approved these changes
Jul 16, 2023
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What will the build pipeline look like ?
What will the release process look like ?
We run the “Prepare release” workflow on the tag (or branch) which we want to use as starting point for the newly created release branch.
After the release branch has been successfully created we can run the “Build all artifacts” workflow manually to generate the release builds.
Improvements :
Possible simplification
We could get rid of the “increment” input in the workflow by adopting a similar behaviour as DB builds.
When “Build all artifacts” runs on a release branch,
Prerequisits before merge
Required secrets:
Optional secrets
Optional Repository variables:
How was the PR tested ?
Run "All (Live Beta Test)" manually on branch prs/unified_artifacts_worflow
https://github.com/gr0vity-dev/nano-node/actions/runs/5444908141/jobs/9903410239
run "Prepare Release" from {tag} to create releases/v26
https://github.com/gr0vity-dev/nano-node/actions/runs/5444720099/jobs/9902956258
run "Prepare Release" again. It's expected to fail because the releases/v26 already exists.
https://github.com/gr0vity-dev/nano-node/actions/runs/5445139689/jobs/9903953960
Run "All (Live Beta Test)" manually on tag V26.0
https://github.com/gr0vity-dev/nano-node/actions/runs/5446028132
Creates docker images:
Creates binaries:
What needs to be tested ?
increment= 0 on a DB buildincrementwith values other than 0 or 1Flowcharts of the intended behaviour for “Build all artifacts” workflow
Flowcharts of the intended behaviour for “Prepare release” workflow