gh-actions: add workflow for release tests

[miri64]

This provides a Github Actions Workflow to run the release tests both weekly currently configured for Sat 3:00 and on triggered events. For the triggered runs one can either use the REST API hook or, once this PR is merged into master, via the "Run workflow" mask in the Actions tab.

As input parameters the riot_version and the docker_version (for the riotbuild image) can be chosen. riot_version can be any git reference in the RIOT-OS/RIOT repo, including commit SHA1s, branches or tags. If riot_version is a release candidate tag (so name format like 2020.07-RC2), the framework will recognize this and will also manipulate the test tracking issue for that RC (see #173). The idea is, that once this PR gets merged a webhook the workflow is run once that tag is created in RIOT-OS/RIOT.

For the weekly cron job, the release tests are always run against the current master of RIOT-OS/RIOT.

The current deployment can be checked here.

Dependencies

This PR includes #164 (and all its dependencies), #173, and #178.

[miri64]

Looking at RIOT-OS/RIOT#14587 it might make sense to move this to RIOT-OS/RIOT as well. This way, reacting to tag creation would be much more easier.

[miri64]

With having it here, some webhook would be required to start the job on tag creation over at RIOT-OS/RIOT. By moving it over it would be as easy as

on:
  push:
    tags:
      - '[0-9][0-9][0-9][0-9].[0-9][0-9]-RC[0-9]*'
      - '[0-9][0-9][0-9][0-9].[0-9][0-9]'

[fjmolinas]

I think although it might be easier it mixes some some concerns, from what I see here it is something github wants to work on. For now it would seem to me that it would be somehow cleaner to have just a ymal setting the trigger in RIOT-OS/RIOT maybe something like webhook-trigger? What do you think?

[miri64]

I think although it might be easier it mixes some some concerns, from what I see here it is something github wants to work on. For now it would seem to me that it would be somehow cleaner to have just a ymal setting the trigger in RIOT-OS/RIOT maybe something like webhook-trigger? What do you think?

So you rather want to create two actions, one in RIOT-OS/RIOT and RIOT-OS/Release-Specs? IMHO this could potentially add some managing overhead: Two yamls equal two yamls to be edited, PR'd, and reviewed in case anything changes in our release procedures.

[fjmolinas]

So you rather want to create two actions, one in RIOT-OS/RIOT and RIOT-OS/Release-Specs? IMHO this could potentially add some managing overhead: Two yamls equal two yamls to be edited, PR'd, and reviewed in case anything changes in our release procedures.

I cant imagine the webhook requiring much editing. On the other side of the argument we would be adding more code into RIOT-OS/RIOT and that yaml file goes through a more costly review procedure (even if skipping ci-builds you have to wait for it to go through the queue) than if the bulk of the configuration is kept in this repo. That being said, I don't have a strong opinion on this so I would no insist on any approach.

[miri64]

[…] if the bulk of the configuration is kept in this repo.

Just to be clear: I am just talking about moving the yaml file to the RIOT repo. The tests and all will of course remain here. As the workflow checks out both repos already

Release-Specs/.github/workflows/release-test.yml

Lines 31 to 41 in d7d7f66

	- name: Checkout Release-Specs
	uses: actions/checkout@v2
	with:
	path: Release-Specs
	- name: Checkout RIOT
	uses: actions/checkout@v2
	with:
	repository: RIOT-OS/RIOT
	path: RIOT
	fetch-depth: 1
	ref: ${{ github.event.inputs.riot_version }}

and none of the hooks are currently repo-dependent

Release-Specs/.github/workflows/release-test.yml

Lines 3 to 15 in d7d7f66

	on:
	schedule:
	- cron: '0 3 * * 6'
	workflow_dispatch:
	inputs:
	riot_version:
	description: 'RIOT version to checkout'
	required: true
	default: 'master'
	docker_version:
	description: 'riot/riotbuild docker image version'
	required: true
	default: 'latest'

the move over there wouldn't even be that hard.

[miri64]

In the end: the system under tests with this workflow is RIOT, not the release specs ;-).

[miri64]

the move over there wouldn't even be that hard.

See RIOT-OS/RIOT#14646

[fjmolinas]

Can you rebase this one to have a better view of the PR?

[miri64]

Can you rebase this one to have a better view of the PR?

Rebased to current master and out-standing dependencies (#173 and #178). However, now that I ported it to RIOT-OS/RIOT#14646, I think I prefer it that way.

[miri64]

77d477c will go into its own PR once confirmed it solves the problems with native. Explanation will follow there.

[miri64]

77d477c will go into its own PR once confirmed it solves the problems with native. Explanation will follow there.

See #180

[miri64]

Rebased and included #181 for re-resting because I dunno .... 😪

[aabadie]

However, now that I ported it to RIOT-OS/RIOT#14646, I think I prefer it that way.

So this PR becomes obsolete once RIOT-OS/RIOT#14646 is in ?

[miri64]

However, now that I ported it to RIOT-OS/RIOT#14646, I think I prefer it that way.

So this PR becomes obsolete once RIOT-OS/RIOT#14646 is in ?

Yes. I mainly use this PR to track and test the remaining PRs required for RIOT-OS/RIOT#14646.

[miri64]

But since #181 is the only relevant left, I think we can close this now.