Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[Feature Request]: Enable concurrency in GHW #2421

Closed
eriqua opened this issue Dec 9, 2022 · 2 comments · Fixed by #2422
Closed

[Feature Request]: Enable concurrency in GHW #2421

eriqua opened this issue Dec 9, 2022 · 2 comments · Fixed by #2422
Assignees
Labels
[cat] github category: GitHub [cat] pipelines category: pipelines enhancement New feature or request

Comments

@eriqua
Copy link
Contributor

eriqua commented Dec 9, 2022

Description

Running multiple instances of the same module validation workflow, may result in conflicts when trying, e.g., to deploy resources while the same resources are getting deleted by a previous run.

concurrency can be enabled at workflow (Ref: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#concurrency) or job level (Ref: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idconcurrency) to ensure that only a single workflow or job using the same concurrency group will run at a time.

In the default behavior, next runs are queued and started as soon as previous runs complete.
Alternatively, the cancel-in-progress setting can be used to cancel previous runs, triggering the most recent instead. This setting does not look useful in the CARML case, since canceling e.g. the deployment validation job, would not cancel the actual test deployment to Azure, resulting anyway in the same deployment conflicts described above.

This issue is about implementing a PoC to add concurrency to GH module validation workflows, proposing, e.g.,

  • On which level (workflow vs job) concurrency should be enabled
  • What would be the most convenient group to set up
@eriqua eriqua added the enhancement New feature or request label Dec 9, 2022
@eriqua eriqua self-assigned this Dec 9, 2022
@eriqua eriqua added [cat] pipelines category: pipelines [cat] github category: GitHub labels Dec 9, 2022
@eriqua
Copy link
Contributor Author

eriqua commented Dec 9, 2022

Tested group: group: ${{ github.workflow }}

Ref: GH context https://docs.github.com/en/actions/learn-github-actions/contexts#github-context
github.workflow: The name of the workflow. If the workflow file doesn't specify a name, the value of this property is the full path of the workflow file in the repository.

Since workflow names are unique for each module, this ensures only a single workflow of the same module runs at the same time.

@eriqua
Copy link
Contributor Author

eriqua commented Dec 9, 2022

Workflow vs job level

Workflow level concurrency tests

The whole workflow is queued
image

image

The workflow starts as soon as the previous run finishes
image

Job level concurrency tests

Concurrency can also be set at job level.
In the CARML case that would be ideal, since concurrency is not strictly required for static validation or publishing, but only for deployment validation.
However, due to the matrix testing variants in different jobs, enabling concurrency at deployment validation level results instead in intended parallel tests to wait for each other
image
In addition, in case the same module workflow is re-triggered meanwhile, deployment validation jobs in the second run gets canceled
image

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
[cat] github category: GitHub [cat] pipelines category: pipelines enhancement New feature or request
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant