Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

fix(applicationset): use requeue after if generate app errors out (cherry-pick #18761) #19089

Closed

Conversation

gcp-cherry-pick-bot[bot]
Copy link

Cherry-picked fix(applicationset): use requeue after if generate app errors out (#18761)

The GenerateApplications can call to external resources like Github
API for instance which might be rate limited or fail. If those requests
somehow fail we should requeue them after some time like (same
reason as

// While some generators may return an error that requires user intervention,
).

For instance, in our environments we were rate limited by Github and the ArgoCD
applicationset controller was logging the following error about every
second or less for every application set using the pull request generator
that we have:

time="2024-06-21T14:17:15Z" level=error msg="error generating params" error="error listing repos: error listing pull requests for LedgerHQ/xxx: GET https://api.github.com/repos/LedgerHQ/xxx/pulls?per_page=100: 403 API rate limit exceeded for installation ID xxx. If you reach out to GitHub Support for help, please include the request ID xxx and timestamp 2024-06-xx xxx UTC. [rate reset in 8m18s]" generator="&{0xc000d652c0 0x289a100 {0xc00087bdd0}  [] true}"
time="2024-06-21T14:17:15Z" level=error msg="error generating application from params" applicationset=argocd/xxx error="error listing repos: error listing pull requests for LedgerHQ/xxxx: GET https://api.github.com/repos/LedgerHQ/xxx/pulls?per_page=100: 403 API rate limit exceeded for installation ID xxx. If you reach out to GitHub Support for help, please include the request ID xxx and timestamp 2024-06-xx xxx UTC. [rate reset in 8m18s]" generator="{nil nil nil nil nil &PullRequestGenerator{Github:&PullRequestGeneratorGithub{Owner:LedgerHQ,Repo:xxx,API:,TokenRef:nil,AppSecretName:xxxx,Labels:[argocd/preview],},GitLab:nil,Gitea:nil,BitbucketServer:nil,Filters:[]PullRequestGeneratorFilter{},RequeueAfterSeconds:*1800,Template:ApplicationSetTemplate{ApplicationSetTemplateMeta:ApplicationSetTemplateMeta{Name:,Namespace:,Labels:map[string]string{},Annotations:map[string]string{},Finalizers:[],},Spec:ApplicationSpec{Source:nil,Destination:ApplicationDestination{Server:,Namespace:,Name:,},Project:,SyncPolicy:nil,IgnoreDifferences:[]ResourceIgnoreDifferences{},Info:[]Info{},RevisionHistoryLimit:nil,Sources:[]ApplicationSource{},},},Bitbucket:nil,AzureDevOps:nil,} nil nil nil nil}"

Signed-off-by: Arthur Outhenin-Chalandre arthur.outhenin-chalandre@ledger.fr

…8761)

The `GenerateApplications` can call to external resources like Github
API for instance which might be rate limited or fail. If those requests
somehow fail we should requeue them after some time like (same
reason as https://github.com/argoproj/argo-cd/blob/e98d3b2a877fb7140880dc1450703a99db4398b5/applicationset/controllers/applicationset_controller.go#L154).

For instance, in our environments we were rate limited by Github and the ArgoCD
applicationset controller was logging the following error about every
second or less for every application set using the pull request generator
that we have:
```
time="2024-06-21T14:17:15Z" level=error msg="error generating params" error="error listing repos: error listing pull requests for LedgerHQ/xxx: GET https://api.github.com/repos/LedgerHQ/xxx/pulls?per_page=100: 403 API rate limit exceeded for installation ID xxx. If you reach out to GitHub Support for help, please include the request ID xxx and timestamp 2024-06-xx xxx UTC. [rate reset in 8m18s]" generator="&{0xc000d652c0 0x289a100 {0xc00087bdd0}  [] true}"
time="2024-06-21T14:17:15Z" level=error msg="error generating application from params" applicationset=argocd/xxx error="error listing repos: error listing pull requests for LedgerHQ/xxxx: GET https://api.github.com/repos/LedgerHQ/xxx/pulls?per_page=100: 403 API rate limit exceeded for installation ID xxx. If you reach out to GitHub Support for help, please include the request ID xxx and timestamp 2024-06-xx xxx UTC. [rate reset in 8m18s]" generator="{nil nil nil nil nil &PullRequestGenerator{Github:&PullRequestGeneratorGithub{Owner:LedgerHQ,Repo:xxx,API:,TokenRef:nil,AppSecretName:xxxx,Labels:[argocd/preview],},GitLab:nil,Gitea:nil,BitbucketServer:nil,Filters:[]PullRequestGeneratorFilter{},RequeueAfterSeconds:*1800,Template:ApplicationSetTemplate{ApplicationSetTemplateMeta:ApplicationSetTemplateMeta{Name:,Namespace:,Labels:map[string]string{},Annotations:map[string]string{},Finalizers:[],},Spec:ApplicationSpec{Source:nil,Destination:ApplicationDestination{Server:,Namespace:,Name:,},Project:,SyncPolicy:nil,IgnoreDifferences:[]ResourceIgnoreDifferences{},Info:[]Info{},RevisionHistoryLimit:nil,Sources:[]ApplicationSource{},},},Bitbucket:nil,AzureDevOps:nil,} nil nil nil nil}"
```

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@ledger.fr>
@gcp-cherry-pick-bot gcp-cherry-pick-bot bot requested a review from a team as a code owner July 17, 2024 00:01
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants