fix(appset): Fix perpetual appset reconciliation

[tjamet]

ScmProvider does not guarantee order of the application resources from the applicationset which causes rapid sync activity for the applicationset as the objects and hence their resourceVersions are updated after each reconcile loop.

This then triggers reconciliation of all objects watching the ApplicationSet.

In order to prevent this behaviour, ensure that the ApplicationSet reconciler provides an idempotent list of resources, ensuring objects are not updated.

Fixes: #19757

This PR is an alternate approach to #19676.

In #19676, the approach is to only reconcile based on generation changes, i.e. spec changes as defined by the generation implementation in CRDs.
This means that, with this PR, any change of metadata (labels, annotations, ...) or status will be taken into consideration on the next run (triggered by requeueAfter, 3mn by default in my trials), which is the way new SCM repos or folders are discovered from my understanding.

This PR takes a different approach, ensuring that status updates are idempotent across reconcile loops and hence prevents from triggering any reconciliation in other reconcilers unless there is an actual change on the ApplicationSet.

Checklist:

• Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
• The title of the PR states what changed and the related issues number (used for the release note).
• The title of the PR conforms to the Toolchain Guide
• I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
• I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
• Does this PR require documentation updates?
• I've updated documentation as required by this PR.
• I have signed off all my commits as required by DCO
• I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
• My build is green (troubleshooting builds).
• My new feature complies with the feature status guidelines.
• I have added a brief description of why this PR is necessary and/or what this PR solves.
• Optional. My organization is added to USERS.md.
• Optional. For bug fixes, I've indicated what older releases this fix should be cherry-picked into (this may or may not happen depending on risk/complexity).

[bunnyshell]

❌ Preview Environment deleted from Bunnyshell

Available commands (reply to this comment):

• 🚀 /bns:deploy to deploy the environment

[bunnyshell]

❌ Preview Environment deleted from Bunnyshell

Available commands (reply to this comment):

• 🚀 /bns:deploy to deploy the environment

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (master@031fb88). Learn more about missing BASE report.
Report is 39 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##             master   #19822   +/-   ##
=========================================
  Coverage          ?   55.84%           
=========================================
  Files             ?      320           
  Lines             ?    44381           
  Branches          ?        0           
=========================================
  Hits              ?    24786           
  Misses            ?    17027           
  Partials          ?     2568           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[tjamet]

I see there is a license check problem, if I read correctly, the problem also appears on other PRs, is there anything needed from us to fix it?

[jetersen]

Looks good to me 👍 Thanks for implementing a fix for #19757

[agaudreault]

LGTM

[jetersen]

Would be great to get a patch release for this, it is very noisy inside our etcd 😓

[gmauleon]

yes please! 👆 🙏

[agaudreault]

/cherry-pick release-2.12