Fix test flakiness resulting from credhub-service-broker asset's improper error handling #863
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some differences in the way the new app works: * Shares service UUID and plan UUID between catalog calls, rather than generating each time. * Logs and returns internal server errors on certain failures. * Only spins up credhub client on startup and reuses throughout. * No manifest, as the old one was just setting a GOPACKAGENAME, which isn't necessary anymore. In particular, that second bullet is important, as there were some consistent flakes when running CATs due to a one-time failure to connect with credhub that wasn't handled appropriately by the old app, and resulted in a panic. Also updates the tests to use this new app more appropriate, with the `--no-start` flag and then starting it up after environment variables are set. This should actually speed up runtime!
davewalter
approved these changes
Jun 21, 2023
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Are you submitting this PR against the develop branch?
Yes
What is this change about?
Refactors the credhub-service-broker asset for clarity, and to handle errors better. We were seeing a good amount of pipeline flakes recently that were resulting from the app going into a panic off one-time failures to connect to CredHub.
Please provide contextual information.
https://concourse.wg-ard.ci.cloudfoundry.org/teams/main/pipelines/cf-deployment/jobs/upgrade-cats/builds/340
What version of cf-deployment have you run this cf-acceptance-test change against?
v29.1.0
Please check all that apply for this PR:
Did you update the README as appropriate for this change?
If you are introducing a new acceptance test, what is your rationale for including it CATs rather than your own acceptance test suite?
N/A
How many more (or fewer) seconds of runtime will this change introduce to CATs?
Actually knocks about 2 seconds off the runtime per test using this app because of the switch from restarting to starting. Not including the windows suite this amounts to about 16 seconds total.
What is the level of urgency for publishing this change?
Tag your pair, your PM, and/or team!
None