Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

scorecard: use TestList for output, exit 1 on test failure #3427

Merged
merged 2 commits into from
Jul 15, 2020

Conversation

joelanford
Copy link
Member

Description of the change:

  1. Instead of aggregating all test results from ALL scorecard tests under a single v1alpha3.Test status, use a new v1alpha3.TestList to output a list of v1alpha3.Test objects, one for each test defined in the scorecard config file.
  2. Exit with status 1 when any test fails.

Motivation for the change:

  1. The test results should not be disassociated from the test spec (image/entrypoint) that provided the result.
  2. To match existing documentation

Checklist

If the pull request includes user-facing changes, extra documentation is required:

@joelanford joelanford added this to the v1.0.0 milestone Jul 15, 2020
@joelanford joelanford force-pushed the scorecard-output-api branch from 800649e to d8ab436 Compare July 15, 2020 03:26
@jmccormick2001
Copy link
Contributor

seems to work for the basic/olm case, the kuttl image needs the updates for that to work again, also the custom test image would need the same updates.

@joelanford
Copy link
Member Author

Just tested kuttl, it also works, which is expected because this PR does not change anything about the pod log interface and result gathering.

@joelanford joelanford merged commit 4d8960c into operator-framework:master Jul 15, 2020
@joelanford joelanford deleted the scorecard-output-api branch July 15, 2020 15:49
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants