sonobuoy-serviceaccount makes the cluster_admin platform test fail

[sysarch-repo]

Describe the bug
The platform test cluster_admin fails with reference to the sonobuoy facility that is part of the testing framework.

To Reproduce
Steps to reproduce the behavior:

1. Install and set up 1.3.3
2. Run the cluster_admin platform test
3. See the failed test referring to sonobuoy-serviceaccount:

🎬 Testing: [cluster_admin]
�[31mFailed resource: ServiceAccount sonobuoy-serviceaccount�[0m
�[31mRemediation: You should apply least privilege principle. Make sure cluster admin permissions are granted only when it is absolutely necessary. Don't use subjects with such high permissions for daily operations.�[0m

Expected behavior
The testing framework shall not interfere with the AUT/cluster.

Device (please complete the following information):

[ec2-user@ip-10-0-110-88 ~]$ uname -a
Linux ip-10-0-110-88.ec2.internal 5.10.223-212.873.amzn2.x86_64 #1 SMP Wed Aug 7 16:53:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

How will this be tested? aka Acceptance Criteria (optional)

Additional context

[martin-mat]

I don't understand why this should be a bug. The test detects service account with cluster admin permissions so it reports it and fails, as per test description
https://github.com/cnti-testcatalog/testsuite/blob/main/docs/TEST_DOCUMENTATION.md#cluster-admin

[sysarch-repo]

@martin-mat Thanks for your comment. The use of Sonobuoy is the decision of the CNF testsuite creator, not the user. Therefore, the tools selected for the execution of tests (Sonobuoy for the k8s_conformance test) should not fail any tests that should solely target the AUT of the user. The user cannot fix the Sonobuoy deployment to make the platform test pass for his/her AUT.

[martin-mat]

I cannot get it reproduced. Can you check and give exact reporoduction steps?

[sysarch-repo]

@martin-mat,
it looks like the issue is with an orphaned Sonobuoy serviceaccount resource. With this, you must run the entire platform test group (the k8s_conformance test before the cluster_admin test) to run into the issue. Just running the platform:cluster_admin test in isolation will not be enough for providing evidence.

Isolated cluster_admin test (no issue as there is no Sonobuoy serviceaccount resource deployed yet):

• deploy-aut.sh dns
  �[32mCNF configuration validated 📋�[0m
  �[32mSuccessfully created directories for cnf-testsuite�[0m
  �[32mKUBECONFIG is already set.�[0m
  �[32mcnf-testsuite namespace already exists on the Kubernetes cluster�[0m
  �[32mClusterTools installed�[0m
  �[32mcnf setup start�[0m
  �[32mWaiting for resource availability, timeout for each resource is 1800 seconds
  �[0m
  �[32m�[1A�[KWaiting for resource (1/2): [Deployment] dns-dserver�[0m
  �[32m�[1A�[KWaiting for resource (2/2): [StatefulSet] dns-drouter�[0m
  �[32m�[1A�[KAll CNF resources are up!�[0m
  �[32mSuccessfully setup dns�[0m
  �[32mcnf setup complete�[0m
  AUT dns deployed.
• cnf-testsuite platform:cluster_admin
  🎬 Testing: [cluster_admin]
  �[32m✔️ PASSED: [cluster_admin] No users with cluster-admin RBAC permissions were found 🔓🔑�[0m

All platform tests (issue due to orphaned Sonobuoy serviceaccount resource after the k8s_conformance test):

• deploy-aut.sh dns
  �[32mCNF configuration validated 📋�[0m
  �[32mSuccessfully created directories for cnf-testsuite�[0m
  �[32mKUBECONFIG is already set.�[0m
  �[32mcnf-testsuite namespace already exists on the Kubernetes cluster�[0m
  �[32mClusterTools installed�[0m
  �[32mcnf setup start�[0m
  �[32mWaiting for resource availability, timeout for each resource is 1800 seconds
  �[0m
  �[32m�[1A�[KWaiting for resource (1/2): [Deployment] dns-dserver�[0m
  �[32m�[1A�[KWaiting for resource (2/2): [StatefulSet] dns-drouter�[0m
  �[32m�[1A�[KAll CNF resources are up!�[0m
  �[32mSuccessfully setup dns�[0m
  �[32mcnf setup complete�[0m
  AUT dns deployed.
• cnf-testsuite platform
  �[32mSuccessfully created directories for cnf-testsuite�[0m
  🎬 Testing: [k8s_conformance]
  �[32m✔️ PASSED: [k8s_conformance] K8s conformance test has no failures �[0m
  🎬 Testing: [kube_state_metrics]
  �[33m⏭️ SKIPPED: [kube_state_metrics] Kube State Metrics not in poc mode 📶☠�[0m
  🎬 Testing: [node_exporter]
  �[33m⏭️ SKIPPED: [node_exporter] node exporter not in poc mode 📶☠�[0m
  🎬 Testing: [prometheus_adapter]
  �[33m⏭️ SKIPPED: [prometheus_adapter] prometheus adapter not in poc mode 📶☠�[0m
  🎬 Testing: [metrics_server]
  �[33m⏭️ SKIPPED: [metrics_server] Metrics server not in poc mode 📶☠�[0m
  �[31mPlatform Observability results: 0 of 4 tests passed
  �[0m
  🎬 Testing: [worker_reboot_recovery]
  �[33m⏭️ SKIPPED: [worker_reboot_recovery] Node not in destructive mode �[0m
  �[31mPlatform Resilience results: 0 of 1 tests passed
  �[0m
  🎬 Testing: [oci_compliant]
  �[32m✔️ PASSED: [oci_compliant] Your platform is using the following runtimes: [containerd://1.7.11] which are OCI compliant runtimes �[0m
  �[32mPlatform Hardware And Scheduling results: 1 of 1 tests passed
  �[0m
  🎬 Testing: [control_plane_hardening]
  �[32m✔️ PASSED: [control_plane_hardening] Insecure port of Kubernetes API server is not enabled 🔓🔑�[0m
  🎬 Testing: [cluster_admin]
  �[31mFailed resource: ServiceAccount sonobuoy-serviceaccount�[0m
  �[31mRemediation: You should apply least privilege principle. Make sure cluster admin permissions are granted only when it is absolutely necessary. Don't use subjects with such high permissions for daily operations.�[0m
  �[31m✖️ FAILED: [cluster_admin] Users with cluster-admin RBAC permissions found 🔓🔑�[0m
  🎬 Testing: [helm_tiller]
  �[32m✔️ PASSED: [helm_tiller] No Helm Tiller containers are running 🔓🔑�[0m
  �[32mPlatform Security results: 2 of 3 tests passed
  �[0m
  �[32mFinal platform score: 20 of 55�[0m
  �[32mTest results have been saved to results/cnf-testsuite-results-20240919-105528-726.yml�[0m