Why am I getting list operation denied errors for a controller created for namespace scoped resources?

[jwatte]

I created a project with kubebuilder init, and added two CRDs, to be managed by namespace.
However, when installing and starting the controller, I get an error message that the controller isn't allowed to list resources at cluster scope.

Where is this disconnect (does it need to list at cluster scope? Or did I somehow manage to not restrict to namespace?), and where should I look to diagnose/update/fix this?

Versions:
Version: main.version{KubeBuilderVersion:"4.1.0", KubernetesVendor:"1.27.1", GitCommit:"de1cc60900b896b2195e403a40c976a892df4921", BuildDate:"2024-07-05T09:07:42Z", GoOs:"darwin", GoArch:"arm64"}

Here's the PROJECT file:

domain: example.com
layout:
- go.kubebuilder.io/v4
projectName: mps-operator
repo: github.com/example-com/mps-operator/v2
resources:
- api:
    crdVersion: v1
    namespaced: true
  controller: true
  domain: example.com
  kind: MPSVersion
  path: github.com/example-com/mps-operator/v2/api/v1alpha1
  version: v1alpha1
- api:
    crdVersion: v1
    namespaced: true
  controller: true
  domain: example.com
  kind: MPSInstance
  path: github.com/reve-ai/reve-core/backend/mps-operator/v2/api/v1alpha1
  version: v1alpha1
version: "3"

Errors I'm getting:

E0718 01:23:08.235559       1 reflector.go:150] sigs.k8s.io/controller-runtime/pkg/cache/internal/informers.go:106: Failed to watch *v1alpha1.MPSInstance: failed to list *v1alpha1.MPSInstance: mpsinstances.example.com is forbidden: User "system:serviceaccount:mps-operator-system:mps-operator-controller-manager" cannot list resource "mpsinstances" in API group "example.com" at the cluster scope

Note that I'd much rather not grant clusterrole permissions to the service account if I don't have to.

[jwatte]

Maybe answering myself here: It looks like the second CRD I added wasn't added to the rbac role?
Or the second one overrode the first one?
The clusterrole that's set up is:

jwatte@Jons-MacBook-Pro mps-operator % lkctl get clusterrole mps-operator-manager-role -o yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: "2024-07-18T01:13:57Z"
  name: mps-operator-manager-role
  resourceVersion: "521137"
  uid: ec5e23dc-29ce-46eb-b8bb-37613b15fb16
rules:
- apiGroups:
  - example.com
  resources:
  - mpsversions
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - example.com
  resources:
  - mpsversions/finalizers
  verbs:
  - update
- apiGroups:
  - example.com
  resources:
  - mpsversions/status
  verbs:
  - get
  - patch
  - update

[jwatte]

I filed a bug about this -- the tool should generate all CRD names in the resources: sections of the apiGroups.

[camilamacedo86]

Hi @jwatte,

First at all, I would like to recommend you follow up the Quick Start (https://book.kubebuilder.io/quick-start) and the Getting Started (it will take 15 min) and will give to you a better idea over how things work. See: https://book.kubebuilder.io/getting-started

Note that to allow we merge ANY PR we run a lot of automated tests as we update the samples under the testdata. The Roles has been generated correctly as should be see, i.e.: https://github.com/kubernetes-sigs/kubebuilder/blob/master/testdata/project-v4/config/rbac/role.yaml . The samples under the testdata are generated via the script: https://github.com/kubernetes-sigs/kubebuilder/blob/master/test/testdata/generate.sh. On top of that, we have e2e tests which covers a lot of scenarios, see: https://github.com/kubernetes-sigs/kubebuilder/tree/master/test/e2e

After create the API you need to run make manifest:

kubebuilder/testdata/project-v4/Makefile

Lines 46 to 48 in c3a5806

	.PHONY: manifests
	manifests: controller-gen ## Generate WebhookConfiguration, ClusterRole and CustomResourceDefinition objects.
	$(CONTROLLER_GEN) rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases

After you change anything in the API you need to run make generate:

kubebuilder/testdata/project-v4/Makefile

Lines 50 to 52 in c3a5806

	.PHONY: generate
	generate: controller-gen ## Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations.
	$(CONTROLLER_GEN) object:headerFile="hack/boilerplate.go.txt" paths="./..."

Note that the required markers in this case will be scaffolded already by the tool. However, you will probably need to get, list, update other secondary resources which are required to manage the CRD (i.e To have MyApp deployed I need a service and etc). So, you will update the marker in the controller of the MyAPP, such as you can check in the controller of the Deploy Image plugin sample testadata:

https://github.com/kubernetes-sigs/kubebuilder/blob/master/testdata/project-v4-with-deploy-image/internal/controller/busybox_controller.go#L63-L68

Another suggestion to know how to do things is check the Deploy Image, by using this plugin to do the scaffold you will see that the whole code to manage an Image is scaffold.

Example:

kubebuilder init 
kubebuilder create api --group example.com --version v1alpha1 --kind Memcached --image=memcached:memcached:1.6.26-alpine3.19 --image-container-command="memcached,-m=64,-o,modern,-v" --image-container-port="11211" --run-as-user="1001" --plugins="deploy-image/v1-alpha" --make=false

So, I am closing this one as sorted out.
If you still needing help then, I would like to ask for you open an issue and add ALL commands executed by you to reproduce the scenario from a new project built with the tool so that we can help you better.