"Cleaning up stale CnsVolumeOperationRequest instances" ignores block-volume-snapshot:false feature states

[sathieu]

Is this a BUG REPORT or FEATURE REQUEST?:

/kind bug

What happened:

We have block-volume-snapshot:false as confirmed by the log:

 {
  "level": "info",
  "time": "2024-12-20T09:03:27.684711955Z",
  "caller": "k8sorchestrator/k8sorchestrator.go:440",
  "msg": "New internal feature states values stored successfully: map[block-volume-snapshot:false csi-migration:false csi-windows-support:false improved-volume-topology:false multi-vcenter-csi-topology:false pv-to-backingdiskobjectid-mapping:false topology-preferential-datastores:false trigger-csi-fullsync:false]",
  "TraceId": "7fdf059a-172b-4b44-a4da-584fbbf0406e"
}

But this evaluates to true:

vsphere-csi-driver/pkg/internalapis/cnsvolumeoperationrequest/cnsvolumeoperationrequest.go

Line 399 in 75c1ffc

if blockVolumeSnapshotEnabled {

We know this because we don't have VolumeSnapshotContents CRD and the following fails:

vsphere-csi-driver/pkg/internalapis/cnsvolumeoperationrequest/cnsvolumeoperationrequest.go

Lines 409 to 414 in 75c1ffc

	vscList, err := snapshotterClient.SnapshotV1().VolumeSnapshotContents().List(ctx, metav1.ListOptions{})
	if err != nil {
	log.Errorf("failed to list VolumeSnapshotContents with error %v. Abandoning "+
	"CnsVolumeOperationRequests clean up ...", err)
	return
	}

What you expected to happen:

The snapshot code should not run when feature is disabled.

How to reproduce it (as minimally and precisely as possible):

Using vanilla, block-volume-snapshot:false and no VolumeSnapshotContents CRD.

Maybe there is some concurency problem because some parts are in different goroutines.

Anything else we need to know?:

Environment:

• csi-vsphere version: 3.3.1 vanilla
• vsphere-cloud-controller-manager version: N/1
• Kubernetes version: 1.30.3
• vSphere version: 8.0
• OS (e.g. from /etc/os-release): Debian
• Kernel (e.g. uname -a):
• Install tools:
• Others: