Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

TempoMonolithic: affinity and tolerations are not respected by the operator #867

Closed
skuethe opened this issue Apr 4, 2024 · 2 comments · Fixed by #880
Closed

TempoMonolithic: affinity and tolerations are not respected by the operator #867

skuethe opened this issue Apr 4, 2024 · 2 comments · Fixed by #880

Comments

@skuethe
Copy link

skuethe commented Apr 4, 2024

We are using tempo-operator v0.9.0 and having issues with the new feature added to define affinity and tolerations.
We can add them to the CR instance, but the operator does not respect the values when creating the statefulset instance.

Output from the applied CR instance named cluster:

$ k get tempomonolithics.tempo.grafana.com cluster -o yaml | yq '.spec.affinity'
nodeAffinity:
  requiredDuringSchedulingIgnoredDuringExecution:
    nodeSelectorTerms:
      - matchExpressions:
          - key: node-role.kubernetes.io/observability
            operator: Exists

$ k get tempomonolithics.tempo.grafana.com cluster -o yaml | yq '.spec.tolerations'
- effect: NoSchedule
  key: dedicated
  operator: Equal
  value: observability

And the statefulset created from the operator:

$ k get sts tempo-cluster -o yaml | yq '.spec.template.spec.affinity'
podAntiAffinity:
  preferredDuringSchedulingIgnoredDuringExecution:
    - podAffinityTerm:
        labelSelector:
          matchLabels:
            app.kubernetes.io/component: tempo
            app.kubernetes.io/instance: cluster
            app.kubernetes.io/managed-by: tempo-operator
            app.kubernetes.io/name: tempo-monolithic
        topologyKey: kubernetes.io/hostname
      weight: 100
    - podAffinityTerm:
        labelSelector:
          matchLabels:
            app.kubernetes.io/component: tempo
            app.kubernetes.io/instance: cluster
            app.kubernetes.io/managed-by: tempo-operator
            app.kubernetes.io/name: tempo-monolithic
        topologyKey: failure-domain.beta.kubernetes.io/zone
      weight: 75
$ k get sts tempo-cluster -o yaml | yq '.spec.template.spec.tolerations'
null

And just for confirmation, CRD property parsing:

$ k get crd tempomonolithics.tempo.grafana.com -o yaml | yq '.spec.versions[].schema.openAPIV3Schema.properties.spec.properties | keys'
- affinity
- extraConfig
- ingestion
- jaegerui
- management
- nodeSelector
- observability
- resources
- storage
- tolerations
@skuethe
Copy link
Author

skuethe commented Apr 4, 2024

For completeness:

$ k exec -it tempo-cluster-0 -- ./tempo --version
tempo, version 2.3.1 (branch: HEAD, revision: 9398fd620)
  build user:       
  build date:       
  go version:       go1.21.3
  platform:         linux/amd64
  tags:             unknown
$ k get pod tempo-cluster-0 -o yaml | yq '.spec.containers[].image'
docker.io/grafana/tempo:2.3.1

$ k get pod tempo-operator-controller-5f545dd674-gfqm5 -o yaml | yq '.spec.containers[].image'
ghcr.io/grafana/tempo-operator/tempo-operator:v0.9.0
gcr.io/kubebuilder/kube-rbac-proxy:v0.13.0

@andreasgerstmayr
Copy link
Collaborator

@skuethe thanks for the bug report! This issue will be fixed in #880.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
2 participants