[Release-1.32] - rotate-ca doesn't work if server is set in config when rotating

[brandond]

Backport fix for rotate-ca doesn't work if server is set in config when rotating

• rotate-ca doesn't work if server is set in config when rotating #11233

[fmoral2]

Master

Validated on Version:

-$ k3s version v1.32.2+k3s-26e3fa97 (26e3fa97)

 

Environment Details

Infrastructure
Cloud EC2 instance

Node(s) CPU architecture, OS, and Version:
sle 15
AMD

Cluster Configuration:
-3 node server

Steps to validate the fix

1. Install 3 server nodes k3s
2. Delete and uninstall k3s on primary node
3. Rejoin primary node to cluster
4. Pull and generate cross-signed certificates
5. Verify dates on certificates
6. Restart k3s on all nodes
7. Verify cluster is healthy
8. Verify that CA certs in /var/lib/rancher/k3s/server/tls is replaced with new ca certs

Reproduction Issue:

  
 On node 2:
  kubectl delete node node1

On node 1:
  ./k3s-uninstall.sh

On node 1:
curl -sfL https://get.k3s.io | INSTALL_K3S_COMMIT=x K3S_TOKEN=x sh -s - server --server https://x:6443

curl -sL https://github.com/k3s-io/k3s/raw/master/contrib/util/rotate-default-ca-certs.sh | bash -

k3s certificate rotate-ca --path=/var/lib/rancher/k3s/server/rotate-ca

ls -al /var/lib/rancher/k3s/server/tls
      -rw------- 1 root root  566 Mar 11 14:00 client-ca.crt
      -rw------- 1 root root  227 Mar 11 14:00 client-ca.key   


systemctl restart k3s

 ls -al /var/lib/rancher/k3s/server/tls 
      -rw------- 1 root root  566 Mar 11 14:00 client-ca.crt
      -rw------- 1 root root  227 Mar 11 14:00 client-ca.key

Validation Results:

  On node 2:
  kubectl delete node node1

On node 1:
  ./k3s-uninstall.sh

On node 1:
curl -sfL https://get.k3s.io | INSTALL_K3S_COMMIT=x K3S_TOKEN=x sh -s - server --server https://x:6443

curl -sL https://github.com/k3s-io/k3s/raw/master/contrib/util/rotate-default-ca-certs.sh | bash -

k3s certificate rotate-ca --path=/var/lib/rancher/k3s/server/rotate-ca


ls -al /var/lib/rancher/k3s/server/tls
      -rw------- 1 root root  570 Mar 11 14:00 client-ca.crt
      -rw------- 1 root root  227 Mar 11 14:00 client-ca.key   


systemctl restart k3s

 ls -al /var/lib/rancher/k3s/server/tls 
      -rw------- 1 root root  570 Mar 11 14:27 client-ca.crt
      -rw------- 1 root root  227 Mar 11 14:27 client-ca.key




 kubectl get pods -A
NAMESPACE     NAME                                      READY   STATUS    RESTARTS   AGE
kube-system   coredns-ff8999cc5-fbhpb                   1/1     Running   0          11m
kube-system   local-path-provisioner-774c6665dc-cv48m   1/1     Running   0          11m
kube-system   metrics-server-6f4c6675d5-gclx7           1/1     Running   0          11m
kube-system   svclb-traefik-cd98fbef-hmmj6              2/2     Running   0          11m
kube-system   svclb-traefik-cd98fbef-v76lj              2/2     Running   0          33m
kube-system   svclb-traefik-cd98fbef-xcr46              2/2     Running   0          33m
kube-system   traefik-67bfb46dcb-srp5t                  1/1     Running   0          11m


 kubectl get nodes 
NAME                                          STATUS   ROLES                       AGE   VERSION
ip-x.us-east-2.compute.internal     Ready    control-plane,etcd,master   33m   v1.32.2+k3s-7034b96c
ip-x                                Ready    control-plane,etcd,master   11m   v1.32.2+k3s-26e3fa97
ip-x.us-east-2.compute.internal     Ready    control-plane,etcd,master   33m   v1.32.2+k3s-7034b96c