You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Datastore is etcd and calico is deployed on kubernetes
There are node1、node2、node3,after delete node2
calico-kube-controllers not sync k8s node
calico-kube-controllers recognized that the node has been removed. Cleaning up IPAM resources for deleted node node="node-223-174-vip-176
but node information still in etcd /calico/resources/v3/projectcalico.org/nodes/node-223-174-vip-176
k8s node not found, because node-223-174-vip-176 deleted
NAME STATUS ROLES AGE VERSION
node-223-163-vip-176 Ready <none> 46h v1.21.14
node-223-173-vip-176 Ready control-plane,master 46h v1.21.14
node-223-175-vip-176 NotReady control-plane,master 46h v1.21.14
pod status service-software seasqlcache-cluster-1 0/1 Pending
set calico-kube-controllers LOG_LEVEL debug, found such code bug, if pod status pending no pod ip, calico node information still in etcd even k8s node delete, forever not delete node information in etcd
kube-controllers/pkg/controllers/node/ipam.go
if c.allocationIsValid(a, true) {
// Allocation is still valid. We can't cleanup the node yet, even
// if it appears to be deleted, because the allocation's validity breaks
// our confidence.
canDelete = false
a.markValid()
continue
}
if p.Status.PodIP == "" || len(p.Status.PodIPs) == 0 {
// The pod hasn't received an IP yet.
log.Debugf("Pod IP has not yet been reported, consider allocation valid")
return true
}
if !kubernetesNodeExists {
if !canDelete {
// There are still valid allocations on the node.
logc.Infof("Can't cleanup node yet - IPs still in use on this node")
continue
}
Debug log such as:
Failed to release block affinities for node calicoNode="node-223-174-vip-176" error=block '177.177.73.64/26' is not empty
Error cleaning up node error=block '177.177.73.64/26' is not empty node="node-223-174-vip-176"
Periodic IPAM sync failed error=block '177.177.73.64/26' is not empty
Checking cache for pod handle="k8s-pod-network.5b1d21c9a440ab14388386a11d41a848e231c9fa01216c87f1de5885d424b1fc" ip="177.177.73.70" node="node-223-174-vip-176" pod="service-software/seasqlcache-slave-0-0"
Pod IP has not yet been reported, consider allocation valid
calico block '177.177.73.64/26' in etcd, but k8s node-223-174-vip-176 deleted
Datastore is etcd and calico is deployed on kubernetes
There are node1、node2、node3,after delete node2
calico-kube-controllers not sync k8s node
calico-kube-controllers recognized that the node has been removed.
Cleaning up IPAM resources for deleted node node="node-223-174-vip-176
but node information still in etcd
/calico/resources/v3/projectcalico.org/nodes/node-223-174-vip-176
k8s node not found, because node-223-174-vip-176 deleted
pod status
service-software seasqlcache-cluster-1 0/1 Pending
set calico-kube-controllers LOG_LEVEL debug, found such code bug, if pod status pending no pod ip, calico node information still in etcd even k8s node delete, forever not delete node information in etcd
kube-controllers/pkg/controllers/node/ipam.go
Debug log such as:
calico block '177.177.73.64/26' in etcd, but k8s node-223-174-vip-176 deleted
Expected Behavior
delete k8s node1
calico-kube-controllers delete etcd node information
Current Behavior
delete k8s node1
calico-kube-controllers not delete etcd node information
Possible Solution
Steps to Reproduce (for bugs)
Context
Your Environment
The text was updated successfully, but these errors were encountered: