Skip to content
This repository has been archived by the owner on Oct 10, 2023. It is now read-only.

Set node-ip in /etc/hosts instead of node-ip flag #4323

Merged

Conversation

christianang
Copy link
Contributor

What this PR does / why we need it

Setting the --node-ip on the kubelet can prevent the vsphere-cpi from configuring the node ip itself if its ip selection process detects a different IP for some reason. One occurance of this happening was the kubelet selecting the slaac address and the vsphere-cpi selecting the dhcpv6 address. This usually shouldn't occur because they both should normally select the first global ip on the nic, but timing may have caused kubelet to not select the dhcpv6 address.

When running with kube-vip, configuing the node ip in kubelet is desireable so that pods on the hostNetwork will not run on the kube-vip IP when the kubelet selects an IP for the node.

Kubelet can be informed of what IP to use by DNS lookup of the hostname instead of using --node-ip. This approach is advantageous because if the CPI were to choose a different IP, kubelet would not reject the IP. Setting a non floating kube-vip IP in the /etc/hosts file gives the desired DNS answer, allowing kubelet to launch pods on the host network, while also avoiding the kube-vip IP.

Which issue(s) this PR fixes

Fixes n/a

Describe testing done for PR

We deployed an ipv6 cluster with kube-vip and everything worked successfully.
We also forced kubelet to select an IP address that vsphere-cpi wouldn't select and see that vsphere-cpi is able to update the node addresses to a different ip successfully.

Release note

Fix issue where vsphere-cpi cannot set the node address after kubelet is configured with a different address on IPv6 clusters.

Additional information

Special notes for your reviewer

Setting the --node-ip on the kubelet can prevent the vsphere-cpi from
configuring the node ip itself if its ip selection process detects a
different IP for some reason. One occurance of this happening was the
kubelet selecting the slaac address and the vsphere-cpi selecting the
dhcpv6 address. This usually shouldn't occur because they both should
normally select the first global ip on the nic, but timing may have
caused kubelet to not select the dhcpv6 address.

When running with kube-vip, configuing the node ip in kubelet is
desireable so that pods on the hostNetwork will not run on the kube-vip
IP when the kubelet selects an IP for the node.

Kubelet can be informed of what IP to use by DNS lookup of the hostname
instead of using --node-ip. This approach is advantageous because if the
CPI were to choose a different IP, kubelet would not reject the IP.
Setting a non floating kube-vip IP in the /etc/hosts file gives the
desired DNS answer, allowing kubelet to launch pods on the host network,
while also avoiding the kube-vip IP.

Co-authored-by: Tyler Schultz <tschultz@vmware.com>
@github-actions
Copy link

github-actions bot commented Feb 2, 2023

Cluster Generation A/B Results:
https://storage.googleapis.com/tkg-clustergen/4323/20230202224049/clustergen.diff.txt
Author/reviewers:
Please review to verify that the effects on the generated cluster configurations are exactly what the PR intended, and give a thumbs-up if so.

@codecov
Copy link

codecov bot commented Feb 3, 2023

Codecov Report

Merging #4323 (bfbcc04) into main (eeda69d) will decrease coverage by 0.84%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #4323      +/-   ##
==========================================
- Coverage   49.53%   48.69%   -0.84%     
==========================================
  Files         452      482      +30     
  Lines       44849    46969    +2120     
==========================================
+ Hits        22214    22871     +657     
- Misses      20521    21941    +1420     
- Partials     2114     2157      +43     
Impacted Files Coverage Δ
cmd/cli/plugin/cluster/delete.go 12.50% <0.00%> (ø)
cmd/cli/plugin/isolated-cluster/main.go 0.00% <0.00%> (ø)
...olated-cluster/imagepushop/publishimagesfromtar.go 73.17% <0.00%> (ø)
...md/cli/plugin/cluster/delete_machinehealthcheck.go 19.23% <0.00%> (ø)
.../cli/plugin/cluster/get_machinehealthcheck_node.go 9.30% <0.00%> (ø)
cmd/cli/plugin/cluster/list.go 11.36% <0.00%> (ø)
cmd/cli/plugin/cluster/delete_node_pool.go 16.66% <0.00%> (ø)
cmd/cli/plugin/isolated-cluster/test/main.go 0.00% <0.00%> (ø)
cmd/cli/plugin/cluster/get_machinehealthcheck.go 11.42% <0.00%> (ø)
cmd/cli/plugin/cluster/get.go 6.27% <0.00%> (ø)
... and 26 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@christianang christianang added the ok-to-merge PRs should be labelled with this before merging label Feb 6, 2023
@christianang christianang merged commit 5567ab4 into vmware-tanzu:main Feb 6, 2023
@christianang christianang deleted the kubelet-node-ip-in-hosts-file branch February 6, 2023 22:15
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
cla-not-required ok-to-merge PRs should be labelled with this before merging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants