Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

automatically clean dummy interface and iptables rule when yurthub is stopped by k8s #530

Merged
merged 1 commit into from
Oct 25, 2021
Merged

automatically clean dummy interface and iptables rule when yurthub is stopped by k8s #530

merged 1 commit into from
Oct 25, 2021

Conversation

Congrool
Copy link
Member

What type of PR is this?

/kind feature

What this PR does / why we need it:

Automatically delete dummy interface and iptables rule set by yurthub, when the yurthub pod is stoped by k8s.

Which issue(s) this PR fixes:

Fixes #526

Special notes for your reviewer:

Does this PR introduce a user-facing change?

other Note

@openyurt-bot
Copy link
Collaborator

@Congrool: GitHub didn't allow me to assign the following users: your_reviewer.

Note that only openyurtio members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

What type of PR is this?

/kind feature

What this PR does / why we need it:

Automatically delete dummy interface and iptables rule set by yurthub, when the yurthub pod is stoped by k8s.

Which issue(s) this PR fixes:

Fixes #526

Special notes for your reviewer:

Does this PR introduce a user-facing change?

other Note

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openyurt-bot openyurt-bot added the kind/feature kind/feature label Oct 23, 2021
@openyurt-bot openyurt-bot added the size/S size/S 10-29 label Oct 23, 2021
rambohe-ch
rambohe-ch reviewed Oct 25, 2021
View reviewed changes
cmd/yurthub/yurthub.go Outdated Show resolved Hide resolved
@rambohe-ch
Copy link
Member

rambohe-ch commented Oct 25, 2021 •
edited
Loading

@Congrool Would you be able to upload the detail logs of Yurthub that iptables rules and dummy interface are deleted or not?

@Congrool Congrool force-pushed the master branch 2 times, most recently from 17d58f6 to 203ef53 Compare October 25, 2021 03:43
@Congrool
Copy link
Member Author

First setup openyurt with local_up_openyurt.sh, then docker exec -it openyurt-e2e-test-worker bash to enter into the container.

Before stopping yurthub, check the current status of network interface and iptables rules:

root@openyurt-e2e-test-worker:/# ip addr | grep dummy
6: yurthub-dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default 
    inet 169.254.2.1/32 brd 169.254.2.1 scope global yurthub-dummy0

root@openyurt-e2e-test-worker:/# iptables -t raw -nvL PREROUTING
Chain PREROUTING (policy ACCEPT 248 packets, 79683 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 CT         tcp  --  *      *       0.0.0.0/0            127.0.0.1            tcp dpt:10268 NOTRACK
    0     0 CT         tcp  --  *      *       0.0.0.0/0            169.254.2.1          tcp dpt:10268 NOTRACK
  689 83081 CT         tcp  --  *      *       0.0.0.0/0            127.0.0.1            tcp dpt:10261 NOTRACK
    0     0 CT         tcp  --  *      *       0.0.0.0/0            169.254.2.1          tcp dpt:10261 NOTRACK
root@openyurt-e2e-test-worker:/# iptables -t raw -nvL OUTPUT    
Chain OUTPUT (policy ACCEPT 181 packets, 27022 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 CT         tcp  --  *      *       127.0.0.1            0.0.0.0/0            tcp spt:10268 NOTRACK
    0     0 CT         tcp  --  *      *       0.0.0.0/0            127.0.0.1            tcp dpt:10268 NOTRACK
    0     0 CT         tcp  --  *      *       169.254.2.1          0.0.0.0/0            tcp spt:10268 NOTRACK
    0     0 CT         tcp  --  *      *       0.0.0.0/0            169.254.2.1          tcp dpt:10268 NOTRACK
  648  258K CT         tcp  --  *      *       127.0.0.1            0.0.0.0/0            tcp spt:10261 NOTRACK
  707 85292 CT         tcp  --  *      *       0.0.0.0/0            127.0.0.1            tcp dpt:10261 NOTRACK
    0     0 CT         tcp  --  *      *       169.254.2.1          0.0.0.0/0            tcp spt:10261 NOTRACK
    0     0 CT         tcp  --  *      *       0.0.0.0/0            169.254.2.1          tcp dpt:10261 NOTRACK
root@openyurt-e2e-test-worker:/# iptables -t filter -nvL INPUT 
Chain INPUT (policy ACCEPT 1884 packets, 929K bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            127.0.0.1            tcp dpt:10268
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            169.254.2.1          /* for container access hub agent */ tcp dpt:10268
  742 89631 ACCEPT     tcp  --  *      *       0.0.0.0/0            127.0.0.1            tcp dpt:10261
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            169.254.2.1          /* for container access hub agent */ tcp dpt:10261
  433 27386 KUBE-EXTERNAL-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW /* kubernetes externally-visible service portals */
 8418 3858K KUBE-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Remove the yurt-hub.yaml from /etc/kubernetes/manifest to stop yurthub. Then we check out the dummy network interface:

root@openyurt-e2e-test-worker:/# mv /etc/kubernetes/manifests/yurt-hub.yaml ./
root@openyurt-e2e-test-worker:/# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: vetha60114ad@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether c6:7d:bd:61:9e:63 brd ff:ff:ff:ff:ff:ff link-netns cni-51e5c3d4-8d8d-0ada-cd08-1ce5796d0971
    inet 10.244.1.1/32 scope global vetha60114ad
       valid_lft forever preferred_lft forever
3: vetha0546b8c@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether aa:19:1f:73:e5:82 brd ff:ff:ff:ff:ff:ff link-netns cni-9b8405a6-5c51-96b3-3539-4041fd853df6
    inet 10.244.1.1/32 scope global vetha0546b8c
       valid_lft forever preferred_lft forever
4: veth80c74d2b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 3a:f1:31:e6:a8:f2 brd ff:ff:ff:ff:ff:ff link-netns cni-a65cc9df-196b-04cb-64f9-4fc0e34afbd5
    inet 10.244.1.1/32 scope global veth80c74d2b
       valid_lft forever preferred_lft forever
27: eth0@if28: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:ac:13:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.19.0.2/16 brd 172.19.255.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fc00:f853:ccd:e793::2/64 scope global nodad 
       valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe13:2/64 scope link 
       valid_lft forever preferred_lft forever

We can find that yurthub-dummy0 interface has been successfully deleted.

Now, check iptables rule:

root@openyurt-e2e-test-worker:/# iptables -t raw -nvL PREROUTING
Chain PREROUTING (policy ACCEPT 2167 packets, 571K bytes)
 pkts bytes target     prot opt in     out     source               destination         
root@openyurt-e2e-test-worker:/# iptables -t raw -nvL OUTPUT    
Chain OUTPUT (policy ACCEPT 1039 packets, 83591 bytes)
 pkts bytes target     prot opt in     out     source               destination         
root@openyurt-e2e-test-worker:/# iptables -t filter -nvL INPUT 
Chain INPUT (policy ACCEPT 1100 packets, 295K bytes)
 pkts bytes target     prot opt in     out     source               destination         
  321 20598 KUBE-EXTERNAL-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW /* kubernetes externally-visible service portals */
 6025 2744K KUBE-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0

We can find that the iptables rules set at iptables.go also be deleted.

@Congrool
Copy link
Member Author

@rambohe-ch hi, the log has been uploaded.

@rambohe-ch
Copy link
Member

/lgtm
/approve

@openyurt-bot openyurt-bot added the lgtm lgtm label Oct 25, 2021
@openyurt-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Congrool, rambohe-ch

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openyurt-bot openyurt-bot added the approved approved label Oct 25, 2021
@openyurt-bot openyurt-bot merged commit ad08d1c into openyurtio:master Oct 25, 2021
@Congrool Congrool mentioned this pull request Jan 26, 2022
Merged
6 tasks
MrGirl pushed a commit to MrGirl/openyurt that referenced this pull request Mar 29, 2022
@Congrool
automatically clean dummy interface and iptables rule when yurthub is…
3793e0a 
… stopped by k8s (openyurtio#530)
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@rambohe-ch rambohe-ch rambohe-ch approved these changes

@huangyuqi huangyuqi Awaiting requested review from huangyuqi

@Peeknut Peeknut Awaiting requested review from Peeknut

Assignees

@rambohe-ch rambohe-ch

Labels
approved approved kind/feature kind/feature lgtm lgtm size/S size/S 10-29
Projects
None yet
Milestone
No milestone
3 participants
@Congrool @openyurt-bot @rambohe-ch