Does loxilb support SR-IOV based Virtual Function?

[ianchen0119]

Hello,

I have deployed the free5GC on the Kubernetes.
The AMF Pod's interface is binding on the virtual function (created by SR-IOV) directly.
All of deployment is very similar to the picture below:

Ref: https://futuredon.medium.com/5g-sctp-loadbalancer-using-loxilb-b525198a9103

When I trying to establish the sctp connection from local to the loxilb, it seems like:

• the virtual function interface has received the sctp init from local
• but these message did not get forwarded to the endpoints (configured by using loxicmd)

If has anyone knows the possible cause, please let me know.

Thanks.

[nik-netlox]

Hi @ianchen0119, Can you provide some additional information?

1. uname -r

2. lsb_release -d

3. loxicmd get lb -o wide

4. loxicmd get ct

5. ip route

6. ip addr

7. ip neigh

[ianchen0119]

Hi @nik-netlox
Thanks for your reply.
The information from loxilb's pod is below:

1. 5.19.0-41-generic
2. Ubuntu 22.04.2 LTS
3. lb info:

| EXTERNAL IP | PORT  | PROTOCOL | BLOCK | SELECT |  MODE  | ENDPOINT IP  | TARGET PORT | WEIGHT | STATE  |
|-------------|-------|----------|-------|--------|--------|--------------|-------------|--------|--------|
| 20.20.20.1  | 38412 | sctp     |     0 | rr     | onearm | 192.168.1.10 |       38412 |      1 | active |
|             |       |          |       |        |        | 192.168.1.11 |       38412 |      1 | active |

4. ct info

| DESTINATIONIP | SOURCEIP | DESTINATIONPORT | SOURCEPORT | PROTOCOL | STATE | ACT | PACKETS | BYTES |
|---------------|----------|-----------------|------------|----------|-------|-----|---------|-------|

5. route info

default via 192.168.1.254 dev net1 
default via 169.254.1.1 dev eth0 
169.254.1.1 dev eth0 scope link 
192.168.1.0/24 dev net1 proto kernel scope link src 192.168.1.18 

6. addr info

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
3: eth0@if116: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1440 qdisc noqueue state UP group default qlen 1000
    link/ether fa:ab:6e:ff:57:23 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.1.168.1/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::f8ab:6eff:feff:5723/64 scope link 
       valid_lft forever preferred_lft forever
4: llb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdpgeneric/id:1566 qdisc fq_codel state UNKNOWN group default qlen 1000
    link/ether 02:e1:06:da:5e:5d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::e1:6ff:feda:5e5d/64 scope link 
       valid_lft forever preferred_lft forever
18: net1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether e6:4e:dc:7b:14:ba brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.18/24 brd 192.168.1.255 scope global net1
       valid_lft forever preferred_lft forever
    inet6 fe80::e44e:dcff:fe7b:14ba/64 scope link 
       valid_lft forever preferred_lft forever

• net1: virtual function created by SR-IOV

7. neigh info

192.168.1.10 dev net1 lladdr 5a:b0:ee:e2:47:6b DELAY
10.10.2.58 dev eth0 lladdr ee:ee:ee:ee:ee:ee STALE
192.168.1.254 dev net1  FAILED
192.168.1.11 dev net1 lladdr 92:bf:a9:c9:45:9f STALE
192.168.1.15 dev net1 lladdr 00:0a:cd:3a:ff:76 STALE
169.254.1.1 dev eth0 lladdr ee:ee:ee:ee:ee:ee STALE

[ianchen0119]

Hi All,

This issue has been solved.
Thanks the assistance from @nik-netlox

The root cause of this problem is the independent issue due to kernel.
The commands below can solve this kind of problem if anyone faces the same issue with me:

1. docker run --privileged ghcr.io/loxilb-io/loxilb
2. replace the dock argument specified in the Makefile with the container id of the container you just created
3. make docker-cp-ebpf
4. docker commit <container id> ghcr.io/loxilb-io/loxilb and retry again