tcp-services configMap add one configuration ， Other services that are accessed through tcp cannot be accessed properly #9943

wcsceshi · 2023-05-10T09:43:23Z

kubectl version 1.23.6, nginx-ingress-controller image version v1.3.0

What happened:
I added a piece of data to the tcp-services configMap，The ipv6 dual-stack nginx service is enabled，then ran the curl nodeIp:port command to get the normal service return data.
However, when I added another piece of data to the configMap of tcp-services（ipv6 dual stack is also enabled for nginx services）, I looped through shell.sh to access the previously added services and got an abnormal return，It took about 30 seconds to get the normal service result.

What you expected to happen:
When I add configuration data to the configMap of tcp-services, I do not affect access to other services that are also configured through the configMap of tcp-services

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.):

NGINX Ingress controller
Release: v1.3.0
Build: 2b7b748
Repository: https://github.com/kubernetes/ingress-nginx
nginx version: nginx/1.19.10

Kubernetes version (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.6", GitCommit:"ad3338546da947756e8a88aa6822e9c11e7eac22", GitTreeState:"clean", BuildDate:"2022-04-14T08:49:13Z", GoVersion:"go1.17.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.6", GitCommit:"ad3338546da947756e8a88aa6822e9c11e7eac22", GitTreeState:"clean", BuildDate:"2022-04-14T08:43:11Z", GoVersion:"go1.17.9", Compiler:"gc", Platform:"linux/amd64"}

Environment:
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Linux 10.19.38.223 4.19.12-1.el7.elrepo.x86_64 #1 SMP Fri Dec 21 11:06:36 EST 2018 x86_64 x86_64 x86_64 GNU/Linux

How was the ingress-nginx-controller installed:
- If helm was used then please show output of helm ls -A | grep -i ingress
- If helm was used then please show output of helm -n <ingresscontrollernamepspace> get values <helmreleasename>
- If helm was not used, then copy/paste the complete precise command used to install the controller, along with the flags and options used
- if you have more than one instance of the ingress-nginx-controller installed in the same cluster, please provide details for all the instances

[root@10 sh]# kubectl describe ingressclasses
Name: nginx
Labels: app.kubernetes.io/component=controller
app.kubernetes.io/instance=ingress-nginx
app.kubernetes.io/name=ingress-nginx
app.kubernetes.io/part-of=ingress-nginx
app.kubernetes.io/version=1.3.0
Annotations:
Controller: k8s.io/ingress-nginx
Events:
[root@10 sh]#
[root@10 sh]# kubectl get po -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-629dv 0/1 Completed 0 62d
ingress-nginx-admission-patch-4rlht 0/1 Completed 0 62d
ingress-nginx-controller-4bfc6 1/1 Running 1 (43d ago) 50d
ingress-nginx-controller-cghx9 1/1 Running 1 (43d ago) 43d

shell script content：
#!/bin/bash
for I in {1..50}; do
curl -v http://x.x.x.x:31687
let SUM=$I
echo "$SUM"
sleep 1
done

The text was updated successfully, but these errors were encountered:

k8s-ci-robot · 2023-05-10T09:43:32Z

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

longwuyuan · 2023-05-10T09:47:24Z

/remove-kind bug

Can you please write a step-by-step instruction set here, that anyone can use to copy/paste from and re-create the problem in a minikube or kind cluster.

wcsceshi · 2023-05-10T11:37:53Z

I created the service and pod through yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx-2
labels:
app.kubernetes.io/name: proxy-2
spec:
containers:

name: nginx
image: nginx:stable
ports:
- containerPort: 80
  name: http-web-svc

apiVersion: v1
kind: Service
metadata:
name: nginx-service-2
spec:
ipFamilies:

IPv6
IPv4
ipFamilyPolicy: PreferDualStack
selector:
app.kubernetes.io/name: proxy-2
ports:
name: name-of-service-port
protocol: TCP
port: 80
targetPort: http-web-svc

configured the host port and service map in the configMap of tcp-services. I accessed the host and host port through curl command and got the correct response

curl nodeIp:31626

Then I create a service and pod with yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx-3
labels:
app.kubernetes.io/name: proxy-3
spec:
containers:

name: nginx
image: nginx:stable
ports:
- containerPort: 80
  name: http-web-svc

apiVersion: v1
kind: Service
metadata:
name: nginx-service-3
spec:
ipFamilies:

IPv6
IPv4
ipFamilyPolicy: PreferDualStack
selector:
app.kubernetes.io/name: proxy-3
ports:
name: name-of-service-port
protocol: TCP
port: 80
targetPort: http-web-svc

Using a shell script, I use the for loop to curl the service I configured in tcp-services' configMap.
#!/bin/bash
for I in {1..50}; do
curl -v http://nodeIp:31626
let SUM=$I
echo "$SUM"
sleep 1
done
During shell script execution, the newly created service is also configured in the configMap of tcp-services. As a result, the curl command in the shell script returns some incorrect result. After a while, the curl command returns some normal result

Obtain correct results first, and then the result is not normal

and then get the right answer

longwuyuan · 2023-05-10T12:22:29Z

@wcsceshi I don't understand the problem you have described. Let us wait if someone else can understand and comment on this issue.

maybe you should not use nginx:stable image because nginx is commonly used for HTTP
maybe you can use mysql as one service to access over TCP
maybe you can use postgres as second service to access over TCP

wcsceshi · 2023-05-10T12:58:56Z

Sorry, maybe I didn't make myself clear.

What I mean by this is that if someone modifs tcp-services configMap, adds a configuration (e.g. 32100: default/app-svc:80) and saves it while I am accessing the service through tcp, I will only get an error result. like "connection reset by peer"。

I use tcp to access the IP address of the node where the ingress-controller pod resides and the port configured by the tcp -services configMap to access app service

longwuyuan · 2023-05-10T13:16:47Z

I still don't understand. I don't want to guess what problem you want the project to solve. So lets wait for some expert comments.

But if you referring to dropping established connections during config reload, then that is ow nginx works. The project maintainers are working on splitting control-plane and data-plane and that may improve the experience.

longwuyuan · 2023-05-21T14:29:18Z

@ghostloda I am not a developer but since you submitted a PR, I was wondering if it is possible for you and @wcsceshi to chat on a zoom session to triage this issue. There are multiple aspects about the description of this issue and the data that is posted in this issue, that can be discussed on chat like arrangement, to make progress.

longwuyuan · 2023-05-21T14:29:53Z

I mean a zoom or slack chat with me

ghostloda · 2023-05-22T02:31:55Z

@ghostloda I am not a developer but since you submitted a PR, I was wondering if it is possible for you and @wcsceshi to chat on a zoom session to triage this issue. There are multiple aspects about the description of this issue and the data that is posted in this issue, that can be discussed on chat like arrangement, to make progress.

Good idea, i agree @longwuyuan

ghostloda · 2023-05-22T03:24:43Z

Hi,@wcsceshi @longwuyuan zoom chat

ghostloda · 2023-05-24T10:53:43Z

Steps to Reproduce the Issue:

Using tcp-controller-configmap for L4 Load Balancing
IPv6 Address endpoint
Delete one from tcp-controller-configmap

While deleting a service from the configmap, Lua will resynchronize the backend services. During this process, there may be an error the ipv6 address is not formatted, resulting in the unavailability of other services as well.

@longwuyuan @wcsceshi

ghostloda · 2023-05-24T12:13:55Z

The tcp-controller-configmap used for testing is as follows:

longwuyuan · 2023-05-24T17:34:36Z

@ghostloda why do you configure nginx webserver for TCP access ? Whole world uses nginx webserver for HTTP/HTTPS, not TCP.

ghostloda · 2023-05-25T01:44:35Z

@ghostloda why do you configure nginx webserver for TCP access ? Whole world uses nginx webserver for HTTP/HTTPS, not TCP.

Just as a backend service for testing.This is not the main focal point of the issue.

I think this issue is critical,when a TCP service(only ipv6) is deleting(or edit), it may affects the normal access of other services.We should promptly investigate and confirm this issue.

Based on the debug logs, the error is quite evident.

ghostloda · 2023-05-25T02:02:45Z

Hi,@longwuyuan I have already chat with @wcsceshi , and this bug was discovered in their production environment. This PR will fix the issue, and they have already tested it in their production environment.
Perhaps we should consider inviting more experienced contributors to help us verify this issue?

tao12345666333 · 2023-05-25T02:42:31Z

After a quick glance, I think reproducing this issue should be relatively simple.

@wcsceshi could you please format your examples? #9943 (comment)

The complete and directly usable reproduction steps will save everyone's time.

longwuyuan · 2023-05-25T03:02:54Z

@wcsceshi & @ghostloda , why don't you try mysql as one tcp service and postgresql as second tcp service. Then keep mysql and psql clients open for test.

Also, how about you disable ipv6 in the cluster.

Thirdly, any chance you can write more details about what is the app that you are using behind TCP ports.

Maybe we will discover more than just a unformatted ipv6 address.

Nginx reconciling during nginx.conf reload will cause some problems for sure. Maybe we will also find out why only you are reporting this problem and other users of TCP services are not reporting this problem.

wcsceshi · 2023-05-25T03:11:18Z

@wcsceshi & @ghostloda , why don't you try mysql as one tcp service and postgresql as second tcp service. Then keep mysql and psql clients open for test.

Also, how about you disable ipv6 in the cluster.

Thirdly, any chance you can write more details about what is the app that you are using behind TCP ports.

Maybe we will discover more than just a unformatted ipv6 address.

Nginx reconciling during nginx.conf reload will cause some problems for sure. Maybe we will also find out why only you are reporting this problem and other users of TCP services are not reporting this problem.

I've seen others discover similar issues #9307

longwuyuan · 2023-05-25T05:13:11Z

That one also is using nginx webserver in example.
I think this issue should be titled specific to ipv6. I will test and update.

wcsceshi · 2023-06-01T06:22:29Z

I re-used two pod of mysql and the corresponding service with ipv6 enabled for testing. The testing procedure and test results are as follows

step1. i create two deployment and service （with ipv6）, the mirror is MySQL ， below is the resource object

[root@10 ingress-ipv6-svc]# kubectl get all |grep mysql
pod/ssx-mysql-dm-2-d8977d7cd-gs566        1/1     Running            0                 2m3s
pod/ssx-mysql-dm-789c4dd484-rc9dv         1/1     Running            2 (4d23h ago)     4d23h
service/ssx-mysql-sv      NodePort    2001:db8:40:1::acc7   <none>        9000:31090/TCP   4d23h
service/ssx-mysql-sv-2    NodePort    2001:db8:40:1::ac8f   <none>        9001:31091/TCP   5h3m
deployment.apps/ssx-mysql-dm         1/1     1            1           4d23h
deployment.apps/ssx-mysql-dm-2       1/1     1            1           5h3m
replicaset.apps/ssx-mysql-dm-2-d8977d7cd        1         1         1       5h3m
replicaset.apps/ssx-mysql-dm-789c4dd484         1         1         1       4d23h

step2. add two pieces of data to the configMap of tcp-services

[root@10 ingress-ipv6-svc]# kubectl get cm -n ingress-nginx tcp-services -o yaml
apiVersion: v1
data:
  "9090": monitoring/prometheus-k8s:9090
  "32001": default/ssx-mysql-sv:9000
  "32002": default/ssx-mysql-sv-2:9001
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"8888":"a/b:9090","9090":"monitoring/prometheus-k8s:9090"},"kind":"ConfigMap","metadata":{"annotations":{},"creationTimestamp":"2023-03-08T10:01:30Z","labels":{"app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/part-of":"ingress-nginx"},"name":"tcp-services","namespace":"ingress-nginx","resourceVersion":"48942888","selfLink":"/api/v1/namespaces/ingress-nginx/configmaps/tcp-services","uid":"106b64b3-f596-4e2c-8014-56c3ac43fa02"}}
  creationTimestamp: "2023-03-08T10:01:30Z"
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  name: tcp-services
  namespace: ingress-nginx
  resourceVersion: "72771677"
  selfLink: /api/v1/namespaces/ingress-nginx/configmaps/tcp-services
  uid: 106b64b3-f596-4e2c-8014-56c3ac43fa02
[root@10 ingress-ipv6-svc]#

step3. use IP of the node where the pod of nginx ingress controller is located，with the configured tcp host port，to access the created service . I was able to log into MySQL successfully and log out safely

[root@10 ingress-ipv6-svc]# mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 737
Server version: 5.7.20 MySQL Community Server (GPL)
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [sys]> exit
Bye
[root@10 ingress-ipv6-svc]#

step4. I tried to log in and out of mysql multiple times using a shell script, using the host ip and port，And execute
./test-1-6-mysql-demo.sh .the script content and execution result are as follows

[root@10 ingress-ipv6-svc]# cat test-1-6-mysql-demo.sh
#!/usr/bin/expect
 for {set i 0} {$i < 10} {incr i} {
        puts " $i"
        spawn  mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
        expect "MySQL"
        send "exit"
        sleep 2
    }

[root@10 ingress-ipv6-svc]# ./test-1-6-mysql-demo.sh
 0
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 872
Server version: 5.7.20 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [sys]>  1
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 873
Server version: 5.7.20 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

 2
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 874
Server version: 5.7.20 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

 3
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 875
Server version: 5.7.20 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [sys]>  4
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 876
Server version: 5.7.20 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

 5
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 877
Server version: 5.7.20 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [sys]>  6
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 878
Server version: 5.7.20 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

 7
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 879
Server version: 5.7.20 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

 8
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 880
Server version: 5.7.20 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

 9
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 881
Server version: 5.7.20 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

[root@10 ingress-ipv6-svc]#

step5. I tried to delete another mysql pod created during the execution of the shell script ,and found many errors in the shell script execution results

[root@10 ingress-ipv6-svc]# ./test-1-6-mysql-demo.sh
 0
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 882
Server version: 5.7.20 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [sys]>  1
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 883
Server version: 5.7.20 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [sys]>  2
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
 3
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
 4
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
 5
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
 6
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
 7
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
 8
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
 9
spawn mysql -h fd69:1:10:f980::a13:26e0 -P32001 -uroot -p1 sys
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
[root@10 ingress-ipv6-svc]#

Viewing the ingress-nginx pod also has a lot of error messages

W0601 03:05:56.828406       7 controller.go:456] Service "default/ssx-mysql-sv-2" does not have any active Endpoint for TCP port 9001
W0601 03:05:56.828580       7 controller.go:423] Error getting Service "foot/nginx-ns": no object matching key "foot/nginx-ns" in local store
I0601 03:05:56.829402       7 controller.go:167] "Configuration changes detected, backend reload required"
I0601 03:05:56.987781       7 controller.go:184] "Backend successfully reloaded"
I0601 03:05:56.989090       7 event.go:285] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-48nw7", UID:"69febc3b-9dc7-4523-9fd0-9012b6a3ee6f", APIVersion:"v1", ResourceVersion:"72961152", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
[fd69:1:10:f980::a13:26df] [01/Jun/2023:03:05:57 +0000] TCP 502 0 0 0.000
2023/06/01 03:05:57 [error] 712#712: *25823 stream [lua] tcp_udp_balancer.lua:196: balance(): error while setting current upstream peer 2001:db8:42:83:318a:6059:3e82:d1d6:3306: invalid port while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, bytes from/to client:0/0, bytes from/to upstream:0/0
2023/06/01 03:05:57 [crit] 712#712: *25823 connect() to 0.0.0.1:1234 failed (22: Invalid argument) while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, upstream: "0.0.0.1:1234", bytes from/to client:0/0, bytes from/to upstream:0/0
[fd69:1:10:f980::a13:26df] [01/Jun/2023:03:05:59 +0000] TCP 502 0 0 0.000
2023/06/01 03:05:59 [error] 712#712: *25859 stream [lua] tcp_udp_balancer.lua:196: balance(): error while setting current upstream peer 2001:db8:42:83:318a:6059:3e82:d1d6:3306: invalid port while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, bytes from/to client:0/0, bytes from/to upstream:0/0
2023/06/01 03:05:59 [crit] 712#712: *25859 connect() to 0.0.0.1:1234 failed (22: Invalid argument) while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, upstream: "0.0.0.1:1234", bytes from/to client:0/0, bytes from/to upstream:0/0
W0601 03:06:00.162047       7 controller.go:423] Error getting Service "foot/nginx-ns": no object matching key "foot/nginx-ns" in local store
I0601 03:06:00.162280       7 controller.go:167] "Configuration changes detected, backend reload required"
I0601 03:06:00.291845       7 controller.go:184] "Backend successfully reloaded"
I0601 03:06:00.292568       7 event.go:285] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-48nw7", UID:"69febc3b-9dc7-4523-9fd0-9012b6a3ee6f", APIVersion:"v1", ResourceVersion:"72961152", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
2023/06/01 03:06:01 [error] 847#847: *25909 stream [lua] tcp_udp_balancer.lua:196: balance(): error while setting current upstream peer 2001:db8:42:83:318a:6059:3e82:d1d6:3306: invalid port while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, bytes from/to client:0/0, bytes from/to upstream:0/0
2023/06/01 03:06:01 [crit] 847#847: *25909 connect() to 0.0.0.1:1234 failed (22: Invalid argument) while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, upstream: "0.0.0.1:1234", bytes from/to client:0/0, bytes from/to upstream:0/0
[fd69:1:10:f980::a13:26df] [01/Jun/2023:03:06:01 +0000] TCP 502 0 0 0.000
[fd69:1:10:f980::a13:26df] [01/Jun/2023:03:06:03 +0000] TCP 502 0 0 0.000
2023/06/01 03:06:03 [error] 847#847: *25955 stream [lua] tcp_udp_balancer.lua:196: balance(): error while setting current upstream peer 2001:db8:42:83:318a:6059:3e82:d1d6:3306: invalid port while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, bytes from/to client:0/0, bytes from/to upstream:0/0
2023/06/01 03:06:03 [crit] 847#847: *25955 connect() to 0.0.0.1:1234 failed (22: Invalid argument) while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, upstream: "0.0.0.1:1234", bytes from/to client:0/0, bytes from/to upstream:0/0
2023/06/01 03:06:05 [error] 847#847: *25989 stream [lua] tcp_udp_balancer.lua:196: balance(): error while setting current upstream peer 2001:db8:42:83:318a:6059:3e82:d1d6:3306: invalid port while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, bytes from/to client:0/0, bytes from/to upstream:0/0
2023/06/01 03:06:05 [crit] 847#847: *25989 connect() to 0.0.0.1:1234 failed (22: Invalid argument) while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, upstream: "0.0.0.1:1234", bytes from/to client:0/0, bytes from/to upstream:0/0
[fd69:1:10:f980::a13:26df] [01/Jun/2023:03:06:05 +0000] TCP 502 0 0 0.000
[fd69:1:10:f980::a13:26df] [01/Jun/2023:03:06:07 +0000] TCP 502 0 0 0.000
2023/06/01 03:06:07 [error] 847#847: *26023 stream [lua] tcp_udp_balancer.lua:196: balance(): error while setting current upstream peer 2001:db8:42:83:318a:6059:3e82:d1d6:3306: invalid port while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, bytes from/to client:0/0, bytes from/to upstream:0/0
2023/06/01 03:06:07 [crit] 847#847: *26023 connect() to 0.0.0.1:1234 failed (22: Invalid argument) while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, upstream: "0.0.0.1:1234", bytes from/to client:0/0, bytes from/to upstream:0/0
[fd69:1:10:f980::a13:26df] [01/Jun/2023:03:06:09 +0000] TCP 502 0 0 0.000
2023/06/01 03:06:09 [error] 847#847: *26059 stream [lua] tcp_udp_balancer.lua:196: balance(): error while setting current upstream peer 2001:db8:42:83:318a:6059:3e82:d1d6:3306: invalid port while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, bytes from/to client:0/0, bytes from/to upstream:0/0
2023/06/01 03:06:09 [crit] 847#847: *26059 connect() to 0.0.0.1:1234 failed (22: Invalid argument) while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, upstream: "0.0.0.1:1234", bytes from/to client:0/0, bytes from/to upstream:0/0
[fd69:1:10:f980::a13:26df] [01/Jun/2023:03:06:11 +0000] TCP 502 0 0 0.000
2023/06/01 03:06:11 [error] 847#847: *26093 stream [lua] tcp_udp_balancer.lua:196: balance(): error while setting current upstream peer 2001:db8:42:83:318a:6059:3e82:d1d6:3306: invalid port while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, bytes from/to client:0/0, bytes from/to upstream:0/0
2023/06/01 03:06:11 [crit] 847#847: *26093 connect() to 0.0.0.1:1234 failed (22: Invalid argument) while connecting to upstream, client: fd69:1:10:f980::a13:26df, server: [::]:32001, upstream: "0.0.0.1:1234", bytes from/to client:0/0, bytes from/to upstream:0/0

@longwuyuan @tao12345666333

tao12345666333 · 2023-06-01T07:02:10Z

Thanks for your update.

But please format the contents, you can refer to this documentation https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks#fenced-code-blocks

wcsceshi · 2023-06-01T07:18:26Z

Thanks for your update.

But please format the contents, you can refer to this documentation https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks#fenced-code-blocks

ok ，I've formatted the content

longwuyuan · 2024-09-12T09:52:33Z

TCP/UDP forwarding is being deprecated so no action item on the project in this issue. Hence I will close it.

/close

k8s-ci-robot · 2024-09-12T09:52:37Z

@longwuyuan: Closing this issue.

In response to this:

TCP/UDP forwarding is being deprecated so no action item on the project in this issue. Hence I will close it.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

wcsceshi added the kind/bug Categorizes issue or PR as related to a bug. label May 10, 2023

k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label May 10, 2023

k8s-ci-robot added the needs-priority label May 10, 2023

strongjz added this to [SIG Network] Ingress NGINX May 10, 2023

k8s-ci-robot added needs-kind Indicates a PR lacks a `kind/foo` label and requires one. and removed kind/bug Categorizes issue or PR as related to a bug. labels May 10, 2023

ghostloda mentioned this issue May 21, 2023

Fix the error while setting upstream peer #9977

Open

10 tasks

k8s-ci-robot closed this as completed Sep 12, 2024

github-project-automation bot moved this to Done in [SIG Network] Ingress NGINX Sep 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tcp-services configMap add one configuration ， Other services that are accessed through tcp cannot be accessed properly #9943

tcp-services configMap add one configuration ， Other services that are accessed through tcp cannot be accessed properly #9943

wcsceshi commented May 10, 2023

k8s-ci-robot commented May 10, 2023

longwuyuan commented May 10, 2023

wcsceshi commented May 10, 2023 •

edited

Loading

longwuyuan commented May 10, 2023

wcsceshi commented May 10, 2023

longwuyuan commented May 10, 2023

longwuyuan commented May 21, 2023

longwuyuan commented May 21, 2023

ghostloda commented May 22, 2023

ghostloda commented May 22, 2023

ghostloda commented May 24, 2023

ghostloda commented May 24, 2023 •

edited

Loading

longwuyuan commented May 24, 2023

ghostloda commented May 25, 2023 •

edited

Loading

ghostloda commented May 25, 2023

tao12345666333 commented May 25, 2023

longwuyuan commented May 25, 2023

wcsceshi commented May 25, 2023

longwuyuan commented May 25, 2023

wcsceshi commented Jun 1, 2023 •

edited

Loading

tao12345666333 commented Jun 1, 2023

wcsceshi commented Jun 1, 2023

longwuyuan commented Sep 12, 2024

k8s-ci-robot commented Sep 12, 2024

tcp-services configMap add one configuration ， Other services that are accessed through tcp cannot be accessed properly #9943

tcp-services configMap add one configuration ， Other services that are accessed through tcp cannot be accessed properly #9943

Comments

wcsceshi commented May 10, 2023

k8s-ci-robot commented May 10, 2023

longwuyuan commented May 10, 2023

wcsceshi commented May 10, 2023 • edited Loading

longwuyuan commented May 10, 2023

wcsceshi commented May 10, 2023

longwuyuan commented May 10, 2023

longwuyuan commented May 21, 2023

longwuyuan commented May 21, 2023

ghostloda commented May 22, 2023

ghostloda commented May 22, 2023

ghostloda commented May 24, 2023

ghostloda commented May 24, 2023 • edited Loading

longwuyuan commented May 24, 2023

ghostloda commented May 25, 2023 • edited Loading

ghostloda commented May 25, 2023

tao12345666333 commented May 25, 2023

longwuyuan commented May 25, 2023

wcsceshi commented May 25, 2023

longwuyuan commented May 25, 2023

wcsceshi commented Jun 1, 2023 • edited Loading

tao12345666333 commented Jun 1, 2023

wcsceshi commented Jun 1, 2023

longwuyuan commented Sep 12, 2024

k8s-ci-robot commented Sep 12, 2024

wcsceshi commented May 10, 2023 •

edited

Loading

ghostloda commented May 24, 2023 •

edited

Loading

ghostloda commented May 25, 2023 •

edited

Loading

wcsceshi commented Jun 1, 2023 •

edited

Loading