Failtoban Increment Counter Issue

[valentino-90]

Hi,
i have SSHPiper running in a Kubernetes pod. With an SFTP client that refreshes the directory tree every 5 seconds, I get this error despite using the correct credentials. I am not banned, but the counter increments, and at the first real error, I actually get banned.

Let me know if you'd like further adjustments or clarifications!

RUNCMD
/sshpiper/sshpiperd --log-level trace--allowed-proxy-addresses 0.0.0.0/0 --port 30003 /sshpiper/kubernetes -- /sshpiper/failtoban

LOG
time="2024-09-18T08:06:59Z" level=debug msg="failtoban: 151.35.123.29 pipe create failed, reason EOF. current status: fail 79 times, max allowed 5"
time="2024-09-18T08:07:00Z" level=debug msg="connection accepted: 10.0.0.13:3640"
time="2024-09-18T08:07:00Z" level=debug msg="connection from 10.0.0.13:3640 establishing failed reason: write tcp 10.244.182.12:30003->10.244.189.64:53346: write: connection reset by peer"
time="2024-09-18T08:07:00Z" level=debug msg="failtoban: 10.0.0.13 pipe create failed, reason write tcp 10.244.182.12:30003->10.244.189.64:53346: write: connection reset by peer. current status: fail 0 times, max allowed 5"
time="2024-09-18T08:07:03Z" level=debug msg="connection accepted: 79.27.251.236:60002"
time="2024-09-18T08:07:05Z" level=debug msg="connection from 79.27.251.236:60002 establishing failed reason: EOF"
time="2024-09-18T08:07:05Z" level=debug msg="failtoban: 79.27.251.236 pipe create failed, reason EOF. current status: fail 341 times, max allowed 5"
time="2024-09-18T08:07:05Z" level=debug msg="connection accepted: 10.0.0.13:51774"
time="2024-09-18T08:07:05Z" level=debug msg="connection from 10.0.0.13:51774 establishing failed reason: write tcp 10.244.182.12:30003->10.244.199.128:25568: write: connection reset by peer"
time="2024-09-18T08:07:05Z" level=debug msg="failtoban: 10.0.0.13 pipe create failed, reason write tcp 10.244.182.12:30003->10.244.199.128:25568: write: connection reset by peer. current status: fail 0 times, max allowed 5"
time="2024-09-18T08:07:06Z" level=debug msg="connection accepted: 79.27.251.236:60035"
time="2024-09-18T08:07:08Z" level=debug msg="connection from 79.27.251.236:60035 establishing failed reason: EOF"
time="2024-09-18T08:07:08Z" level=debug msg="failtoban: 79.27.251.236 pipe create failed, reason EOF. current status: fail 342 times, max allowed 5"
time="2024-09-18T08:07:08Z" level=debug msg="connection from 151.35.123.29:26152 establishing failed reason: EOF"
time="2024-09-18T08:07:08Z" level=debug msg="failtoban: 151.35.123.29 pipe create failed, reason EOF. current status: fail 0 times, max allowed 5"
time="2024-09-18T08:07:09Z" level=debug msg="connection accepted: 79.27.251.236:60055"
time="2024-09-18T08:07:09Z" level=debug msg="connection accepted: 79.27.251.236:60063"
time="2024-09-18T08:07:09Z" level=debug msg="connection accepted: 151.35.123.29:26215"
time="2024-09-18T08:07:11Z" level=debug msg="connection from 79.27.251.236:60055 establishing failed reason: EOF"
time="2024-09-18T08:07:11Z" level=debug msg="failtoban: 79.27.251.236 pipe create failed, reason EOF. current status: fail 343 times, max allowed 5"
time="2024-09-18T08:07:11Z" level=debug msg="connection from 79.27.251.236:60063 establishing failed reason: EOF"
time="2024-09-18T08:07:11Z" level=debug msg="failtoban: 79.27.251.236 pipe create failed, reason EOF. current status: fail 344 times, max allowed 5"

[tg123]

this is likely a bug, sorry too busy these days
i will try to fix it this weekend

[valentino-90]

I found the issue in the Fail2Ban plugin code. A generic SFTP client opens and closes the connection to list directories, so EOF is not an error. I fixed it with this test code:

import "strings" on top
Edit function in failtoban plugin

`PipeCreateErrorCallback: func(remoteAddr string, err error) {
    ip, _, _ := net.SplitHostPort(remoteAddr)

    // BYPASS IF ERROR IS EOF OR UNSTABLE CONNECTION
    if strings.Contains(err.Error(), "connection reset by peer") || strings.Contains(err.Error(), "EOF") {
        log.Debugf("failtoban: %v pipe create failed, but error not counted for ban due to connection issue: %v", ip, err)
        //return
    }else{
        failed, _ := cache.IncrementInt(ip, 1)
        log.Debugf("failtoban: %v pipe create failed, reason %v. current status: fail %v times, max allowed %v", ip, err, failed, maxFailures)
    }
},`

[tg123]

looked into your logs carefully, seems it is by design
ip 79.27.251.236 connect to piper and did not say anything then close conn (EOF)
piper failed to create the pipe and log the err

i would say, the logs is confusing anyway

[valentino-90]

Hi,
i was under an SSH attack where multiple user attempts were being made, but I was also getting banned myself. I investigated the issue for quite some time, and while it's not noticeable in those logs. I had an SFTP client listing folders every 5 seconds, which was erroneously increasing the failure counter ( maybe an alive connection check ? ) .

Now, I only get banned if I enter the wrong password or if I use an undeclared source username in the Kubernetes pipes.

[tg123]

it bans per ip basis, what did you see about your ip?
you can ignore other ips scanning your server

[valentino-90]

Actually, my IP was getting banned simply by opening VSCode, because I had an SFTP/FTP extension that was performing a directory listing at regular intervals. Specifically, the error was logged as:

time="2024-09-18T08:07:11Z" level=debug msg="failtoban: 79.27.251.236 pipe create failed, reason EOF. current status: fail 343 times, max allowed 5"

So, the error counter for that IP kept increasing. I’m not entirely sure if this was an internal issue of extension, but the ban would consistently occur right after opening VSCode every time.

In my opinion, the issue was caused by the extension repeatedly opening the connection to check for the presence of folders and then closing it (whether it was done regularly or irregularly), which seemed to trigger the problem each time.

[tg123]

thanks after carefully review, i believe this is your case

the client side disconnected (eof or rest by peer) and failtoban got an err i agreed that the ip should not get punished in such case

let me think about how to handle it, maybe removing PipeCreateErrorCallback totally

[tg123]

hi @valentino-90 could you please share your sftp vscode ext, i am not able to repro

[valentino-90]

hi @valentino-90 could you please share your sftp vscode ext, i am not able to repro

I think it would be enough to use any SSH client and terminate the connection by simulating an EOF. In any case, if it helps you, the extension was XPLOT - SSH Tools.

[tg123]

hi @valentino-90

This is due to XPLOT impl
it has a ping host settings

which periodically sends tcp handshake to sshpiper then close the connection (EOF) to test if tcp is alive
however, this behavior is something like tcp handshake flood attack
you can imagine someone knocks your door every 3s to see if you are home then go away without saying anything.

[valentino-90]

hi @valentino-90

This is due to XPLOT impl
it has a ping host settings

which periodically sends tcp handshake to sshpiper then close the connection (EOF) to test if tcp is alive
however, this behavior is something like tcp handshake flood attack
you can imagine someone knocks your door every 3s to see if you are home then go away without saying anything.

Yes, I had suspected that the extension was doing this and that the issue was related to it, but a ping/handshake without any further action isn’t necessarily an attack. The main attacks are connection attempts with incorrect usernames or passwords. Either way, the mystery is solved.

Thanks