Too Many Connections, Internet down then up, DNSCrypt Does NOT Recover

[MNLierman]

I can't comment in the last issue from 2021 but this is most definitely still not fixed, not even in the slightly. A better implementation would be to create a variable in the .conf file that says something like drop connections when there is no connectivity. Like don't keep filling it. Right now my internet is now restored but DNSCrypt will not work again until ssh into it and do systemctl restart dnscrypt-proxy. Until then, my internet perpetually is down because DNS is down because DNSCrypt remembers all of the failed DNS requests and just holds onto them. There should be a TTL for failed requests, I think that's what I'm going for here. A TTL for that request, a DNS request from a client to a server is only good for so long because the client will keep requesting over and over again trying to check when internet is back up. At some point it needs to drop all of the DNS requests that it can't possibly fulfil, and why would a client want them minutes later? I was hoping to find a timeout or TTL variable that I could uncomment in the .conf to enable it to drop all requests that are just sitting there. I even upped my connection limit and it doesn't matter how high you make it.

Steps to reproduce:

1. Set your router to exclusively use DNSCrypt.
2. Disconnect your internet from your router or otherwise make it so that the requests timeout and DNSCrypt doesn't get an answer.
3. After some time, 5-10 minutes, the connection limit will fill up, and DNSCrypt gets stuck in a state that it cannot recover from without manual intervention.

[MNLierman]

It was stuck in this state, and after taking my screenshots and writing my post, I did systemctl restart dnscrypt-proxy and this is the log now, it's back to normal and everything is fine. This is clearly a bug that has not been fixed and my idea of a TTL for queries and then dropping old queries is a perfect solution.

And let me just clarify @jedisct1, no this is not just a message in the log, as you had indicated in 2021. DNSCrypt full stops and does not resolve in queries when the internet is restored, after it reaches this connection limit. It gets stuck. I let it sit there for 20 minutes while I researched this issue and then made my post here.

[lifenjoiner]

Hi @MNLierman,

I can feel your eagerness to have the problem solved from #2713 (reply in thread). But, take it easy. We should get to know what's going on exactly first.

Didn't reproduce it in a Windows sandbox. I'm waiting to see if there is more information provided from you or any others.

The more details, the better. Device, OS, config, version, topology of your network and etc. Mask your private info.
Although showing the same error message, they may be caused by different reasons. And "long-standing" unresolved issues require more than adequate leads.

[MNLierman]

@lifenjoiner Set the limit low enough in your VM that you can easily overwhelm DNSCrypt, and get it to say that there are too many connections, then keep sending lots of DNS requests. It's best if you set it up on your own network with the default DNS pointing to DNS crypt, and then pull plug for half an hour – but you could speed this up by lowering the limit so that DNSCrypt obviously gets overwhelmed. I'm able to consistently reproduce the problem. DNSCrypt should be removing/expiring/purging it's requests list as time goes on, with or without internet connectivity. The requests are no good after a few seconds. OR, my next idea was, it keeps a counter of the number of failed DNS requests in a row, that counter resets every 20 seconds, or just wait for the too many connections to occur and after that happens, it should check the bootstrap servers for connectivity. If both conditions are true, that the connections limit was reached and then the bootstrap servers ping fails, it should assume internet connectivity is down, clear all requests and connections, reject all from thereon, and open a ping loop against the bootstrap servers and wait for connectivity to be restored. Once connectivity is restored, it can start accepting requests. That's what I think the proper solution, it's the easiest method rather than trying to add a TTL to requests and trying to keep track of all of those timers or time entries.

Here is an example on how this could be done:

# We've hit a certain threshold of failed DNS requests, we just pinged the bootstrap servers and they are also down.
# We assume internet is down, clear all DNS requests, and stop taking requests.
# All requests should be rejected until connectivity restored.

# Variable allows us to cleanly and safely stop the loop if we need to call it from another thread
do_connectivity_check=true

# Main loop to check connectivity
while $do_connectivity_check; do
    for server in "${bootstrap_servers[@]}"; do
        if ping -c 1 -W 1 "$server" &> /dev/null; then
            log "Internet connectivity restored via $server"
            do_connectivity_check=false
            accept_dnsrequests=true
            break
        fi
    done
    echo "No connectivity. Retrying in 5 seconds..." && sleep 5
done

That gives you an idea of what I was thinking.

[jedisct1]

accept_dnsrequests=true

that would translate to timeouts, so clients are going to retry forever. Or maybe you mean closing the port, but that's not an option with systemd. All we can do is return an error.

[jedisct1]

On overflow, we only respond to cached responses. But if everything has expired, some clients apparently keep retrying over and over if they don't get a response, constantly filling the queue.
For responses that are not cached, we can try to return an error before closing the connection. Maybe some clients keep retrying like crazy if they get an empty response.