[PLATFORM-2274]: Investigate possible DNS caching issues on our rabbitmq libraries

[cottinisimone]

https://prima-assicurazioni-spa.myjetbrains.com/youtrack/issue/PLATFORM-2274

This PR modifies the connection behavior of amqpx when connecting to a RabbitMQ cluster or host:

• If the host field in the connection parameters or the RabbitMQ URL is an IP address, the library will establish a direct connection to that IP.
• If the host field in the connection parameters or the RabbitMQ URL is a DNS record, the library will resolve the associated A records and attempt to connect to the first available IP address.

[neslinesli93]

nice work! a few comments:

1. does it work only for the first connection opening, or also for re-connections after errors?
2. do you think it's possible to add a test that actually reproduces the issue where the first resolution fails but the second succeeds? you know, actually testing the reduce_while behavior
3. can you add a short description in the PR please please :c

[cottinisimone]

nice work! a few comments:

1. does it work only for the first connection opening, or also for re-connections after errors?
2. do you think it's possible to add a test that actually reproduces the issue where the first resolution fails but the second succeeds? you know, actually testing the reduce_while behavior
3. can you add a short description in the PR please please :c

Ehi @neslinesli93

1. It only handles first connection opening. I thought about reconnection but i feel like the reconnection logic/strategy should be implemented by the client itself. Wdyt?
2. You are right. I tested it locally and it requires something like a local dns server. I'll do another commit adding
3. Yes lol

[neslinesli93]

nice work! a few comments:

1. does it work only for the first connection opening, or also for re-connections after errors?
2. do you think it's possible to add a test that actually reproduces the issue where the first resolution fails but the second succeeds? you know, actually testing the reduce_while behavior
3. can you add a short description in the PR please please :c

Ehi @neslinesli93

1. It only handles first connection opening. I thought about reconnection but i feel like the reconnection logic/strategy should be implemented by the client itself. Wdyt?

if the library handles the reconnection itself (i don't remember tbh) then the process should be the same... otherwise yeah the clients should handle that

[cpiemontese]

Nice 👍

Tests of actual resolution would be really cool, not sure if modifying the /etc/hosts file or something like that could work 🤷

[cpiemontese]

Mocking for the win

[neslinesli93]

❤️

[francesco995]

@andreausu @cottinisimone i'm worried about

If the host field in the connection parameters or the RabbitMQ URL is a DNS record, the library will resolve the associated A records and attempt to connect to the first available IP address.

don't we risk to, in case of normal operations, connect always to the first node?

[cottinisimone]

@andreausu @cottinisimone i'm worried about

If the host field in the connection parameters or the RabbitMQ URL is a DNS record, the library will resolve the associated A records and attempt to connect to the first available IP address.

don't we risk to, in case of normal operations, connect always to the first node?

It seems is not. The load balancer is balancing the ips during the resolution too.

Here's the code i run to test it:

1..100
|> Enum.map(fn _ ->
    'loud-azure-centipede.in.rmq4.cloudamqp.com'
    |> Amqpx.DNS.resolve_ips()
    |> Enum.map(&to_string/1)
    |> Enum.join("-")
end)
|> Enum.group_by(fn v -> v end)
|> Enum.map(fn {k, v} -> {k, Enum.count(v)} end)

[andreausu]

@andreausu @cottinisimone i'm worried about

If the host field in the connection parameters or the RabbitMQ URL is a DNS record, the library will resolve the associated A records and attempt to connect to the first available IP address.

don't we risk to, in case of normal operations, connect always to the first node?

I've always witnessed DNS servers returning answers for A records with multiple IPs associated in random order, but I'm not sure if that's by RFC or just a common implementation.