[sentinel] update master address if it changes.

[jhgg]

Otherwise we'll end up spamming the connection pool because each time the master is discovered, it'll trigger the disconnect of the pool - as self.master_address never gets updated.

[rolette]

Can we get this merged to master? SentinelConnectionPool eventually ends up causing OSError 99's under load.

The connections in the pool all become one-shot connections if the master moves. When those connections are flushed quickly like that, you end up with tons of connections in TIME_WAIT state. Eventually this leads to having no source ports available for your connection, which results in this error:

redis.exceptions.ConnectionError: Error 99 connecting to 100.64.0.3:6379. Cannot assign requested address.

[phenomax]

LGTM!

[nickwilliams-eventbrite]

+1 here. Really need this merged. Having to monkey-patch.

[avgerin0s]

@andymccurdy Are there any plans on merging this?

[jhgg]

@andymccurdy bump on this one!

[jobec]

Would be great if it could be merged.

We're running into issues when running a lot of tasks in Celery with Gevent and Redis as it's result backend. Whenever the master moves, we get errors like these (only a restart of Celery fixes it):

• ConcurrentObjectUseError: This socket is already used by another greenlet: <bound method Waiter.switch of <gevent.__waiter.Waiter object at 0x7efcfb1eab38>>
• AttributeError: 'NoneType' object has no attribute 'sendall'
• ConnectionError: Error 32 while writing to socket. Broken pipe.
• ConnectionError: Error 9 while writing to socket. File descriptor was closed in another greenlet.
• ConnectionError: Error 104 while writing to socket. Connection reset by peer.

We waited until 3.2.0 to see if it fixed things but it didn't. By applying this patch, all tasks run fine again after a master move. (Except the first task after the move, complaining about "connection reset by peer")

[samschlegel]

@andymccurdy Could we get this merged?

[andymccurdy]

Thanks!