Make read timeout configurable #763
Conversation
rzvoncek
force-pushed
the
radovan/s3-read-timeout
branch
from
97e62d7 to
9f263ef
Compare
| @@ -136,7 +136,8 @@ def connect(self): | |||
| region_name=self.credentials.region, | |||
| signature_version='v4', | |||
| tcp_keepalive=True, | |||
| max_pool_connections=max_pool_size | |||
| max_pool_connections=max_pool_size, | |||
| read_timeout=int(self.config.read_timeout), | |||
There was a problem hiding this comment.
Question: is there a similar feature for GCP and Azure? It would be nice to apply the same settings for all 3.
There was a problem hiding this comment.
Yes, they are available. I've added commits applying the read timeout config value to Azure and GCP read operations.
rzvoncek
force-pushed
the
radovan/s3-read-timeout
branch
from
07680a5 to
998d9da
Compare
|
| @@ -151,7 +154,7 @@ async def _download_blob(self, src: str, dest: str): | |||
| stream = await self.gcs_storage.download_stream( | |||
| bucket=self.bucket_name, | |||
| object_name=object_key, | |||
| timeout=-1, | |||
| timeout=self.read_timeout if self.read_timeout is not None else -1, | |||
There was a problem hiding this comment.
Hello @rzvoncek,
I am a user of the k8ssandra operator on gcp and this change can make a breaking change in the restore case if the files are big. We are switching from infinite to 60 seconds timeout if I am not mistaken.
Do you think we should rollback the change for the gcp case or should I post a comment here k8ssandra/k8ssandra-operator#1353 to know if we know when this will be work on ?
Fixes #762