Skip to content

Commit

Permalink
Add clarification on multithreading and multiprocessing for resources
Browse files Browse the repository at this point in the history
  • Loading branch information
ryansonshine committed May 5, 2021
1 parent 8a6bd89 commit ef48773
Showing 1 changed file with 10 additions and 4 deletions.
14 changes: 10 additions & 4 deletions docs/source/guide/resources.rst
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ keyword arguments. Examples of waiters include::


Multithreading and multiprocessing
--------------------------------
----------------------------------
It is recommended to create a resource instance for each thread / process in a multithreaded or multiprocess application rather than sharing a single instance among the threads / processes. For example::

import boto3
Expand All @@ -215,6 +215,12 @@ It is recommended to create a resource instance for each thread / process in a m
In the example above, each thread would have its own Boto3 session and its own instance of the S3 resource. This is a good idea because resources contain shared data when loaded and calling actions, accessing properties, or manually loading or reloading the resource can modify this data.

.. note::
Resources are **not** thread safe. These special classes contain additional meta data that cannot be shared between threads. When using a Resource, it is recommended to instantiate a new Resource for each thread, as is shown in the example above.

Low-level clients **are** thread safe. When using a low-level client, it is recommended to instantiate your client then pass that client object to each of your threads.
Resources are **not** thread safe. These special classes contain additional meta data that cannot be shared between threads. When using a Resource, it is recommended to instantiate a new Resource for each thread, as is shown in the example above.

Sessions are **not** thread safe. This is due to shared meta data, similar to Resource.

Low-level clients **are** thread safe in a **threaded environment** assuming the clients are **not** used to interact with the underlying Session. When using a low-level client, it is recommended to instantiate your client then pass that client object to each of your threads.

Low-level clients are **not** safe in a **multiprocess environment**. This is due to `forking issues <https://github.com/psf/requests/issues/4323>`__ with urllib3's connection pool.


0 comments on commit ef48773

Please # to comment.