Skip to content

Commit

Permalink
update 'dask_executor' docstring. (#2684)
Browse files Browse the repository at this point in the history
  • Loading branch information
DavidKatz-il authored Jul 6, 2020
1 parent bfbf490 commit 55b31db
Showing 1 changed file with 7 additions and 8 deletions.
15 changes: 7 additions & 8 deletions python_modules/libraries/dagster-dask/dagster_dask/executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,25 +60,24 @@
def dask_executor(init_context):
'''Dask-based executor.
The 'cluster' can be one of the following:
('local', 'yarn', 'ssh', 'pbs', 'moab', 'sge', 'lsf', 'slurm', 'oar', 'kube').
If the Dask executor is used without providing executor-specific config, a local Dask cluster
will be created (as when calling :py:class:`dask.distributed.Client() <dask:distributed.Client>`
without specifying the scheduler address).
with :py:class:`dask.distributed.LocalCluster() <dask:distributed.LocalCluster>`).
The Dask executor optionally takes the following config:
.. code-block:: none
cluster:
{
local?: # The cluster type, one of the following ('local', 'yarn', 'ssh', 'pbs', 'moab', 'sge', 'lsf', 'slurm', 'oar', 'kube').
local?: # takes distributed.LocalCluster parameters
{
address?: '127.0.0.1:8786', # The address of a Dask scheduler
timeout?: 5, # Timeout duration for initial connection to the scheduler
scheduler_file?: '/path/to/file' # Path to a file with scheduler information
# Whether to connect directly to the workers, or ask the scheduler to serve as
# intermediary
direct_to_workers?: False,
heartbeat_interval?: 1000, # Time in milliseconds between heartbeats to scheduler
n_workers?: 4 # Number of workers to start
threads_per_worker?: 1 # Number of threads per each worker
}
}
Expand Down

0 comments on commit 55b31db

Please # to comment.