Skip to content

Latest commit

 

History

History
97 lines (77 loc) · 4.67 KB

README.md

File metadata and controls

97 lines (77 loc) · 4.67 KB
Raw
                  __                                                            __
       .,-;-;-,. /'_\     +-----------------------------------------------+    /_'\.,-;-;-,.
     _/_/_/_|_\_\) /      |  CROMSHELL : run Cromwell jobs from the shell |     \ (/_/__|_\_\_
   '-<_><_><_><_>=/\      +-----------------------------------------------+     /\=<_><_><_><_>-'
     `/_/====/_/-'\_\                                                          /_/'-\_\====\_\'
      ""     ""    ""                                                          ""    ""     ""

Installation

cromshell and it's dependencies can be installed on OSX with brew install broadinstitute/dsp/cromshell

or through bioconda with conda install cromshell

Alternatively, download the script and put it somewhere...

cromshell

A script for submitting workflows to a cromwell server and monitoring / querying their results.

requires column, curl, mail, and jq

Examples:

         cromshell submit workflow.wdl inputs.json options.json dependencies.zip
         cromshell status
         cromshell -t 20 metadata
         cromshell logs -2

Supported Flags:

  • -t TIMEOUT
    • Set the curl connect timeout to TIMEOUT seconds.
    • Also sets the curl max timeout to 2*TIMEOUT seconds.
    • TIMEOUT must be an integer.

Supported Subcommands:

Start/Stop workflows

  • submit [-w] <wdl> <inputs_json> [options_json] [included_wdl_zip_file]
    • Submit a new workflow
    • -w Wait for workflow to transition from 'Submitted' to some other status before ${SCRIPTNAME} exits
    • included_wdl_zip_file Zip file containing any WDL files included in the input WDL
  • abort [workflow-id] [[workflow-id]...]
    • Abort a running workflow

Query workflow status:

  • status [workflow-id] [[workflow-id]...]
    • Check the status of a workflow
  • metadata [workflow-id] [[workflow-id]...]
    • Get the full metadata of a workflow
  • slim-metadata [workflow-id] [[workflow-id]...]
    • Get a subset of the metadata from a workflow
  • execution-status-count [workflow-id] [[workflow-id]...]
    • Get the summarized status of all jobs in the workflow
  • timing [workflow-id] [[workflow-id]...]
    • Open the timing diagram in a browser

Logs

  • logs [workflow-id] [[workflow-id]...]
    • List the log files produced by a workflow
  • fetch-logs [workflow-id] [[workflow-id]...]
    • Download all logs produced by a workflow

Job Outputs

  • list-outputs [workflow-id] [[workflow-id]...]
    • List all output files produced by a workflow
  • fetch-all [workflow-id] [[workflow-id]...]
    • Download all output files produced by a workflow

Get email notification on job completion

  • notify [workflow-id] [daemon-server] email [cromwell-server]
    • daemon-server server to run the notification daemon on

Display a list jobs submitted through cromshell

  • list [-c] [-u]
    • -c Color the output by completion status
    • -u Check completion status of all unfinished jobs

Clean up local cached list

  • cleanup [-s STATUS]
    • Remove completed jobs from local list. Will remove all jobs from the local list that are in a completed state, where a completed state is one of: Succeeded, Failed, Aborted
    • -s STATUS If provided, will only remove jobs with the given STATUS from the local list.

Features:

  • Running submit will create a new folder in the ~/.cromshell/${CROMWELL_URL}/ directory named with the cromwell job id of the newly submitted job.
    It will copy your wdl and json inputs into the folder for reproducibility.
  • It keeps track of your most recently submitted jobs by storing their ids in ./cromshell/
    You may ommit the job ID of the last job submitted when running commands, or use negative numbers to reference previous jobs, e.g. "-1" will track the last job, "-2" will track the one before that, and so on.
  • You can override the default cromwell server by setting the environmental variable CROMWELL_URL to the appropriate URL.
  • Most commands takes multiple workflow-ids, which you can specify both in relative and absolute ID value (i.e. ./cromwell status -1 -2 -3 c2db2989-2e09-4f2c-8a7f-c3733ae5ba7b).