Skip to content

Commit

Permalink
Change name of contents to ASTRID instead of DJS.
Browse files Browse the repository at this point in the history
  • Loading branch information
Samuel Cox committed May 24, 2018
1 parent ed19520 commit 2a56974
Show file tree
Hide file tree
Showing 3 changed files with 13 additions and 12 deletions.
13 changes: 7 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,18 @@
# Dependent Job Submission
# ASTRID - Automated Submission Tool foR jobs with InterDependencies

This small Python 3 script solves the problem of how to submit many jobs to a job scheduler with multiple dependencies
between jobs. Based on an input json file, the user defines which job dependencies, and the script submits each in the
correct order. DJS does not wait for jobs to launch or complete before submitting dependent jobs. It is left up to the
job scheduler to schedule and launch the jobs, and to cancel any jobs dependent on failed jobs.
between jobs. This can be a particular issue when a multi-stage pipeline requires separate scheduled jobs for each stage.
Based on an input json file, the user defines the job dependencies, and the script submits each in the correct order.
ASTRID does not wait for jobs to launch or complete before submitting dependent jobs - it is left up to the job scheduler
to schedule and launch the jobs, and to cancel any jobs dependent on failed jobs.

Job dependencies are resolved before submission, to ensure that the full pipeline can be submitted.

Helpful error messages are output if the jobs cannot be submitted due to mutual dependencies.

## Usage

Call DJS with python3 djs.py /name/of/input/file . The input file should be a JSON file with structure as given in
Call ASTRID with python3 astrid.py /name/of/input/file . The input file should be a JSON file with structure as given in
input.json. Jobs are listed as keys, with their dependencies as comma-separated strings in the values. Jobs with no
dependencies use the empty string "" as their value.

Expand All @@ -24,4 +25,4 @@ in your installation:
* subprocess
* numpy
* collections


4 changes: 2 additions & 2 deletions djs.py → astrid.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ def print_summary(jobids):
return


def djs(args):
def astrid(args):
json_data = open(args.input_file).read()

data = json.loads(json_data, object_pairs_hook=collections.OrderedDict)
Expand Down Expand Up @@ -134,4 +134,4 @@ def create_parser():
if __name__ == '__main__':
parser = create_parser()
args = parser.parse_args()
djs(args)
astrid(args)
8 changes: 4 additions & 4 deletions input.json
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
{
"trim": "",
"alignment": "trim",
"VC1": "alignment",
"CNA1": "alignment",
"annotation": "VC1,VC2",
"VC2": "alignment",
"alignment": "trim",
"annotation": "VC1,VC2",
"CNA1": "alignment",
"CNA2": "alignment"
}
}

0 comments on commit 2a56974

Please # to comment.