Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

use localized copy of vep custom inputs #47

Merged
merged 1 commit into from
Aug 1, 2022

Conversation

chrisamiller
Copy link
Member

Handle issue that affects localization of input files on cluster. This has not been tested on the cloud, but I can't see why it wouldn't work there.

To explain what changes here (because it's complicated), we set the --custom string to include a variable in the first parsing step: $custom_inputs_dir. Then, in the second "run vep" step, we populate that variable with the correct inputs directory for that step (as opposed to pointing back to the inputs dir from the first step).

Copy link
Member

@tmooney tmooney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, I guess this makes sense as much as anything here does!

This WDL is really convoluted, but some of the comments to speak to frustrations I've experienced working on a different WDL workflow, so maybe this is what we're stuck with? It feels extra painful to be injecting a bash variable into a script from another step.

We're at least lucky that the inputs in the file array are all being staged to the same directory. (At least, I'm assuming that's what happens in the cloud, too--I haven't tried it.)

@chrisamiller
Copy link
Member Author

Totally agree that the whole process is convoluted. I would really like to see if we can figure out the cromwell reference-disks implementation, which allows common files to just be mounted instead of copied and staged locally. (reference fastas, vep cache, etc would all benefit from this). In the meantime, I'll merge this as a short-term fix

@chrisamiller chrisamiller merged commit 2fe0a9b into wustl-oncology:main Aug 1, 2022
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants