Adding Class II HLA typing to immuno #57

Layth17 · 2022-07-08T20:35:11Z

Environment
Docker image docker(mgibio/phlat:1.1_withindex) (note: mem must be >= 16G for bowtie2 to work properly. I ran it with 20G) bsub -Is -M 20G -G $GROUP -n 1 -R 'select[mem>20000] rusage[mem=20000] span[hosts=1]' -q oncology-interactive -a 'docker(mgibio/phlat:1.1_withindex)' /bin/bash
Within docker, bash script run.b38.sh is working.
Create phlat.wdl
Implement phlat.wdl within immuno.wdl
Input/Output cleaning
Extract Alleles -- extract class I from OptiType and class II from PHLAT

testing the steps below will require me to switch pVACtools docker image to laljorani20/pvactools:latest (temporary until pvactools is updated)

HLA Consensus -- must output a list of valid HLAs for class I and II
Relevant sources:
Allele X not valid for Method Y griffithlab/pVACtools#375
Feature: Auto-generate MHC class II combinations griffithlab/pVACtools#528
Auto-generate MHC class II combinations griffithlab/pVACtools#813 (HLA class II pairing)
For class II, HLA- prefix not needed and in fact would cause issues.
pVACseq and pVACfuse working properly with the input provided
grab the outputs into final_output

chrisamiller · 2022-07-11T16:29:26Z

phlat/run.b38.sh

+
+# extract hla regions and unmapped reads
+echo "extracting hla region and unmapped reads ..."
+$SAMTOOLS view -h -T $REF_FASTA $BAM chr6:29836259-33148325 >$tmpdir/reads.sam


In the context of the larger pipelines, (as opposed to a one-off bash script it came from) I don't like that this has a hardcoded human build38 path. (if we tried to run this on b37 data (or mouse data), it would happily do the wrong thing!). The ideal thing to do here would probably be to use the annotation build info to look up the HLA locus, I think? That could look something like this:

grep the GTF file for the proper gene names (HLA-A, HLA-B, HLA-C,, etc) to get coordinates

use those coordinates in this command

To be clear, I don't think phlat works on mouse data, so we can ignore that for now, but being build-agnostic for human would be good!

Layth17 · 2022-07-11T18:20:47Z

phlat folder should be deleted ultimately. it is only in this repo atm for quick testing purposes.
run.b38.sh and phlat-release/ belong within the docker image

tmooney · 2022-09-29T23:10:11Z

definitions/tools/extract_hla_alleles.wdl

-    /usr/bin/awk '{FS="\t";getline;for(n=2;n<=NF-2;n++){if($n==""){}else{printf "HLA-"$n"\n"}}}' ~{file} > ~{outname}
+    /usr/bin/awk '{FS="\t";getline;for(n=2;n<=NF-2;n++){if($n==""){}else{printf "HLA-"$n"\n"}}}' ~{file} > ~{temp}
+    grep "HLA_D" ~{phlat_file} | /usr/bin/awk '{FS="\t";if($2==""){}else{printf $2"\n"};if($3==""){}else{printf $3"\n"}}' >> ~{temp}
+    /usr/bin/awk -F":" '{print $1 ":" $2}' ~{temp} > ~{outname}


If you want, this can be done without a temporary file by grouping the commands together for the pipe. Something like:

(/usr/bin/awk ... ~{file}; grep ... ~{phlat_file} | /usr/bin/awk ...) | /usr/bin/awk ...

tmooney · 2022-09-29T23:12:55Z

definitions/tools/pvacfuse.wdl

@@ -27,7 +27,7 @@ task pvacfuse {

  Int space_needed_gb = 10 + round(size([input_fusions_zip], "GB") * 3)
  runtime {
-    docker: "griffithlab/pvactools:3.0.0"
+    docker: "laljorani20/pvactools:latest"


We should switch this back to an official release before merging.

I believe this is blocked on @susannasiebert switching back to pvactools effort and creating a release. But yeah we don't want to be pointing to personal docker images like this. I have been meaning to search the whole repo and see if we are doing it anywhere else.

I just released pVACtools 3.1.0 that has the combinatorial class II alleles enabled.

tmooney · 2022-09-29T23:14:23Z

definitions/tools/pvacseq.wdl

@@ -50,7 +50,7 @@ task pvacseq {
  runtime {
    memory: "16GB"
    cpu: n_threads
-    docker: "griffithlab/pvactools:3.0.0"
+    docker: "laljorani20/pvactools:latest"


(Same here.)

tmooney · 2022-09-29T23:31:37Z

definitions/tools/phlat.wdl

+    cpu: nthreads
+    docker: "mgibio/phlat:1.1_withindex"
+    disks: "local-disk ~{space_needed_gb} HDD"
+    bootDiskSizeGb: 3*space_needed_gb


Does the boot disk need to scale this severely with the size of the input data?

I was mimicking optitype.wdl iirc when I created phlat.wdl

If there is a better way to do this, I am all for it. Would times 2 be better?

I'm working on a resource usage script that I hope will help us see how much disk space is actually being used so that we can fine tune some of these resource requests.

I guess I was asking more if the boot disk really needs to scale in the size of the inputs at all. The disks entry should cover space for staging the inputs and working on the outputs. The boot disk needs to hold the docker image and OS files for the VM, but the work ought to get done in the space defined by disks.

tmooney · 2022-09-29T23:35:02Z

definitions/tools/pvacseq.wdl

+    Array[File?] combined = [combined_all_epitopes, combined_aggregated_report, combined_filtered_epitopes, combined_aggregated_metrics_file]
+
+    # Don't think this works the way it is intended. Refer to documentation.
+    # https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#globs


What is intended and what is happening instead?

From the looks of the the command that uses glob, it seemed like it intends to iterate through pvacfuse_predictions/ like this:

| PVACfuse_predictions/
-->| mhc_i
----> grabs all files
-->| mhc_ii
---->grabs all files
-->| combined
---->grabs all files

And output a similar structure to the above. Or dump all contents in one file.

However the real result of that was like this:

| PVACfuse_predictions/
-->| mhc_i
---->grabs all files

Leading to the output:

PVACfuse_predictions/content of mhc_i

And the documentation does state that glob skips folders and their contents. Only thing that is slightly vague to me is the use of the double * and why did it pick mhc_i as opposed to another folder unless that was just purely due to order.

malachig

As mentioned by Tom above, I think we want to switch to a docker image for an official pvactools release before we consider this final.

malachig · 2022-10-07T02:06:43Z

I have tested this PR on real patient data and it seems to be working nicely:

I got a successful immuno.wdl run.
I manually examined the phlat HLA typing results and compared class I calls to optitype calls (100% agreement)
I examined the organization of class I and II HLA typing results into final folders with improved file names. This looks good to me
I confirmed that class I pvacseq and pvacfuse results are using the class I HLA types predicted
I confirmed that class II pvacseq and pvacfuse results are using the class II HLA types predicted
I loaded both the class I and class II neoantigen results in pvacview and manually examined them

I think other than the issue with using a personal docker image for pvactools this is ready to merge... We should ask @susannasiebert what she thinks about a timeline for a release that would include the pvactools improvements needed here. I believe the main thing was the code that produces classII allele pairings automatically?

susannasiebert · 2022-10-17T20:23:15Z

3.1.0 is out now and has the required feature.

malachig

Looks good to me.

adding bash script for phlat

caebbe7

Layth17 linked an issue Jul 8, 2022 that may be closed by this pull request

Add class II HLA typing to immuno analysis #37

Closed

Layth17 self-assigned this Jul 8, 2022

Layth17 requested a review from malachig July 8, 2022 20:42

Aljorani added 2 commits July 11, 2022 11:13

working bash script for phlat

cc4535d

cleaning

48bcaad

chrisamiller reviewed Jul 11, 2022

View reviewed changes

stage 0 of implementing phlat

8169d68

Aljorani added 21 commits July 11, 2022 16:06

added one more input

3d82e8f

setting up

1644ee1

removed fastq in

7a9c7ca

cleaned command and inputs

d2bff8e

cleaned the phlat task

2298862

implementing phlat within immuno -- in progress

da16a42

moved to the formal docker image

5eea43c

fixing variables for phlat in immuno

bab2be7

adding phlat output

3b40c4f

added .cram and .fai files

8816ffe

changed results dir

3c503f8

fix spacing

7174080

attempting fix to command

5a603db

fixed phlat output

9077f30

implemented phlat hla typing normal

36c0f67

removed workflows.zip

71146e4

extract alleles takes in phlat

91e9529

reformating PHLAT output for consensus

3f5c47f

added another input to extract alleles

4569b4c

feeding extract alleles 2 input files

d627993

optitype_file -> file

1cc56c7

no HLA- for class II

1be4515

Layth17 force-pushed the phlat branch from 493191d to 1be4515 Compare August 5, 2022 16:21

Aljorani added 18 commits September 4, 2022 12:55

Merge branch 'main' of github.com:Layth17/analysis-wdls into phlat

e672635

switching pvacseq/fuse docker image for testing

7d49f49

2 field alleles

ba11047

Merge branch 'main' of github.com:Layth17/analysis-wdls into phlat

fc33f6f

Merge branch 'main' of github.com:Layth17/analysis-wdls into phlat

7f64fe5

changed output names and structure

d2d0e4a

grabbing mhc_ii and restructuring output

3465bfe

fixing syntax

95a51a2

fixing syntax

f3d4d74

mapping outputs correctly

72fec8e

fixing phlat & optitype output name

f3ad522

fixing variable type

8d93462

fixing variable type

e61cb79

fixing variable type

d643e90

fixing variable type

7022561

fixed phlat output name

1205d9b

fixed folder name

63aefae

changed method for extract alleles for mhc class II

de19ed1

tmooney reviewed Sep 29, 2022

View reviewed changes

grabbing all the content of predictions

6618e3c

malachig requested changes Oct 7, 2022

View reviewed changes

switching back to original docker image

5841e66

Layth17 requested a review from malachig October 18, 2022 00:15

Merge branch 'griffithlab:main' into phlat

7559b3a

malachig approved these changes Oct 19, 2022

View reviewed changes

Layth17 merged commit 018f5c0 into wustl-oncology:main Oct 19, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding Class II HLA typing to immuno #57

Adding Class II HLA typing to immuno #57

Layth17 commented Jul 8, 2022 •

edited

Loading

chrisamiller Jul 11, 2022 •

edited

Loading

Layth17 commented Jul 11, 2022 •

edited

Loading

tmooney Sep 29, 2022

tmooney Sep 29, 2022

malachig Oct 7, 2022

susannasiebert Oct 17, 2022

tmooney Sep 29, 2022

tmooney Sep 29, 2022

Layth17 Sep 30, 2022 •

edited

Loading

malachig Oct 7, 2022

tmooney Oct 12, 2022

tmooney Sep 29, 2022

Layth17 Sep 30, 2022 •

edited

Loading

malachig left a comment

malachig commented Oct 7, 2022

susannasiebert commented Oct 17, 2022

malachig left a comment

Adding Class II HLA typing to immuno #57

Adding Class II HLA typing to immuno #57

Conversation

Layth17 commented Jul 8, 2022 • edited Loading

chrisamiller Jul 11, 2022 • edited Loading

Choose a reason for hiding this comment

Layth17 commented Jul 11, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Layth17 Sep 30, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Layth17 Sep 30, 2022 • edited Loading

Choose a reason for hiding this comment

malachig left a comment

Choose a reason for hiding this comment

malachig commented Oct 7, 2022

susannasiebert commented Oct 17, 2022

malachig left a comment

Choose a reason for hiding this comment

Layth17 commented Jul 8, 2022 •

edited

Loading

chrisamiller Jul 11, 2022 •

edited

Loading

Layth17 commented Jul 11, 2022 •

edited

Loading

Layth17 Sep 30, 2022 •

edited

Loading

Layth17 Sep 30, 2022 •

edited

Loading