leftalignindels: java.lang.IllegalArgumentException: the range cannot contain negative indices

[gbrandt6]

Bug Report

Affected tool(s) or class(es)

LeftAlignIndels

Affected version(s)

• [Exists in version 4.1.7.0 and 4.1.8.1 ]
• [No issue in version 4.1.4.1 ]

Description

LeftAlignIndels exits with a java IllegalArgumentException. This issue was discussed at the GATK Office Hours. A workaround is using an older GATK version (4.1.4.1), however it still persists in the current GATK version.

Steps to reproduce

Issue was reproduced by the support team using just chr2

Associated forum post

https://gatk.broadinstitute.org/hc/en-us/community/posts/360072506131-leftalignindels-java-lang-IllegalArgumentException-the-range-cannot-contain-negative-indices

Command:
gatk LeftAlignIndels
-R /projects/beck-lab/walawi/GATK/hg38.fa
-I /projects/beck-lab/walawi/GATK/sorted_bam/H1-L_GT19_35266_GACCTGAA-TTGGTGAG_S2.withRG.sorted.bam
-O /projects/beck-lab/walawi/GATK/sorted_bam/leftalignindels/H1-L_GT19_35266_GACCTGAA-TTGGTGAG_S2_leftaligned.withRG.sorted.bam

gatk LeftAlignIndels -R /projects/beck-lab/walawi/GATK/hg38.fa -I /projects/beck-lab/walawi/GATK/sorted_bam/H1-L_GT19_35266_GACCTGAA-TTGGTGAG_S2.withRG.sorted.bam -O /projects/beck-lab/walawi/GATK/sorted_bam/leftalignindels/H1-L_GT19_35266_GACCTGAA-TTGGTGAG_S2_leftaligned.withRG.sorted.bam

10:43:19.190 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/walawi/miniconda3/envs/gatk4_venv/share/gatk4-4.1.7.0-0/gatk-package-4.1.7.0-local.jar!/com/intel/gkl/native/libgkl_compression.so

Aug 18, 2020 10:43:19 AM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine

INFO: Failed to detect whether we are running on Google Compute Engine.

10:43:19.667 INFO  LeftAlignIndels - ------------------------------------------------------------

10:43:19.668 INFO  LeftAlignIndels - The Genome Analysis Toolkit (GATK) v4.1.7.0

10:43:19.668 INFO  LeftAlignIndels - For support and documentation go to https://software.broadinstitute.org/gatk/

10:43:19.668 INFO  LeftAlignIndels - Executing as walawi@sumner055 on Linux v3.10.0-1062.1.2.el7.x86_64 amd64

10:43:19.668 INFO  LeftAlignIndels - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_192-b01

10:43:19.668 INFO  LeftAlignIndels - Start Date/Time: August 18, 2020 10:43:19 AM EDT

10:43:19.668 INFO  LeftAlignIndels - ------------------------------------------------------------

10:43:19.668 INFO  LeftAlignIndels - ------------------------------------------------------------

10:43:19.668 INFO  LeftAlignIndels - HTSJDK Version: 2.21.2

10:43:19.669 INFO  LeftAlignIndels - Picard Version: 2.21.9

10:43:19.669 INFO  LeftAlignIndels - HTSJDK Defaults.COMPRESSION_LEVEL : 2

10:43:19.669 INFO  LeftAlignIndels - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false

10:43:19.669 INFO  LeftAlignIndels - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true

10:43:19.669 INFO  LeftAlignIndels - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false

10:43:19.669 INFO  LeftAlignIndels - Deflater: IntelDeflater

10:43:19.669 INFO  LeftAlignIndels - Inflater: IntelInflater

10:43:19.669 INFO  LeftAlignIndels - GCS max retries/reopens: 20

10:43:19.669 INFO  LeftAlignIndels - Requester pays: disabled

10:43:19.669 INFO  LeftAlignIndels - Initializing engine

10:43:20.394 INFO  LeftAlignIndels - Done initializing engine

10:43:20.414 INFO  ProgressMeter - Starting traversal

10:43:20.414 INFO  ProgressMeter -        Current Locus  Elapsed Minutes       Reads Processed     Reads/Minute

10:43:30.418 INFO  ProgressMeter -         chr1:4677047              0.2                389000        2333533.3

10:43:40.433 INFO  ProgressMeter -        chr1:11374345              0.3                956000        2865421.1

10:43:50.434 INFO  ProgressMeter -        chr1:19249299              0.5               1666000        3329891.1

10:44:00.439 INFO  ProgressMeter -        chr1:27418567              0.7               2376000        3562129.9

10:44:10.443 INFO  ProgressMeter -        chr1:35737560              0.8               3091000        3707124.0

10:44:20.443 INFO  ProgressMeter -        chr1:44727422              1.0               3873000        3871193.4

10:44:30.447 INFO  ProgressMeter -        chr1:56778753              1.2               4884000        4184313.1

10:44:40.454 INFO  ProgressMeter -        chr1:70088547              1.3               5979000        4482009.0

10:44:50.464 INFO  ProgressMeter -        chr1:83764904              1.5               7064000        4706770.8

10:45:00.480 INFO  ProgressMeter -        chr1:92264119              1.7               7763000        4654774.4

10:45:10.491 INFO  ProgressMeter -       chr1:100266449              1.8               8410000        4584105.5

10:45:20.499 INFO  ProgressMeter -       chr1:109363126              2.0               9112000        4552850.9

10:45:30.506 INFO  ProgressMeter -       chr1:117191576              2.2               9772000        4506964.3

10:45:40.514 INFO  ProgressMeter -       chr1:123559934              2.3              10534000        4511542.3

10:45:50.517 INFO  ProgressMeter -       chr1:125181638              2.5              11376000        4547368.4

10:46:00.518 INFO  ProgressMeter -       chr1:144497246              2.7              12131000        4546170.0

10:46:10.524 INFO  ProgressMeter -       chr1:152171724              2.8              12801000        4515105.0

10:46:20.529 INFO  ProgressMeter -       chr1:160529043              3.0              13508000        4499816.8

10:46:30.536 INFO  ProgressMeter -       chr1:169448951              3.2              14260000        4500268.2

10:46:40.543 INFO  ProgressMeter -       chr1:178147829              3.3              14969000        4487827.8

10:46:50.551 INFO  ProgressMeter -       chr1:186125500              3.5              15636000        4464516.0

10:47:00.555 INFO  ProgressMeter -       chr1:194986011              3.7              16297000        4441809.8

10:47:10.565 INFO  ProgressMeter -       chr1:203560084              3.8              17001000        4432191.5

10:47:20.577 INFO  ProgressMeter -       chr1:211951736              4.0              17700000        4421996.7

10:47:30.580 INFO  ProgressMeter -       chr1:220837654              4.2              18418000        4417386.9

10:47:40.592 INFO  ProgressMeter -       chr1:229536819              4.3              19156000        4417642.0

10:47:50.595 INFO  ProgressMeter -       chr1:237917173              4.5              19861000        4410598.8

10:48:00.598 INFO  ProgressMeter -       chr1:246682719              4.7              20561000        4403066.6

10:48:10.604 INFO  ProgressMeter -         chr2:6733404              4.8              21270000        4397838.6

10:48:20.605 INFO  ProgressMeter -        chr2:15144861              5.0              21942000        4385607.8

10:48:30.607 INFO  ProgressMeter -        chr2:24131740              5.2              22650000        4381143.4

10:48:40.610 INFO  ProgressMeter -        chr2:32916253              5.3              23467000        4397382.8

10:48:43.217 INFO  LeftAlignIndels - Shutting down engine

[August 18, 2020 10:48:43 AM EDT] org.broadinstitute.hellbender.tools.LeftAlignIndels done. Elapsed time: 5.40 minutes.

Runtime.totalMemory()=2076049408

java.lang.IllegalArgumentException: the range cannot contain negative indices

at org.broadinstitute.hellbender.utils.Utils.validateArg(Utils.java:727)

at org.broadinstitute.hellbender.utils.IndexRange.validate(IndexRange.java:108)

at org.broadinstitute.hellbender.utils.IndexRange.shift(IndexRange.java:73)

at org.broadinstitute.hellbender.utils.IndexRange.shiftLeft(IndexRange.java:77)

at org.broadinstitute.hellbender.utils.read.AlignmentUtils.leftAlignIndels(AlignmentUtils.java:735)

at org.broadinstitute.hellbender.tools.LeftAlignIndels.apply(LeftAlignIndels.java:78)

at org.broadinstitute.hellbender.engine.ReadWalker.lambda$traverse$0(ReadWalker.java:96)

at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)

at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)

at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)

at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)

at java.util.Iterator.forEachRemaining(Iterator.java:116)

at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)

at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)

at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)

at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)

at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)

at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)

at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)

at org.broadinstitute.hellbender.engine.ReadWalker.traverse(ReadWalker.java:94)

at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1048)

at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)

at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)

at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)

at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:163)

at org.broadinstitute.hellbender.Main.mainEntry(Main.java:206)

at org.broadinstitute.hellbender.Main.main(Main.java:292)

Using GATK jar /home/walawi/miniconda3/envs/gatk4_venv/share/gatk4-4.1.7.0-0/gatk-package-4.1.7.0-local.jar

Running:

    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /home/walawi/miniconda3/envs/gatk4_venv/share/gatk4-4.1.7.0-0/gatk-package-4.1.7.0-local.jar LeftAlignIndels -R /projects/beck-lab/walawi/GATK/hg38.fa -I /projects/beck-lab/walawi/GATK/sorted_bam/H1-L_GT19_35266_GACCTGAA-TTGGTGAG_S2.withRG.sorted.bam -O /projects/beck-lab/walawi/GATK/sorted_bam/leftalignindels/H1-L_GT19_35266_GACCTGAA-TTGGTGAG_S2_leftaligned.withRG.sorted.bam

[droazen]

@davidbenjamin Since the error doesn't occur with GATK 4.1.4.1, it's possible that this is a side effect of the changes to LeftAlignIndels that went into GATK 4.1.5.0 in #6427.

@gbrandt6 Could you or the user test whether the error occurs in GATK 4.1.5.0? If it does, that would definitely implicate #6427 as the source of the regression.

[gbrandt6]

Yes, this issue occurs in 4.1.5.0 so it looks like #6427 is the source.

[droazen]

@davidbenjamin Can you let us know whether this is something you'll have a chance to look at, or should we assign it to someone else? It looks like your PR #6427 is the culprit.

[davidbenjamin]

@droazen I will be done with this in a few days. I see what's going on in the debugger and know how to fix it.