Add dump reliability check if MAT's loaded DTFJ doesn't match the dump's DTFJ

[eclipsewebmaster]

| --- | --- |
| Bugzilla Link | 582736 |
| Status | ASSIGNED |
| Importance | P3 normal |
| Reported | Dec 13, 2023 13:31 EDT |
| Modified | Dec 18, 2023 13:53 EDT |
| Reporter | Kevin Grigorenko |

Description

The DTFJ implementations for IBM Java and IBM Semeru Runtimes are different and usually incompatible, particularly for core dumps.

MAT ships with a built-in update site for IBM Java DTFJ at https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/runtimes/tools/dtfj/

As of this writing, that update site was last updated in 2020 and is not actively maintained.

IBM also publishes standalone update sites and packaged MAT builds for both IBM Java DTFJ and IBM Semeru Runtimes DTFJ at https://www.ibm.com/support/pages/eclipse-memory-analyzer-tool-dtfj-and-ibm-extensions

However, as of this writing, those update sites are not signed.

In addition, I have not found time to restart the work to add an option to MAT to specify the DTFJ implementation or pass through to the underlying JVM: https://bugs.eclipse.org/bugs/show_bug.cgi?id=567819

So that is the current and unfortunate state of things.

With that noted, as IBM Semeru Runtimes usage picks up, it is becoming a more common issue that users running with MAT + IBM Java DTFJ are trying to load dumps produced by IBM Semeru Runtimes. This produces confusing fatal errors such as the following:

java.lang.NoClassDefFoundError: com.ibm.j9ddr.vm29.pointer.generated.GC_ArrayObjectModelPointer
at com.ibm.j9ddr.vm29.j9.gc.GCArrayObjectModel.from(GCArrayObjectModel.java:52)
at com.ibm.j9ddr.vm29.j9.gc.GCObjectModel.(GCObjectModel.java:43)
at com.ibm.j9ddr.vm29.j9.gc.GCObjectModel_V1.(GCObjectModel_V1.java:80)
at com.ibm.j9ddr.vm29.j9.gc.GCObjectModel.from(GCObjectModel.java:59)
at com.ibm.j9ddr.vm29.j9.ObjectModel.(ObjectModel.java:56)
at com.ibm.j9ddr.vm29.j9.gc.GCObjectHeapIteratorAddressOrderedList_V1.advanceScanPointer(GCObjectHeapIteratorAddressOrderedList_V1.java:200)
at com.ibm.j9ddr.vm29.j9.gc.GCObjectHeapIteratorAddressOrderedList_V1.next(GCObjectHeapIteratorAddressOrderedList_V1.java:283)
at com.ibm.j9ddr.vm29.j9.walkers.HeapWalker.walk(HeapWalker.java:118)
at com.ibm.j9ddr.vm29.view.dtfj.java.j9.HeapObjectIterator.hasNext(HeapObjectIterator.java:50)
at com.ibm.j9ddr.vm29.view.dtfj.java.DTFJJavaHeap$1.hasNext(DTFJJavaHeap.java:118)
at org.eclipse.mat.dtfj.DTFJIndexBuilder.fill(DTFJIndexBuilder.java:1236) [...]

MAT should have some detection of this and provide a better error.

[eclipsewebmaster]

By Kevin Grigorenko on Dec 14, 2023 08:48

https://git.eclipse.org/c/mat/org.eclipse.mat.git/commit/?id=a192f1be11e2b005001eca1cea26b7150b6cf2bb

https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/205821

[eclipsewebmaster]

By Andrew Johnson on Dec 17, 2023 05:31

Minor warning from the build:

Should org.eclipse.mat.dtfj.DTFJIndexBuilder$DumpReliabilityResult be a static inner class?\

This class is an inner class, but does not use its embedded reference to the object which created it. This reference makes the instances of the class larger, and may keep the reference to the creator object alive longer than necessary. If possible, the class should be made static.

Probably doesn't matter much.

[eclipsewebmaster]

By Kevin Grigorenko on Dec 18, 2023 13:53

Good catch. Made the change and verified locally and all tests pass.

https://git.eclipse.org/c/mat/org.eclipse.mat.git/commit/?id=42b50511976f6072f4f66c7a7a42e39c14ad7d7e

https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/205880

[pavelkokush]

Any update on this one?
Getting same error: The loaded DTFJ implementation for IBM Java does not match the file source of IBM Semeru Runtimes. Extracted Java version: JRE 21 Linux amd64-64 (build 21.0.5+11-LTS)

[kgibm]

@pavelkokush The DTFJ update site at https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/runtimes/tools/dtfj/ is only supported for IBM Java (and it's also a very old build). If you want to parse IBM Semeru Runtimes dumps, then you need to use https://www.ibm.com/support/pages/eclipse-memory-analyzer-tool-dtfj-and-ibm-extensions

So, it's basically the same status though we're working on open sourcing the DTFJ plugin which will at some point hopefully allow integrating the Semeru plugin directly into MAT.