Skip to content

Commit 8ccb837

Browse files
authored
[class-parse] .jmod file support (#891)
Context: #858 Context: https://stackoverflow.com/questions/44732915/why-did-java-9-introduce-the-jmod-file-format/64202720#64202720 Context: https://openjdk.java.net/projects/jigsaw/ Context: https://github.com/xamarin/monodroid/commit/c9e5cbd61fe80e183b44356149abe95578a13d06 JDK 9 replaced the "venerable" (and huge, ~63MB) `jre/lib/rt.jar` with a set of `.jmod` files. Thus, as of JDK 9, there is no `.jar` file to try to parse with `class-parse`, only `.jmod` files! A `.jmod` file, in turn, is still a ZIP container, much like `.jar` files, but: 1. With a different internal directory structure, and 2. With a custom file header. The result of (2) is that while `unzip -l` can show and extract the contents of a `.jmod` file -- with a warning -- `System.IO.Compression.ZipArchive` cannot process the file: % mono …/class-parse.exe $HOME/android-toolchain/jdk-11/jmods/java.base.jmod class-parse: Unable to read file 'java.base.jmod': Number of entries expected in End Of Central Directory does not correspond to number of entries in Central Directory. <api api-source="class-parse" /> Update `Xamarin.Android.Tools.Bytecode.ClassPath` to support `.jmod` files by using `PartialStream` (73096d9) to skip the first 4 bytes. Once able to read a `.jmod` file, lots of debug messages appeared while parsing `java.base.jmod`, a'la: class-parse: method com/xamarin/JavaType$1MyStringList.<init>(Lcom/xamarin/JavaType;Ljava/lang/String;ILjava/lang/StringBuilder;)V: Local variables array has 2 entries ('LocalVariableTableAttribute( LocalVariableTableEntry(Name='this', Descriptor='Lcom/xamarin/JavaType$1MyStringList;', StartPC=0, Index=0), LocalVariableTableEntry(Name='this$0', Descriptor='Lcom/xamarin/JavaType;', StartPC=0, Index=1), LocalVariableTableEntry(Name='a', Descriptor='Ljava/lang/String;', StartPC=0, Index=2), LocalVariableTableEntry(Name='b', Descriptor='I', StartPC=0, Index=3))' ); descriptor has 3 entries! class-parse: method com/xamarin/JavaType$1MyStringList.<init>(Lcom/xamarin/JavaType;Ljava/lang/String;ILjava/lang/StringBuilder;)V: Signature ('Signature((Ljava/lang/String;I)V)') has 2 entries; Descriptor '(Lcom/xamarin/JavaType;Ljava/lang/String;ILjava/lang/StringBuilder;)V' has 3 entries! This was a variation on the "JDK 8?" block that previously didn't have much detail, in part because it didn't have a repro. Now we have a repro, based on [JDK code][0] which contains a class declaration within a method declaration // Java public List<String> staticActionWithGenerics(…) { class MyStringList extends ArrayList<String> { public MyStringList(String a, int b) { } public String get(int index) { return unboundedList.toString() + value1.toString(); } } } The deal is that `staticActionWithGenerics()` contains a `MyStringList` class, which in turn contains a constructor with two parameters. *However*, as far as Java bytecode is concerned, the constructor contains *3* local variables with StartPC==0, which is what we use to infer parameter names. Refactor, cleanup, and otherwise modify huge swaths of `Methods.cs` to get to a "happy medium" of: * No warnings from our unit tests, ensured by updating `ClassFileFixture` to have a `[SetUp]` method which sets the `Log.OnLog` field to a delegate which may call `Assert.Fail()` when invoked. This asserts for all messages starting with `class-parse: methods`, which are produced by `Methods.cs`. * No warnings when processing `java.base.jmod`: % mono bin/Debug/class-parse.exe $HOME/android-toolchain/jdk-11/jmods/java.base.jmod >/dev/null # no error messages * No warnings when processing Android API-31: % mono bin/Debug/class-parse.exe $HOME/android-toolchain/sdk/platforms/android-31/android.jar >/dev/null # no error messages Additionally, improve `Log.cs` so that there are `M(string)` overloads for the existing `M(string, params object[])` methods. This is a "sanity-preserving" change, as "innocuous-looking" code such as `Log.Debug("{foo}")` will throw `FormatException` when the `(string, params object[])` overload is used. Aside: closures are *weird* and finicky. Consider the following Java code: class ClosureDemo { public void m(String a) { class Example { public Example(int x) { System.out.println (a); } } } } It looks like the JNI signature for the `Example` constructor might be `(I)V`, but isn't. It is instead: (LClosureDemo;ILjava/lang/String;)V Breaking that down: * `LClosureDemo;`: `Example` is an inner class, and thus has an implicit reference to the containing type. OK, easy to forget. * `I`: the `int x` parameter. Expected. * `Ljava/lang/String`: the `String a` parameter from the enclosing scope! This is the closure parameter. This does make sense. The problem is that it's *selective*: only variables used within `Example` become extra parameters. If the `Example` constructor is updated to remove the `System.out.println(a)` statement, then `a` is no longer used, and is no longer present as a constructor parameter. The only way I found to "reasonably" determine if a constructor parameter was a closure parameter was by checking all fields in the class with names starting with `val$`, and then comparing the types of those fields to types within the enclosing method's descriptor. I can't think of a way to avoid using `val$`. :-( Another aside: closure parameter behavior *differs* between JDK 1.8 and JDK-11: there appears to be a JDK 1.8 `javac` bug in which it assigns the *wrong* parameter names. Consider `MyStringList`: The Java constructor declaration is: public static <T, …> void staticActionWithGenerics ( T value1, … List<?> unboundedList, …) { class MyStringList extends ArrayList<String> { public MyStringList(String a, int b) { } // … } } The JNI signature for the `MyStringList` constructor is: (Ljava/lang/String;ILjava/util/List;Ljava/lang/Object;)V which is: * `String`: parameter `a` * `I`: parameter `b` * `List`: closure parameter for `unboundedList` * `Object`: closure parameter for `value1`. If we build with JDK 1.8 `javac -parameters`, the `MethodParameters` annotation states that the closure parameters are: MyStringList(String a, int b, List val$value1, Object val$unboundedList); which is *wrong*; `unboundedList` is the `List`, `value1` is `Object`! This was fixed in JDK-11, with the `MethodParameters` annotations specifying: MyStringList(String a, int b, List val$unboundedList, Object val$value1); This means that the unit tests need to take this into consideration. Add a new `ConfiguredJdkInfo` class, which contains code similar to `tests/TestJVM/TestJVM.cs`: it will read `bin/Build*/JdkInfo.props` to find the path to the JDK found in `make prepare`, and then determine the JDK version from that directory's `release` file. [0]: https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/util/stream/WhileOps.java#L334
1 parent 7c8d463 commit 8ccb837

15 files changed

+912
-106
lines changed

src/Xamarin.Android.Tools.Bytecode/AttributeInfo.cs

+2
Original file line numberDiff line numberDiff line change
@@ -360,6 +360,8 @@ public string InnerName {
360360
}
361361
}
362362

363+
public string OuterClassName => OuterClass?.Name?.Value;
364+
363365
public override string ToString ()
364366
{
365367
return string.Format ("InnerClass(InnerClass='{0}', OuterClass='{1}', InnerName='{2}', InnerClassAccessFlags={3})",

src/Xamarin.Android.Tools.Bytecode/ClassPath.cs

+31-6
Original file line numberDiff line numberDiff line change
@@ -41,14 +41,22 @@ public ClassPath (string path = null)
4141
Load (path);
4242
}
4343

44-
public void Load (string jarFile)
44+
public void Load (string filePath)
4545
{
46-
if (!IsJarFile (jarFile))
47-
throw new ArgumentException ("'jarFile' is not a valid .jar file.", "jarFile");
48-
49-
using (var jarStream = File.OpenRead (jarFile)) {
50-
Load (jarStream);
46+
if (IsJmodFile (filePath)) {
47+
using (var source = File.OpenRead (filePath)) {
48+
var slice = new PartialStream (source, 4);
49+
Load (slice);
50+
}
51+
return;
52+
}
53+
if (IsJarFile (filePath)) {
54+
using (var jarStream = File.OpenRead (filePath)) {
55+
Load (jarStream);
56+
}
57+
return;
5158
}
59+
throw new ArgumentException ($"`{filePath}` is not a supported file format.", nameof (filePath));
5260
}
5361

5462
public void Load (Stream jarStream, bool leaveOpen = false)
@@ -113,6 +121,23 @@ public static bool IsJarFile (string jarFile)
113121
}
114122
}
115123

124+
public static bool IsJmodFile (string jmodFile)
125+
{
126+
if (jmodFile == null)
127+
throw new ArgumentNullException (nameof (jmodFile));
128+
try {
129+
var f = File.OpenRead (jmodFile);
130+
var h = new byte[4];
131+
if (f.Read (h, 0, h.Length) != 4) {
132+
return false;
133+
}
134+
return h[0] == 0x4a && h[1] == 0x4d && h[2] == 0x01 && h[3] == 0x00;
135+
}
136+
catch (Exception) {
137+
return false;
138+
}
139+
}
140+
116141
XAttribute GetApiSource ()
117142
{
118143
if (string.IsNullOrEmpty (ApiSource))

src/Xamarin.Android.Tools.Bytecode/Log.cs

+8
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ public static void Warning (int verbosity, string format, params object[] args)
1515
log (TraceLevel.Warning, verbosity, format, args);
1616
}
1717

18+
public static void Warning (int verbosity, string message) => Warning (verbosity, "{0}", message);
19+
1820
public static void Error (string format, params object[] args)
1921
{
2022
var log = OnLog;
@@ -23,6 +25,8 @@ public static void Error (string format, params object[] args)
2325
log (TraceLevel.Error, 0, format, args);
2426
}
2527

28+
public static void Error (string message) => Error ("{0}", message);
29+
2630
public static void Message (string format, params object[] args)
2731
{
2832
var log = OnLog;
@@ -31,13 +35,17 @@ public static void Message (string format, params object[] args)
3135
log (TraceLevel.Info, 0, format, args);
3236
}
3337

38+
public static void Message (string message) => Message ("{0}", message);
39+
3440
public static void Debug (string format, params object[] args)
3541
{
3642
var log = OnLog;
3743
if (log == null)
3844
return;
3945
log (TraceLevel.Verbose, 0, format, args);
4046
}
47+
48+
public static void Debug (string message) => Debug ("{0}", message);
4149
}
4250
}
4351

0 commit comments

Comments
 (0)