Unable to retrolambda scala 2.12.0-M2: not a method descriptor: scala/Serializable

[pfn]

I realize processing scala isn't a primary use-case, but I am toying with getting it (scala) to continue working on Android. This is currently a showstopper as I don't know what the correct behavior in this situation should be (ignore the exception? return itself?)

[error] Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalArgumentException: not a method descriptor: scala/Serializable
[error]         at net.orfjackal.retrolambda.lambdas.LambdaReifier.reifyLambdaClass(LambdaReifier.java:42)
[error]         at net.orfjackal.retrolambda.lambdas.BackportLambdaInvocations$InvokeDynamicInsnConverter.backportLambda(BackportLambdaInvocations.java:118)
[error]         at net.orfjackal.retrolambda.lambdas.BackportLambdaInvocations$InvokeDynamicInsnConverter.visitInvokeDynamicInsn(BackportLambdaInvocations.java:107)
[error]         at net.orfjackal.retrolambda.asm.ClassReader.readCode(ClassReader.java:1452)
[error]         at net.orfjackal.retrolambda.asm.ClassReader.readMethod(ClassReader.java:1017)
[error]         at net.orfjackal.retrolambda.asm.ClassReader.accept(ClassReader.java:693)
[error]         at net.orfjackal.retrolambda.asm.ClassReader.accept(ClassReader.java:506)
[error]         at net.orfjackal.retrolambda.Transformers.lambda$transform$4(Transformers.java:106)
[error]         at net.orfjackal.retrolambda.Transformers$$Lambda$2/1724731843.accept(Unknown Source)
[error]         at net.orfjackal.retrolambda.Transformers.transform(Transformers.java:121)
[error]         at net.orfjackal.retrolambda.Transformers.transform(Transformers.java:106)
[error]         at net.orfjackal.retrolambda.Transformers.backportClass(Transformers.java:46)
[error]         at net.orfjackal.retrolambda.Retrolambda.run(Retrolambda.java:77)
[error]         at android.RetroMain$.main(retrolambda.scala:95)
[error]         at android.RetroMain.main(retrolambda.scala)
[error] Caused by: java.lang.IllegalArgumentException: not a method descriptor: scala/Serializable
[error]         at java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:910)
[error]         at net.orfjackal.retrolambda.lambdas.Types.toMethodType(Types.java:26)
[error]         at net.orfjackal.retrolambda.lambdas.Types.asmToJdkType(Types.java:17)
[error]         at net.orfjackal.retrolambda.lambdas.LambdaReifier.callBootstrapMethod(LambdaReifier.java:106)
[error]         at net.orfjackal.retrolambda.lambdas.LambdaReifier.reifyLambdaClass(LambdaReifier.java:37)
[error]         ... 14 more
[info] [error] Retrolambda failure: exit 1

[pfn]

If I try to just return 'arg' if the exception occurs, this error ensues. Don't know if it's related, or if it's further issues trying to retrolambda scala-library. The stack makes me think it's related, though.

Exception in thread "main" java.lang.RuntimeException: java.lang.ArrayStoreException
        at net.orfjackal.retrolambda.lambdas.LambdaReifier.reifyLambdaClass(LambdaReifier.java:42)
        at net.orfjackal.retrolambda.lambdas.BackportLambdaInvocations$InvokeDynamicInsnConverter.backportLambda(BackportLambdaInvocations.java:118)
        at net.orfjackal.retrolambda.lambdas.BackportLambdaInvocations$InvokeDynamicInsnConverter.visitInvokeDynamicInsn(BackportLambdaInvocations.java:107)
        at net.orfjackal.retrolambda.asm.ClassReader.readCode(ClassReader.java:1452)
        at net.orfjackal.retrolambda.asm.ClassReader.readMethod(ClassReader.java:1017)
        at net.orfjackal.retrolambda.asm.ClassReader.accept(ClassReader.java:693)
        at net.orfjackal.retrolambda.asm.ClassReader.accept(ClassReader.java:506)
        at net.orfjackal.retrolambda.Transformers.lambda$transform$5(Transformers.java:106)
        at net.orfjackal.retrolambda.Transformers$$Lambda$2/2143192188.accept(Unknown Source)
        at net.orfjackal.retrolambda.Transformers.transform(Transformers.java:121)
        at net.orfjackal.retrolambda.Transformers.transform(Transformers.java:106)
        at net.orfjackal.retrolambda.Transformers.backportClass(Transformers.java:46)
        at net.orfjackal.retrolambda.Retrolambda.run(Retrolambda.java:77)
        at android.RetroMain$.main(retrolambda.scala:95)
        at android.RetroMain.main(retrolambda.scala)
Caused by: java.lang.ArrayStoreException
        at java.lang.System.arraycopy(Native Method)
        at java.lang.invoke.LambdaMetafactory.altMetafactory(LambdaMetafactory.java:441)
        at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
        at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:633)
        at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:658)
        at net.orfjackal.retrolambda.lambdas.LambdaReifier.callBootstrapMethod(LambdaReifier.java:110)
        at net.orfjackal.retrolambda.lambdas.LambdaReifier.reifyLambdaClass(LambdaReifier.java:37)
        ... 14 more

[luontola]

Please modify net.orfjackal.retrolambda.lambdas.BackportLambdaInvocations.InvokeDynamicInsnConverter#visitInvokeDynamicInsn so that it will print all the arguments of the visitInvokeDynamicInsn method, in order to find out what is the instruction that is causing this problem.

This could be related to scala.Serializable extending java.io.Serializable. Retrolambda supports serializable lambdas since version 1.7.0, but maybe there is a corner case that I've missed.

[pfn]

Input directory:  C:\Users\pfnguyen\src\sbt\sbt-test\android-sdk-plugin\hello-world-scala-2.12\target\android-bin\retrolambda
Output directory: C:\Users\pfnguyen\src\sbt\sbt-test\android-sdk-plugin\hello-world-scala-2.12\target\android-bin\retrolambda
Classpath:        C:\Users\pfnguyen\android-sdk-windows\platforms\android-17\android.jar;C:\Users\pfnguyen\.ivy2\cache\org.scala-lang\scala-library\jars\scala-library-2.12.0-M2.jar;C:\Users\pfnguyen\src\sbt\sbt-test\android-sdk-plugin\hello-world-scala-2.12\target\android-bin\classes.jar
Included files:   3032
name: apply, desc: (Lscala/collection/immutable/Stream$;Ljava/lang/Object;Lscala/Function1;)Lscala/runtime/java8/JFunction0;, handle: java/lang/invoke/LambdaMetafactory.altMetafactory(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite; (6), args: ()Ljava/lang/Object;,scala/collection/immutable/Stream$.scala$collection$immutable$Stream$$$anonfun$26(Ljava/lang/Object;Lscala/Function1;)Lscala/collection/immutable/Stream; (5),()Lscala/collection/immutable/Stream;,3,1,Lscala/Serializable;,0
Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalArgumentException: not a method descriptor: scala/Serializable

[pfn]

System.out.println(String.format("name: %s, desc: %s, handle: %s, args: %s", name, desc, bsm, Joiner.on(",").join(bsmArgs)));

[luontola]

Seems like the asmToJdkType doesn't properly handle Class parameters to the metafactory. According to altMetafactory's docs that's related to marker interfaces (i.e. additional interfaces for the lambda to implement). Should be simple to fix.

[luontola]

This is fixed in Retrolambda 2.0.5.

I tested backporting scala-library-2.12.0-M2 and with went through without errors. I didn't try running Scala applications. I'm curious to hear whether you succeed in your effors. Maybe you could write a blog post about it. ;)

Due to Scala's use of default methods and Retrolambda's limitations of processing them, you will probably need to unpack scala-library and your application classes to a single directory, so that you can process them all together with Retrolambda. This could be made easier with #52.

[pfn]

You mean #44? :)

Anyway, I gave this a whirl, the retrolambda processing ran through fine, however I ended up with code that still referenced some stuff out of java.lang.invoke.*, etc. (e.g. java.lang.invoke.MethodHandles)

I had to ignore classes like

  "-dontwarn java.lang.invoke.**" ::
  "-dontwarn java.lang.FunctionalInterface" ::
  "-dontwarn java.lang.ReflectiveOperationException" ::

And eventually got an error while dexing:

EXCEPTION FROM SIMULATION:
local variable type mismatch: attempt to set or access a value of type java.lang.Object using a local variable of type int. This is symptomatic of .class transformation tools that ignore local variable information.

...at bytecode offset 00000020
locals[0000]: Lscala/collection/immutable/Vector;
locals[0001]: [Ljava/lang/Object;
locals[0002]: I
locals[0003]: [Ljava/lang/Object;
locals[0004]: Lscala/compat/Platform$;
locals[0005]: [Ljava/lang/Object;
locals[0006]: I
locals[0007]: [Ljava/lang/Object;
locals[0008]: I
locals[0009]: I
...while working on block 001e
...while working on method copyLeft:([Ljava/lang/Object;I)[Ljava/lang/Object;
...while processing copyLeft ([Ljava/lang/Object;I)[Ljava/lang/Object;
...while processing scala/collection/immutable/Vector.class

Thanks, but it looks like I'm stuck.

[luontola]

Sounds like the dexer doesn't like some of the bytecode generated by the Scala compiler, in addition to which Scala uses Java 8 APIs. It might be possible to write a tool that processes the bytecode to fix that local variable information. Good luck if you're planning on continuing. :)