1.0.0 RC7 breaks support for Micronaut/Netty: No instances are allowed in the image heap

[graemerocher]

With RC7 and attempting to build a native image that previously worked in RC6 results in:

error: No instances are allowed in the image heap for a class that is initialized or reinitialzied at image runtime: sun.security.provider.NativePRNG
Detailed message:
Error: No instances are allowed in the image heap for a class that is initialized or reinitialzied at image runtime: sun.security.provider.NativePRNG
Trace: 	object io.netty.handler.ssl.util.ThreadLocalInsecureRandom
	method io.netty.handler.ssl.util.ThreadLocalInsecureRandom.current()
Call path from entry point to io.netty.handler.ssl.util.ThreadLocalInsecureRandom.current(): 
	at io.netty.handler.ssl.util.ThreadLocalInsecureRandom.current(ThreadLocalInsecureRandom.java:35)
	at io.netty.handler.ssl.util.SelfSignedCertificate.<init>(SelfSignedCertificate.java:110)
	at io.netty.handler.ssl.util.SelfSignedCertificate.<init>(SelfSignedCertificate.java:88)
	at io.netty.handler.ssl.util.SelfSignedCertificate.<init>(SelfSignedCertificate.java:79)
	at io.micronaut.http.server.netty.ssl.NettyServerSslBuilder.build(NettyServerSslBuilder.java:67)
	at io.micronaut.http.server.netty.NettyHttpServer.<init>(NettyHttpServer.java:207)
	at 

Further trace:

com.oracle.svm.reflect.proxies.Proxy_io_micronaut_http_server_netty_NettyHttpServer_constructor_io_micronaut_http_server_netty_configuration_NettyHttpServerConfiguration_io_micronaut_context_ApplicationContext_io_micronaut_web_router_Router_io_micronaut_http_server_binding_RequestArgumentSatisfier_io_micronaut_http_codec_MediaTypeCodecRegistry_io_micronaut_http_server_netty_types_NettyCustomizableResponseTypeHandlerRegistry_io_micronaut_web_router_resource_StaticResourceResolver_java_util_concurrent_ExecutorService_java_util_concurrent_ThreadFactory_io_micronaut_scheduling_executor_ExecutorSelector_io_micronaut_http_server_netty_ssl_NettyServerSslBuilder_java_util_List_430.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at sun.security.x509.OtherName.getGNI(OtherName.java:132)
	at sun.security.x509.OtherName.equals(OtherName.java:184)
	at io.micronaut.caffeine.cache.UnboundedLocalCache$EntrySetView.contains(UnboundedLocalCache.java:729)
	at java.util.AbstractCollection.containsAll(AbstractCollection.java:318)
	at java.util.RegularEnumSet.containsAll(RegularEnumSet.java:199)
	at com.oracle.svm.core.amd64.AMD64CPUFeatureAccess.verifyHostSupportsArchitecture(AMD64CPUFeatureAccess.java:158)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:138)
	at com.oracle.svm.core.code.CEntryPointCallStubs.com_002eoracle_002esvm_002ecore_002eJavaMainWrapper_002erun_0028int_002corg_002egraalvm_002enativeimage_002ec_002etype_002eCCharPointerPointer_0029(generated:0)

Error: Processing image build request failed

Are there any new flags that should be passed to avoid this? Or am I missing something

[graemerocher]

Possibly relates to #712

[cstancu]

Can you try pushing the initialization of ThreadLocalInsecureRandom to runtime using --delay-class-initialization-to-runtime=ThreadLocalInsecureRandom?

[graemerocher]

@cstancu If I do that it now fails with:

error: No instances are allowed in the image heap for a class that is initialized or reinitialzied at image runtime: sun.security.provider.NativePRNG
Detailed message:
Error: No instances are allowed in the image heap for a class that is initialized or reinitialzied at image runtime: sun.security.provider.NativePRNG
Trace: 	object java.security.SecureRandom
	field sun.security.jca.JCAUtil$CachedSecureRandomHolder.instance

Error: Processing image build request failed

If I try add java.security.SecureRandom then it fails with:

error: Class is already initialized, so it is too late to register delaying class initialization: java.security.SecureRandom
Error: Processing image build request failed

[cstancu]

You cannot delay SecureRandom, we need that in the image. You need --rerun-class-initialization-at-runtime=sun.security.jca.JCAUtil$CachedSecureRandomHolder. Notice that this is --rerun-class-initialization-at-runtime instead of --delay-class-initialization-to-runtime since this is a JDK class and might be already initialized during image building.

[graemerocher]

I made the following change:

--- a/app-with-discovery/build-native-image.sh
+++ b/app-with-discovery/build-native-image.sh
@@ -8,4 +8,5 @@ native-image --class-path build/libs/app-with-discovery-0.1-all.jar \
                         -H:Class=app.with.discovery.Application \
                         -H:+ReportUnsupportedElementsAtRuntime \
                         -H:+AllowVMInspection \
-                        --delay-class-initialization-to-runtime=io.netty.handler.codec.http.HttpObjectEncoder,io.netty.handler.codec.http.websocketx.WebSocket00FrameEncoder,io.netty.handler.codec.http.HttpObjectDecoder
+                        --delay-class-initialization-to-runtime=io.netty.handler.codec.http.HttpObjectEncoder,io.netty.handler.codec.http.websocketx.WebSocket00FrameEncoder,io.netty.handler.codec.http.HttpObjectDecoder,io.netty.handler.ssl.util.ThreadLocalInsecureRandom \
+                        --rerun-class-initialization-at-runtime=sun.security.jca.JCAUtil$CachedSecureRandomHolder

But it still fails with the same error:

error: No instances are allowed in the image heap for a class that is initialized or reinitialzied at image runtime: sun.security.provider.NativePRNG
Detailed message:
Error: No instances are allowed in the image heap for a class that is initialized or reinitialzied at image runtime: sun.security.provider.NativePRNG
Trace: 	object java.security.SecureRandom
	field sun.security.jca.JCAUtil$CachedSecureRandomHolder.instance

[cstancu]

Can you point me to the repo/branch to replicate this?

[graemerocher]

@cstancu you can replicate with the steps in #714 just using RC7 instead of RC6

[cstancu]

Trying to build I fail with

$ ./gradlew assemble

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':shadowJar'.
> Could not resolve all dependencies for configuration ':runtimeClasspath'.
   > Could not find io.micronaut:graal:.
     Required by:
         project :

[graemerocher]

@cstancu are you behind a proxy? I cleaned my Maven and Gradle caches and it seems to work fine. You need to able to resolve snapshots from https://oss.sonatype.org/content/repositories/snapshots

[graemerocher]

@cstancu maybe the step to install the Graal SVM dependency is missing?

mvn install:install-file -Dfile=${JAVA_HOME}/jre/lib/svm/builder/svm.jar \
                           -DgroupId=com.oracle.substratevm \
                           -DartifactId=svm \
                           -Dversion=GraalVM-1.0.0-rc6 \
                           -Dpackaging=jar

If you install the RC7 SVM jar you will need to update the reference in the build.gradle

[mavilein]

We are hitting the same issue. We have a reproduction available here. This is the error we are seeing:

[main:498]     analysis:  14,269.80 ms
error: No instances are allowed in the image heap for a class that is initialized or reinitialzied at image runtime: sun.security.provider.NativePRNG
Detailed message:
Error: No instances are allowed in the image heap for a class that is initialized or reinitialzied at image runtime: sun.security.provider.NativePRNG
Trace: 	object java.security.SecureRandom
	object sun.security.ssl.SSLContextImpl$DefaultSSLContext
	object javax.net.ssl.SSLContext
	field javax.net.ssl.SSLContext.defaultContext

Error: Processing image build request failed

I am also happy to jump on a call if this easier for you @cstancu

[cstancu]

@graemerocher I already had the SVM dependency installed. What I needed was to clean the maven and gradle caches. Now ./gradlew assemble builds successfully, however the second step java -cp build/libs/app-with-discovery-0.1-all.jar io.micronaut.graal.reflect.GraalClassLoadingAnalyzer doesn't produce the build/reflect.json:

$ ./gradlew assemble

BUILD SUCCESSFUL in 1s
10 actionable tasks: 10 up-to-date
$ java -cp build/libs/app-with-discovery-0.1-all.jar io.micronaut.graal.reflect.GraalClassLoadingAnalyzer
$ ls build/
classes  distributions  generated  libs  resources  scripts  scriptsShadow  tmp

[cstancu]

@graemerocher never mind, it looks like the GraalClassLoadingAnalyzer tool needs the java from GraalVM.

[cstancu]

@graemerocher the problem is the $ sign on the command line, you need to add the class name between '':-rerun-class-initialization-at-runtime='sun.security.jca.JCAUtil$CachedSecureRandomHolder'. Then it works, I can build the image, however when I run it I get io.micronaut.http.client.exceptions.HttpClientException: Connect Error: Connection refused: localhost/127.0.0.1:8500. Do I need a server running?

[graemerocher]

@cstancu great news, yes the example needs consul running. If you want to try it further you can do

docker run -p 8500:8500 consul

[cstancu]

@mavilein you need --rerun-class-initialization-at-runtime=javax.net.ssl.SSLContext to get past that error and you also need --enable-all-security-services since your code is using the JCA. With this the image builds but it fails at run time due to missing reflection configurations. Please open another issue if you still have problems.

[cstancu]

@graemerocher I tried your app further but it looks like the image is missing some reflection config: io.micronaut.http.codec.CodecException: Error encoding object [io.micronaut.discovery.consul.client.v1.NewServiceEntry@7f2f1d2c4420] to JSON: Class com.fasterxml.jackson.databind.PropertyNamingStrategy$UpperCamelCaseStrategy has no default (no arg) constructor.

[cstancu]

As a resolution for this issue:

• I'll make --rerun-class-initialization-at-runtime='sun.security.jca.JCAUtil$CachedSecureRandomHolder' a default via SecurityServicesFeature. In general any class that has a static reference to a SecureRandom needs to be re-initialized at run time because the SecureRandom implementations open the /dev/random and /dev/urandom files which are used as sources for entropy.
• --delay-class-initialization-to-runtime=ThreadLocalInsecureRandom seems to be needed by netty ssl.
• I'll make --rerun-class-initialization-at-runtime=javax.net.ssl.SSLContext a default for when --enable-all-security-services is used (which is enabled automatically when --enable-https is used).

[graemerocher]

@cstancu btw when I add --rerun-class-initialization-at-runtime=javax.net.ssl.SSLContext a default for when --enable-all-security-services to the app above (just as a test to the impact on it). The application fails to build with:

fatal error: java.lang.NullPointerException
	at com.oracle.svm.hosted.code.CompileQueue.defaultCompileFunction(CompileQueue.java:893)
	at com.oracle.svm.hosted.code.CompileQueue.doCompile(CompileQueue.java:882)
	at com.oracle.svm.hosted.code.CompileQueue$CompileTask.run(CompileQueue.java:257)
	at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:174)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	a

[cstancu]

@graemerocher I cannot replicate the NPE trace that you posted above. I pulled in your latest changes to the micronaut-graal-experiments/app-with-discovery and tried with rc7. Maybe you were using rc6?

[archerz]

@graemerocher @cstancu Are there any updates on this issue?

[graemerocher]

@archerz are you seeing issues with Micronaut 1.0 related to this?

[archerz]

@graemerocher With 1.0.0.RC2. I'm using Graal 1.0.0-rc8.

The command I'm using is:

./gradlew assemble
java -cp build/libs/appbilling-api-all.jar io.micronaut.graal.reflect.GraalClassLoadingAnalyzer
native-image --class-path build/libs/app-all.jar \
			 -H:ReflectionConfigurationFiles=build/reflect.json \
			 -H:EnableURLProtocols=http \
			 -H:IncludeResources="logback.xml|application.yml|META-INF/services/*.*" \
			 -H:Name=app \
			 -H:Class=com.app.App \
			 -H:+ReportUnsupportedElementsAtRuntime \
			 -H:+AllowVMInspection \
			 --rerun-class-initialization-at-runtime=io.netty.handler.ssl.util.ThreadLocalInsecureRandom \
			 --delay-class-initialization-to-runtime= \
			    io.netty.handler.codec.http.HttpObjectEncoder, \
			    io.netty.handler.ssl.util.ThreadLocalInsecureRandom, \
			    io.netty.handler.codec.http.websocketx.WebSocket00FrameEncoder

And the error:

error: unsupported features in 2 methods
Detailed message:
Error: Detected a direct/mapped ByteBuffer in the image heap. A direct ByteBuffer has a pointer to unmanaged C memory, and C memory from the image generator is not available at image run time. A mapped ByteBuffer references a file descriptor, which is no longer open and mapped at run time. The object was probably created by a class initializer and is reachable from a static field. By default, all class initialization is done during native image building.You can manually delay class initialization to image run time by using the option --delay-class-initialization-to-runtime=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Trace:  object io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf
        object io.netty.buffer.UnreleasableByteBuf
        method io.netty.handler.codec.http.HttpObjectEncoder.encodeChunkedContent(ChannelHandlerContext, Object, long, List)
Call path from entry point to io.netty.handler.codec.http.HttpObjectEncoder.encodeChunkedContent(ChannelHandlerContext, Object, long, List): 
        at io.netty.handler.codec.http.HttpObjectEncoder.encodeChunkedContent(HttpObjectEncoder.java:196)
        at io.netty.handler.codec.http.HttpObjectEncoder.encode(HttpObjectEncoder.java:169)
        at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:88)
        at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:738)
        at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:730)
        at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:38)
        at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1081)
        at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1070)
        at java.lang.Thread.run(Thread.java:748)
        at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:476)
        at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:199)
        at com.oracle.svm.core.code.CEntryPointCallStubs.com_002eoracle_002esvm_002ecore_002eposix_002ethread_002ePosixJavaThreads_002epthreadStartRoutine_0028com_002eoracle_002esvm_002ecore_002ethread_002eJavaThreads_0024ThreadStartData_0029(generated:0)
Error: No instances are allowed in the image heap for a class that is initialized or reinitialized at image runtime: sun.security.provider.NativePRNG
Trace:  object io.netty.handler.ssl.util.ThreadLocalInsecureRandom
        method io.netty.handler.ssl.util.ThreadLocalInsecureRandom.current()
Call path from entry point to io.netty.handler.ssl.util.ThreadLocalInsecureRandom.current(): 
        at io.netty.handler.ssl.util.ThreadLocalInsecureRandom.current(ThreadLocalInsecureRandom.java:35)
        at io.netty.handler.ssl.util.SelfSignedCertificate.<init>(SelfSignedCertificate.java:110)
        at io.netty.handler.ssl.util.SelfSignedCertificate.<init>(SelfSignedCertificate.java:88)
        at io.netty.handler.ssl.util.SelfSignedCertificate.<init>(SelfSignedCertificate.java:79)
        at io.micronaut.http.server.netty.ssl.NettyServerSslBuilder.build(NettyServerSslBuilder.java:67)
        at io.micronaut.http.server.netty.NettyHttpServer.<init>(NettyHttpServer.java:208)
        at com.oracle.svm.reflect.proxies.Proxy_io_micronaut_http_server_netty_NettyHttpServer_constructor_io_micronaut_http_server_netty_configuration_NettyHttpServerConfiguration_io_micronaut_context_ApplicationContext_io_micronaut_web_router_Router_io_micronaut_http_server_binding_RequestArgumentSatisfier_io_micronaut_http_codec_MediaTypeCodecRegistry_io_micronaut_http_server_netty_types_NettyCustomizableResponseTypeHandlerRegistry_io_micronaut_web_router_resource_StaticResourceResolver_java_util_concurrent_ExecutorService_java_util_concurrent_ThreadFactory_io_micronaut_scheduling_executor_ExecutorSelector_io_micronaut_http_server_netty_ssl_NettyServerSslBuilder_java_util_List_522.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at sun.security.x509.OtherName.getGNI(OtherName.java:132)
        at sun.security.x509.OtherName.equals(OtherName.java:184)
        at java.util.HashMap.getNode(HashMap.java:579)
        at java.util.HashMap.get(HashMap.java:557)
        at com.oracle.svm.jni.access.JNIReflectionDictionary.getFieldNameByID(JNIReflectionDictionary.java:150)
        at com.oracle.svm.jni.functions.JNIFunctions.ToReflectedField(JNIFunctions.java:839)
        at com.oracle.svm.core.code.CEntryPointCallStubs.com_002eoracle_002esvm_002ejni_002efunctions_002eJNIFunctions_002eToReflectedField_0028com_002eoracle_002esvm_002ejni_002enativeapi_002eJNIEnvironment_002ccom_002eoracle_002esvm_002ejni_002enativeapi_002eJNIObjectHandle_002ccom_002eoracle_002esvm_002ejni_002enativeapi_002eJNIFieldId_0029(generated:0)

Error: Processing image build request failed

[graemerocher]

@archerz Try 1.0.0 final create a new project and use 1.0 RC7 of Graal. For RC8 came out the same day as 1.0.0 of Micronaut so we have some updates to do for that due to some changes with META-INF/services handling

[graemerocher]

Although some folks have got Micronaut working with RC8, think you need to pass -H:-UseServiceLoaderFeature additional flag to the compiler. See https://gitlab.com/puravida-software/hello-micronaut-graalvm/blob/master/.gitlab-ci.yml

[cstancu]

I'll close this since it is an old issue that should be fixed in the latest GraalVM RC. Please reopen or open a new issue if you still have problems.