Getting java.lang.NoSuchMethodException when loading PageFactory elements in Java16+

[ayuryev]

Description

Failed to switch to Java 17 due to issue with running Appium tests:

Caused by:
           java.lang.reflect.InvocationTargetException
               at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
               at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
               at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
               at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
               at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)

               Caused by:
               java.lang.RuntimeException: java.lang.NoSuchMethodException: jdk.proxy2.$Proxy119.proxyClassLookup()
                   at io.appium.java_client.pagefactory.bys.builder.AppiumByBuilder.prepareAnnotationMethods(AppiumByBuilder.java:84)
               Caused by:

                   at io.appium.java_client.pagefactory.bys.builder.AppiumByBuilder.getFilledValue(AppiumByBuilder.java:91)
                   at io.appium.java_client.pagefactory.bys.builder.AppiumByBuilder.createBy(AppiumByBuilder.java:147)
                   at io.appium.java_client.pagefactory.DefaultElementByBuilder.getBys(DefaultElementByBuilder.java:133)
                   at io.appium.java_client.pagefactory.DefaultElementByBuilder.buildMobileNativeBy(DefaultElementByBuilder.java:170)
                   at io.appium.java_client.pagefactory.DefaultElementByBuilder.buildBy(DefaultElementByBuilder.java:204)
                   at io.appium.java_client.pagefactory.AppiumElementLocatorFactory.createLocator(AppiumElementLocatorFactory.java:66)
                   at io.appium.java_client.pagefactory.AppiumElementLocatorFactory.createLocator(AppiumElementLocatorFactory.java:53)
                   at io.appium.java_client.pagefactory.AppiumElementLocatorFactory.createLocator(AppiumElementLocatorFactory.java:1)
                   at org.openqa.selenium.support.pagefactory.DefaultFieldDecorator.decorate(DefaultFieldDecorator.java:56)
                   at io.appium.java_client.pagefactory.AppiumFieldDecorator.decorate(AppiumFieldDecorator.java:154)
                   at org.openqa.selenium.support.PageFactory.proxyFields(PageFactory.java:113)
                   at org.openqa.selenium.support.PageFactory.initElements(PageFactory.java:105)
...
                   at ui.pageobject.ringapp.common.AbstractMobileViewWithHeader.<init>(AbstractMobileViewWithHeader.java:104)
                   at ui.pageobject.ringapp.devicedetail.ActivateSirenPopupView.<init>(ActivateSirenPopupView.java:9)
                   ... 60 more

                   Caused by:
                   java.lang.NoSuchMethodException: jdk.proxy2.$Proxy119.proxyClassLookup()
                       at java.base/java.lang.Class.getMethod(Class.java:2227)
                   Caused by:

                       at io.appium.java_client.pagefactory.bys.builder.AppiumByBuilder.prepareAnnotationMethods(AppiumByBuilder.java:82)
                       ... 76 more

Java 16 also failed. Java 15 is OK.

Environment

• Java client build version or git revision if you use some snapshot: 17
• Appium server version or git revision if you use some snapshot: 1.22.0
• Desktop OS/version used to run Appium if necessary: MacOs Big Sur 11.6
• Node.js version (unless using Appium.app|exe) or Appium CLI or Appium.app|exe: 12.18.3
• Mobile platform/version under test: iOS/Android
• Real device or emulator/simulator: real

[ayuryev]

UPD: the reason of this particular issue with Java 16 and 17 is in the following code: io.appium.java_client.pagefactory.bys.builder.AppiumByBuilder#METHODS_TO_BE_EXCLUDED_WHEN_ANNOTATION_IS_READ
Since Java 16 Proxy class has now new method proxyClassLookup. And this method should be also added to the METHODS_TO_BE_EXCLUDED_WHEN_ANNOTATION_IS_READ.
And the failure happens because this particular method has arguments which is not foreseen by setting DEFAULT_ANNOTATION_METHOD_ARGUMENTS to an empty array.
I'd propose to do as following:

    private static final List<String> METHODS_TO_BE_EXCLUDED_WHEN_ANNOTATION_IS_READ =
        new ArrayList<String>() {
            private static final long serialVersionUID = 1L; {
                List<String> objectClassMethodNames =
                    getMethodNames(Object.class.getDeclaredMethods());
                addAll(objectClassMethodNames);
                List<String> annotationClassMethodNames =
                    getMethodNames(Annotation.class.getDeclaredMethods());
                annotationClassMethodNames.removeAll(objectClassMethodNames);
                addAll(annotationClassMethodNames);
                List<String> proxyClassMethodNames =
                        getMethodNames(Proxy.class.getDeclaredMethods());
                proxyClassMethodNames.removeAll(objectClassMethodNames);
                addAll(proxyClassMethodNames);
            }
        };

It will add proxyClassLookup method (among many others from Proxy class) to the exclusion list and it will avoid failure on annotation invocations.

[mykola-mokhnach]

Thanks for checking this @ayuryev
Feel free to push a PR.

[ayuryev]

UPD2: but then new issue appears - since Java 9 there is more strict rules for encapsulation and especially in Java 17 it is not possible to access encapsulated modules from other libraries (AFAIK). It becomes an issue here in Appium Java Client with cglib:

...
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
...
	... 2 more
Caused by: java.lang.ExceptionInInitializerError
	at io.appium.java_client.pagefactory.utils.ProxyFactory.getEnhancedProxy(ProxyFactory.java:53)
	at io.appium.java_client.pagefactory.utils.ProxyFactory.getEnhancedProxy(ProxyFactory.java:33)
	at io.appium.java_client.pagefactory.AppiumFieldDecorator.proxyForAnElement(AppiumFieldDecorator.java:216)
	at io.appium.java_client.pagefactory.AppiumFieldDecorator.access$0(AppiumFieldDecorator.java:214)
	at io.appium.java_client.pagefactory.AppiumFieldDecorator$1.proxyForLocator(AppiumFieldDecorator.java:107)
	at org.openqa.selenium.support.pagefactory.DefaultFieldDecorator.decorate(DefaultFieldDecorator.java:62)
	at io.appium.java_client.pagefactory.AppiumFieldDecorator.decorate(AppiumFieldDecorator.java:154)
	at org.openqa.selenium.support.PageFactory.proxyFields(PageFactory.java:113)
	at org.openqa.selenium.support.PageFactory.initElements(PageFactory.java:105)
...
	... 8 more
Caused by: net.sf.cglib.core.CodeGenerationException: java.lang.reflect.InaccessibleObjectException-->Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @1e7ba8d9
	at net.sf.cglib.core.ReflectUtils.defineClass(ReflectUtils.java:464)
	at net.sf.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:339)
	at net.sf.cglib.core.AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:96)
	at net.sf.cglib.core.AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:94)
	at net.sf.cglib.core.internal.LoadingCache$2.call(LoadingCache.java:54)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at net.sf.cglib.core.internal.LoadingCache.createEntry(LoadingCache.java:61)
	at net.sf.cglib.core.internal.LoadingCache.get(LoadingCache.java:34)
	at net.sf.cglib.core.AbstractClassGenerator$ClassLoaderData.get(AbstractClassGenerator.java:119)
	at net.sf.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:294)
	at net.sf.cglib.core.KeyFactory$Generator.create(KeyFactory.java:221)
	at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:174)
	at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:153)
	at net.sf.cglib.proxy.Enhancer.<clinit>(Enhancer.java:73)
	... 20 more
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @1e7ba8d9
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:199)
	at java.base/java.lang.reflect.Method.setAccessible(Method.java:193)
	at net.sf.cglib.core.ReflectUtils$1.run(ReflectUtils.java:61)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)
	at net.sf.cglib.core.ReflectUtils.<clinit>(ReflectUtils.java:52)
	at net.sf.cglib.core.KeyFactory$Generator.generateClass(KeyFactory.java:243)
	at net.sf.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
	at net.sf.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:332)
	... 32 more

It makes Appium incompatible with Java 17 by the moment.

[ayuryev]

@mykola-mokhnach cglib (and probably other reflection issues) has to be resolved first I guess. Otherwise it will be hollow fix - it will not make Appium work for Java 17

[ayuryev]

cglib/cglib#191 - known issue
However workaround exists and it works for me - https://dev.to/jjbrt/how-to-make-reflection-fully-work-on-jdk-16-and-later-ihp
@mykola-mokhnach Could you please add me to the list of contributors? Currently I'm not able to push the changes

ERROR: Permission to appium/java-client.git denied to ayuryev.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

[mykola-mokhnach]

Please create a fork of this project and then you'd be able to create a PR from that fork. This is a safer approach.

[mykola-mokhnach]

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork

[ayuryev]

Fixed

[margasan]

@mykola-mokhnach could you merge this fix into 7.6.x version?
Thank you

[mykola-mokhnach]

@margasan We currently aim to release version 8 and will only be supporting that one. Consider creating a custom fork if you'd like to continue using v7

[AlexanderBaravulia]

@mykola-mokhnach Dear Mykola, we've faced with the similar problem on our project, it will be wonderful if it is possible to fix into 7 version. Thank you in advance.

[omprakashchavan01]

@mykola-mokhnach Is this available in current 8.0.0 Beta? I see the fix was done on Nov 5, but 8.0.0 Beta was released prior to that. If this is not available yet, when do you think the next Beta with the fix will be available? I'm just looking for a tentative date. Would be really helpful for me.

[mykola-mokhnach]

We'd like to publish a new beta as soon as there is a new selenium lib patch available. See #1608 for more details

[omprakashchavan01]

@mykola-mokhnach @ayuryev

So I tested 8.0.0 Beta 2 and I have got the same cglib error with Java JDK 17.0.1 which is the LTS.

Am I missing something here?

Please find the exception log below. I get this exception for Selenium's PageFactory.initElements(driver, this) command;

java.lang.ExceptionInInitializerError
at io.appium.java_client.pagefactory.utils.ProxyFactory.getEnhancedProxy(ProxyFactory.java:53)
at io.appium.java_client.pagefactory.utils.ProxyFactory.getEnhancedProxy(ProxyFactory.java:33)
at io.appium.java_client.pagefactory.AppiumFieldDecorator.proxyForAnElement(AppiumFieldDecorator.java:209)
at io.appium.java_client.pagefactory.AppiumFieldDecorator.access$000(AppiumFieldDecorator.java:61)
at io.appium.java_client.pagefactory.AppiumFieldDecorator$1.proxyForLocator(AppiumFieldDecorator.java:100)
at org.openqa.selenium.support.pagefactory.DefaultFieldDecorator.decorate(DefaultFieldDecorator.java:63)
at io.appium.java_client.pagefactory.AppiumFieldDecorator.decorate(AppiumFieldDecorator.java:147)
at org.openqa.selenium.support.PageFactory.proxyFields(PageFactory.java:111)
at org.openqa.selenium.support.PageFactory.initElements(PageFactory.java:103)
at base.BaseTest.(BaseTest.java:98)
at pages.LoginPage.(LoginPage.java:9)
at com.qa.tests.LoginTests.beforeMethod(LoginTests.java:48)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:133)
at org.testng.internal.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:62)
at org.testng.internal.ConfigInvoker.invokeConfigurationMethod(ConfigInvoker.java:385)
at org.testng.internal.ConfigInvoker.invokeConfigurations(ConfigInvoker.java:321)
at org.testng.internal.TestInvoker.runConfigMethods(TestInvoker.java:700)
at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:527)
at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:173)
at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:824)
at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:146)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.testng.TestRunner.privateRun(TestRunner.java:794)
at org.testng.TestRunner.run(TestRunner.java:596)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:377)
at org.testng.SuiteRunner.access$000(SuiteRunner.java:28)
at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:418)
at org.testng.internal.thread.ThreadUtil.lambda$execute$0(ThreadUtil.java:64)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: net.sf.cglib.core.CodeGenerationException: java.lang.reflect.InaccessibleObjectException-->Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @6442b0a6
at net.sf.cglib.core.ReflectUtils.defineClass(ReflectUtils.java:464)
at net.sf.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:339)
at net.sf.cglib.core.AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:96)
at net.sf.cglib.core.AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:94)
at net.sf.cglib.core.internal.LoadingCache$2.call(LoadingCache.java:54)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at net.sf.cglib.core.internal.LoadingCache.createEntry(LoadingCache.java:61)
at net.sf.cglib.core.internal.LoadingCache.get(LoadingCache.java:34)
at net.sf.cglib.core.AbstractClassGenerator$ClassLoaderData.get(AbstractClassGenerator.java:119)
at net.sf.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:294)
at net.sf.cglib.core.KeyFactory$Generator.create(KeyFactory.java:221)
at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:174)
at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:153)
at net.sf.cglib.proxy.Enhancer.(Enhancer.java:73)
... 39 more
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @6442b0a6
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:199)
at java.base/java.lang.reflect.Method.setAccessible(Method.java:193)
at net.sf.cglib.core.ReflectUtils$1.run(ReflectUtils.java:61)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)
at net.sf.cglib.core.ReflectUtils.(ReflectUtils.java:52)
at net.sf.cglib.core.KeyFactory$Generator.generateClass(KeyFactory.java:243)
at net.sf.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
at net.sf.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:332)
... 51 more

[joao-borges]

Hi @mykola-mokhnach sorry for circling back to an old issue. We have an extensive usage of appium-java-client on our application, I'd say more than 1000 classes, and you could imagine the size of the effort is to upgrade to v8.

We already tried and we bumped into some selenium issues, that we already reported, but we are held back on the java upgrade because of this specific issue.

I know you're probably gonna say no, but doesn't hurt to ask, any chance this can be pulled into a patch fix for the 7.x?

I've already talked to my stakeholders and keeping a fork is not an option.

Thanks

[vishala84]

Hi @mykola-mokhnach
We have same issue as @joao-borges .
Any fix given in 7.x?

Thanks

[abdoemad62]

its not working java 21