Loading jnidispatch on Android 15 with 16 KB page size leads to crash

[lisa-bella97]

1. Version of JNA and related jars
   5.16.0@aar
2. Version and vendor of the java virtual machine
   ART, Android 15
   System.getProperty("java.vm.version") = "2.1.0"
3. Operating system
   Android 15 with 16 KB page size
4. System architecture (CPU type, bitness of the JVM)
   arm64-v8a
5. Complete description of the problem
   Loading jnidispatch using System.loadLibrary("jnidispatch") is OK on Android 15 or less with 4 KB page size, but is not OK (leads to SIGSEGV crash) on Android 15 with 16 KB page size (support of this page size in Android 15 is described in docs).
   System.loadLibrary("jnidispatch") is called on Android platform in loadNativeDispatchLibrary function that is called in static block of JNA class Native, so you cannot properly use, for example, Native.load function on Android 15 with 16 KB page size.

To test this behaviour, we used the corresponding Android Studio emulator and Google Pixel 8 and Google Pixel 9 physical devices with 16 KB mode enabled.

Info about emulator:
Emulator version: 35.1.13-11943732 (HVF 14.5.0)
Host machine: macOS 14.5
Api level: 35
Type: Google APIs PlayStore Page Size 16 KB

Info about Google Pixel 8:
Api level: 35
Build number: AP41.240925.009

Info about Google Pixel 9:
Api level: 35
Build number: BP11.241121.010

6. Steps to reproduce
   You can find minimal sample here. Launching this app on Android 15, 16-KB-based emulator or some Google Pixel physical devices with 16 KB mode enabled will lead to SIGSEGV crash, full log with dump.

Related issue: #1618

[matthiasblaesing]

This should have fixed the issue: 17f4e59

It is unclear why it does not and it is unclear why that was not found when testing. This needs someone with interest in Android and willing to see where this goes wrong.

[matthiasblaesing]

I can't reproduce the problem. I modified the onCreate method in the sample app (btw: thanks, very useful) to this:

        Log.d("MY_LOGS", "Loading library jnidispatch")
        System.loadLibrary("jnidispatch") // This causes crash on Android 15 emulator with 16 KB page size
        var p = Runtime.getRuntime().exec(arrayOf("getconf", "PAGE_SIZE"));
        p.waitFor();
        var pageSize = p.inputStream.readAllBytes().toString(Charsets.UTF_8);
        Log.d("MY_LOGS", "jnidispatch is loaded. Version: " + Native.VERSION + " / Version Native: " + Native.VERSION_NATIVE + " / Page size: " + pageSize);

And get the expected result

Loading library jnidispatch
Load /data/app/~~QPQEWTIZJtTKYiMg6KMS_w==/com.example.testapp-8ycvvd3YvKiQlClPrJpUJw==/base.apk!/lib/x86_64/libjnidispatch.so using ns clns-7 from class loader (caller=/data/data/com.example.testapp/code_cache/.overlay/base.apk/classes4.dex): ok
jnidispatch is loaded. Version: 5.16.0 / Version Native: 7.0.3 / Page size: 16384

This looks sane to me. I can not check on arm64 though as the emulator can't be started:

matthias@enterprise:~$ bin/android-sdk-linux_86/emulator/emulator @Pixel_9_API_35_arm64_-_16k_pages
INFO    | Android emulator version 35.2.10.0 (build_id 12414864) (CL:N/A)
INFO    | Graphics backend: gfxstream
INFO    | Found systemPath /home/matthias/bin/android-sdk-linux_86/system-images/android-35/google_apis_ps16k/arm64-v8a/
PANIC: Avd's CPU Architecture 'arm64' is not supported by the QEMU2 emulator on x86_64 host.
matthias@enterprise:~$

[lisa-bella97]

What device (or emulator) do you use for testing? With x64 architecture?

I check your code on my Google Pixel 9 ARM64 physical device with 4 KB page size and 16 KB page size, Runtime.getRuntime().exec(arrayOf("getconf", "PAGE_SIZE")) returned 4096 for 4 KB page size and 16384 for 16 KB page size (as expected), but SIGSEGV crash occured in case of 16 KB page size.

I recorded videos to show this behaviour: https://drive.google.com/drive/folders/12vZfRMl4FKeDgM4lqOTGZLhHY_KK54G-?usp=sharing

[matthiasblaesing]

@lisa-bella97 I used the emulator for x86-64. Google did not only decided to break ABI they also ensured, that it is ugly to debug. The emulator is not able to emulate arm64 on x86-64 (at least it claims so and starting does not work). My Pixel 7a is not among the few blessed devises that can be switched to 16k mode, so that ends my journey.

[rvandermeulen]

FYI, NDK r28 was released today which enables 16KB page size by default. Maybe worth rebuilding with that?
https://github.com/android/ndk/wiki/Changelog-r28

[kingsword09]

I can't reproduce the problem. I modified the onCreate method in the sample app (btw: thanks, very useful) to this:

    Log.d("MY_LOGS", "Loading library jnidispatch")
    System.loadLibrary("jnidispatch") // This causes crash on Android 15 emulator with 16 KB page size
    var p = Runtime.getRuntime().exec(arrayOf("getconf", "PAGE_SIZE"));
    p.waitFor();
    var pageSize = p.inputStream.readAllBytes().toString(Charsets.UTF_8);
    Log.d("MY_LOGS", "jnidispatch is loaded. Version: " + Native.VERSION + " / Version Native: " + Native.VERSION_NATIVE + " / Page size: " + pageSize);

When I call this code in the onCreate, System.loadLibrary("jnidispatch") crashed directly.

D  Loading library jnidispatch
D  Load /data/app/~~jaqWmGyrcRhNN-TDaAtWMw==/io.github.kingsword09.dweblib-u2Sivirptb7GzpkKIiFQOQ==/base.apk!/lib/arm64-v8a/libjnidispatch.so using class loader ns clns-7 (caller=/data/data/io.github.kingsword09.dweblib/code_cache/.overlay/base.apk/classes3.dex): ok
A  Fatal signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x7e479e57e590 in tid 5410 (sword09.dweblib), pid 5410 (sword09.dweblib)

After commenting out System.loadLibrary("jnidispatch"), my log output is also

jnidispatch is loaded. Version: 5.16.0 / Version Native: 7.0.3 / Page size: 16384

my arm64 emulator:

Properties
avd.ini.displayname              Pixel 9 Pro API Baklava
avd.ini.encoding                 UTF-8
AvdId                            Pixel_9_Pro_API_Baklava
disk.dataPartition.size          6442450944
fastboot.chosenSnapshotFile      
fastboot.forceChosenSnapshotBoot no
fastboot.forceColdBoot           no
fastboot.forceFastBoot           yes
hw.accelerometer                 yes
hw.arc                           false
hw.audioInput                    yes
hw.battery                       yes
hw.camera.back                   virtualscene
hw.camera.front                  emulated
hw.cpu.ncore                     4
hw.device.hash2                  MD5:73e7b35d09e3a8055043aca4688e0dad
hw.device.manufacturer           Google
hw.device.name                   pixel_9_pro
hw.dPad                          no
hw.gps                           yes
hw.gpu.enabled                   yes
hw.gpu.mode                      auto
hw.initialOrientation            portrait
hw.keyboard                      yes
hw.lcd.density                   480
hw.lcd.height                    2856
hw.lcd.width                     1280
hw.mainKeys                      no
hw.ramSize                       11548
hw.sdCard                        yes
hw.sensors.orientation           yes
hw.sensors.proximity             yes
hw.trackBall                     no
image.androidVersion.api         34
image.androidVersion.codename    Baklava
image.sysdir.1                   system-images/android-Baklava/google_apis_playstore_ps16k/arm64-v8a/
PlayStore.enabled                true
runtime.network.latency          none
runtime.network.speed            full
showDeviceFrame                  yes
skin.dynamic                     yes
tag.display                      16 KB Page Size
tag.displaynames                 16 KB Page Size,Google APIs PlayStore
tag.id                           page_size_16kb
tag.ids                          page_size_16kb,google_apis_playstore
vm.heapSize                      256

[kingsword09]

@lisa-bella97 I used the emulator for x86-64. Google did not only decided to break ABI they also ensured, that it is ugly to debug. The emulator is not able to emulate arm64 on x86-64 (at least it claims so and starting does not work). My Pixel 7a is not among the few blessed devises that can be switched to 16k mode, so that ends my journey.

When I use an x86-64 emulator, everything works fine.

[rvandermeulen]

@matthiasblaesing Is there a way to get a build created with NDK r28 to see if that improves things?

[matthiasblaesing]

@rvandermeulen currently the android native binaries are build with NDK 12b. If I remember correctly at some point google switched from gcc to llvm. I have doubts, that newer SDKs can currently build JNA.

[BugsBeGone]

Reading the Android docs again , I just noticed this bit:

Note: Due to bugs in some lower versions of ld, you might also need to set common-page-size=16384. However, we strongly recommended that you update your tooling to avoid these bugs.

Oddly, I can't find any usage of common-page-size after NDK r22b, so presumably all mention was removed along with GNU binutils in r23.

[BugsBeGone]

The note above mentioning common-page-size seems to relate to this bug in GNU ld: https://sourceware.org/bugzilla/show_bug.cgi?id=28689

Here's a GH mirror of the fix, since gitweb can be very slow: bminor/binutils-gdb@74e315d

This tracks with what I noticed looking at the ELF alignment in libjnidispatch.so: previous to 5.16, the LOAD section alignment for arm64 was already 64KB. Setting the max page size to 16KB actually reduced the p_align values.

[matthiasblaesing]

@BugsBeGone if you tell me where to modify the Makefile, I'll create an updated build.

[BugsBeGone]

@BugsBeGone if you tell me where to modify the Makefile, I'll create an updated build.

@matthiasblaesing I'd love to test this, but don't have any new enough devices. Since @lisa-bella97 found it on real hardware, maybe they can check on ones we know for sure have crashed already?

I think putting LDFLAGS+=-Wl,-z,common-page-size=16384 somewhere here should fix it. For example:

ifeq ($(ARCH),aarch64)
PREFIX=aarch64-linux-android-
HOST=aarch64-linux-android
AARCH=arm64
ALIBDIR=/usr/lib64
LDFLAGS+=-Wl,-z,common-page-size=16384
else

Also not sure if the previous LDFLAGS fix #1618 belongs in the ifeq ($(OS),android) condition instead of just for ifeq ($(ARCH),aarch64), but maybe it is harmless on other targets? I still don't fully understand the binutils bug, or why it made no difference at all on previous ARM builds.

[BugsBeGone]

Ah, of course, you still want testing on x86-64 VMs to work. As it happens, recent NDKs just always set 64-bit targets for ARM and x86 to default to 16KB: https://android.googlesource.com/platform/ndk/+/02d2d9adccdf353bcea0a678c28b3de62dc2cf33^!/

So perhaps then, the fix is to remove max-page-size from the generic Android LDFLAGS, and instead have:

ifeq ($(ARCH),aarch64)
PREFIX=aarch64-linux-android-
HOST=aarch64-linux-android
AARCH=arm64
ALIBDIR=/usr/lib64
LDFLAGS+=-Wl,-z,common-page-size=16384 -Wl,-z,max-page-size=16384
else
...
ifeq ($(ARCH),x86-64)
PREFIX=x86_64-linux-android-
COPT+= -m64
HOST=x86_64-linux-android
AARCH=x86_64
ALIBDIR=/usr/lib64
LDFLAGS+=-Wl,-z,common-page-size=16384 -Wl,-z,max-page-size=16384
else

Keeping them separate also makes it easier for custom builds to undo the change only for x86-64, if that does causes any problems.

[BugsBeGone]

Not to go off topic, but it can't be a coincidence that the note about common-page-size was only added some time between 2024-12-03 and 2024-12-16. Seems like the NDK devs didn't really test this with old toolchains. But if you want to support older versions of Android, there is no choice.

Edit: I finally found the bug tracker for the docs. Here is the issue that prompted adding the note.

[matthiasblaesing]

@BugsBeGone thank you for the reference and idea. Based on your suggestion I updated the original change to include the common-page-size fix. The change can be found in #1654.

A prebuild binary can be found here: jna.zip (JAR and AAR are both included).

It would be great if the people having access to an aarch64 emulator could test this (and also the ones just doing android development)

[Thomyrock]

Found a way to test it on aarch64 emulator, with the sample of this issue, and it seems to have fixed it, well done @BugsBeGone.

the note about common-page-size was only added some time between 2024-12-03 and 2024-12-16

Interesting, it explains why I didn't see it when doing the first fix attempt.

[zlmr]

Also tested it now with provided jna.aar binary and it finally works now on arm64 emulators running on macOS (M3 MacBook). Previous versions resulted in app crashing on Android 15 with 16KB page size, with same SIGSEGV crash as already reported.

I am using it to load libsodium binaries for Android app, and for now it all seems to work fine on other Android versions as well.
Will do more testing but it seems I can finally start to target Android 15, thank you!!

[BugsBeGone]

Eventually managed to get a 16KB page size x64 VM to work with arm64 libjnidispatch.so in ARM translation mode. For prebuild libs at least, Android Studio tries to be clever, so by default only includes the native libs that exactly match the target. The trick is to manually set the ABIs you want in build.gradle, for example:

android {
    ...
    defaultConfig {
        ...
        ndk {
            abiFilters ("armeabi-v7a", "arm64-v8a")
        }
    }
}

Also, only the Google API & Play Store (typically larger) images support ARM translation: https://android-developers.googleblog.com/2020/03/run-arm-apps-on-android-emulator.html

Edit: If you use API level 28 (9.0) or 30 (11.0) images with ARM translation, they should work with native 32-bit arm & x86 libs as well:

But of course, you can't test the new 16 KB page size requirement on images that old.

[shn1233]

@BugsBeGone thank you for the reference and idea. Based on your suggestion I updated the original change to include the common-page-size fix. The change can be found in #1654.

A prebuild binary can be found here: jna.zip (JAR and AAR are both included).

It would be great if the people having access to an aarch64 emulator could test this (and also the ones just doing android development)

When will the correct 16kb jna version be released? Our app strongly depends on this so～～

[rvandermeulen]

@matthiasblaesing Just curious, is there an ETA on the 5.17.0 release? Not sure what your release cadence normally is 😄

[matthiasblaesing]

Release 5.17.0 is out. I wanted to have a look at some other things, but time ran out, so just a small release.

[rvandermeulen]

Thank you very much! Appreciate all the hard work you do to maintain this dependency!