Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Crashes on startup significantly increased after migration to .NET6 #7335

Closed
tipa opened this issue Sep 5, 2022 · 41 comments · Fixed by #7732
Closed

Crashes on startup significantly increased after migration to .NET6 #7335

tipa opened this issue Sep 5, 2022 · 41 comments · Fixed by #7732
Assignees
Labels
Area: App Runtime Issues in `libmonodroid.so`.

Comments

@tipa
Copy link

tipa commented Sep 5, 2022

Android application type

Android for .NET (net6.0-android, etc.)

Affected platform version

VS 2022 17.3

Description

After I ported my Xamarin app to .NET6 and publishing it to the store, I noticed a significant increase in startup crashes reported in the Play Store developer console:

  #00  pc 0x0000000000051894  /apex/com.android.runtime/lib64/bionic/libc.so (abort)
  #00  pc 0x0000000000029b1c  /data/app/~~x4yqp23VTm2FT1FLIxOjlg==/myapp.name-joOsfd1aZ0MWgfx31__DBw==/split_config.arm64_v8a.apk!libmono-android.release.so (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*))
  #00  pc 0x00000000002680bc  /data/app/~~x4yqp23VTm2FT1FLIxOjlg==/myapp.name-joOsfd1aZ0MWgfx31__DBw==/split_config.arm64_v8a.apk!libmonosgen-2.0.so
  #00  pc 0x00000000002681e8  /data/app/~~x4yqp23VTm2FT1FLIxOjlg==/myapp.name-joOsfd1aZ0MWgfx31__DBw==/split_config.arm64_v8a.apk!libmonosgen-2.0.so
  #00  pc 0x0000000000161ae8  /data/app/~~x4yqp23VTm2FT1FLIxOjlg==/myapp.name-joOsfd1aZ0MWgfx31__DBw==/split_config.arm64_v8a.apk!libmonosgen-2.0.so
  #00  pc 0x00000000001dac28  /data/app/~~x4yqp23VTm2FT1FLIxOjlg==/myapp.name-joOsfd1aZ0MWgfx31__DBw==/split_config.arm64_v8a.apk!libmonosgen-2.0.so
  #00  pc 0x00000000001dbe64  /data/app/~~x4yqp23VTm2FT1FLIxOjlg==/myapp.name-joOsfd1aZ0MWgfx31__DBw==/split_config.arm64_v8a.apk!libmonosgen-2.0.so
  #00  pc 0x00000000001ee468  /data/app/~~x4yqp23VTm2FT1FLIxOjlg==/myapp.name-joOsfd1aZ0MWgfx31__DBw==/split_config.arm64_v8a.apk!libmonosgen-2.0.so
  #00  pc 0x0000000000004bf8 

I have also experienced these crashes locally on my own devices, e.g. when launching the app repeatedly while profiling using this script.
The crash seems to appear randomly, that means it can crash even when it previously launched successfully and the app will launch successfully even after the crash happened.

This is the logcat output of when the crash happened locally on my own device:

2022-09-02 06:17:59.197 2727-2727/? E/myapp.name: Unknown bits set in runtime_flags: 0x20000
2022-09-02 06:17:59.206 2727-2727/? E/myapp.name: Not starting debugger since process cannot load the jdwp agent.
2022-09-02 06:17:59.363 604-611/? E/cutils: Nothing there yet; let's create it: /storage/emulated/0/Android/data/myapp.name
2022-09-02 06:17:59.364 604-611/? E/cutils: Nothing there yet; let's create it: /storage/emulated/0/Android/data/myapp.name/cache
2022-09-02 06:18:00.272 2727-3092/? A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 3092 (.NET ThreadPool), pid 2727 (myapp.name)
2022-09-02 06:18:00.348 3109-3109/? A/DEBUG: pid: 2727, tid: 3092, name: .NET ThreadPool  >>> myapp.name <<<
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #01 pc 0000000000029b1c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmono-android.release.so (offset 0x103d000) (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+144) (BuildId: 29c5a3805a0bedee1eede9b6668d7c676aa63371)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #02 pc 00000000002680bc  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #03 pc 00000000002681e8  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #04 pc 000000000008555c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (mono_metadata_string_heap+188) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #05 pc 00000000000cade4  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #06 pc 00000000000cb164  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #07 pc 0000000000045878  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #08 pc 00000000000440d0  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #09 pc 0000000000043d94  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #10 pc 000000000003db64  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (mono_class_get_field+44) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #11 pc 00000000001dd374  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #12 pc 00000000001dfc88  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #13 pc 00000000001dab24  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #14 pc 00000000001dbe64  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
2022-09-02 06:18:00.544 3109-3109/? A/DEBUG:       #15 pc 00000000001ee468  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)

I also found it weird that when I ported multiple of my apps to .NET6, first only one app was affected by this increased crash rate.
Later on I pushed an update to an app that was not affected previously and it then started to show the same problem and the crashes jumped significantly. So maybe the process how the app bundle was built has some influence on this. I use both the stable as well as the Preview version of VS 22 and create the app bundle using the IDE and the "Archive..." menu button

Steps to Reproduce

Unfortunately I have no way to reliably reproduce this crash, but when automating app launches and launching the app many times (e.g. using this) I can occasionally see the crash in the logcat output.

@tipa tipa added Area: App Runtime Issues in `libmonodroid.so`. needs-triage Issues that need to be assigned. labels Sep 5, 2022
@grendello
Copy link
Contributor

@tipa unfortunately, the log fragments don't include enough information. We can see that it's a failed assertion, most likely int he runtime, but we can't see where it happens and what's the
actual message. We will need a more detailed logcat output, alas. On the devices where you can reproduce the issue, could you make sure these commands are ran:

$ adb shell setprop debug.mono.log default,assembly,mono_log_level=debug,mono_log_mask=all
$ adb logcat -G 16M
$ adb logcat -c
# Run the app here and when it crashes, invoke
$ adb logcat -d > log.txt

@grendello grendello added need-info Issues that need more information from the author. and removed needs-triage Issues that need to be assigned. labels Sep 5, 2022
@ghost
Copy link

ghost commented Sep 5, 2022

Hi @tipa. We have added the "need-info" label to this issue, which indicates that we have an open question for you before we can take further action. This issue will be closed automatically in 7 days if we do not hear back from you by then - please feel free to re-open it if you come back to this issue after that time.

@tipa
Copy link
Author

tipa commented Sep 6, 2022

@grendello thanks for the info on how to produced more detailed output. The log.txt contained this part:

--------- beginning of crash
09-06 10:41:36.004  3269  3975 F libc    : Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 3975 (.NET ThreadPool), pid 3269 (partl.Diarium)
09-06 10:41:36.015  3269  3977 E         : AOT Runtime could not load method due to Method not found: !!0 System.Text.Json.JsonSerializer.Deserialize<!0>(System.ReadOnlySpan`1<byte>,System.Text.Json.JsonSerializerOptions) Due to: Signature claims method has generic parameters, but generic_params table says it doesn't for method 0x00000332 from image System.Text.Json.dll
09-06 10:41:36.016  1548  1548 V SettingsProvider: Notifying for 0: content://settings/system/rading_mode_status_auto
09-06 10:41:36.017  2634 14535 I OIMC    : notified , mode ColorReadMode changeTo 2
09-06 10:41:36.017  2634  2681 I OIMC_CORE: handleMessage:   MODE_EXIT, arg1: 0, arg2: 0, obj: com.oneplus.server.oimc.gwy@7d43ca6
09-06 10:41:36.017  2634  2681 I OIMC_CORE: The mode: ColorReadMode is not entered
09-06 10:41:36.018  2634 14535 I OIMC    : notified , mode ReadMode changeTo 2
09-06 10:41:36.018  2634  2681 I OIMC_CORE: handleMessage:   MODE_EXIT, arg1: 0, arg2: 0, obj: com.oneplus.server.oimc.gwy@e3c96e7
09-06 10:41:36.018  2634  2681 I OIMC_CORE: The mode: ReadMode is not entered
09-06 10:41:36.018  1548  1548 V SettingsProvider: Notifying for 0: content://settings/system/rading_mode_status_auto
09-06 10:41:36.044  3981  3981 I crash_dump64: obtaining output fd from tombstoned, type: kDebuggerdTombstone
09-06 10:41:36.044  1250  1250 I /system/bin/tombstoned: received crash request for pid 3975
09-06 10:41:36.045  3981  3981 I crash_dump64: performing dump of process 3269 (target tid = 3975)
09-06 10:41:36.053  3981  3981 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
09-06 10:41:36.053  3981  3981 F DEBUG   : Build fingerprint: 'OnePlus/OnePlus5/OnePlus5:10/QKQ1.191014.012/2010292059:user/release-keys'
09-06 10:41:36.053  3981  3981 F DEBUG   : Revision: '0'
09-06 10:41:36.053  3981  3981 F DEBUG   : ABI: 'arm64'
09-06 10:41:36.054  3981  3981 F DEBUG   : Timestamp: 2022-09-06 10:41:36+0200
09-06 10:41:36.054  3981  3981 F DEBUG   : pid: 3269, tid: 3975, name: .NET ThreadPool  >>> partl.Diarium <<<
09-06 10:41:36.054  3981  3981 F DEBUG   : uid: 10766
09-06 10:41:36.054  3981  3981 F DEBUG   : signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
09-06 10:41:36.054  3981  3981 F DEBUG   :     x0  0000000000000000  x1  0000000000000f87  x2  0000000000000006  x3  00000072ef530250
09-06 10:41:36.054  3981  3981 F DEBUG   :     x4  0080000000000000  x5  0080000000000000  x6  0080000000000000  x7  0000000000008000
09-06 10:41:36.054  3981  3981 F DEBUG   :     x8  00000000000000f0  x9  608d0c77eedf1b8d  x10 0000000000000001  x11 0000000000000000
09-06 10:41:36.054  3981  3981 F DEBUG   :     x12 fffffff0fffffbdf  x13 0000000000000030  x14 ffffffffffffffff  x15 0000c8b7ded40c7a
09-06 10:41:36.054  3981  3981 F DEBUG   :     x16 00000073ea2098c0  x17 00000073ea1e5900  x18 00000072ef002000  x19 0000000000000cc5
09-06 10:41:36.054  3981  3981 F DEBUG   :     x20 0000000000000f87  x21 00000000ffffffff  x22 00000072fa2dea6c  x23 00000072f9f10000
09-06 10:41:36.054  3981  3981 F DEBUG   :     x24 00000073ed8a8640  x25 00000072fa3ab240  x26 0000000000000001  x27 00000072fa2b35b8
09-06 10:41:36.054  3981  3981 F DEBUG   :     x28 0000000000000028  x29 00000072ef5302f0
09-06 10:41:36.054  3981  3981 F DEBUG   :     sp  00000072ef530230  lr  00000073ea1970c4  pc  00000073ea1970f0
09-06 10:41:36.131   595   595 I hwservicemanager: getTransport: Cannot find entry vendor.qti.hardware.iop@2.0::IIop/default in either framework or device manifest.
09-06 10:41:36.131  1055 30963 E ANDR-IOP: IIop:: Iop HAL Service is not available.
09-06 10:41:36.143  3981  3981 F DEBUG   : 
09-06 10:41:36.143  3981  3981 F DEBUG   : backtrace:
09-06 10:41:36.143  3981  3981 F DEBUG   :       #00 pc 00000000000830f0  /apex/com.android.runtime/lib64/bionic/libc.so (abort+160) (BuildId: e55e6e4c631509598633769798683023)
09-06 10:41:36.143  3981  3981 F DEBUG   :       #01 pc 0000000000029b1c  /data/app/partl.Diarium-p2XRiJL6KmXM75MHlE41hQ==/split_config.arm64_v8a.apk!libmono-android.release.so (offset 0xb37000) (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+144) (BuildId: 29c5a3805a0bedee1eede9b6668d7c676aa63371)
09-06 10:41:36.143  3981  3981 F DEBUG   :       #02 pc 00000000002680bc  /data/app/partl.Diarium-p2XRiJL6KmXM75MHlE41hQ==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0xb95000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
09-06 10:41:36.143  3981  3981 F DEBUG   :       #03 pc 0000000000268144  /data/app/partl.Diarium-p2XRiJL6KmXM75MHlE41hQ==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0xb95000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
09-06 10:41:36.143  3981  3981 F DEBUG   :       #04 pc 00000000001dc9e0  /data/app/partl.Diarium-p2XRiJL6KmXM75MHlE41hQ==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0xb95000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
09-06 10:41:36.143  3981  3981 F DEBUG   :       #05 pc 00000000001dfc88  /data/app/partl.Diarium-p2XRiJL6KmXM75MHlE41hQ==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0xb95000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
09-06 10:41:36.143  3981  3981 F DEBUG   :       #06 pc 00000000001dab24  /data/app/partl.Diarium-p2XRiJL6KmXM75MHlE41hQ==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0xb95000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
09-06 10:41:36.144  3981  3981 F DEBUG   :       #07 pc 00000000001dbe64  /data/app/partl.Diarium-p2XRiJL6KmXM75MHlE41hQ==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0xb95000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
09-06 10:41:36.144  3981  3981 F DEBUG   :       #08 pc 00000000001db500  /data/app/partl.Diarium-p2XRiJL6KmXM75MHlE41hQ==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0xb95000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
09-06 10:41:36.144  3981  3981 F DEBUG   :       #09 pc 0000000000162e24  /data/app/partl.Diarium-p2XRiJL6KmXM75MHlE41hQ==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0xb95000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
09-06 10:41:36.144  3981  3981 F DEBUG   :       #10 pc 0000000000162ae4  /data/app/partl.Diarium-p2XRiJL6KmXM75MHlE41hQ==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0xb95000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
09-06 10:41:36.144  3981  3981 F DEBUG   :       #11 pc 00000000001edf78  /data/app/partl.Diarium-p2XRiJL6KmXM75MHlE41hQ==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0xb95000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
09-06 10:41:36.144  3981  3981 F DEBUG   :       #12 pc 00000000001eda9c  /data/app/partl.Diarium-p2XRiJL6KmXM75MHlE41hQ==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0xb95000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
09-06 10:41:36.144  3981  3981 F DEBUG   :       #13 pc 00000000000042f8  <anonymous:734c588000>

Appears to be some problem with AOT and System.Text.Json

@ghost ghost added need-attention A xamarin-android contributor needs to review and removed need-info Issues that need more information from the author. labels Sep 6, 2022
@tipa
Copy link
Author

tipa commented Sep 6, 2022

Running it again produces a similar but different stack trace:

09-06 10:54:18.553 11416 12275 E         : AOT Runtime could not load method due to Could not load file or assembly 'System.Private.CoreLib, Version=6.0.0.0, Culture=£D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010, PublicKeyToken=7cec85d7bea7798e' or one of its dependencies.
--------- beginning of crash
09-06 10:54:18.554 11416 12275 F libc    : Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 12275 (.NET ThreadPool), pid 11416 (partl.Diarium)
09-06 10:54:18.575  1548  1548 V SettingsProvider: Notifying for 0: content://settings/system/rading_mode_status_auto
09-06 10:54:18.576  2634 14535 I OIMC    : notified , mode ColorReadMode changeTo 2
09-06 10:54:18.577  2634  2681 I OIMC_CORE: handleMessage:   MODE_EXIT, arg1: 0, arg2: 0, obj: com.oneplus.server.oimc.gwy@7239afe
09-06 10:54:18.577  2634  2681 I OIMC_CORE: The mode: ColorReadMode is not entered
09-06 10:54:18.577  2634 14535 I OIMC    : notified , mode ReadMode changeTo 2
09-06 10:54:18.578  2634  2681 I OIMC_CORE: handleMessage:   MODE_EXIT, arg1: 0, arg2: 0, obj: com.oneplus.server.oimc.gwy@1d6925f
09-06 10:54:18.578  2634  2681 I OIMC_CORE: The mode: ReadMode is not entered
09-06 10:54:18.578 11416 12274 E         : AOT Runtime could not load method due to Could not resolve type with token 01000150 from typeref (expected class 'System.Text.Json.JsonSerializerOptions' in assembly 'System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51') assembly:System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 type:System.Text.Json.JsonSerializerOptions member:(null)

log.txt

Edit: Another few tests and it shows this now:

2022-09-06 10:59:01.206 14770-15174/? E/: AOT Runtime could not load method due to Could not resolve type with token 01000366 from typeref (expected class 'System.Text.Json.JsonSerializer' in assembly 'System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51') assembly:System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 type:System.Text.Json.JsonSerializer member:(null)

So all somehow related to System.Text.Json.
Is there any quick fix I can do to prevent these crashes? I am currently using profiled AOT with a custom AOT profile. AndroidLinkMode is set to Full

@grendello
Copy link
Contributor

grendello commented Sep 6, 2022

@tipa in your log I spotted this weird thing:

09-06 10:54:18.553 11416 12275 W monodroid-assembly: Assembly '£D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010/System.Private.CoreLib' (hash 0xdb08fc0a5fd4f2ea) not found
09-06 10:54:18.553 11416 12275 W monodroid-assembly: open_from_bundles: failed to load assembly £D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010/System.Private.CoreLib
09-06 10:54:18.553 11416 12275 W monodroid-assembly: Assembly '£D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010/System.Private.CoreLib' (hash 0xdb08fc0a5fd4f2ea) not found
09-06 10:54:18.553 11416 12275 W monodroid-assembly: open_from_bundles: failed to load assembly £D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010/System.Private.CoreLib
09-06 10:54:18.553 11416 12275 E         : AOT Runtime could not load method due to Could not load file or assembly 'System.Private.CoreLib, Version=6.0.0.0, Culture=£D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010, PublicKeyToken=7cec85d7bea7798e' or one o
f its dependencies.

Note the assembly name: £D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010/System.Private.CoreLib - the part before / should not be there. The runtime fails to load it because I doubt we saw this kind of
name at the build time when we process assemblies. Would you be able to come up with a small repro for this issue? I'd need to examine the generated code.

Edit: the part before / is normally treated as culture name, which it definitely is not what it is in this instance

@grendello
Copy link
Contributor

@tipa also, would you be able to find full log output of this crash? This is likely a failed assertion or abort() as well, but I'd like to see the message (which should be somewhere above the SIGABRT line)

@tipa
Copy link
Author

tipa commented Sep 6, 2022

Not sure what part exactly I should include in a small repro...
Are you sure the problem is caused by the line with System.Private.CoreLib? In my other crashes there is no mention of this, e.g. here:

09-06 10:59:00.942 14770 14770 W monodroid-gc: GREF GC Threshold: 46080
09-06 10:59:01.009 14770 14770 V Font    : Change font:1
09-06 10:59:01.009 14770 14770 V Font    : Default family:android.graphics.Typeface@91f6875e
09-06 10:59:01.094  3120  3334 D MainServiceImplHandler: : handleMessage msg.what = 1002
09-06 10:59:01.095  3120  3334 D MainServiceImplHandler: : EVENT_STATE_INIT
09-06 10:59:01.138  1548  1587 W AlarmManager: Unrecognized alarm listener com.android.server.b@f9a8816
09-06 10:59:01.139  2636  2636 D KeyguardUpdateMonitor: received broadcast android.intent.action.BATTERY_CHANGED
09-06 10:59:01.189 14770 15171 E         : AOT Runtime could not load method due to Could not resolve type with token 01000366 from typeref (expected class 'System.Text.Json.JsonSerializer' in assembly 'System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51') assembly:System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 type:System.Text.Json.JsonSerializer member:(null)
--------- beginning of crash
09-06 10:59:01.190 14770 15171 F libc    : Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 15171 (.NET ThreadPool), pid 14770 (partl.Diarium)
09-06 10:59:01.206 14770 15174 E         : AOT Runtime could not load method due to Could not resolve type with token 01000366 from typeref (expected class 'System.Text.Json.JsonSerializer' in assembly 'System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51') assembly:System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 type:System.Text.Json.JsonSerializer member:(null)
09-06 10:59:01.215  1548  1548 V SettingsProvider: Notifying for 0: content://settings/system/rading_mode_status_auto
09-06 10:59:01.216  2634  2691 I OIMC    : notified , mode ColorReadMode changeTo 2
09-06 10:59:01.216  2634  2681 I OIMC_CORE: handleMessage:   MODE_EXIT, arg1: 0, arg2: 0, obj: com.oneplus.server.oimc.gwy@a902d36
09-06 10:59:01.216  2634  2681 I OIMC_CORE: The mode: ColorReadMode is not entered
09-06 10:59:01.216  2634  2691 I OIMC    : notified , mode ReadMode changeTo 2
09-06 10:59:01.216  2634  2681 I OIMC_CORE: handleMessage:   MODE_EXIT, arg1: 0, arg2: 0, obj: com.oneplus.server.oimc.gwy@aede437
09-06 10:59:01.216  2634  2681 I OIMC_CORE: The mode: ReadMode is not entered
09-06 10:59:01.217  1548  1548 V SettingsProvider: Notifying for 0: content://settings/system/rading_mode_status_auto
09-06 10:59:01.227 15178 15178 I crash_dump64: obtaining output fd from tombstoned, type: kDebuggerdTombstone
09-06 10:59:01.228  1250  1250 I /system/bin/tombstoned: received crash request for pid 15171
09-06 10:59:01.229 15178 15178 I crash_dump64

@tipa also, would you be able to find full log output of this crash? This is likely a failed assertion or abort() as well, but I'd like to see the message (which should be somewhere above the SIGABRT line)

I don't have the full output of that particular crash any more, but as I am now able to reproduce it faster, I did another log:
log.txt

@grendello
Copy link
Contributor

Not sure what part exactly I should include in a small repro... Are you sure the problem is caused by the line with System.Private.CoreLib? In my other crashes there is no mention of this, e.g. here:

As sure as I can be without seeing the app source (or at least its obj/ build directory) - the "culture" name is definitely invalid and I doubt it was this particular string at the build time. Something's broken there.

09-06 10:59:01.189 14770 15171 E         : AOT Runtime could not load method due to Could not resolve type with token 01000366 from typeref (expected class 'System.Text.Json.JsonSerializer' in assembly 'System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51') assembly:System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 type:System.Text.Json.JsonSerializer member:(null)

The above line and another one:

09-06 11:15:06.831 16500 17035 E         : AOT Runtime could not load method due to Method not found: !!0 System.Text.Json.JsonSerializer.Deserialize<!0>(string,System.Text.Json.JsonSerializerOptions)

suggest that it's a problem with the linker, it is probably linking out too much. You might try turning off linking completely, just to see if the app works then.

@tipa
Copy link
Author

tipa commented Sep 6, 2022

What would be the best way to share my obj folder with you?

I also assumed some problem with the linker, but turning it off completely is not an option for my app (to be uploaded in the store) as it would increase app size too much. I will do some tests to see if excluding (parts of) the System.Text.Json assembly would fix the problem.
The weird thing however is: The app doesn't crash every time, just like every 10th to 20th launch. And when it launches, the app works correctly. So the code must be in the app and not have been linked away.

@grendello
Copy link
Contributor

@tipa are you on the DotNetEvolution Discord server? If yes, I'm grendel there - DM me for details on how to send obj/ contents to me

Regarding the intermittent crashes - are they crashes of the same build of the app, or do you build it each time and one of those instances crashes?

@tipa
Copy link
Author

tipa commented Sep 8, 2022

I was now able to create a repro (-> Neuer_Ordner.zip) to reproduce the crash with. The project folder contains a "profile.ps1" script that helps mass-launching the app and with that, reproduce the crash ever 20th or 30th launch or so.
It appears to be some race condition problem as I was able to workaround the crash by not creating tasks all at once during startup, but awaiting them one after the other.
Please let me know if there are any additional info you need from my side.

@tipa
Copy link
Author

tipa commented Sep 19, 2022

@grendello Please let me know if there's anything I can help with for this issue to get resolved. Several of my Android apps are still crashing at a very high rate after migrating them to a .NET6 project - without much that I can do against it. Thanks!

@grendello
Copy link
Contributor

@tipa I hope next week will bring something new, thanks for your patience!

@jpobst jpobst added this to the .NET 7 milestone Sep 19, 2022
@yunusefendi52
Copy link

yunusefendi52 commented Oct 8, 2022

I also use custom profile with net 6 and hit similar issue but mine was can't find ctor (or something) but for System.Formats.Asn1, System.Memory and sometimes System.Private.CoreLib, but when I remove the "System" from custom profile the app is not crashing

remove System from custom profile using https://github.com/radekdoulik/aotprofile-tool

aotprofile-tool -sd --filter-module='^(?!System)' custom.aprof -o startup.aotprofile

logcat showing like:

 Assertion at /__w/1/s/src/mono/mono/mini/mini-runtime.c:1490, condition `is_ok (error)' not met, function:mono_resolve_patch_target_ext, Could not load type of field 'System.Formats.Asn1.AsnWriter:_nestingStack' (2) due to: Could not resolve type with token 01000004 from typeref (expected class 'System.ValueTyp?' in assembly 'System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e') assembly:System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e type:System.ValueTyp? member:(null) assembly:System.Formats.Asn1.dll type:AsnWriter member:(null)
10-08 08:45:33.453 13173 13213 F libc    : Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 13213 (.NET ThreadPool), pid 13173 (dyalabs.keresto)
10-08 08:45:33.465 13173 13215 E         : * Assertion at /__w/1/s/src/mono/mono/mini/mini-runtime.c:1490, condition `is_ok (error)' not met, function:mono_resolve_patch_target_ext, Could not load type of field 'System.Formats.Asn1.AsnWriter:_nestingStack' (2) due to: Could not resolve type with token 01000004 from typeref (expected class 'System.ValueTyp?' in assembly 'System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e') assembly:System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e type:System.ValueTyp? member:(null) assembly:System.Formats.Asn1.dll type:AsnWriter member:(null)
10-08 08:45:33.515 13237 13237 I crash_dump64: obtaining output fd from tombstoned, type: kDebuggerdTombstone

UPDATE:
just got another one, restarting app multiple times really fast

Can't find custom attr constructor image: System.Memory.dll mtoken: 0x0a00008b due to: Could not resolve type with token 0100002a from typeref (expected class 'System.Runtime.Versioning.TargetFramework in assembly 'System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e') assembly:System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e type:System.Runtime.Versioning.TargetFrameworkmember:(null)

@tranb3r
Copy link

tranb3r commented Oct 28, 2022

I'm also experiencing similar random crashes with AOT.
It only happens with <AndroidEnableProfiledAot>False</AndroidEnableProfiledAot> (full aot).
No crash with <AndroidEnableProfiledAot>True</AndroidEnableProfiledAot> (default profile).

Here are a few errors I've encountered (each one being a different crash):

Assertion at /__w/1/s/src/mono/mono/metadata/metadata.c:1101, condition `index < meta->heap_strings.size' not met, function:mono_metadata_string_heap,  index = 0x0000b411 size = 0x000088b0
Assertion at /__w/1/s/src/mono/mono/metadata/metadata.c:3384, condition `mono_class_get_generic_container (container_class)->type_argc == inst->type_argc' not met
AOT Runtime could not load method due to Could not load file or assembly 'System.Private.CoreLib, Version=6.0.0.0, Culture=�D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010, PublicKeyToken=7cec85d7bea7798e' or one of its dependencies.
AOT Runtime could not load method due to Could not find method 'Serialize' due to a type load error: Could not load file or assembly 'System.Private.CoreLib, Version=6.0.0.0, Culture=�D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010, PublicKeyToken=7cec85d7bea7798e' or one of its dependencies. assembly:System.Text.Json.dll type:JsonSerializer member:(null) assembly:System.Text.Json.dll type:JsonSerializer member:(null)
Assertion at /__w/1/s/src/mono/mono/metadata/class.c:6426, condition `*sig == 0x06' not met

@grendello grendello assigned lambdageek and unassigned grendello Nov 7, 2022
@grendello grendello added Area: Mono Runtime Mono-related issues: BCL bugs, AOT issues, etc. and removed Area: App Runtime Issues in `libmonodroid.so`. need-attention A xamarin-android contributor needs to review labels Nov 7, 2022
@grendello
Copy link
Contributor

I'm sorry for the delayed response, we're getting back to this issue. To summarize, here are all the asserts/errors mentioned above:

AOT Runtime could not load method due to Method not found: !!0 System.Text.Json.JsonSerializer.Deserialize<!0>(System.ReadOnlySpan`1<byte>,System.Text.Json.JsonSerializerOptions) Due to: Signature claims method has generic parameters, but generic_params table says it doesn't for method 0x00000332 from image System.Text.Json.dll

AOT Runtime could not load method due to Could not resolve type with token 01000366 from typeref (expected class 'System.Text.Json.JsonSerializer' in assembly 'System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51') assembly:System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 type:System.Text.Json.JsonSerializer member:(null)

AOT Runtime could not load method due to Could not load file or assembly 'System.Private.CoreLib, Version=6.0.0.0, Culture=£D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010, PublicKeyToken=7cec85d7bea7798e' or one of its dependencies.
Assembly '£D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010/System.Private.CoreLib' (hash 0xdb08fc0a5fd4f2ea) not found

AOT Runtime could not load method due to Method not found: !!0 System.Text.Json.JsonSerializer.Deserialize<!0>(string,System.Text.Json.JsonSerializerOptions)

* Assertion at /__w/1/s/src/mono/mono/mini/mini-runtime.c:1490, condition `is_ok (error)' not met, function:mono_resolve_patch_target_ext, Could not load type of field 'System.Formats.Asn1.AsnWriter:_nestingStack' (2) due to: Could not resolve type with token 01000004 from typeref (expected class 'System.ValueTyp?' in assembly 'System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e') assembly:System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e type:System.ValueTyp? member:(null) assembly:System.Formats.Asn1.dll type:AsnWriter member:(null)

Can't find custom attr constructor image: System.Memory.dll mtoken: 0x0a00008b due to: Could not resolve type with token 0100002a from typeref (expected class 'System.Runtime.Versioning.TargetFramework in assembly 'System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e') assembly:System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e type:System.Runtime.Versioning.TargetFrameworkmember:(null)

Assertion at /__w/1/s/src/mono/mono/metadata/metadata.c:1101, condition `index < meta->heap_strings.size' not met, function:mono_metadata_string_heap,  index = 0x0000b411 size = 0x000088b0

Assertion at /__w/1/s/src/mono/mono/metadata/metadata.c:3384, condition `mono_class_get_generic_container (container_class)->type_argc == inst->type_argc' not met

AOT Runtime could not load method due to Could not load file or assembly 'System.Private.CoreLib, Version=6.0.0.0, Culture=�D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010, PublicKeyToken=7cec85d7bea7798e' or one of its dependencies.

AOT Runtime could not load method due to Could not find method 'Serialize' due to a type load error: Could not load file or assembly 'System.Private.CoreLib, Version=6.0.0.0, Culture=�D0067CAD9A63E0813759A2BB841051CA73570C0DA2E08E840A8EB45DB6A7A010, PublicKeyToken=7cec85d7bea7798e' or one of its dependencies. assembly:System.Text.Json.dll type:JsonSerializer member:(null) assembly:System.Text.Json.dll type:JsonSerializer member:(null)

Assertion at /__w/1/s/src/mono/mono/metadata/class.c:6426, condition `*sig == 0x06' not met

@lambdageek would you mind taking a look? I'll try to get more info from the repro app.

@grendello
Copy link
Contributor

I ran the repro application 100 times with the current Xamarin.Android/main (net8) and got a single crash:

* Assertion at /__w/1/s/src/mono/mono/metadata/exception.c:172, condition `method' not met

create_exception_two_strings
/__w/1/s/src/mono/mono/metadata/exception.c:172

0xd23d4
mono_exception_from_name_two_strings_checked
/__w/1/s/src/mono/mono/metadata/exception.c:236

0xd4844
mono_corlib_exception_new_with_args
/__w/1/s/src/mono/mono/metadata/exception.c:1271

0x179a70
mono_error_prepare_exception
/__w/1/s/src/mono/mono/utils/mono-error.c:0

0x179e84
mono_error_convert_to_exception
/__w/1/s/src/mono/mono/utils/mono-error.c:793

0x1d3fe8
mono_jit_compile_method_inner
/__w/1/s/src/mono/mono/mini/mini.c:4147

0x1d86bc
mono_jit_compile_method_with_opt
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2707
jit_compile_method_with_opt_cb
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2762
jit_compile_method_with_opt
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2778

0x1d7bd4
mono_jit_compile_method
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2797

0x265f88
common_call_trampoline
/__w/1/s/src/mono/mono/mini/mini-trampolines.c:618

0x265aec
mono_magic_trampoline
/__w/1/s/src/mono/mono/mini/mini-trampolines.c:759

@grendello
Copy link
Contributor

Full logcat output for all the crashes above:
crash-logs.zip

@yunusefendi52
Copy link

@grendello do you still need stacktrace? just built the apk and I always got the crash on my device (xiaomi redmi note 7), I am using net 8 hoping that it would not crash but it's still crashing

@grendello
Copy link
Contributor

grendello commented Nov 7, 2022

@yunusefendi52 yes, please - the more the better! Please also attach libmonosgen-2.0.so from the apk (for the arm64 architecture)

@yunusefendi52
Copy link

yunusefendi52 commented Nov 7, 2022

here you go @grendello
logcat-d.zip

btw, I am using full AOT

@grendello
Copy link
Contributor

here you go @grendello logcat-d.zip

btw, I am using full AOT

The crash here is (two concurrent aborts, it appears):

Could not load signature of System.IO.Compression.GZipStream:CopyToAsync due to: Could not resolve type with token 0100001e from typeref (expected class 'System.Threading.JancellationToken' in assembly 'System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e') assembly:System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e type:System.Threading.JancellationToken member:(null)
 * Assertion at /__w/1/s/src/mono/mono/mini/mini-runtime.c:1556, condition `is_ok (error)' not met, function:mono_resolve_patch_target_ext, VTable setup of type System.IO.Compression.GZipStream failed assembly:System.IO.Compression.dll type:GZipStream member:(null)
Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 23584 (.NET TP Worker), pid 23548 (mycoreappmobile)
 * Assertion at /__w/1/s/src/mono/mono/mini/mini-runtime.c:1556, condition `is_ok (error)' not met, function:mono_resolve_patch_target_ext, VTable setup of type System.IO.Compression.GZipStream failed assembly:System.IO.Compression.dll type:GZipStream member:(null)

0x2db7a8
monoeg_g_logstr
/__w/1/s/src/mono/mono/eglib/goutput.c:151
monoeg_g_logv_nofree
/__w/1/s/src/mono/mono/eglib/goutput.c:166

0x2db8d4
monoeg_assertion_message
/__w/1/s/src/mono/mono/eglib/goutput.c:207

0x1d66a4
mono_resolve_patch_target_ext
/__w/1/s/src/mono/mono/mini/mini-runtime.c:1556

0x251de0
init_method
/__w/1/s/src/mono/mono/mini/aot-runtime.c:4653

0x252fac
load_method
/__w/1/s/src/mono/mono/mini/aot-runtime.c:4242

0x266268
mono_aot_trampoline
/__w/1/s/src/mono/mono/mini/mini-trampolines.c:876

@grendello
Copy link
Contributor

Note, all the crashes so far have been on aarch64 devices.

@grendello
Copy link
Contributor

I ran the test app in an x86_64 emulator (100 runs), got 5 crashes:

3 segfaults are the same:

Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x726b42522098 in tid 10762 (.NET ThreadPool), pid 10734 (me.Neuer_Ordner)

0x175c46
get_basic_blocks
/__w/1/s/src/mono/mono/mini/method-to-ir.c:4849
mono_method_to_ir
/__w/1/s/src/mono/mono/mini/method-to-ir.c:6440

0x15db60
mini_method_compile
/__w/1/s/src/mono/mono/mini/mini.c:3460

0x16026a
mono_jit_compile_method_inner
/__w/1/s/src/mono/mono/mini/mini.c:4081

0x1648f2
mono_jit_compile_method_with_opt
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2645
jit_compile_method_with_opt_cb
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2700
jit_compile_method_with_opt
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2716

0x163d08
mono_jit_compile_method
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2735

0x1f98e2
common_call_trampoline
/__w/1/s/src/mono/mono/mini/mini-trampolines.c:630

0x1f93ae
mono_magic_trampoline
/__w/1/s/src/mono/mono/mini/mini-trampolines.c:771

2 sigabrt are the same as well:

Method 'System.Text.Json.Reflection.ReflectionExtensions:IsNullableOfT (System.Type)' in assembly 'System.Text.Json.dll' contains native code that cannot be executed by Mono on this platform. The assembly was probably c
reated using C++/CLI.
Method 'System.Text.Json.Reflection.ReflectionExtensions:IsNullableOfT (System.Type)' in assembly 'System.Text.Json.dll' contains native code that cannot be executed by Mono on this platform. The assembly was probably c
reated using C++/CLI.
Method 'System.Text.Json.Reflection.ReflectionExtensions:IsNullableOfT (System.Type)' in assembly 'System.Text.Json.dll' contains native code that cannot be executed by Mono on this platform. The assembly was probably c
reated using C++/CLI.
 * Assertion at /__w/1/s/src/mono/mono/metadata/marshal-ilgen.c:6723, condition `piinfo->addr' not met
 * Assertion at /__w/1/s/src/mono/mono/metadata/marshal-ilgen.c:6723, condition `piinfo->addr' not met
Method 'System.Text.Json.Reflection.ReflectionExtensions:IsNullableOfT (System.Type)' in assembly 'System.Text.Json.dll' contains native code that cannot be executed by Mono on this platform. The assembly was probably c
reated using C++/CLI.
 * Assertion at /__w/1/s/src/mono/mono/metadata/marshal-ilgen.c:6723, condition `piinfo->addr' not met

0x29625a
monoeg_g_logstr
/__w/1/s/src/mono/mono/eglib/goutput.c:151
monoeg_g_logv_nofree
/__w/1/s/src/mono/mono/eglib/goutput.c:166

0x2963e4
monoeg_assertion_message
/__w/1/s/src/mono/mono/eglib/goutput.c:207

0x296426
mono_assertion_message
/__w/1/s/src/mono/mono/eglib/goutput.c:226

0xefe65
emit_native_icall_wrapper_ilgen
/__w/1/s/src/mono/mono/metadata/marshal-ilgen.c:6723

0x7d63c
mono_marshal_get_native_wrapper
/__w/1/s/src/mono/mono/metadata/marshal.c:3360

0x1642b3
compile_special
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2406
mono_jit_compile_method_with_opt
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2610
jit_compile_method_with_opt_cb
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2700
jit_compile_method_with_opt
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2716

0x163d08
mono_jit_compile_method
/__w/1/s/src/mono/mono/mini/mini-runtime.c:2735

0x1f98e2
common_call_trampoline
/__w/1/s/src/mono/mono/mini/mini-trampolines.c:630

0x1f93ae
mono_magic_trampoline
/__w/1/s/src/mono/mono/mini/mini-trampolines.c:771

@Manish-Pradhan-FP
Copy link

Any updates on this?

@vargaz
Copy link
Contributor

vargaz commented Dec 8, 2022

Did some debugging, and it appears that when xamarin-android calls
mono_image_open_from_data_alc () from embedded-assemblies.cc, for most assemblies, the byte array is the same, but for System.Text.Json.dll, the byte array is different from run to run, or even differs between calls since from_data_alc () can be called multiple times for the same assemblies because the test case is multi threaded.

@tranb3r
Copy link

tranb3r commented Jan 13, 2023

Any update?

@zeyangl
Copy link

zeyangl commented Jan 15, 2023

Any news on this?

@grendello
Copy link
Contributor

PR #7732 fixes the issue for me. I ran the test app provided by @tipa 5000 times in quick succession (on Pixel 6 Pro) without any SIGABRT being signaled.

Before that fix lands, you might work around it by disabling assembly compression (set the AndroidEnableAssemblyCompression MSBuild property to False)

Commit d236af5 introduced embedded assembly compression,
using the lz4 algorithm, which speeds up startup and reduces final package size.

Assemblies are compressed at build time and, at the same time, pre-allocated buffers for
the decompressed data are allocated in libxamarin-app.so. The buffers are then passed
to the LZ4 APIs, all threads using the same output buffer. The assumption was that we can
do fine without locking as even if overlapped decompression happens, the output data will be
the same and so even if two threads do the same thing at the same time, the data will be valid
at all times, so long as at least one thread completes the decompression.

This assumption proved to be largely true, but it appears that in high concurrency cases it
is possible that the data in decompression buffer differs. My guess is that LZ4 either uses the
output buffer as a scratchpad area when decompressing or that it initializes/modifies the buffer
before writing actual data in it. With overlapped decompression, it may lead to one thread
overwriting valid data previously written by another thread, so that when the latter returns the
buffer it thought to have had valid data may contain certain bytes temporarily overwritten by the
decompression session in the other, still running, thread. It may happen that MonoVM reads the
corrupted data just when it is still invalid (before the still running decompression session actually
writes the valid data), a classic race condition.

To fix this, the decompression block is now protected with a startup-aware mutex. Mutex will be held
only after the initial startup phase is completed, so there should not be much loss of startup
performance.

@grendello grendello added Area: App Runtime Issues in `libmonodroid.so`. and removed Area: Mono Runtime Mono-related issues: BCL bugs, AOT issues, etc. labels Jan 24, 2023
jonpryor pushed a commit that referenced this issue Jan 26, 2023
…7732)

Fixes: #7335

Context: d236af5

Commit d236af5 introduced embedded assembly compression, using LZ4,
which speeds up startup and reduces final package size.

Assemblies are compressed at build time and, at the same time, pre-
allocated buffers for the **decompressed** data are allocated in
`libxamarin-app.so`.  The buffers are then passed to the LZ4 APIs,
all threads using the same output buffer.  The assumption was that we
can do fine without locking as even if overlapped decompression
happens, the output data will be the same and so even if two threads
do the same thing at the same time, the data will be valid at all
times, so long as at least one thread completes the decompression.

This assumption proved to be **largely** true, but it appears that in
high concurrency cases it is possible that the data in the
decompression buffer differs.  This can result in app crashes:

	A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 3092 (.NET ThreadPool), pid 2727 (myapp.name)
	A/DEBUG: pid: 2727, tid: 3092, name: .NET ThreadPool  >>> myapp.name <<<
	A/DEBUG:       #1 pc 0000000000029b1c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmono-android.release.so (offset 0x103d000) (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+144) (BuildId: 29c5a3805a0bedee1eede9b6668d7c676aa63371)
	A/DEBUG:       #2 pc 00000000002680bc  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       #3 pc 00000000002681e8  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       #4 pc 000000000008555c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (mono_metadata_string_heap+188) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	…

My guess is that LZ4 either uses the output buffer as a scratchpad
area when decompressing or that it initializes/modifies the buffer
before writing actual data in it.  With overlapped decompression, it
may lead to one thread overwriting valid data previously written by
another thread, so that when the latter returns the buffer it thought
to have had valid data may contain certain bytes temporarily
overwritten by the decompression session in the other, still running,
thread.  It may happen that MonoVM reads the corrupted data just when
it is still invalid (before the still running decompression session
actually writes the valid data), a classic race condition.

To fix this, the decompression block is now protected with a startup-
aware mutex.  Mutex will be held only after the initial startup phase
is completed, so there should not be much loss of startup performance.
jonpryor pushed a commit to jonpryor/xamarin-android that referenced this issue Jan 26, 2023
…otnet#7732)

Fixes: dotnet#7335

Context: d236af5

Commit d236af5 introduced embedded assembly compression, using LZ4,
which speeds up startup and reduces final package size.

Assemblies are compressed at build time and, at the same time, pre-
allocated buffers for the **decompressed** data are allocated in
`libxamarin-app.so`.  The buffers are then passed to the LZ4 APIs,
all threads using the same output buffer.  The assumption was that we
can do fine without locking as even if overlapped decompression
happens, the output data will be the same and so even if two threads
do the same thing at the same time, the data will be valid at all
times, so long as at least one thread completes the decompression.

This assumption proved to be **largely** true, but it appears that in
high concurrency cases it is possible that the data in the
decompression buffer differs.  This can result in app crashes:

	A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 3092 (.NET ThreadPool), pid 2727 (myapp.name)
	A/DEBUG: pid: 2727, tid: 3092, name: .NET ThreadPool  >>> myapp.name <<<
	A/DEBUG:       #1 pc 0000000000029b1c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmono-android.release.so (offset 0x103d000) (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+144) (BuildId: 29c5a3805a0bedee1eede9b6668d7c676aa63371)
	A/DEBUG:       #2 pc 00000000002680bc  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       dotnet#3 pc 00000000002681e8  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       dotnet#4 pc 000000000008555c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (mono_metadata_string_heap+188) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	…

My guess is that LZ4 either uses the output buffer as a scratchpad
area when decompressing or that it initializes/modifies the buffer
before writing actual data in it.  With overlapped decompression, it
may lead to one thread overwriting valid data previously written by
another thread, so that when the latter returns the buffer it thought
to have had valid data may contain certain bytes temporarily
overwritten by the decompression session in the other, still running,
thread.  It may happen that MonoVM reads the corrupted data just when
it is still invalid (before the still running decompression session
actually writes the valid data), a classic race condition.

To fix this, the decompression block is now protected with a startup-
aware mutex.  Mutex will be held only after the initial startup phase
is completed, so there should not be much loss of startup performance.
jonpryor pushed a commit to jonpryor/xamarin-android that referenced this issue Jan 27, 2023
…otnet#7732)

Fixes: dotnet#7335

Context: d236af5

Commit d236af5 introduced embedded assembly compression, using LZ4,
which speeds up startup and reduces final package size.

Assemblies are compressed at build time and, at the same time, pre-
allocated buffers for the **decompressed** data are allocated in
`libxamarin-app.so`.  The buffers are then passed to the LZ4 APIs,
all threads using the same output buffer.  The assumption was that we
can do fine without locking as even if overlapped decompression
happens, the output data will be the same and so even if two threads
do the same thing at the same time, the data will be valid at all
times, so long as at least one thread completes the decompression.

This assumption proved to be **largely** true, but it appears that in
high concurrency cases it is possible that the data in the
decompression buffer differs.  This can result in app crashes:

	A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 3092 (.NET ThreadPool), pid 2727 (myapp.name)
	A/DEBUG: pid: 2727, tid: 3092, name: .NET ThreadPool  >>> myapp.name <<<
	A/DEBUG:       #1 pc 0000000000029b1c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmono-android.release.so (offset 0x103d000) (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+144) (BuildId: 29c5a3805a0bedee1eede9b6668d7c676aa63371)
	A/DEBUG:       #2 pc 00000000002680bc  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       dotnet#3 pc 00000000002681e8  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       dotnet#4 pc 000000000008555c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (mono_metadata_string_heap+188) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	…

My guess is that LZ4 either uses the output buffer as a scratchpad
area when decompressing or that it initializes/modifies the buffer
before writing actual data in it.  With overlapped decompression, it
may lead to one thread overwriting valid data previously written by
another thread, so that when the latter returns the buffer it thought
to have had valid data may contain certain bytes temporarily
overwritten by the decompression session in the other, still running,
thread.  It may happen that MonoVM reads the corrupted data just when
it is still invalid (before the still running decompression session
actually writes the valid data), a classic race condition.

To fix this, the decompression block is now protected with a startup-
aware mutex.  Mutex will be held only after the initial startup phase
is completed, so there should not be much loss of startup performance.
jonpryor pushed a commit to jonpryor/xamarin-android that referenced this issue Jan 27, 2023
…otnet#7732)

Fixes: dotnet#7335

Context: d236af5

Commit d236af5 introduced embedded assembly compression, using LZ4,
which speeds up startup and reduces final package size.

Assemblies are compressed at build time and, at the same time, pre-
allocated buffers for the **decompressed** data are allocated in
`libxamarin-app.so`.  The buffers are then passed to the LZ4 APIs,
all threads using the same output buffer.  The assumption was that we
can do fine without locking as even if overlapped decompression
happens, the output data will be the same and so even if two threads
do the same thing at the same time, the data will be valid at all
times, so long as at least one thread completes the decompression.

This assumption proved to be **largely** true, but it appears that in
high concurrency cases it is possible that the data in the
decompression buffer differs.  This can result in app crashes:

	A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 3092 (.NET ThreadPool), pid 2727 (myapp.name)
	A/DEBUG: pid: 2727, tid: 3092, name: .NET ThreadPool  >>> myapp.name <<<
	A/DEBUG:       #1 pc 0000000000029b1c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmono-android.release.so (offset 0x103d000) (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+144) (BuildId: 29c5a3805a0bedee1eede9b6668d7c676aa63371)
	A/DEBUG:       #2 pc 00000000002680bc  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       dotnet#3 pc 00000000002681e8  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       dotnet#4 pc 000000000008555c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (mono_metadata_string_heap+188) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	…

My guess is that LZ4 either uses the output buffer as a scratchpad
area when decompressing or that it initializes/modifies the buffer
before writing actual data in it.  With overlapped decompression, it
may lead to one thread overwriting valid data previously written by
another thread, so that when the latter returns the buffer it thought
to have had valid data may contain certain bytes temporarily
overwritten by the decompression session in the other, still running,
thread.  It may happen that MonoVM reads the corrupted data just when
it is still invalid (before the still running decompression session
actually writes the valid data), a classic race condition.

To fix this, the decompression block is now protected with a startup-
aware mutex.  Mutex will be held only after the initial startup phase
is completed, so there should not be much loss of startup performance.
jonpryor pushed a commit that referenced this issue Jan 28, 2023
…7732)

Fixes: #7335

Context: d236af5

Commit d236af5 introduced embedded assembly compression, using LZ4,
which speeds up startup and reduces final package size.

Assemblies are compressed at build time and, at the same time, pre-
allocated buffers for the **decompressed** data are allocated in
`libxamarin-app.so`.  The buffers are then passed to the LZ4 APIs,
all threads using the same output buffer.  The assumption was that we
can do fine without locking as even if overlapped decompression
happens, the output data will be the same and so even if two threads
do the same thing at the same time, the data will be valid at all
times, so long as at least one thread completes the decompression.

This assumption proved to be **largely** true, but it appears that in
high concurrency cases it is possible that the data in the
decompression buffer differs.  This can result in app crashes:

	A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 3092 (.NET ThreadPool), pid 2727 (myapp.name)
	A/DEBUG: pid: 2727, tid: 3092, name: .NET ThreadPool  >>> myapp.name <<<
	A/DEBUG:       #1 pc 0000000000029b1c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmono-android.release.so (offset 0x103d000) (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+144) (BuildId: 29c5a3805a0bedee1eede9b6668d7c676aa63371)
	A/DEBUG:       #2 pc 00000000002680bc  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       #3 pc 00000000002681e8  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       #4 pc 000000000008555c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (mono_metadata_string_heap+188) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	…

My guess is that LZ4 either uses the output buffer as a scratchpad
area when decompressing or that it initializes/modifies the buffer
before writing actual data in it.  With overlapped decompression, it
may lead to one thread overwriting valid data previously written by
another thread, so that when the latter returns the buffer it thought
to have had valid data may contain certain bytes temporarily
overwritten by the decompression session in the other, still running,
thread.  It may happen that MonoVM reads the corrupted data just when
it is still invalid (before the still running decompression session
actually writes the valid data), a classic race condition.

To fix this, the decompression block is now protected with a startup-
aware mutex.  Mutex will be held only after the initial startup phase
is completed, so there should not be much loss of startup performance.
grendello added a commit to grendello/xamarin-android that referenced this issue Feb 21, 2023
…otnet#7732)

Fixes: dotnet#7335

Context: d236af5

Commit d236af5 introduced embedded assembly compression, using LZ4,
which speeds up startup and reduces final package size.

Assemblies are compressed at build time and, at the same time, pre-
allocated buffers for the **decompressed** data are allocated in
`libxamarin-app.so`.  The buffers are then passed to the LZ4 APIs,
all threads using the same output buffer.  The assumption was that we
can do fine without locking as even if overlapped decompression
happens, the output data will be the same and so even if two threads
do the same thing at the same time, the data will be valid at all
times, so long as at least one thread completes the decompression.

This assumption proved to be **largely** true, but it appears that in
high concurrency cases it is possible that the data in the
decompression buffer differs.  This can result in app crashes:

	A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 3092 (.NET ThreadPool), pid 2727 (myapp.name)
	A/DEBUG: pid: 2727, tid: 3092, name: .NET ThreadPool  >>> myapp.name <<<
	A/DEBUG:       #1 pc 0000000000029b1c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmono-android.release.so (offset 0x103d000) (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+144) (BuildId: 29c5a3805a0bedee1eede9b6668d7c676aa63371)
	A/DEBUG:       #2 pc 00000000002680bc  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       #3 pc 00000000002681e8  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       dotnet#4 pc 000000000008555c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (mono_metadata_string_heap+188) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	…

My guess is that LZ4 either uses the output buffer as a scratchpad
area when decompressing or that it initializes/modifies the buffer
before writing actual data in it.  With overlapped decompression, it
may lead to one thread overwriting valid data previously written by
another thread, so that when the latter returns the buffer it thought
to have had valid data may contain certain bytes temporarily
overwritten by the decompression session in the other, still running,
thread.  It may happen that MonoVM reads the corrupted data just when
it is still invalid (before the still running decompression session
actually writes the valid data), a classic race condition.

To fix this, the decompression block is now protected with a startup-
aware mutex.  Mutex will be held only after the initial startup phase
is completed, so there should not be much loss of startup performance.
@ghost ghost locked as resolved and limited conversation to collaborators Feb 26, 2023
jonpryor pushed a commit that referenced this issue Feb 27, 2023
…7817)

Fixes: #7335

Context: d236af5

Commit d236af5 introduced embedded assembly compression, using LZ4,
which speeds up startup and reduces final package size.

Assemblies are compressed at build time and, at the same time, pre-
allocated buffers for the **decompressed** data are allocated in
`libxamarin-app.so`.  The buffers are then passed to the LZ4 APIs,
all threads using the same output buffer.  The assumption was that we
can do fine without locking as even if overlapped decompression
happens, the output data will be the same and so even if two threads
do the same thing at the same time, the data will be valid at all
times, so long as at least one thread completes the decompression.

This assumption proved to be **largely** true, but it appears that in
high concurrency cases it is possible that the data in the
decompression buffer differs.  This can result in app crashes:

	A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 3092 (.NET ThreadPool), pid 2727 (myapp.name)
	A/DEBUG: pid: 2727, tid: 3092, name: .NET ThreadPool  >>> myapp.name <<<
	A/DEBUG:       #1 pc 0000000000029b1c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmono-android.release.so (offset 0x103d000) (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+144) (BuildId: 29c5a3805a0bedee1eede9b6668d7c676aa63371)
	A/DEBUG:       #2 pc 00000000002680bc  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       #3 pc 00000000002681e8  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	A/DEBUG:       #4 pc 000000000008555c  /data/app/myapp.name-B9t_3dF9i8mDxJEKodZw5w==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (offset 0x109b000) (mono_metadata_string_heap+188) (BuildId: 4a5dd4396e8816b7f69881838bd549285213d53b)
	…

My guess is that LZ4 either uses the output buffer as a scratchpad
area when decompressing or that it initializes/modifies the buffer
before writing actual data in it.  With overlapped decompression, it
may lead to one thread overwriting valid data previously written by
another thread, so that when the latter returns the buffer it thought
to have had valid data may contain certain bytes temporarily
overwritten by the decompression session in the other, still running,
thread.  It may happen that MonoVM reads the corrupted data just when
it is still invalid (before the still running decompression session
actually writes the valid data), a classic race condition.

To fix this, the decompression block is now protected with a startup-
aware mutex.  Mutex will be held only after the initial startup phase
is completed, so there should not be much loss of startup performance.
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
Area: App Runtime Issues in `libmonodroid.so`.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants