Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Symbols required for AOT are private in libmonosgen-2.0.dylib #55000

Closed
rolfbjarne opened this issue Jul 1, 2021 · 18 comments · Fixed by #55224, #55344, #56144 or #56545
Closed

Symbols required for AOT are private in libmonosgen-2.0.dylib #55000

rolfbjarne opened this issue Jul 1, 2021 · 18 comments · Fixed by #55224, #55344, #56144 or #56545
Assignees
Milestone

Comments

@rolfbjarne
Copy link
Member

Description

Mono's AOT compiler generates references to numerous functions in lib mono, but those functions are not public in libmonosgen-2.0.dylib, only libmonosgen-2.0.a.

Example for mono_array_new_1:

$ nm microsoft.netcore.app.runtime.mono.iossimulator-x64/6.0.0-preview.7.21329.1/runtimes/iossimulator-x64/native/libmonosgen-2.0.dylib | grep mono_array_new_1 
00000000001a2b40 t _mono_array_new_1

$ nm microsoft.netcore.app.runtime.mono.iossimulator-x64/6.0.0-preview.7.21329.1/runtimes/iossimulator-x64/native/libmonosgen-2.0.a | grep mono_array_new_1
                 U _mono_array_new_1
00000000000004e0 T _mono_array_new_1

The end result is that we can't link dynamically with libmonosgen-2.0.dylib when using the AOT compiler, we have to link with the static libmonosgen-2.0.a library.

Example native link failures:

Undefined symbols for architecture x86_64:
  "_mini_llvm_init_method", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvm_init_method_intptr_intptr_intptr_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_get_interp_entry", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_get_interp_entry_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_init_delegate", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_init_delegate_object in System.Private.CoreLib.dll.o
  "_mini_llvmonly_init_delegate_virtual", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_init_delegate_virtual_object_object_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_init_vtable_slot", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_init_vtable_slot_intptr_int in System.Private.CoreLib.dll.o
  "_mini_llvmonly_pop_lmf", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_pop_lmf_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_generic_virtual_call", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_generic_virtual_call_intptr_int_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_generic_virtual_iface_call", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_generic_virtual_iface_call_intptr_int_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_iface_call_gsharedvt", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_iface_call_gsharedvt_object_int_intptr_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_vcall_gsharedvt", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_vcall_gsharedvt_object_int_intptr_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_throw_aot_failed_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_throw_aot_failed_exception_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_throw_nullref_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_throw_nullref_exception in System.Private.CoreLib.dll.o
  "_mono_array_new_1", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_array_new_1_intptr_int in System.Private.CoreLib.dll.o
[...]

Example binlog: msbuild.binlog.zip

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Jul 1, 2021
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@rolfbjarne
Copy link
Member Author

CC @steveisok @akoeplinger

@steveisok
Copy link
Member

@lambdageek @vargaz Can either one of you take a look and help us triage?

@lambdageek
Copy link
Member

mono-aot-cross execution started with arguments: --aot=mtriple=x86_64-ios,data-outfile=obj/Debug/net6.0-ios/iossimulator-x64/nativelibraries/aot-output/x86_64/System.Private.CoreLib.aotdata,static,asmonly,direct-icalls,interp,soft-debug,dwarfdebug,no-direct-calls,outfile=obj/Debug/net6.0-ios/iossimulator-x64/nativelibraries/aot-output/x86_64/System.Private.CoreLib.dll.s --debug /Users/rolf/work/maccore/arm64-dyn-msgsend/xamarin-macios/tests/dotnet/MyInterpretedApp/iOS/obj/Debug/net6.0-ios/iossimulator-x64/linked/System.Private.CoreLib.dll

@vargaz do these arguments make sense? I'm not sure if static,direct-icalls,interp is a combination that makes sense with dynamic linking.

@rolfbjarne Do you also get problems if you're doing a fullaot build, or just interp-only?

@vargaz vargaz self-assigned this Jul 6, 2021
@vargaz
Copy link
Contributor

vargaz commented Jul 6, 2021

Those are declared using ICALL_EXTERN_C, the runtime probably needs to be built with
ENABLE_ICALL_EXPORT.

@vargaz
Copy link
Contributor

vargaz commented Jul 6, 2021

Or direct-icalls needs to be dropped from the aot command line.

@steveisok steveisok removed the untriaged New issue has not been triaged by the area owner label Jul 6, 2021
@steveisok steveisok added this to the 6.0.0 milestone Jul 6, 2021
vargaz added a commit to vargaz/runtime that referenced this issue Jul 6, 2021
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 6, 2021
@vargaz
Copy link
Contributor

vargaz commented Jul 6, 2021

Are there any tests in dotnet/runtime which run into this problem ?

@lambdageek
Copy link
Member

Are there any tests in dotnet/runtime which run into this problem ?

I think no, because we've been doing static linking in dotnet/runtime. I have a PR to switch to dynamic linking (#54241) but I didn't get far enough to finish it. I suspect it probably hits this issue.

@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 6, 2021
@akoeplinger
Copy link
Member

We're actually missing <_MonoCMakeArgs Include="-DENABLE_ICALL_EXPORT=1"/> in mono.proj around L345 for iOS device builds to match the --enable-icall-export that we had in mono/mono.

@akoeplinger akoeplinger reopened this Jul 8, 2021
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 9, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 9, 2021
@rolfbjarne
Copy link
Member Author

This is still an issue with 6.0.0-rc.1.21369.14:

$ nm  packages/microsoft.netcore.app.runtime.mono.iossimulator-x64/6.0.0-rc.1.21369.14/runtimes/iossimulator-x64/native/libmonosgen-2.0.dylib | grep mono_array_new_1
00000000001a3d70 t _mono_array_new_1

@rolfbjarne rolfbjarne reopened this Jul 22, 2021
@vargaz
Copy link
Contributor

vargaz commented Jul 22, 2021

The PR above only fixes it for device builds, simulator builds need the same define.

@vargaz
Copy link
Contributor

vargaz commented Jul 22, 2021

The problem is that those are jit icalls which are only marked with ICALL_EXTERN_C which doesn't actually export them.

@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 22, 2021
vargaz added a commit to vargaz/runtime that referenced this issue Jul 22, 2021
vargaz added a commit that referenced this issue Jul 22, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 22, 2021
@rolfbjarne
Copy link
Member Author

This is still a problem in Mac Catalyst and macOS (but not on iOS):

$ for i in packages/microsoft.netcore.app.runtime.*/6.0.0-*/runtimes/*/native/libmonosgen-2.0.dylib packages/microsoft.netcore.app.runtime.*/6.0.0-*/runtimes/*/native/libcoreclr.dylib; do echo $i; nm $i | grep mono_array_new_1; done
packages/microsoft.netcore.app.runtime.mono.ios-arm64/6.0.0-rc.1.21376.24/runtimes/ios-arm64/native/libmonosgen-2.0.dylib
000000000013e264 T _mono_array_new_1
packages/microsoft.netcore.app.runtime.mono.iossimulator-x64/6.0.0-rc.1.21376.24/runtimes/iossimulator-x64/native/libmonosgen-2.0.dylib
00000000001a3cb0 T _mono_array_new_1
packages/microsoft.netcore.app.runtime.mono.maccatalyst-arm64/6.0.0-rc.1.21376.24/runtimes/maccatalyst-arm64/native/libmonosgen-2.0.dylib
0000000000185f24 t _mono_array_new_1
packages/microsoft.netcore.app.runtime.mono.maccatalyst-x64/6.0.0-rc.1.21376.24/runtimes/maccatalyst-x64/native/libmonosgen-2.0.dylib
00000000001ace70 t _mono_array_new_1
packages/microsoft.netcore.app.runtime.mono.osx-x64/6.0.0-rc.1.21376.24/runtimes/osx-x64/native/libcoreclr.dylib
00000000001afb60 t _mono_array_new_1

@rolfbjarne rolfbjarne reopened this Jul 29, 2021
vargaz added a commit to vargaz/runtime that referenced this issue Jul 29, 2021
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 29, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 29, 2021
@rolfbjarne
Copy link
Member Author

This isn't fully fixed yet - it's fixed for some symbols, but not others. I tested for mono_array_new_1 previously, and that seems to be public on all platforms now. However there are numerous other symbols which are still private (on all platforms):

Undefined symbols for architecture arm64:
  "_mini_llvm_init_method", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvm_init_method_intptr_intptr_intptr_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_get_interp_entry", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_get_interp_entry_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_init_delegate", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_init_delegate_object in System.Private.CoreLib.dll.o
  "_mini_llvmonly_init_delegate_virtual", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_init_delegate_virtual_object_object_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_init_vtable_slot", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_init_vtable_slot_intptr_int in System.Private.CoreLib.dll.o
  "_mini_llvmonly_pop_lmf", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_pop_lmf_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_generic_virtual_call", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_generic_virtual_call_intptr_int_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_generic_virtual_iface_call", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_generic_virtual_iface_call_intptr_int_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_iface_call_gsharedvt", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_iface_call_gsharedvt_object_int_intptr_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_resolve_vcall_gsharedvt", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_resolve_vcall_gsharedvt_object_int_intptr_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_throw_aot_failed_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_throw_aot_failed_exception_intptr in System.Private.CoreLib.dll.o
  "_mini_llvmonly_throw_nullref_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mini_llvmonly_throw_nullref_exception in System.Private.CoreLib.dll.o
  "_mono_debug_personality", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_debug_personality_int_int_intptr_intptr_intptr in System.Private.CoreLib.dll.o
  "_mono_gc_wbarrier_range_copy", referenced from:
      MonoTouchFixtures_Security_KeyTest_BenchmarkManaged4096 in monotouchtest.dll.o
      System_Collections_Immutable_ImmutableHashSet_1_T_REF_GetEnumerator in System.Collections.Immutable.dll.o
      System_Collections_Immutable_ImmutableHashSet_1_T_REF_System_Collections_Generic_IEnumerable_T_GetEnumerator in System.Collections.Immutable.dll.o
      System_Collections_Immutable_ImmutableHashSet_1_T_REF_System_Collections_IEnumerable_GetEnumerator in System.Collections.Immutable.dll.o
      System_Collections_Immutable_ImmutableHashSet_1_Builder_T_REF_GetEnumerator in System.Collections.Immutable.dll.o
      System_Collections_Immutable_ImmutableHashSet_1_Builder_T_REF_System_Collections_Generic_IEnumerable_T_GetEnumerator in System.Collections.Immutable.dll.o
      System_Collections_Immutable_ImmutableHashSet_1_Builder_T_REF_System_Collections_IEnumerable_GetEnumerator in System.Collections.Immutable.dll.o
      ...
  "_mono_interp_entry_from_trampoline", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_interp_entry_from_trampoline_intptr_intptr in System.Private.CoreLib.dll.o
  "_mono_interp_to_native_trampoline", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_interp_to_native_trampoline_intptr_intptr in System.Private.CoreLib.dll.o
  "_mono_llvm_load_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_llvm_load_exception in System.Private.CoreLib.dll.o
  "_mono_llvm_match_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_llvm_match_exception_intptr_int_int_intptr_object in System.Private.CoreLib.dll.o
  "_mono_llvm_resume_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_llvm_resume_exception in System.Private.CoreLib.dll.o
  "_mono_llvm_rethrow_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_llvm_rethrow_exception_object in System.Private.CoreLib.dll.o
  "_mono_llvm_throw_corlib_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_llvm_throw_corlib_exception_int in System.Private.CoreLib.dll.o
  "_mono_llvm_throw_exception", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_llvm_throw_exception_object in System.Private.CoreLib.dll.o
  "_mono_resume_unwind", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_resume_unwind_intptr in System.Private.CoreLib.dll.o
  "_mono_tls_get_domain_extern", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_tls_get_domain_extern in System.Private.CoreLib.dll.o
  "_mono_tls_get_jit_tls_extern", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_tls_get_jit_tls_extern in System.Private.CoreLib.dll.o
  "_mono_tls_get_lmf_addr_extern", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_tls_get_lmf_addr_extern in System.Private.CoreLib.dll.o
  "_mono_tls_get_sgen_thread_info_extern", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_tls_get_sgen_thread_info_extern in System.Private.CoreLib.dll.o
  "_mono_tls_get_thread_extern", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_mono_tls_get_thread_extern in System.Private.CoreLib.dll.o
  "_monoeg_g_free", referenced from:
      wrapper_managed_to_native_object___icall_wrapper_monoeg_g_free_intptr in System.Private.CoreLib.dll.o

Testing for the first (mini_llvm_init_method) and last (monoeg_g_free) symbol shows they're private for all platforms:

$ for i in $(find microsoft.* -name libmonosgen-2.0.dylib -or -name libcoreclr.dylib | grep -v DWARF | grep mono); do echo $i; nm $i | grep monoeg_g_free; done
microsoft.netcore.app.runtime.mono.ios-arm/6.0.0-rc.1.21380.13/runtimes/ios-arm/native/libmonosgen-2.0.dylib
0015330a t _monoeg_g_free
0015b186 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.ios-arm64/6.0.0-rc.1.21380.13/runtimes/ios-arm64/native/libmonosgen-2.0.dylib
00000000001b0e18 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.iossimulator-x64/6.0.0-rc.1.21380.13/runtimes/iossimulator-x64/native/libmonosgen-2.0.dylib
00000000002a78a0 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.iossimulator-x86/6.0.0-rc.1.21380.13/runtimes/iossimulator-x86/native/libmonosgen-2.0.dylib
0028dd50 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.maccatalyst-arm64/6.0.0-rc.1.21380.13/runtimes/maccatalyst-arm64/native/libmonosgen-2.0.dylib
000000000025335c t _monoeg_g_free
microsoft.netcore.app.runtime.mono.maccatalyst-x64/6.0.0-rc.1.21380.13/runtimes/maccatalyst-x64/native/libmonosgen-2.0.dylib
00000000002ad0a0 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.osx-arm64/6.0.0-rc.1.21380.13/runtimes/osx-arm64/native/libcoreclr.dylib
0000000000256df8 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.osx-x64/6.0.0-rc.1.21380.13/runtimes/osx-x64/native/libcoreclr.dylib
00000000002b4200 t _monoeg_g_free
microsoft.netcore.app.runtime.mono.tvos-arm64/6.0.0-rc.1.21380.13/runtimes/tvos-arm64/native/libmonosgen-2.0.dylib
00000000001b1dec t _monoeg_g_free
microsoft.netcore.app.runtime.mono.tvossimulator-x64/6.0.0-rc.1.21380.13/runtimes/tvossimulator-x64/native/libmonosgen-2.0.dylib
00000000002a4910 t _monoeg_g_free

$ for i in $(find microsoft.* -name libmonosgen-2.0.dylib -or -name libcoreclr.dylib | grep -v DWARF | grep mono); do echo $i; nm $i | grep mini_llvm_init_method; done
microsoft.netcore.app.runtime.mono.ios-arm/6.0.0-rc.1.21380.13/runtimes/ios-arm/native/libmonosgen-2.0.dylib
0011cf8c t _mini_llvm_init_method
00121b9c t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.ios-arm64/6.0.0-rc.1.21380.13/runtimes/ios-arm64/native/libmonosgen-2.0.dylib
000000000016b618 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.iossimulator-x64/6.0.0-rc.1.21380.13/runtimes/iossimulator-x64/native/libmonosgen-2.0.dylib
0000000000235d30 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.iossimulator-x86/6.0.0-rc.1.21380.13/runtimes/iossimulator-x86/native/libmonosgen-2.0.dylib
00233580 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.maccatalyst-arm64/6.0.0-rc.1.21380.13/runtimes/maccatalyst-arm64/native/libmonosgen-2.0.dylib
0000000000200c10 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.maccatalyst-x64/6.0.0-rc.1.21380.13/runtimes/maccatalyst-x64/native/libmonosgen-2.0.dylib
000000000023b130 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.osx-arm64/6.0.0-rc.1.21380.13/runtimes/osx-arm64/native/libcoreclr.dylib
00000000002046ac t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.osx-x64/6.0.0-rc.1.21380.13/runtimes/osx-x64/native/libcoreclr.dylib
0000000000242350 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.tvos-arm64/6.0.0-rc.1.21380.13/runtimes/tvos-arm64/native/libmonosgen-2.0.dylib
000000000016ca00 t _mini_llvm_init_method
microsoft.netcore.app.runtime.mono.tvossimulator-x64/6.0.0-rc.1.21380.13/runtimes/tvossimulator-x64/native/libmonosgen-2.0.dylib
00000000002331c0 t _mini_llvm_init_method

@vargaz
Copy link
Contributor

vargaz commented Aug 4, 2021

Maybe we should avoid using direct-icalls in non-static mode as a simpler fix.

@rolfbjarne
Copy link
Member Author

Maybe we should avoid using direct-icalls in non-static mode as a simpler fix.

I can have a look and see if that works.

rolfbjarne added a commit to rolfbjarne/xamarin-macios that referenced this issue Aug 4, 2021
@vargaz
Copy link
Contributor

vargaz commented Aug 8, 2021

Did it work out ?

rolfbjarne added a commit to dotnet/macios that referenced this issue Aug 9, 2021
* [dotnet] Remove workaround for private symbols for AOT.

* [tools] Make Application.AotArguments a list of string.

This is just a simple refactoring to make Application.AotArguments a list of
strings instead of a comma-separated list of values.

* [tools] Only use direct-icalls when linking mono statically.

Ref: dotnet/runtime#55000

* [mtouch] Fix aot arguments comparison.

* [tests] Adjust mtouch test according to mtouch changes.

* [tests] Add minimum OS version to the Mac Catalyst variation of the MySimpleApp test case.
@rolfbjarne
Copy link
Member Author

Did it work out ?

Yes, it did, so this can be closed.

@ghost ghost locked as resolved and limited conversation to collaborators Sep 8, 2021
# for free to subscribe to this conversation on GitHub. Already have an account? #.