Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Optimize type & member remapping storage, lookup #7020

Closed
jonpryor opened this issue May 20, 2022 · 0 comments · Fixed by #7059
Closed

Optimize type & member remapping storage, lookup #7020

jonpryor opened this issue May 20, 2022 · 0 comments · Fixed by #7059
Assignees
@grendello
Labels
Area: App Runtime Issues in `libmonodroid.so`.
Milestone
.NET 7 Servicing

Comments

@jonpryor
Copy link
Member

Android application type

Android for .NET (net6.0-android, etc.)

Affected platform version

Description

Context: #6591

Background

PR #6591 fixes dotnet/java-interop#867, but it's largely a "prototype" with great cost to app size:

  • It uses XML as the intermediate format.
  • This XML is parsed during app startup, before most other managed init.
  • Which caused the CheckIncludedAssemblies() test to require updates, adding the assemblies System.Collections.dll, System.Collections.Concurrent.dll, System.Collections.NonGeneric.dll, System.Console.dll, System.IO.Compression.dll, System.Net.Http.dll, System.Net.Primitives.dll, System.Net.Requests.dll, System.Private.Uri.dll, System.Private.Xml.dll, System.Security.Cryptography.dll, and System.Text.RegularExpressions.dll
  • which in turn increase the .apk size; BuildReleaseArm64SimpleDotNet.apkdesc saw PackageSize increase from 2,959,252 bytes to 3,451,764 bytes, an increase of ~17%.

PR #6591 exists to allow further testing. It is not the end state of how things should be.

Proposal

Instead of using XML as an intermediary packaged and parsed within the app, the data should instead be stored within libxamarin-app.so, a'la typemap data.

We would add two new exports to libmonodroid.so:

extern "C" const char *
_monodroid_lookup_replacement_type (const char *jniSimpleReference);

extern "C" const char *
_monodroid_lookup_replacement_method_info (const char *key);

Update/extend the _CollectAndroidRemapMembers target so that instead of creating xa-remap-members.xml as an @(AndroidAsset), xa-remap-members.xml is instead provided as input into whatever creates libxamarin-app.so.

The assembly generator will read <replace-type/> to populate the table used by _monodroid_lookup_replacement_type(). //replace-type/@from will be the key, while //replace-type/@to will be the value.

The assembly generator will read <replace-method/> to populate the table used by _monodroid_lookup_replacement_method_info(). The key to search on will be a tab separated value of:

{@source-type}\t{@source-method-name}\t{@source-method-signature}

The value will be a tab separated value of:

{@target-type}\t{@target-method-name}\t{@target-method-signature}\t{@target-method-parameter-count}\t{@target-method-instance-to-static}

Placing the data into libxamarin-app.so will allow us to revert all changes to JNIEnv.cs, most changes to src/monodroid. AndroidRuntime.cs will need to be updated to use the new _monodroid_lookup_replacement_*() functions instead of the current dictionaries.

Steps to Reproduce

Did you find any workaround?

No response

Relevant log output

No response
@jonpryor jonpryor added Area: App Runtime Issues in `libmonodroid.so`. needs-triage Issues that need to be assigned. labels May 20, 2022
@jonpryor jonpryor added this to the .NET 7 milestone May 20, 2022
jonpryor added a commit that referenced this issue May 20, 2022
@jonpryor
[Xamarin.Android.Build.Tasks] MAM Member Remapping? (#6591)
f6f11a5 
Fixes: dotnet/java-interop#867

Context: dotnet/java-interop@1f27ab5
Context: #6142 (comment)
Context: #7020

Changes: dotnet/java-interop@843f3c7...1f27ab5

  * dotnet/java-interop@1f27ab55: [Java.Interop] Type & Member Remapping Support (#936)
  * dotnet/java-interop@02aa54e0: [Java.Interop.Tools.JavaCallableWrappers] marshal method decl types (#987)
  * dotnet/java-interop@e7bacc37: [ci] Update azure-pipelines.yaml to Pin .NET 6.0.202 (#986)
  * dotnet/java-interop@fb94d598: [Java.Interop.Tools.JavaCallableWrappers] Collect overriden methods (#985)
  * dotnet/java-interop@3fcce746: [Java.Interop.{Dynamic,Export}] Nullable Reference Type support (#980)

~~ The Scenarios ~~

Our Java binding infrastructure involves looking up types and methods
via [JNI][0], and assumes that types and methods won't "move" in an
unexpected manner.  Methods which move from a subclass to a
superclass works transparently.  Methods which are moved to an
entirely different class causes us problems.

Case in point: [desugaring][0], which can *move* Java types to places
that our bindings don't expect.  For example, [`Arrays.stream(T[])`][1]
may be moved into a `DesugarArrays` class, or a default interface
method `Example.m()` may be moved into an `Example$-CC` type.
Java.Interop has not supported such changes, resulting in throwing a
`Java.Lang.NoSuchMethodError` on Android versions where methods are
not where we expect.

Additionally, the [InTune Mobile Application Management][3] team
needs a expanded type and member lookup mechanism in order to
simplify how they maintain their product.  Currently they make things
work by rewriting IL, which can be brittle.


~~ Build actions ~~

To improve support for this, dotnet/java-interop#936 introduces
new `virtual` methods into `Java.Interop.JniRuntime.JniTypeManager`
which are called as part of type and member lookup, allowing
`AndroidTypeManager` to participate in the type and member resolution
process.

`AndroidTypeManager` in turn needs to know what types and members can
be remapped, and what they should be remapped to.  Some of these can
be algorithmic, such as pre-pending `Desugar` or appending `$-CC` for
the Desugar case.

The InTune use case involves a table, contained within the
[Microsoft.Intune.MAM.Remapper.Tasks NuGet package][4].

Update `src/Xamarin.Android.Build.Tasks` to add a new
`@(_AndroidRemapMembers)` Build action.  This build action is not
externally supported; it's to help test the feature.  Files with this
build action are XML files which control type and member remapping:

	<replacements>
	  <replace-type
	      from="android/app/Activity"
	      to="com/microsoft/intune/mam/client/app/MAMActivity" />
	  <replace-method
	      source-type="com/microsoft/intune/mam/client/app/MAMActivity"
	      source-method-name="onCreate" source-method-signature="(Landroid/os/Bundle;)V"
	      target-type="com/microsoft/intune/mam/client/app/MAMActivity"
	      target-method-name="onMAMCreate" target-method-instance-to-static="false" />
	</replacements>

`//replacements/replace-method` is structured with each attribute
corresponding to a member on the `JniRuntime.ReplacementMethodInfo`
structure, in dotnet/java-interop@1f27ab55.

  * `//replace-method/@source-type` is
    `JniRuntime.ReplacementMethodInfo.SourceJniType`
  * `//replace-method/@source-method-name` is
    `JniRuntime.ReplacementMethodInfo.SourceJniMethodName`
  * `//replace-method/@source-method-signature` is
    `JniRuntime.ReplacementMethodInfo.SourceJniMethodSignature`
  * `//replace-method/@target-type` is
    `JniRuntime.ReplacementMethodInfo.TargetJniType`
  * `//replace-method/@target-method-name` is
    `JniRuntime.ReplacementMethodInfo.TargetJniMethodName`
  * `//replace-method/@target-method-signature` is
    `JniRuntime.ReplacementMethodInfo.TargetJniMethodSignature`
    This attribute is optional.
  * `//replace-method/@target-method-parameter-count` is
    `JniRuntime.ReplacementMethodInfo.TargetJniMethodParameterCount`.
    This attribute is optional.
  * `//replace-method/@target-method-instance-to-static` is
    `JniRuntime.ReplacementMethodInfo.TargetJniMethodIsStatic`

`@source-type`, `@source-method-name`, and `@source-method-signature`
combined serve as a "key" for looking up the associated `@target-*`
information.

Update `src/Xamarin.Android.Build.Tasks` to add a new
`@(_AndroidMamMappingFile)` Build action.  This build action is not
externally supported; it's to help test the feature.  Files with this
build action are expected to be JSON documents which follow the
current conventions of `remapping-config.json`, within the
`Microsoft.Intune.MAM.Remapper.Tasks` NuGet package.  This build
action is not externally supported; this is currently for testing
purposes.  `@(_AndroidMamMappingFile)` files are processed at build
time into `@(_AndroidRemapMembers)` XML files.

During App builds, all `@(_AndroidRemapMembers)` files are merged
into an `@(AndrodAsset)` named `xa-internal/xa-mam-mapping.xml`.
This asset is opened and provided to `JNIEnv.Initialize()` as part of
native app startup.


~~ Putting it all together ~~

This will only work on .NET 7+.

App project has a `@(_AndroidRemapMembers)` file.  This item is
processed during App build, stored into the `.apk`, and read during
app startup on an Android device.

Given a Java binding such as:

	public partial class Activity {
	    static readonly JniPeerMembers _members = new XAPeerMembers ("android/app/Activity", typeof (Activity));
	}

when the `JniPeerMembers` constructor runs, it will call
`JniEnvironment.Runtime.TypeManager.GetReplacementType("android/app/Activity")`.
If `@(_AndroidRemapMembers)` is based on the InTune
`remapping-config.json` file, then `android/app/Activity` is mapped
to `com/microsoft/intune/mam/client/app/MAMActivity`, and
`JNIEnv::FindClass()` will be told to lookup `MAMActivity`, *not*
`Activity`.

*If `MAMActivity` can't be found*, e.g. you're testing this all out,
the app will ~immediately crash, as `MAMActivity` doesn't exist. 😅

If `MAMActivity` can be found, eventually `Activity.OnCreate()` will
need to be invoked:

	partial class Activity {
	    protected virtual unsafe void OnCreate (Android.OS.Bundle? savedInstanceState)
	    {
	        const string __id = "onCreate.(Landroid/os/Bundle;)V";
	        try {
	            JniArgumentValue* __args = stackalloc JniArgumentValue [1];
	            __args [0] = new JniArgumentValue ((savedInstanceState == null) ? IntPtr.Zero : ((global::Java.Lang.Object) savedInstanceState).Handle);
	            _members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
	        } finally {
	            global::System.GC.KeepAlive (savedInstanceState);
	        }
	    }
	}

`_members.InstanceMethods.InvokeVirtualVoidMethod()` will internally
make a call similar to:

	var r = JniEnvironment.Runtime.TypeManager.GetReplacementMethodInfo (
	    "com/microsoft/intune/mam/client/app/MAMActivity",
	    "onCreate",
	    "(Landroid/os/Bundle;)V"
	);

The data returned will be equivalent to:

	var r = new JniRuntime.ReplacementMethodInfo {
	    SourceJniType                   = "com/microsoft/intune/mam/client/app/MAMActivity",    // from input parameter
	    SourceJniMethodName             = "onCreate",                                           // from input parameter
	    SourceJniMethodSignature        = "(Landroid/os/Bundle;)V",                             // from input parameter

	    TargetJniType                   = "com/microsoft/intune/mam/client/app/MAMActivity",    // from //replace-method/@target-type
	    TargetJniMethodName             = "onMAMCreate",                                        // from //replace-method/@target-method-name
	    TargetJniMethodSignature        = "(Landroid/os/Bundle;)V",                             // from input parameter, as signature didn't change
	    TargetJniMethodParameterCount   = 1,                                                    // computed based on signature
	    TargetJniMethodIsStatic         = false,                                                // from //replace-method/@target-method-instance-to-static
	}

This will allow `_members.InstanceMethods.InvokeVirtualVoidMethod()`
to instead resolve and invoke `MAMActivity.onMAMCreate()`.


~~ Tools ~~

`tools/remap-mam-json-to-xml` is added, and will process the InTune
JSON file into `@(_AndroidRemapMembers)` XML:

	$ dotnet run --project tools/remap-mam-json-to-xml  -- \
	  $HOME/.nuget/packages/microsoft.intune.mam.remapper.tasks/0.1.4635.1/content/MonoAndroid10/remapping-config.json
	<replacements>…


~~ Unit Tests ~~

`@(_AndroidRemapMembers)` usage is required by
`Mono.Android.NET-Tests.apk`, as `Java.Interop-Tests.dll` exercises
the type and member remapping logic.


~~ Unrelated `gref+` logging fixes ~~

When `debug.mono.log` = `gref+`, the app could crash:

	signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x79c045dead

This was likely because a constant string was provided to
`OSBridge::_write_stack_trace()`, which tried to write into the
constant string, promptly blowing things up.

Workaround: don't use `gref+` logging when a GC occurs?
(Fortunately, `gref+` logging isn't the default.)

Better workaround: Don't Do That™. Don't write to const strings.


~~ About `@(_AndroidRemapMembers)` Semantics… ~~

 1. Changing the Java hierarchy "requires" changing the managed
    hierarchy to mirror it.

    If we rename `Activity` to `RemapActivity` but *don't* change
    `MainActivity` to inherit the (bound!) `Example.RemapActivity`,
    the app *crashes*:

        JNI DETECTED ERROR IN APPLICATION: can't call void example.RemapActivity.onMyCreate(android.os.Bundle) on instance of example.MainActivity

    This can be "fixed" *without* changing the base class
    of `MainActivity` by instead changing the base class of the
    Java Callable Wrapper for `MainActivity` to `example.RemapActivity`.
    This can be done manually (just edit the files in `obj/…`!), but
    isn't really supported in "normal" xamarin-android usage
    (the next Clean will wipe your changes).

    Presumably InTune would make this Just Work by e.g. patching the
    `MainActivity.class` file.

 2. `/replacements/replace-type` interacts with
    `/replacements/replace-method`: at runtime, `//replace-type@from`
    *no longer exists*, meaning you ***cannot*** use that name
    in `//replace-method/@source-type` either!

    If `Activity` is remapped to `RemapActivity`, then *there is no*
    `Activity.onCreate()` method to similarly remap.  Instead, you
    need to specify `RemapActivity.onCreate()`.

    This warps the brain a bit.

    This:

        <replace-method
            source-type="example/RemapActivity"
            source-method-name="onCreate"
            target-type="example/RemapActivity"
            target-method-name="onMyCreate" target-method-instance-to-static="false" />

    not this:

        <replace-method
            source-type="android/app/Activity"
            source-method-name="onCreate"
            target-type="example/RemapActivity"
            target-method-name="onMyCreate" target-method-instance-to-static="false" />

 3. Don't intermix type renames with
    `/replace-method/@target-method-instance-to-static='true']`.
    It *can* be done, but also warps the brain.

    The deal with `@target-method-instance-to-static` is that it
    it changes the target method signature -- unless explicitly
    provided in `/replace-method/@target-method-signature` --
    so that the "source declaring type" is a prefix.

    Thus given

        <replace-method
            source-type="android/view/View"
            source-method-name="setOnClickListener"
            target-type="example/ViewHelper"
            target-method-name="mySetOnClickListener" target-method-instance-to-static="true" />

    we'll look for `ViewHelper.mySetOnClickListener(View, View.OnClickListener)`.

    If we renamed `View` to `MyView`, we would instead look for
    `ViewHelper.mySetOnClickListener(MyView, View.OnClickListener)`
    (note changed parameter type).

    This almost certainly *won't* work.


~~ InTune Integration Testing? ~~

For "more complete" InTune integration testing, one will want the
path to `remapping-config.json`, without hardcoding things.
This can be done with `%(PackageReference.GeneratePathProperty)`=True
and using `$(PkgMicrosoft_Intune_MAM_Remapper_Tasks)`:

	<ItemGroup>
	  <PackageReference
	      Include="Microsoft.Intune.MAM.Remapper.Tasks"
	      Version="0.1.4635.1"
	      IncludeAssets="none"
	      GeneratePathProperty="True"
	      ReferenceOutputAssembly="False"
	  />
	</ItemGroup>
	<Target Name="_AddMamFiles"
	    BeforeTargets="_AddAndroidCustomMetaData">
	  <ItemGroup>
	    <_AndroidMamMappingFile Include="$(PkgMicrosoft_Intune_MAM_Remapper_Tasks)/content/MonoAndroid10/remapping-config.json" />
	  </ItemGroup>
	</Target>

This is still fraught with some peril, as it likely also depends on
getting the right "inner" build, which may require using the plural
`$(TargetFrameworks)` property, not the singular `$(TargetFramework)`.
This might still be a useful start.


~~ TODO ~~

Optimize this mess:
#7020

[0]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html
[1]: https://developer.android.com/studio/write/java8-support#library-desugaring
[2]: https://developer.android.com/reference/java/util/Arrays#stream(T[])
[3]: https://docs.microsoft.com/en-us/mem/intune/fundamentals/what-is-intune
[4]: https://www.nuget.org/packages/Microsoft.Intune.MAM.Remapper.Tasks/
@grendello grendello removed the needs-triage Issues that need to be assigned. label May 20, 2022
simonrozsival pushed a commit to simonrozsival/xamarin-android that referenced this issue May 31, 2022
@jonpryor @simonrozsival
[Xamarin.Android.Build.Tasks] MAM Member Remapping? (dotnet#6591)
e73dd78 
Fixes: dotnet/java-interop#867

Context: dotnet/java-interop@1f27ab5
Context: dotnet#6142 (comment)
Context: dotnet#7020

Changes: dotnet/java-interop@843f3c7...1f27ab5

  * dotnet/java-interop@1f27ab55: [Java.Interop] Type & Member Remapping Support (dotnet#936)
  * dotnet/java-interop@02aa54e0: [Java.Interop.Tools.JavaCallableWrappers] marshal method decl types (dotnet#987)
  * dotnet/java-interop@e7bacc37: [ci] Update azure-pipelines.yaml to Pin .NET 6.0.202 (dotnet#986)
  * dotnet/java-interop@fb94d598: [Java.Interop.Tools.JavaCallableWrappers] Collect overriden methods (dotnet#985)
  * dotnet/java-interop@3fcce746: [Java.Interop.{Dynamic,Export}] Nullable Reference Type support (dotnet#980)

~~ The Scenarios ~~

Our Java binding infrastructure involves looking up types and methods
via [JNI][0], and assumes that types and methods won't "move" in an
unexpected manner.  Methods which move from a subclass to a
superclass works transparently.  Methods which are moved to an
entirely different class causes us problems.

Case in point: [desugaring][0], which can *move* Java types to places
that our bindings don't expect.  For example, [`Arrays.stream(T[])`][1]
may be moved into a `DesugarArrays` class, or a default interface
method `Example.m()` may be moved into an `Example$-CC` type.
Java.Interop has not supported such changes, resulting in throwing a
`Java.Lang.NoSuchMethodError` on Android versions where methods are
not where we expect.

Additionally, the [InTune Mobile Application Management][3] team
needs a expanded type and member lookup mechanism in order to
simplify how they maintain their product.  Currently they make things
work by rewriting IL, which can be brittle.


~~ Build actions ~~

To improve support for this, dotnet/java-interop#936 introduces
new `virtual` methods into `Java.Interop.JniRuntime.JniTypeManager`
which are called as part of type and member lookup, allowing
`AndroidTypeManager` to participate in the type and member resolution
process.

`AndroidTypeManager` in turn needs to know what types and members can
be remapped, and what they should be remapped to.  Some of these can
be algorithmic, such as pre-pending `Desugar` or appending `$-CC` for
the Desugar case.

The InTune use case involves a table, contained within the
[Microsoft.Intune.MAM.Remapper.Tasks NuGet package][4].

Update `src/Xamarin.Android.Build.Tasks` to add a new
`@(_AndroidRemapMembers)` Build action.  This build action is not
externally supported; it's to help test the feature.  Files with this
build action are XML files which control type and member remapping:

	<replacements>
	  <replace-type
	      from="android/app/Activity"
	      to="com/microsoft/intune/mam/client/app/MAMActivity" />
	  <replace-method
	      source-type="com/microsoft/intune/mam/client/app/MAMActivity"
	      source-method-name="onCreate" source-method-signature="(Landroid/os/Bundle;)V"
	      target-type="com/microsoft/intune/mam/client/app/MAMActivity"
	      target-method-name="onMAMCreate" target-method-instance-to-static="false" />
	</replacements>

`//replacements/replace-method` is structured with each attribute
corresponding to a member on the `JniRuntime.ReplacementMethodInfo`
structure, in dotnet/java-interop@1f27ab55.

  * `//replace-method/@source-type` is
    `JniRuntime.ReplacementMethodInfo.SourceJniType`
  * `//replace-method/@source-method-name` is
    `JniRuntime.ReplacementMethodInfo.SourceJniMethodName`
  * `//replace-method/@source-method-signature` is
    `JniRuntime.ReplacementMethodInfo.SourceJniMethodSignature`
  * `//replace-method/@target-type` is
    `JniRuntime.ReplacementMethodInfo.TargetJniType`
  * `//replace-method/@target-method-name` is
    `JniRuntime.ReplacementMethodInfo.TargetJniMethodName`
  * `//replace-method/@target-method-signature` is
    `JniRuntime.ReplacementMethodInfo.TargetJniMethodSignature`
    This attribute is optional.
  * `//replace-method/@target-method-parameter-count` is
    `JniRuntime.ReplacementMethodInfo.TargetJniMethodParameterCount`.
    This attribute is optional.
  * `//replace-method/@target-method-instance-to-static` is
    `JniRuntime.ReplacementMethodInfo.TargetJniMethodIsStatic`

`@source-type`, `@source-method-name`, and `@source-method-signature`
combined serve as a "key" for looking up the associated `@target-*`
information.

Update `src/Xamarin.Android.Build.Tasks` to add a new
`@(_AndroidMamMappingFile)` Build action.  This build action is not
externally supported; it's to help test the feature.  Files with this
build action are expected to be JSON documents which follow the
current conventions of `remapping-config.json`, within the
`Microsoft.Intune.MAM.Remapper.Tasks` NuGet package.  This build
action is not externally supported; this is currently for testing
purposes.  `@(_AndroidMamMappingFile)` files are processed at build
time into `@(_AndroidRemapMembers)` XML files.

During App builds, all `@(_AndroidRemapMembers)` files are merged
into an `@(AndrodAsset)` named `xa-internal/xa-mam-mapping.xml`.
This asset is opened and provided to `JNIEnv.Initialize()` as part of
native app startup.


~~ Putting it all together ~~

This will only work on .NET 7+.

App project has a `@(_AndroidRemapMembers)` file.  This item is
processed during App build, stored into the `.apk`, and read during
app startup on an Android device.

Given a Java binding such as:

	public partial class Activity {
	    static readonly JniPeerMembers _members = new XAPeerMembers ("android/app/Activity", typeof (Activity));
	}

when the `JniPeerMembers` constructor runs, it will call
`JniEnvironment.Runtime.TypeManager.GetReplacementType("android/app/Activity")`.
If `@(_AndroidRemapMembers)` is based on the InTune
`remapping-config.json` file, then `android/app/Activity` is mapped
to `com/microsoft/intune/mam/client/app/MAMActivity`, and
`JNIEnv::FindClass()` will be told to lookup `MAMActivity`, *not*
`Activity`.

*If `MAMActivity` can't be found*, e.g. you're testing this all out,
the app will ~immediately crash, as `MAMActivity` doesn't exist. 😅

If `MAMActivity` can be found, eventually `Activity.OnCreate()` will
need to be invoked:

	partial class Activity {
	    protected virtual unsafe void OnCreate (Android.OS.Bundle? savedInstanceState)
	    {
	        const string __id = "onCreate.(Landroid/os/Bundle;)V";
	        try {
	            JniArgumentValue* __args = stackalloc JniArgumentValue [1];
	            __args [0] = new JniArgumentValue ((savedInstanceState == null) ? IntPtr.Zero : ((global::Java.Lang.Object) savedInstanceState).Handle);
	            _members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
	        } finally {
	            global::System.GC.KeepAlive (savedInstanceState);
	        }
	    }
	}

`_members.InstanceMethods.InvokeVirtualVoidMethod()` will internally
make a call similar to:

	var r = JniEnvironment.Runtime.TypeManager.GetReplacementMethodInfo (
	    "com/microsoft/intune/mam/client/app/MAMActivity",
	    "onCreate",
	    "(Landroid/os/Bundle;)V"
	);

The data returned will be equivalent to:

	var r = new JniRuntime.ReplacementMethodInfo {
	    SourceJniType                   = "com/microsoft/intune/mam/client/app/MAMActivity",    // from input parameter
	    SourceJniMethodName             = "onCreate",                                           // from input parameter
	    SourceJniMethodSignature        = "(Landroid/os/Bundle;)V",                             // from input parameter

	    TargetJniType                   = "com/microsoft/intune/mam/client/app/MAMActivity",    // from //replace-method/@target-type
	    TargetJniMethodName             = "onMAMCreate",                                        // from //replace-method/@target-method-name
	    TargetJniMethodSignature        = "(Landroid/os/Bundle;)V",                             // from input parameter, as signature didn't change
	    TargetJniMethodParameterCount   = 1,                                                    // computed based on signature
	    TargetJniMethodIsStatic         = false,                                                // from //replace-method/@target-method-instance-to-static
	}

This will allow `_members.InstanceMethods.InvokeVirtualVoidMethod()`
to instead resolve and invoke `MAMActivity.onMAMCreate()`.


~~ Tools ~~

`tools/remap-mam-json-to-xml` is added, and will process the InTune
JSON file into `@(_AndroidRemapMembers)` XML:

	$ dotnet run --project tools/remap-mam-json-to-xml  -- \
	  $HOME/.nuget/packages/microsoft.intune.mam.remapper.tasks/0.1.4635.1/content/MonoAndroid10/remapping-config.json
	<replacements>…


~~ Unit Tests ~~

`@(_AndroidRemapMembers)` usage is required by
`Mono.Android.NET-Tests.apk`, as `Java.Interop-Tests.dll` exercises
the type and member remapping logic.


~~ Unrelated `gref+` logging fixes ~~

When `debug.mono.log` = `gref+`, the app could crash:

	signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x79c045dead

This was likely because a constant string was provided to
`OSBridge::_write_stack_trace()`, which tried to write into the
constant string, promptly blowing things up.

Workaround: don't use `gref+` logging when a GC occurs?
(Fortunately, `gref+` logging isn't the default.)

Better workaround: Don't Do That™. Don't write to const strings.


~~ About `@(_AndroidRemapMembers)` Semantics… ~~

 1. Changing the Java hierarchy "requires" changing the managed
    hierarchy to mirror it.

    If we rename `Activity` to `RemapActivity` but *don't* change
    `MainActivity` to inherit the (bound!) `Example.RemapActivity`,
    the app *crashes*:

        JNI DETECTED ERROR IN APPLICATION: can't call void example.RemapActivity.onMyCreate(android.os.Bundle) on instance of example.MainActivity

    This can be "fixed" *without* changing the base class
    of `MainActivity` by instead changing the base class of the
    Java Callable Wrapper for `MainActivity` to `example.RemapActivity`.
    This can be done manually (just edit the files in `obj/…`!), but
    isn't really supported in "normal" xamarin-android usage
    (the next Clean will wipe your changes).

    Presumably InTune would make this Just Work by e.g. patching the
    `MainActivity.class` file.

 2. `/replacements/replace-type` interacts with
    `/replacements/replace-method`: at runtime, `//replace-type@from`
    *no longer exists*, meaning you ***cannot*** use that name
    in `//replace-method/@source-type` either!

    If `Activity` is remapped to `RemapActivity`, then *there is no*
    `Activity.onCreate()` method to similarly remap.  Instead, you
    need to specify `RemapActivity.onCreate()`.

    This warps the brain a bit.

    This:

        <replace-method
            source-type="example/RemapActivity"
            source-method-name="onCreate"
            target-type="example/RemapActivity"
            target-method-name="onMyCreate" target-method-instance-to-static="false" />

    not this:

        <replace-method
            source-type="android/app/Activity"
            source-method-name="onCreate"
            target-type="example/RemapActivity"
            target-method-name="onMyCreate" target-method-instance-to-static="false" />

 3. Don't intermix type renames with
    `/replace-method/@target-method-instance-to-static='true']`.
    It *can* be done, but also warps the brain.

    The deal with `@target-method-instance-to-static` is that it
    it changes the target method signature -- unless explicitly
    provided in `/replace-method/@target-method-signature` --
    so that the "source declaring type" is a prefix.

    Thus given

        <replace-method
            source-type="android/view/View"
            source-method-name="setOnClickListener"
            target-type="example/ViewHelper"
            target-method-name="mySetOnClickListener" target-method-instance-to-static="true" />

    we'll look for `ViewHelper.mySetOnClickListener(View, View.OnClickListener)`.

    If we renamed `View` to `MyView`, we would instead look for
    `ViewHelper.mySetOnClickListener(MyView, View.OnClickListener)`
    (note changed parameter type).

    This almost certainly *won't* work.


~~ InTune Integration Testing? ~~

For "more complete" InTune integration testing, one will want the
path to `remapping-config.json`, without hardcoding things.
This can be done with `%(PackageReference.GeneratePathProperty)`=True
and using `$(PkgMicrosoft_Intune_MAM_Remapper_Tasks)`:

	<ItemGroup>
	  <PackageReference
	      Include="Microsoft.Intune.MAM.Remapper.Tasks"
	      Version="0.1.4635.1"
	      IncludeAssets="none"
	      GeneratePathProperty="True"
	      ReferenceOutputAssembly="False"
	  />
	</ItemGroup>
	<Target Name="_AddMamFiles"
	    BeforeTargets="_AddAndroidCustomMetaData">
	  <ItemGroup>
	    <_AndroidMamMappingFile Include="$(PkgMicrosoft_Intune_MAM_Remapper_Tasks)/content/MonoAndroid10/remapping-config.json" />
	  </ItemGroup>
	</Target>

This is still fraught with some peril, as it likely also depends on
getting the right "inner" build, which may require using the plural
`$(TargetFrameworks)` property, not the singular `$(TargetFramework)`.
This might still be a useful start.


~~ TODO ~~

Optimize this mess:
dotnet#7020

[0]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html
[1]: https://developer.android.com/studio/write/java8-support#library-desugaring
[2]: https://developer.android.com/reference/java/util/Arrays#stream(T[])
[3]: https://docs.microsoft.com/en-us/mem/intune/fundamentals/what-is-intune
[4]: https://www.nuget.org/packages/Microsoft.Intune.MAM.Remapper.Tasks/
@grendello grendello mentioned this issue Jun 1, 2022
Merged
jonpryor pushed a commit that referenced this issue Jun 7, 2022
@grendello @jonathanpeppers
[Xamarin.Android.Build.Tasks, monodroid] Optimize member remapping (#…
f99fc81 
…7059)

Fixes: #7020

Context: f6f11a5

Commit f6f11a5 introduced type & member remapping.  The problem with
its approach was that it used XML as the representation format, which
needed to be parsed during process startup.  This would contribute to
app startup slowdowns if there were any remappings, but even if there
weren't any remappings, the minimum App size increased by ~490KB, due
to the added dependencies on `System.Private.Xml.dll` & more.

This app size increase is not ideal.

Remove most of the `.apk` size increases by moving the remapping info
into `libxamarin-app.so`.  If no remappings are used, then the size
increase to `libxamarin-app.so` is negligible, and the `.apk` size is
only 61KB larger than pre-f6f11a5a.


~~ API Changes ~~

`Android.Runtime.AndroidTypeManager` gains the following members:

	partial class AndroidTypeManager {
	    struct JniRemappingReplacementMethod {
	        public string target_type, target_name;
	        public bool   is_static;
	    }
	    static extern byte* _monodroid_lookup_replacement_type (
	            string jniSimpleReference
	    );
	    static extern JniRemappingReplacementMethod* _monodroid_lookup_replacement_method_info (
	            string jniSourceType,
	            string jniMethodName,
	            string jniMethodSignature
	    );
	}

`AndroidTypeManager._monodroid_lookup_replacement_type()` replaces
the `JNIEnv.ReplacementTypes` dictionary from f6f11a5.

`AndroidTypeManager._monodroid_lookup_replacement_method_info()`
replaces the `JNIEnv.ReplacementMethods` dictionary from f6f11a5.

Both `_monodroid_lookup_replacement_type()` and
`_monodroid_lookup_replacement_method_info()` are P/Invokes into
`libxamarin-app.so`.


~~ `libxamarin-app.so` Changes ~~

The contents of the `@(_AndroidRemapMembers)` item group are now
stored within `libxamarin-app.so`, with the following structure:

	const uint32_t                          jni_remapping_replacement_method_index_entry_count;
	const JniRemappingIndexTypeEntry        jni_remapping_method_replacement_index[];

	const uint32_t                          jni_remapping_replacement_type_count;
	const JniRemappingTypeReplacementEntry  jni_remapping_type_replacements[];

	struct JniRemappingString {
	    const uint32_t                  length;
	    const char                     *str;
	};
	struct JniRemappingReplacementMethod {
	    const char                     *target_type, *target_name;
	    const bool                      is_static;
	};
	struct JniRemappingIndexMethodEntry {
	    JniRemappingString              name, signature;
	    JniRemappingReplacementMethod   replacement
	};
	struct struct JniRemappingIndexTypeEntry {
	    JniRemappingString              name;
	    uint32_t                        method_count;
	    JniRemappingIndexMethodEntry   *methods;
	};
	struct JniRemappingTypeReplacementEntry {
	    JniRemappingString              name;
	    const char                     *replacement;
	};
	const char *
	_monodroid_lookup_replacement_type (const char *);

	const JniRemappingReplacementMethod*
	_monodroid_lookup_replacement_method_info (const char *jniSourceType, const char *jniMethodName, const char *jniMethodSignature);

Referring to the `<replacements/>` XML from f6f11a5 in
`@(_AndroidRemapMembers)`:

  * `//replace-type/@from` fills `JniRemappingTypeReplacementEntry::name`,
    `//replace-type/@to` fills `JniRemappingTypeReplacementEntry::replacement`,
    and `_monodroid_lookup_replacement_type()` performs a linear
    search over `jni_remapping_type_replacements`.

  * `//replace-method/@source-type` fills `JniRemappingIndexTypeEntry::name`,
    `//replace-method/@source-method-name` fills `JniRemappingIndexMethodEntry::name`,
    `//replace-method/@source-method-signature` fills `JniRemappingIndexMethodEntry::signature`,
    `//replace-method/@target-type` fills `JniRemappingReplacementMethod::target_type`,
    `//replace-method/@target-method-name` fills `JniRemappingReplacementMethod::target_name`,
    `//replace-method/@target-method-signature` and
    `//replace-method/@target-method-parameter-count` are *ignored*,
    `//replace-method/@target-method-instance-to-static` fills `JniRemappingReplacementMethod::is_static`,
    and `_monodroid_lookup_replacement_method_info()` performs a
    search over `jni_remapping_method_replacement_index` looking for
    entries with "matching" type names, method names, and method
    signatures, and once a match is found it returns a pointer to the
    `JniRemappingReplacementMethod` instance.

Co-authored-by: Jonathan Peppers <jonathan.peppers@gmail.com>
@ghost ghost locked as resolved and limited conversation to collaborators Jul 8, 2022
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Assignees

@grendello grendello

Labels
Area: App Runtime Issues in `libmonodroid.so`.
Projects
None yet
Milestone
.NET 7 Servicing
Development

Successfully merging a pull request may close this issue.

Mam remapping opt grendello/xamarin-android
2 participants
@jonpryor @grendello