[xcode16] Merge main into xcode16. #20989

rolfbjarne · 2024-08-06T17:54:38Z

No description provided.

…g. (#20907) I have cleaned the yaml files while I was trying to debug the issue with the provisioning of the simulators on Xcode 16. We should back port this change there.

This pull request updates the following dependencies [marker]: <> (Begin:Coherency Updates) ## Coherency Updates The following updates ensure that dependencies with a *CoherentParentDependency* attribute were produced in a build used as input to the parent dependency's build. See [Dependency Description Format](https://github.com/dotnet/arcade/blob/master/Documentation/DependencyDescriptionFormat.md#dependency-description-overview) [DependencyUpdate]: <> (Begin) - **Coherency Updates**: - **Microsoft.NET.ILLink.Tasks**: from 8.0.5 to 8.0.7 (parent: Microsoft.Dotnet.Sdk.Internal) - **Microsoft.AspNetCore.App.Ref**: from 8.0.5 to 8.0.7 (parent: Microsoft.Dotnet.Sdk.Internal) - **Microsoft.NETCore.App.Ref**: from 8.0.5 to 8.0.7 (parent: Microsoft.Dotnet.Sdk.Internal) - **Microsoft.NET.Workload.Emscripten.Current.Manifest-8.0.100**: from 8.0.5 to 8.0.7 (parent: Microsoft.NETCore.App.Ref) - **Microsoft.NETCore.App.Ref**: from 8.0.5 to 8.0.7 (parent: Microsoft.Dotnet.Sdk.Internal) [DependencyUpdate]: <> (End) [marker]: <> (End:Coherency Updates) [marker]: <> (Begin:80cb9ffd-f92f-4fc8-9f8b-08dbca46abfb) ## From https://github.com/dotnet/installer - **Subscription**: 80cb9ffd-f92f-4fc8-9f8b-08dbca46abfb - **Build**: 20240717.6 - **Date Produced**: July 17, 2024 5:52:17 PM UTC - **Commit**: b77bfa17bd4c52d32d5bdbd031ddf66da44a06a1 - **Branch**: refs/heads/release/8.0.1xx [DependencyUpdate]: <> (Begin) - **Updates**: - **Microsoft.Dotnet.Sdk.Internal**: [from 8.0.107-servicing.24324.1 to 8.0.108-servicing.24367.6][21] - **Microsoft.NET.ILLink.Tasks**: [from 8.0.5 to 8.0.7][22] - **Microsoft.AspNetCore.App.Ref**: [from 8.0.5 to 8.0.7][23] - **Microsoft.NETCore.App.Ref**: [from 8.0.5 to 8.0.7][22] - **Microsoft.NET.Workload.Emscripten.Current.Manifest-8.0.100**: [from 8.0.5 to 8.0.7][24] - **Microsoft.NETCore.App.Ref**: [from 8.0.5 to 8.0.7][22] [21]: dotnet/installer@03065ca...b77bfa1 [22]: https://dev.azure.com/dnceng/internal/_git/dotnet-runtime/branches?baseVersion=GC087e15321bb712ef6fe8b0ba6f8bd12facf92629&targetVersion=GC2aade6beb02ea367fd97c4070a4198802fe61c03&_a=files [23]: https://dev.azure.com/dnceng/internal/_git/dotnet-aspnetcore/branches?baseVersion=GCc9e3996173cec136bc2e9f3b4ec45f2a323b1d63&targetVersion=GC2f1db20456007c9515068a35a65afdf99af70bc6&_a=files [24]: dotnet/emsdk@71359b1...a64772f [DependencyUpdate]: <> (End) [marker]: <> (End:80cb9ffd-f92f-4fc8-9f8b-08dbca46abfb) --------- Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions@xamarin.com> Co-authored-by: Alex Soto <alex@alexsoto.me>

@ivanpovazan

fixes: dotnet/maui#23577 ## Cause: `MtouchInterpreter` is set as https://github.com/xamarin/xamarin-macios/blob/d1ec7a793f46f615f5f08f0a60f4c6daa6a1b671/msbuild/Xamarin.Shared/Xamarin.Shared.props#L308 `DynamicCodeSupport` is set by https://github.com/xamarin/xamarin-macios/blob/d1ec7a793f46f615f5f08f0a60f4c6daa6a1b671/dotnet/targets/Xamarin.Shared.Sdk.targets#L146 `Xamarin.Shared.Sdk.targets` is imported *before* `Xamarin.Shared.props` as shown below (courtesy of @ivanpovazan) <img src="https://github.com/user-attachments/assets/c97b7f01-2372-4f9d-bedf-83060eed1c50"> so unless the value of `MtouchInterpreter` is set in the project, it's value will be empty when the `DynamicCodeSupport` property is evaluated. ## Resolution: To minimize the impact of this change, until it can be investigated more fully, the value of `MtouchInterpreter` is evaluated as https://github.com/xamarin/xamarin-macios/blob/d1ec7a793f46f615f5f08f0a60f4c6daa6a1b671/msbuild/Xamarin.Shared/Xamarin.Shared.props#L308 So adding `( '$(MtouchInterpreter)' == '' And '$(UseInterpreter)' == 'false' )` to the definition of `DynamicCodeSupport` to match the `MtouchInterpreter` definition. A further fix might be to either: 1. Reorder the imports so that the props are included before the targets, although the ramifications of that change could be significant 2. Move the definition of `DynamicCodeSupport` to `Xamarin.Shared.props`. This at first glance seems to be less significant than 1) but would need review and testing. --------- Co-authored-by: Ivan Povazan <55002338+ivanpovazan@users.noreply.github.com>

This pull request updates the following dependencies [marker]: <> (Begin:601bc5e1-1cae-44b5-cf5f-08db9342aa2f) ## From https://github.com/dotnet/xharness - **Subscription**: 601bc5e1-1cae-44b5-cf5f-08db9342aa2f - **Build**: 20240708.1 - **Date Produced**: July 8, 2024 9:23:17 PM UTC - **Commit**: 2c5fdbed5ea74da70649c03084bc01035f35a3fe - **Branch**: refs/heads/main [DependencyUpdate]: <> (Begin) - **Updates**: - **Microsoft.DotNet.XHarness.iOS.Shared**: [from 9.0.0-prerelease.24354.1 to 9.0.0-prerelease.24358.1][1] [1]: dotnet/xharness@0302f46...2c5fdbe [DependencyUpdate]: <> (End) [marker]: <> (End:601bc5e1-1cae-44b5-cf5f-08db9342aa2f) Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Alex Soto <alex@alexsoto.me>

## Description As part of the fix for: dotnet/runtime#99248 we disabled dedup optimization in partial/hybrid AOT mode (when both interpreter and AOT compiler are enabled). This change got backported to .NET 8 and with the latest servicing release regressed build times and app sizes significantly as reported in: #20848 However, it turns out that disabling dedup optimization is not required to fix dotnet/runtime#99248 but instead we should correct the Xamarin SDK integration with this optimization which this PR is doing. The following section describes the initial problem in more details. ## Overview of AOT modes and dedup optimization When the repro project from dotnet/runtime#99248 is built with dedup enabled in hybrid AOT+interpreter mode, the app crashes with: ``` 024-07-23 14:32:37.524110+0200 IvansApp[12711:20244208] debug: AOT NOT FOUND: (wrapper other) object:gsharedvt_out_sig (intptr). 2024-07-23 14:32:37.524120+0200 IvansApp[12711:20244208] error: * Assertion at /Users/ivan/repos/runtime-mono-iOS/src/mono/mono/mini/interp/interp.c:2667, condition `is_ok (error)' not met, function:init_jit_call_info, Attempting to JIT compile method '(wrapper other) void object:gsharedvt_out_sig (intptr)' while running in aot-only mode. See https://learn.microsoft.com/xamarin/ios/internals/limitations for more information. ``` To track down why these wrappers which are used to transition from interpreter to AOT code, are not generated we need to understand when they are compiled in different AOT modes with and without dedup optimization enabled: - In full AOT setup - all assemblies AOT compiled - `gsharedvt_out_sig` methods are never generated - In hybrid AOT + interpreter setup - all assemblies AOT compiled: `MtouchInterpreter=-all` - Dedup OFF: - `gsharedvt_out_sig` methods are generated in AOT images of every assembly (to enable interpreter calling into each specific assembly - here wrappers with same signatures are duplicated) - Dedup ON: - `gsharedvt_out_sig` methods are generated only in `aot-instances` AOT image - during AOT compilation of individual assemblies generation of `gsharedvt_out_sig` is skipped - during AOT compilation of `aot-instances` assembly we collect all `gsharedvt_out_sig` variants from the full program scope and generate code for them in `aot-instances` AOT image - In hybrid AOT + interpreter setup - all assemblies interpreted except a given assembly: `MtouchInterpreter=all,-MyAssembly` - Dedup OFF: - `gsharedvt_out_sig` methods are generated in AOT image of `MyAssembly` (to enable interpreter calling into it) - Dedup ON: <- $${\color{red} ISSUE }$$ - `gsharedvt_out_sig` methods *should* be generated only in `aot-instances` AOT image, but the `aot-instances` image is missing - explanation: - what happens is that generation of `gsharedvt_out_sig` is skipped during AOT compilation of `MyAssembly` (as expected). - But, the build does not mark `aot-instances` assembly as the one that should be AOT compiled. - The reason for this is that we have a global `_IsDedupEnabled` flag, but when custom linker step analysis `aot-instances.dll` it does not see it as an assembly which should not be interpreted. - To explain that better: we mark *all* assemblies as to be interpreted (via: `MtouchInterpreter=all`), but exclude only `MyAssembly` (via: `MtouchInterpreter=all,-MyAssembly`). - So when custom linker step processes `aot-instaces.dll` it treats it as an assembly to be interpreted, so it does not mark it for AOT compilation. - This further results with `aot-instances` AOT image missing, and all the methods which we skipped during AOT compilation never get generated. ## The fix To fix this and address regressions reported in: #20848 we are reenabling dedup optimization whenever AOT compilation is requested and fixing the issue where the custom linker step for generating AOT parameters always treates the dedup assembly as the one to be AOTed. Once approved this should be backported to .NET 8 as servicing releases are also affected with it. --------- Co-authored-by: GitHub Actions Autoformatter <github-actions-autoformatter@xamarin.com>

## Description This is a follow-up PR to: #20936 We should not enable dedup when targeting `maccatalyst-x64` because in case when the project file specifies `MtouchInterpreter=all,-System.Private.CoreLib`, the build will run the full AOT compiler with interpreter enabled. In this setup the runtime is configured to run in interp only mode: https://github.com/xamarin/xamarin-macios/blob/97a91cc4e3bf7cf8a4c657ca00ece620f2e28e91/tools/common/Target.cs#L812-L813 which means that during runtime, AOT images will be ignored - aot runtime will load them but mark them as unusuable since they are compiled with `full` compiler switch and the code falls back to interpreter (ref: https://github.com/dotnet/runtime/blob/efebf202a4a9bd78702bf4bf28a027f093e15d89/src/mono/mono/mini/aot-runtime.c#L2131-L2148 ) This is problematic for the `aot-instances` container image, which has a special handling at the runtime and does not accept it to be marked as unusable (we might want to revisit this in the future): https://github.com/dotnet/runtime/blob/efebf202a4a9bd78702bf4bf28a027f093e15d89/src/mono/mono/mini/aot-runtime.c#L2527-L2529 ## Changes This PR disables dedup optimization when targeting `maccatalyst-x64` and updates the required tests to match the behavior. --- Backports: - [x] #20946 - [x] Original reenabling of dedup for .NET 9 already includes these changes: #20941 --------- Co-authored-by: GitHub Actions Autoformatter <github-actions-autoformatter@xamarin.com>

This post-build pipeline should only run against net8.0 servicing branches that don't have the commit (2abbaf9) that moved maestro publishing into the regular build pipeline.

Backport of #20952 -------- ## Description Previous fix #20945 did not take into account that when we target non arm64 apple mobile platforms we are using `MONO_AOT_MODE_INTERP_ONLY` which cannot use dedup optimization as it discards AOT images during runtime. ## Changes - Enable dedup only when targeting ARM64 - Fix tests to cover builds for both ARM64 and X64 builds Finally, the change was tested against MAUI iossimulator-x64 template app --------- Co-authored-by: Ivan Povazan <ivan.povazan@gmail.com> Co-authored-by: Ivan Povazan <55002338+ivanpovazan@users.noreply.github.com>

This is mainly to update the suppression files, we don't need any of the changes in maccore. Diff: https://github.com/xamarin/maccore/compare/05609a34411fc8a0bc5224dabd2e6897e65167cf..d3ed4b343e2f39f7afc5d54b30aab74895a13dca

This pull request updates the following dependencies ## From https://github.com/dotnet/xharness - **Subscription**: 601bc5e1-1cae-44b5-cf5f-08db9342aa2f - **Build**: 20240729.1 - **Date Produced**: July 29, 2024 12:18:31 PM UTC - **Commit**: edae8a9491f747ba2a36023075aa3ac57be4d32e - **Branch**: refs/heads/main - **Updates**: - **Microsoft.DotNet.XHarness.iOS.Shared**: [from 9.0.0-prerelease.24358.1 to 9.0.0-prerelease.24379.1][2] [2]: dotnet/xharness@2c5fdbe...edae8a9 --------- Co-authored-by: Alex Soto <alex@soto.dev>

While not strictly necessary for a successful build, the API diff will fail unless we have the correct global.json contents checked in.

It didn't work at some point, but seems to be working fine now. --------- Co-authored-by: Alex Soto <alex@soto.dev>

rolfbjarne · 2024-08-08T08:58:08Z

This PR needs an updated ObjectiveSharpie.

…-in-xcode16-2024-08-06

rolfbjarne · 2024-08-12T09:25:10Z

/azp run

azure-pipelines · 2024-08-12T09:25:34Z

Azure Pipelines successfully started running 1 pipeline(s).

vs-mobiletools-engineering-service2 · 2024-08-12T11:40:07Z

💻 [CI Build] Tests on macOS X64 - Mac Sonoma (14) passed 💻

✅ All tests on macOS X64 - Mac Sonoma (14) passed.

Pipeline on Agent
Hash: 1fca27f2bccab0537780a96fae00f3828f03abdf [PR build]

vs-mobiletools-engineering-service2 · 2024-08-12T11:42:27Z

💻 [CI Build] Tests on macOS M1 - Mac Big Sur (11) passed 💻

✅ All tests on macOS M1 - Mac Big Sur (11) passed.

Pipeline on Agent
Hash: 1fca27f2bccab0537780a96fae00f3828f03abdf [PR build]

vs-mobiletools-engineering-service2 · 2024-08-12T11:43:49Z

💻 [CI Build] Tests on macOS M1 - Mac Monterey (12) passed 💻

✅ All tests on macOS M1 - Mac Monterey (12) passed.

Pipeline on Agent
Hash: 1fca27f2bccab0537780a96fae00f3828f03abdf [PR build]

vs-mobiletools-engineering-service2 · 2024-08-12T11:44:30Z

📚 [CI Build] Artifacts 📚

Packages generated

View packages

Pipeline on Agent
Hash: [PR build]

vs-mobiletools-engineering-service2 · 2024-08-12T11:44:31Z

💻 [CI Build] Tests on macOS M1 - Mac Ventura (13) passed 💻

✅ All tests on macOS M1 - Mac Ventura (13) passed.

Pipeline on Agent
Hash: 1fca27f2bccab0537780a96fae00f3828f03abdf [PR build]

vs-mobiletools-engineering-service2 · 2024-08-12T12:05:04Z

❌ [CI Build] Windows Integration Tests failed ❌

❌ Failed ❌

Pipeline on Agent
Hash: 1fca27f2bccab0537780a96fae00f3828f03abdf [PR build]

vs-mobiletools-engineering-service2 · 2024-08-12T15:25:51Z

🚀 [CI Build] Test results 🚀

Test results

✅ All tests passed on VSTS: test results.

🎉 All 173 tests passed 🎉

Tests counts

✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (MacCatalyst): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 6 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 7 tests passed. Html Report (VSDrops) Download
✅ generator: All 2 tests passed. Html Report (VSDrops) Download
✅ install-source: All 1 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 7 tests passed. [attempt 2] Html Report (VSDrops) Download
✅ introspection: All 13 tests passed. Html Report (VSDrops) Download
✅ linker: All 65 tests passed. Html Report (VSDrops) Download
✅ mac-binding-project: All 1 tests passed. Html Report (VSDrops) Download
✅ mmp: All 2 tests passed. Html Report (VSDrops) Download
✅ mononative: All 6 tests passed. Html Report (VSDrops) Download
✅ monotouch (iOS): All 11 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 7 tests passed. Html Report (VSDrops) Download
✅ monotouch (macOS): All 8 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 11 tests passed. Html Report (VSDrops) Download
✅ monotouch (watchOS): All 4 tests passed. [attempt 2] Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ mtouch: All 1 tests passed. [attempt 2] Html Report (VSDrops) Download
✅ xammac: All 3 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 8 tests passed. [attempt 2] Html Report (VSDrops) Download
✅ xtro: All 2 tests passed. Html Report (VSDrops) Download

Pipeline on Agent
Hash: 1fca27f2bccab0537780a96fae00f3828f03abdf [PR build]

rolfbjarne · 2024-08-12T15:32:37Z

Test failure is unrelated (#20117).

mandel-macaque and others added 13 commits July 17, 2024 13:25

[CI] Clean a number of yamllint issues and update a name to help debu…

ce67ab3

…g. (#20907) I have cleaned the yaml files while I was trying to debug the issue with the provisioning of the simulators on Xcode 16. We should back port this change there.

[ci] Update post-build pipeline triggers (#20950)

11f2ed6

This post-build pipeline should only run against net8.0 servicing branches that don't have the commit (2abbaf9) that moved maestro publishing into the regular build pipeline.

Bump maccore. (#20960)

3cdedfd

This is mainly to update the suppression files, we don't need any of the changes in maccore. Diff: https://github.com/xamarin/maccore/compare/05609a34411fc8a0bc5224dabd2e6897e65167cf..d3ed4b343e2f39f7afc5d54b30aab74895a13dca

Fail the build if global.json was updated. (#20983)

bb5cc93

While not strictly necessary for a successful build, the API diff will fail unless we have the correct global.json contents checked in.

[xtro] Enable Mac Catalyst. (#20974)

47137d6

It didn't work at some point, but seems to be working fine now. --------- Co-authored-by: Alex Soto <alex@soto.dev>

[xcode16] Merge main into xcode16.

2dc0efe

dalexsoto approved these changes Aug 6, 2024

View reviewed changes

This comment has been minimized.

# to view

mandel-macaque approved these changes Aug 6, 2024

View reviewed changes

This comment has been minimized.

# to view

[xtro] Fix Mac Catalyst stuff for Xcode 16.

fe71a5f

This comment has been minimized.

# to view

rolfbjarne mentioned this pull request Aug 9, 2024

[Xcode16] Bump Xcode to beta 5. #20998

Merged

rolfbjarne added 2 commits August 9, 2024 18:27

Merge remote-tracking branch 'origin/xcode16' into dev/rolf/bump-main…

5492bd1

…-in-xcode16-2024-08-06

[xtro] Fix Mac Catalyst stuff for Xcode 16 b5.

1fca27f

haritha-mohan approved these changes Aug 9, 2024

View reviewed changes

This comment has been minimized.

# to view

rolfbjarne merged commit 9f04f08 into xcode16 Aug 12, 2024
28 checks passed

rolfbjarne deleted the dev/rolf/bump-main-in-xcode16-2024-08-06 branch August 12, 2024 15:32

[xcode16] Merge main into xcode16. #20989

[xcode16] Merge main into xcode16. #20989

Conversation

rolfbjarne commented Aug 6, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rolfbjarne commented Aug 8, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rolfbjarne commented Aug 12, 2024

azure-pipelines bot commented Aug 12, 2024

vs-mobiletools-engineering-service2 commented Aug 12, 2024

💻 [CI Build] Tests on macOS X64 - Mac Sonoma (14) passed 💻

vs-mobiletools-engineering-service2 commented Aug 12, 2024

💻 [CI Build] Tests on macOS M1 - Mac Big Sur (11) passed 💻

vs-mobiletools-engineering-service2 commented Aug 12, 2024

💻 [CI Build] Tests on macOS M1 - Mac Monterey (12) passed 💻

vs-mobiletools-engineering-service2 commented Aug 12, 2024

📚 [CI Build] Artifacts 📚

Packages generated

vs-mobiletools-engineering-service2 commented Aug 12, 2024

💻 [CI Build] Tests on macOS M1 - Mac Ventura (13) passed 💻

vs-mobiletools-engineering-service2 commented Aug 12, 2024

❌ [CI Build] Windows Integration Tests failed ❌

This comment has been minimized.

vs-mobiletools-engineering-service2 commented Aug 12, 2024

🚀 [CI Build] Test results 🚀

Test results

Tests counts

rolfbjarne commented Aug 12, 2024