Android SDK XML warnings when using a user-provided SDK

[mhsmith]

Describe the bug
I used the ANDROID_SDK_ROOT environment variable to make Briefcase use my existing Android SDK. This resulted in the following warnings when Briefcase runs sdkmanager --list_installed (duplicate lines removed):

Warning: This version only understands SDK XML versions up to 2 but an SDK XML file of version 3 was encountered. This can happen if you use versions of Android Studio and the command-line tools that were released at different times.
Warning: unexpected element (uri:"", local:"base-extension"). Expected elements are <{}codename>,<{}layoutlib>,<{}api-level>
Warning: unexpected element (uri:"", local:"base-extension"). Expected elements are <{}codename>,<{}api-level>
Warning: unexpected element (uri:"", local:"base-extension"). Expected elements are <{}vendor>,<{}codename>,<{}abi>,<{}api-level>,<{}tag>
Warning: Errors limit exceeded. To receive all errors set com.sun.xml.bind logger to FINEST level.

This is because I usually manage the SDK using Android Studio's graphical SDK Manager (current version: Chipmunk), not the command-line sdkmanager. The cmdline-tools "latest" directory in my SDK is version 5, because I've never had any reason to update it. Version 6 and 7 both work without warnings, so this doesn't affect Briefcase-installed SDKs, which use version 6.

The warnings appear to be harmless, but Briefcase could fix them by installing and using a specific version of cmdline-tools, as discussed at #766 (comment).

Environment:

• Operating System: Windows 10
• Python version: 3.8
• Software versions:
  • Briefcase: 7a95f4b

[mhsmith]

When Briefcase runs Gradle, I get the same "unexpected element" warnings as above, plus a few more:

WARNING:: Mapping new ns http://schemas.android.com/repository/android/common/02 to old ns http://schemas.android.com/repository/android/common/01
Mapping new ns http://schemas.android.com/repository/android/common/02 to old ns http://schemas.android.com/repository/android/common/01
WARNING:: Mapping new ns http://schemas.android.com/repository/android/generic/02 to old ns http://schemas.android.com/repository/android/generic/01
Mapping new ns http://schemas.android.com/repository/android/generic/02 to old ns http://schemas.android.com/repository/android/generic/01
WARNING:: Mapping new ns http://schemas.android.com/sdk/android/repo/addon2/02 to old ns http://schemas.android.com/sdk/android/repo/addon2/01
Mapping new ns http://schemas.android.com/sdk/android/repo/addon2/02 to old ns http://schemas.android.com/sdk/android/repo/addon2/01
WARNING:: Mapping new ns http://schemas.android.com/sdk/android/repo/repository2/02 to old ns http://schemas.android.com/sdk/android/repo/repository2/01
Mapping new ns http://schemas.android.com/sdk/android/repo/repository2/02 to old ns http://schemas.android.com/sdk/android/repo/repository2/01
WARNING:: Mapping new ns http://schemas.android.com/sdk/android/repo/sys-img2/02 to old ns http://schemas.android.com/sdk/android/repo/sys-img2/01
Mapping new ns http://schemas.android.com/sdk/android/repo/sys-img2/02 to old ns http://schemas.android.com/sdk/android/repo/sys-img2/01
WARNING:: Mapping new ns http://schemas.android.com/sdk/android/repo/addon2/03 to old ns http://schemas.android.com/sdk/android/repo/addon2/01
Mapping new ns http://schemas.android.com/sdk/android/repo/addon2/03 to old ns http://schemas.android.com/sdk/android/repo/addon2/01
WARNING:: Mapping new ns http://schemas.android.com/sdk/android/repo/repository2/03 to old ns http://schemas.android.com/sdk/android/repo/repository2/01
Mapping new ns http://schemas.android.com/sdk/android/repo/repository2/03 to old ns http://schemas.android.com/sdk/android/repo/repository2/01
WARNING:: Mapping new ns http://schemas.android.com/sdk/android/repo/sys-img2/03 to old ns http://schemas.android.com/sdk/android/repo/sys-img2/01
Mapping new ns http://schemas.android.com/sdk/android/repo/sys-img2/03 to old ns http://schemas.android.com/sdk/android/repo/sys-img2/01

This is because the Android template uses Android Gradle plugin version 4.2, which presumably has an old version of sdkmanager built into it.

Again, these warnings appear to be harmless, and they don't affect Briefcase-installed SDKs. But they could be fixed by moving the Android template to a newer version of the Android Gradle plugin (I've confirmed that version 7.2 works without warnings). Unfortunately this won't be simple, because all newer versions require Java 11, but Briefcase only supports Java 8.

[freakboy3742]

I've been wondering about those warnings on the Briefcase build - I assumed that it was a Gradle/Gradle plugin version thing, but I wasn't certain.

Given we're moving to a Chaquopy base, it's not worth modifying the template to fix it; but if Chaquopy will benefit from a version bump on the Android Gradle plugin, then that's worth pursing.

If upgrading the Gradle plugin also requires (or would benefit from) a Java 11 update, then that's worth pursuing as well. It shouldn't be too hard to manage - if we modify the Java integration to use a java11 directory, and update the download paths, it will essentially identify as a "new" tool on next run. As an added benefit, Java 11 actually has aarch64 binaries for macOS, so we won't need to run in Rosetta mode.

Regarding the cmdline-tools version - it is inevitable that it will eventually become a problem; the introduction of M1 support, for example, was complicated by having an older version of the cmdline-tools. So - although it's not critical right now, we should definitely address this.

AIUI, cmdline-tools should be self-upgradable with sdkmanager. It looks like there might be a directory structure in the cmdline-tools folder that is designed to accomodate multiple versions; but I don't know if we'll be able to use that reliably to identify which cmdline-tools are installed. Alternatively, we couId run sdkmanager to confirm the version of cmdline-tools that is installed - but, as you've established, that can be slow, so it wouldn't be ideal. Another approach would be to use the marker file that Briefcase drops at time of install; it creates an empty file with the build number referenced in the download URL (currently 8092744) in the cmdline-tools folder to identify which version of the cmdline-tools was originally installed.

[mhsmith]

if Chaquopy will benefit from a version bump on the Android Gradle plugin, then that's worth pursing

At the moment, there's no benefit, but I'm sure we'll have to do it at some point.

Android Gradle plugin version 4.2 is compatible with both Java 8 and 11, so Briefcase could move from 8 to 11 without breaking compatibility with most existing projects, and without needing to support both versions at the same time.

It would, however, break compatibility with projects created before August 2021, which used an even older Android Gradle plugin.

In general it's acceptable for a Briefcase update to require a project regeneration, but we should come up with a proper way of managing this, rather than just hitting the user with a random error message from the underlying tools. For example, the briefcase.toml could contain a line indicating which version of Briefcase it was generated with, and each Briefcase platform module would have a minimum requirement for that version. If the requirement wasn't met, Briefcase would tell the user they need to either rerun briefcase create, or go back to an older version of Briefcase.

[mhsmith]

Since #832, Briefcase no longer routinely runs sdkmanager --list_installed. The warnings from Gradle still appear, though I think only on the first build after starting the Gradle daemon.

[mhsmith]

This is because the Android template uses Android Gradle plugin version 4.2
...
all newer versions require Java 11, but Briefcase only supports Java 8.

Android Gradle plugin 8.0 will soon be released along with Android Studio Flamingo, which comes with Java 17. So whenever we decide to update Briefcase's Java version, we might as well skip over 11 and go straight to 17 (#1271).