android: Update Gradle, Android plugin, build tools, NDK #942
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Bumping Android Gradle plugin to 7.2.2 so that it picks up new SDKs and bumping Gradle to 7.3.3 so that it works with the new plugin, and Android build tools to 30.0.3 so that Gradle works, and with those new tools NDK has to be configured differently (finally, it can be pinned in the project configuration itself, instead of being a free variable in the build machine configuration). This is quite a YOLO upgrade over multiple major versions, but I don't think we have a choice. The treadmill must roll forward, because how else we can Innovate™ to keep the number going up. Otherwise, I can't build Android Themis anymore with today's tooling.
Because Android Studio is being annoying about it, *strongly* suggesting you target the minimum allowed SDK on Google Play. Coincidentally, this is also the version of the SDK that my phone uses, so I let it pass. Note that *minimum* SDK level is still set at 21, we should still be able to run on older devices. Hopefully someone tests that, lol.
New Android stuff is finally available in Google repos and we don't need JCenter anymore. Good night, sweet prince.
Because why not, I'm on a roll. Clicking clickety things in the IDE. These are only used in tests. Unfortunately, Kotlin cannot be updated past 1.4.x branch (current one is 1.7.x) because it desugars into some too new Java 1.8 stuff which is not supported on Android API 21. We'd have to bump our minimum API level to use newer versions. So not today.
Android has its own annotations under "android" and they conflict with Jetbrains annotations. I was never motivated enough to unify them so instead we've been filtering them out for Android builds. New Gradle -- new approach. Such innovative. Much simpler.
Here's a thing. New SDK and NDK build native code differently because of course they do. For example, no more .externalNativeBuild directory with a single build, instead there is .cxx directory with separate debug and release builds under pseudorandom subdirectory. Good news is that Google has finally implemented a way to share native libraries via AARs. Right now we're building BoringSSL as a separate project and it technically produced an AAR before. But it could not be reused by other AARs, we directed the build to look at headers in our source tree and picked up binaries from the build tree. The new way -- Prefab -- adds a whole bunch of complexity that abstracts out native library building process, introduces a bunch of structure into how native libraries are distributed, ultimately allowing to pack your native libraries into reusable AAR artifacts. Bad news is that this is Google project, so obviously it was implemented by someone aiming for a promotion, adds a whole bunch of complexity to solve 80% of the problem -- except that *our* project is in that 20% which does not fucking work out of the fucking box and there is not way to fix it because of all abstraction. Main problem here is that BoringSSL uses CMake build system while libthemis_jni is still built using ndk-build, and that combo is not very well-supported withing the same project. There are reports that Prefab actually works in the consumer project is build using CMake. However. If I were to implemement CMake build for libthemis_jni, then I can simply add_subdirectory() with BoringSSL and build it right there. As long as we don't actually planning to distribute our builds of BoringSSL separately -- and we probably don't, judging how well that worked out for OpenSSL on iOS -- as long as we don't need AARs, we don't really need Prefab. So that's what I'm gonna do. Write a CMake build script for libthemis_jni and use it for all native code of Android Themis.
Now that the script is there, replace ndk-build with CMake. While we're here, pin the verison of CMake in build.gradle so that it's more reproducible.
I don't think we'd ever want to go back to ndk-build, drop it.
Since now BoringSSL is a part of ":android" native code, there is no need to build it as a separate project. We don't have other users.
ilammy
added
O-Android 🤖
vixentael
reviewed
Aug 30, 2022
| // on this Gradle version until it breaks or we *need* an upgrade: | ||
| classpath 'com.android.tools.build:gradle:3.2.1' | ||
| // Android Gradle plugin | ||
| classpath 'com.android.tools.build:gradle:7.2.2' |
vixentael
approved these changes
Aug 30, 2022
Comment on lines
-71
to
-77
| tasks.whenTaskAdded { task -> | ||
| def excludeBoringSSL = [ | ||
| 'publish', | ||
| 'publishToMavenLocal', | ||
| 'publishProductionPublicationToMavenRepository', | ||
| 'publishProductionPublicationToMavenLocal', | ||
| 'generatePomFileForProductionPublication', |
There was a problem hiding this comment.
will this change the way we publish Android Themis?
There was a problem hiding this comment.
will this change the way we publish Android Themis?
No. Same tasks for :android should be used and should work.
I didn't test tho. Later, since this needs to be tested against that golden Docker image, which is probably royally rotten by now and will need to be updated as well to work with all this new SDK stuff.
This removed block was in fact a hack setting up dependencies between :android and :boringssl projects, so that BoringSSL binaries are build before Android JNI code is build. That broke down with new NDK since the task names for building CMake and ndk-build are now different. However, with the build unified into CMake, this hack become unnecessary in principle – thus off it goes.
Harmonize NDK version with what's preinstalled in GitHub Actions environments, shaving off some minutes from build time. We don't particularly care about the NDK version, as long as it's some modern version that might be supported for some next years.
It has newer Android build tools. And yes, still macOS, because running Android emulator on Linux still sucks on GitHub Actions.
No longer necessary. Newer Android tooling is able to filter that out, and it's not installed in newer environments anyway.
But Java 8 for Android's sdkmanager. IKR
Just in case CI environment changes. They typically don't change defaults and keep stuff installed, but who knows. Be explicit.
|
Woops, I saw |
|
omigod, omigod!
|
|
Thank you very much! |
vixentael
approved these changes
Aug 30, 2022
|
@shadinua do you have time to take a look, or shall we merge? |
New SDK calls it "android-release.aar", not just "android.aar".
|
Alrighty. I've checked the release publishing. Docker image indeed required some updates (cossacklabs/dockerfiles#6). After that I was able to build AAR in that image, sign it with a random key, and upload it to Sonatype staging repo. Of course, can't get past since I don't have the real signing key. But the pipeline should be still working. |
|
Linking product docs update https://github.com/cossacklabs/product-docs/pull/265/files |
shadinua
approved these changes
Sep 2, 2022
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These weekly emails about Android build tick me off. Of course Google has to keep breaking stuff, because how else we can progress as a species if Googlers aren't getting promotions for improving something in Android ecosystem. Anyway... you can read some rants in commit messages if you like.
Changes for users of AndroidThemis
None. If you simply get binaries from Maven, those will work. No action required.
We still support minimum Android API 21.
...but if you're building AndroidThemis from source
Then you're in for an update:
Gradle wrapper will download everything automatically. If you open the project in Android Studio, it should build right away.
But why?
Because existing build does not build with modern Android SDKs. Our CI does not built it either.
AndroidThemis has to target at least SDK 30 to be allowed into Play store. People will start nagging us about that. I've upgraded to the latest stuff I can test against. SDK 30 is not the latest SDK, but that's what my phone supports 😞
Other changes
In addition to all this, update the way BoringSSL is built. Newer NDK changes the way native binaries are compiled and reused. Instead of fighting gravity and innovation, I've replaced the old ndk-build with CMake scripts, integrating BoringSSL build into libthemis_jni.so build directly. Hopefully, that's all we need for the forseeable future. Until Google decides to deprecate CMake for Android.
Checklist
Example projects and code samples are up-to-date(nope, maybe when we make a new release?)