1.11.337+ Android cURL Build No Longer Compiles #3014
Comments
AMZN-Gene
added
bug
that PR removed the dependency on OpenSSL from the SDK all together, that parameter should be removed from documentation. can you give me more context on why you need that parameter there? we now use CRT for all crypto related functionalities. |
Cool! If OpenSSL is not longer necessary that would be great. It's possible the "Could NOT find OpenSSL" warning is just a red herring. My build ultimately fails because the Even though ANDROID_BUILD_OPENSSL is gone, I discovered AWS SDK checks for a BUILD_OPENSSL=ON variable. This moves the build further along, but still get a compiler error. |
|
The error comes from a CURL test. Question: If we're not using OpenSSL do we also need cURL? If we do still need cURL, did cURL get updated to not rely on OpenSSL? |
yeah this is what we are trying to do, less dependencies, less trouble....hopefully so lets sort this out.
to elaborate a little bit, openssl was used by the SDK directly for some hashing functions, and we used to directly link against it for cryptographic functions. this is no longer done. this is instead done in the CRT. you definitely still need curl, we plan on using the crt http client in the future but thats in the future. curl has always and will always need some sort of openssl to use tls. We do not prescribe how you get your curl dependency. we just link against it.
this error looks like it happens in your code, if you are brining curl into your dependency set, how are you bringing in openssl? i dont see that test anywhere in our code. |
|
this error looks like it happens in your code, if you are brining curl into your dependency set, how are you bringing in openssl? i dont see that test anywhere in our code. |
|
Looks like AWS Android tools still uses "BUILD_OPENSSL" to build/install OpenSSL for us. Maybe someone just forgot to add some CMake that CURL uses to find where OpenSSL was installed. |
|
this is just out of curiosity by the way, but why are you relying on us building curl and openssl for you? it would seemingly be easier to just use whatever curl and openssl was on the path? it would seem that Give me a bit and i can try to reproduce this, until then I would recommend using a curl or openssl on your path rather than building it alongside the SDK.
where is the test |
|
I maintain a game engine which ships with OpenSSL as a 3rdParty, so I need to be careful about ensuring the same OpenSSL is used across all our libraries. I don't have OpenSSL on my path. AWS's tools/android-build/CMakeLists.txt made it really easy to build and use a specific OpenSSL when using Android (specifically my engine uses OpenSSL 1.1.1g) |
|
so im curious at this point too, what fails to compile as |
|
CURL runs these tests which just tries to compile some simple code and it fails. It's the same error as before (which I was open to solve by letting AWS handle OpenSSL for me :c) |
|
Closer... The build continues, but fails when compiling to missing openssl symbols. for example: I'm able to get around this if I simply hand edit <build_folder>\CMakeFiles\aws-cpp-sdk-core.rsp to include Hand edit fix: Hand editing isn't a viable option, but there's probably a simple flag I'm missing that would place libssl into the build. |
|
I get passed the undefined symbol setting more flags, but run into a missing include dir. Need to inject the proper openSSL include folder into clang's -I parameter. It looks like it's attempting to provide the include path below... but just not quite right.
So it's trying to build an include path, but it's just wrong. Error: |
jmklix
added
p2
|
So even when providing my own OpenSSL, the CURL step is failing. I wonder if it's because we're overwriting OPENSSL_ROOT_DIR to AWS's $OPENSSL_SRC_DIR variable. OPENSSL_SRC_DIR is the variable AWS sets when building OpenSSL, but if OpenSSL is already built (ie, i'm using my own) I need that OPENSSL_ROOT_DIR variable to be the location of OpenSSL already on my hard drive. |
|
if curl is failing to build with that openssl, you could also just bring in your own curl instead of building it, most of our customers bring their own http and crypto deps. We do want remove all deps except the CRT. removing openssl is acutally the first step to that. Android is the only one that offers a "we build dependencies for you". Im still working on reproducing it. |
|
alright replicated with the dockerfils # Use nomral ubuntu base
FROM public.ecr.aws/amazonlinux/amazonlinux:2023
#Install g++
RUN yum groupinstall "Development Tools" -y
#Install required dependencies
RUN yum install -y ninja-build cmake3 wget
#Install NDK
RUN wget -q https://dl.google.com/android/repository/android-ndk-r26d-linux.zip && \
unzip -q android-ndk-r26d-linux.zip
#Try to build SDK
RUN git clone --recurse-submodules https://github.com/aws/aws-sdk-cpp && \
cd aws-sdk-cpp && \
mkdir build &&\
cd build &&\
cmake -DCMAKE_BUILD_TYPE="Debug" \
-DNDK_DIR="/android-ndk-r26d" \
-DTARGET_ARCH=ANDROID \
-DANDROID_NATIVE_API_LEVEL=21 \
-DANDROID_ABI=arm64-v8a \
-DCPP_STANDARD=17 \
-DCMAKE_C_FLAGS="-fPIC" \
-DBUILD_ONLY="s3" \
-DENABLE_TESTING=OFF \
-DENABLE_RTTI=ON \
-DCUSTOM_MEMORY_MANAGEMENT=O \
-DANDROID_BUILD_OPENSSL=ON \
-DANDROID_BUILD_ZLIB=OFF \
-DANDROID_BUILD_CURL=ON \
-DCMAKE_CXX_FLAGS="-fPIC" .. &&\
cmake --build . && \
cmake --install .looking into fixes |
|
created a branch fix-android, im seeing some linker errors in my docker build curious if you have any success with it. although i think that is where you got in your testing, looking into why things arent getting linked |
|
I've been working towards compiling CURL by hand and passing in my own OpenSSL. If I provided a path to my own OpenSSL I get past the recv error, but hit a new error: The new error looked liked: CURL fixed this in January 2019: |
|
I just realized, none of my environment variables are making it to the portion of cmake that's responsible for building CURL: \tools\android-build\CMakeLists.txt For example, if I add a new variable ${GENE}, adding -DGENE to the AWS cmake build command line wont give access to the android-build CURL CMakeList. This prints "GENE: " $GENE is empty. Need to figure out how to get variables into that android-build/CMakeLists.txt EDIT: Nevermind, I found this is set via \cmake\build_external.cmake |
|
alright little update, the branch i have going fix-android does fix the build. im 99 percent sure its because i accidentally removed the lines when I made the PR. Im going to work a little more on that branch to whittle down the "rollback" as im pretty sure those are the only lines that it needs. If you need a working branch right now use that branch at this commit. but I should have a fix ready sooner than later, or rather as quick as my testing feedback loops will allow for it. |
|
created a pull request, you can use/test that branch if you would like to, will give a shout when its been released |
aws/aws-sdk-cpp#3014) Signed-off-by: AMZN-Gene <genewalt@amazon.com>
- Upgrading the AWS SDK 11.361 with AWS_APPSTORE_SAFE flag - 1.11.361 also contains this fix: [Visual Studio 2022 STD::Allocator Warning Treated as Error · Issue #3001 · aws/aws-sdk-cpp (github.com)](aws/aws-sdk-cpp#3001) - 1.11.361 also contains this fix: [1.11.337+ Android cURL Build No Longer Compiles · Issue #3014 · aws/aws-sdk-cpp (github.com)](aws/aws-sdk-cpp#3014) Tested by 1) Grep'ing libraries for non-public api "CryptorGCM" before and after enabling AWS_APPSTORE_SAFE flag 2) Help from external developer re-submitting to Apple AppStore and passing Fixes o3de/o3de#18029 --------- Signed-off-by: Gene Walters <genewalt@amazon.com> Signed-off-by: Gene Walters <32776221+AMZN-Gene@users.noreply.github.com> Signed-off-by: AMZN-Gene <genewalt@amazon.com>
|
This issue is now closed. Comments on closed issues are hard for our team to see. |
Describe the bug
Compiling the AWS SDK for Android no longer works for releases after 1.11.337. The build fails during the cURL build step.
The previous release (1.11.336) still compiles. I think this is because it's missing OpenSSL
Expected Behavior
I should be able to build the SDK successfully for Android
Current Behavior
Compilation fails: notice "Could NOT find OpenSSL"
Reproduction Steps
cmake 3.29
Possible Solution
No response
Additional Information/Context
This PR removed the ANDROID_BUILD_OPENSSL flag, but the readme still says it's available. Is there a replacement?
AWS CPP SDK version used
1.11.337
Compiler and Version used
clang++ Android (9352603, based on r450784d1) clang version 14.0.7
Operating System and version
Windows 10 Enterprise
The text was updated successfully, but these errors were encountered: