SSLHandshake fails on Linux platform when root certificates are not in the platform's default location

[katmsft]

From @Dyutighosh :

I tried running the test cases for cpprest with openssl1.1 version and below is the failed output for https and open ssl test cases:
1.
"error: Failure in set_user_options_asio_https: Unhandled exception: Error in SSL handshake FAILED
Test case authentication_tests:set_user_options_asio_https FAILED"

"** authentication_tests:set_user_options_asio_https FAILED ****
**** outside_tests:multiple_https_requests FAILED ****
**** outside_tests:outside_ssl_json FAILED ****
**** authentication_tests:ssl_test FAILED ****
**** authentication_tests:sni_with_older_server_test FAILED ****
**** authentication_tests:sni_explicit_hostname FAILED **"

[BillyONeal]

We just rely on OpenSSL here -- it needs to be built with the correct path to the configuration / certificates, and that path gets hard coded into the OpenSSL binaries.

vcpkg sets the /etc/ssl as the location because that's where it was on the Ubuntu boxes we tested with -- that'll have to get fixed up however you're building that dependency. https://github.com/Microsoft/vcpkg/pull/4473/files

[mobileben]

Argh, I think this just hit me or perhaps something related to it.

What is interesting about this is that I run this on my Mac with no problem. However when I go to Ubuntu, the same code won't work properly.

On Ubuntu, I'm running in a docker container though on an EC2 instance trying to get to an S3 bucket. I can curl the endpoint from my ssh session. But I fail my call in the then block

client->request(request).then([=] (web::http::http_response response) mutable {
}).then([=] (pplx::task<void> previous_task) mutable {
});

I mounted my ubuntu /etc/ssl to the container's /usr/local/ssl. But it still fails with this error

Error resolving address

The address I've provided actually should be correct.

If I log into an interactive shell of the running container, I can curl the endpoint. And I verified the mounted directory does exist.

I do believe it should be pointing to /usr/local/ssl

I got this output

CERT DIR /usr/local/ssl/certs

When running

        const char *dir;
        
        dir = X509_get_default_cert_dir();

        printf("CERT DIR %s\n", dir);

I will try rebuilding openssl to have openssldir at /etc/local

[mobileben]

I rebuilt openssl to use /etc/ssl

I can see this with this output.

CERT DIR /etc/ssl/certs

Unfortunately I still get the same

Error resolving address

for https calls. I just added support for pulling some config files from S3. So previously I never had code using https (since it didn't require any outbound calls).

Sorry not trying to hijack this thread. I think this is perhaps somewhat related still. I don't use asio or openssl enough to know if I'm missing something...

[BillyONeal]

Are you building openssl yourself or using the .so provided by the platform?

[mobileben]

I've built openssl by myself. I also build myself when running the Mac version too. I do my localhost testing on my Mac. All versions I use are the same.

I also just determined that http doesn't work either. Same issue of not being able to resolve the address. That would seem to eliminate openssl as the culprit? I do notice in code the "Error resolving address" is not the SSL proxy resolve error, so maybe the issue is elsewhere.

The curious thing is this does behave okay on the Mac. It's essentially the same code just complied for different platforms.

[mobileben]

I'm in a container, so I'm double checking if there is some boneheaded mistake I made where I need to enable outbound 443....

Actually scratch that. I can ssh into the container and I can curl my S3 bucket.

[mobileben]

I output the error code and category name for the http_exception. Doesn't seem too helpful at this point.

Error resolving address 1 generic

[BillyONeal]

@mobileben The only place we emit that message is in DNS resolution, nothing to do with SSL https://github.com/Microsoft/cpprestsdk/blob/master/Release/src/http/client/http_client_asio.cpp#L1022

[mobileben]

@BillyONeal yes. I'm trying to figure out why http and https don't work for this particular build configuration but yet works fine on the Mac. Oh, also I have built boost myself as well as openssl. These are both built as statics. I don't need any other special build flags or anything? For example, does boost rely on openssl being built prior to building boost (I currently build it after).

I'll take a step back and make some simple client program standalone. This is the networking for a game server so the code makes it difficult to experiment on.

[mobileben]

@BillyONeal quick update. This seems related to the image I'm basing the docker container on. I had been using busybox and then alpine (to get an interactive shell to examine the file system) as my base image. Those two seem to have some issue in being able to properly resolve. I also added some debug.

DEBUG encoded_resource /bucket/config/ui_manifest_infos.json
DEBUG calling async_resolve tcp_host s3-us-west-2.amazonaws.com tcp_port 80
DEBUG query host name s3-us-west-2.amazonaws.com service name 80
DEBUG handle_resolve error 1 Host not found (authoritative)

I did discover that if I run it on my Ubuntu VM, it works fine. And also, if I use ubuntu:16.04 as the base image for my docker container, it also works. Which is why it seems like the minimal images of busybox/alpine are lacking something.

When it works, the debug output looks like

DEBUG encoded_resource /bucket/config/ui_manifest_infos.json
DEBUG calling async_resolve tcp_host s3-us-west-2.amazonaws.com tcp_port 80
DEBUG query host_name s3-us-west-2.amazonaws.com service_name 80
DEBUG handle_connect

I've tried a few permutations such as using numeric_service when constructing query. I'm doing a rebuild now trying to use "http" versus "80". I still need to verify https at some point.

[BillyONeal]

If you run nslookup in the container does that work? Maybe it has no DNS configured or is being firewalled?