authentication at proxy is not supported

[milesqi]

Hello, looks like java sdk doesn't support aws service authentication on a proxy. I have a proxy server with a iam role for s3 bucket access attached. When I use aws cli to access my s3 bucket via this proxy(HTTP_PROXY=http://proxy_host:proxy_port), it works. I don't have any credentials configured one the machine where I issue aws cli command. However, on same machine, using aws java sdk, I can't access s3 bucket via same proxy because aws java sdk requires a credentials. there doesn't seem to be any credential providers available for this scenario.

In a short:

• aws cli (no credentials -> proxy server (iam role configured) -> s3 -> SUCCESS
• aws java sdk (no credential) -> proxy server (iam role configured) -> s3 -> FAILURE (can't create client without credential)

[debora-ito]

@milesqi what do you mean by "proxy server (iam role configured)"? How is the IAM role being configured?

Can you provide a sample code of how you are creating the client?
And what AWS SDK version are you using?

[milesqi]

Hi @debora-ito ,
I created a proxy server (running in ec2 instance) in aws vpc, it's assigned with a iam role for a s3 bucket access. I run aws cli from a vm in our on-premise data center (aws vpc is connected to on-premise data center through some secured tunnel). The idea is that any apps running in on-premise data center can access s3 bucket without local credential configuration if it's allowed to use this proxy server. It's like a remote InstanceProfileCredentialsProvider to me. AWS SDK that I use is 2.16.7. I tried a few different ways for client code. none of them works.

• Option Avoid spurious warning when uploading empty file using an InputStream. #1
  create default s3 client S3Client.create() with DefaultCredentialsProvider but without giving any credentials, environment variable HTTP_PROXY is configured.

• Option ".CidrIp" missing from {Authorize|Revoke}SecurityGroup{In|E}gressRequestMashallers #2
  create s3 client with StaticCredentialsProvider, but given empty keyid and secret. environment variable HTTP_PROXY is configured.

• Option explicit inheritance of com.amazonaws.AmazonWebServiceClient's methods fo #3
  create s3 client with StatciCredentialProvider, but given empty keyid and secret. use ProxyConfiguration to configure proxy.
  final SdkHttpClient httpClient = ApacheHttpClient
  .builder()
  .proxyConfiguration(
  ProxyConfiguration
  .builder()
  .useSystemPropertyValues(false)
  .endpoint(proxy)
  .build())
  .build();
  S3Client client = S3Client
  .builder()
  .httpClient(httpClient)
  .credentialsProvider(credentialsProvider)
  .region(Region.of(region))
  .build();

• Option Files needed for OSGi and Eclipse #4
  create s3 client with InstanceProfileCredentialsProvider.builder().build()

I have a feeling that aws java skd is always expecting to find credential locally. But it looks like aws cli can find credentials from remote. Here is logs from aws cli for your reference.
2021-03-15 11:27:49,807 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2021-03-15 11:27:49,807 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2021-03-15 11:27:49,808 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role-with-web-identity
2021-03-15 11:27:49,808 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: sso
2021-03-15 11:27:49,808 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2021-03-15 11:27:49,808 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: custom-process
2021-03-15 11:27:49,808 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: config-file
2021-03-15 11:27:49,808 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: ec2-credentials-file
2021-03-15 11:27:49,809 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: boto-config
2021-03-15 11:27:49,809 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: container-role
2021-03-15 11:27:49,809 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: iam-role
2021-03-15 11:27:49,811 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): .elb.us-west-2.amazonaws.com:8888
2021-03-15 11:27:50,073 - MainThread - urllib3.connectionpool - DEBUG - http://.elb.us-west-2.amazonaws.com:8888 "PUT http://169.254.169.254/latest/api/token HTTP/1.1" 200 56
2021-03-15 11:27:50,074 - MainThread - urllib3.connectionpool - DEBUG - Resetting dropped connection: ac94f7012ab5e4405a28a41f48a02c0d-317134de08b48933.elb.us-west-2.amazonaws.com
2021-03-15 11:27:50,343 - MainThread - urllib3.connectionpool - DEBUG - http://.elb.us-west-2.amazonaws.com:8888 "GET http://169.254.169.254/latest/meta-data/iam/security-credentials/ HTTP/1.1" 200 63
2021-03-15 11:27:50,344 - MainThread - urllib3.connectionpool - DEBUG - Resetting dropped connection: ac94f7012ab5e4405a28a41f48a02c0d-317134de08b48933.elb.us-west-2.amazonaws.com
2021-03-15 11:27:50,615 - MainThread - urllib3.connectionpool - DEBUG - http://.elb.us-west-2.amazonaws.com:8888 "GET http://169.254.169.254/latest/meta-data/iam/security-credentials/arn:aws:iam::838711373148:role/etl-s3-dev-iam-role HTTP/1.1" 200 615

[milesqi]

Hi @debora-ito
It looks like it's caused by this line, didn't find similar thing in aws cli code. Instead, aws cli uses NO_PROXY env var to control that rather than hard code it. (https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-proxy.html)

• for v1:

  aws-sdk-java/aws-java-sdk-core/src/main/java/com/amazonaws/internal/ConnectionUtils.java

  Line 40 in c491416

  HttpURLConnection connection = (HttpURLConnection) endpoint.toURL().openConnection(Proxy.NO_PROXY);

• for v2: https://github.com/aws/aws-sdk-java-v2/blob/35267ca707c3fb5cdf7e3e98758a8ef969269183/core/regions/src/main/java/software/amazon/awssdk/regions/internal/util/ConnectionUtils.java#L38

[debora-ito]

@milesqi Apologies for the delayed reply, and thank you for the follow-up comments. I see the issue now, you are trying to use an InstanceProfile credential but not able to because of proxy configurations.

Have you tried to set the instance metadata endpoint to the http.nonProxyHosts system property? If you're using Apache HTTP Client the proxy configuration is also available in the ApacheHttpClient.Builder.

[github-actions]

It looks like this issue hasn’t been active in longer than a week. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.