Download from publicly readable S3 bucket with no credentials

[gyulavoros]

Enhancement Suggestion

I would like to set up a workflow, where only a CI service would have proper AWS credentials configured (and have write access to an S3 bucket) and all the other developers in the team would be able to download cached dependencies without providing any environment variables (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY) or stored credentials (.aws/credentials).

Current behavior

With no credentials provided, I got: Unable to read ENV variable: AWS_ACCESS_KEY_ID. Could not find key aws_access_key_idfor profiledefault in file ~/.aws/credentials

With empty access key and secret access key values, I got: The authorization header is malformed; a non-empty Access Key (AKID) must be provided in the credential.

Suggested behavior

There should be a way to download dependencies from publicly accessible S3 bucket without providing credentials.

Rome version:

0.23.1.61 - Romam uno die non fuisse conditam.

[tmspzz]

Have you tried with fake credentials?

also, you can implement your own engine that does exactly what you're asking for. See https://github.com/tmspzz/Rome#custom-engine and an example at: https://github.com/tmspzz/Rome/blob/master/example-engines/google.py

[gyulavoros]

Fake values are ending up with: The AWS Access Key Id you provided does not exist in our records..

Thx, I'll take a look at the custom engine. Do you think it would worth baking it into the default S3 engine itself?

[tmspzz]

I could add it, but for one it would contradict this other ticket #201 i believe. Second, it would take me much longer than it would take you to make a couple of curl or wget requests 😄

[gyulavoros]

If I'm going to create a custom engine with simple curl or wget requests, then I'll need to take care of uploading as well right? There is no way to tell Rome to use the S3 engine when running on CI so I can easily upload dependencies, but switch to the custom engine for downloading, so I can skip AWS authentication in a local environment.

[tmspzz]

Yes, you will have to take care of uploading and downloading. You could use the AWS cli inside your engine for example.

[gyulavoros]

Roger, thx for all the information!