Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat(gcs): allow setting a token directly #4978

Merged
merged 13 commits into from
Aug 13, 2024

Conversation

jdockerty
Copy link
Contributor

Which issue does this PR close?

Closes #4876

Rationale for this change

An OAuth2 token cannot be set directly at the moment, instead other mechanisms are required such as the credential or credential_path.

What changes are included in this PR?

Inclusion of a token within the GcsConfig and corresponding methods so that a bearer token is set within the signed requests to GCP.

Are there any user-facing changes?

Direct token being available as a option for authentication

@Xuanwo
Copy link
Member

Xuanwo commented Aug 8, 2024

Hi, #4979 has been merged, we can continue this one now.

@@ -354,6 +364,8 @@ impl Builder for GcsBuilder {
client,
signer,
token_loader,
token: self.config.token,
scope: self.config.scope,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scope will be DEFAULT_GCS_SCOPE if not set.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I've altered this now.

I skimmed over that it was being set already with a default, I'll combine this with the direct String usage too. Much appreciated 👍

@@ -76,6 +79,18 @@ static BACKOFF: Lazy<ExponentialBuilder> =

impl GcsCore {
async fn load_token(&self) -> Result<Option<GoogleToken>> {
match (&self.token, &self.scope) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scope must be valid, we can store as String directly.

@@ -116,6 +131,15 @@ impl GcsCore {
}

pub async fn sign<T>(&self, req: &mut Request<T>) -> Result<()> {
if let Some(token) = &self.token {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to change the logic here since load_token has already handled this.

Copy link
Contributor Author

@jdockerty jdockerty Aug 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense 👍

Do I understand rightly that the same is also true for sign_query? I.e. my addition here should also be removed.

if let Some(token) = &self.token {
req.headers_mut().remove(HOST);
let header_value = format!("Bearer {}", token);
req.headers_mut()
.insert(header::AUTHORIZATION, header_value.parse().unwrap());
return Ok(());
}

I've added ☝️, but the sign_query is handled through load_credential and not load_token

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, this part is a bit complex. Let me elaborate.


GCS's token-based authorization doesn't support signed queries. Signed queries involve credentials to build presigned URLs, similar to AWS S3. We can't build such presigned URLs using a token. So if users only set a token but not credentials, the signed query won't work.

However, I believe the changes here are the same, and we don't need to alter code sign_xxx. Just let load_xxx handle it.

Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks!

@Xuanwo Xuanwo merged commit eb7d430 into apache:main Aug 13, 2024
79 checks passed
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

gcs: Should allow setting token directly
2 participants