Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat: OPTIC-1938: Proxy for storages when presigned urls are off #7354

Open
wants to merge 45 commits into
base: develop
Choose a base branch
from

Conversation

makseq
Copy link
Member

@makseq makseq commented Apr 8, 2025

Storage proxy added

Our current implementation with presigned urls:

image

This PR adds proxying from storages instead of redirecting to presigned urls:

image

Minor fixes

Fixed race condition on Project => Settings => Cloud Storages page when project=undefined on the first page loading.

image

@github-actions github-actions bot added the feat label Apr 8, 2025
Copy link

netlify bot commented Apr 8, 2025

Deploy Preview for label-studio-storybook canceled.

Name Link
🔨 Latest commit ae4da82
🔍 Latest deploy log https://app.netlify.com/sites/label-studio-storybook/deploys/67f9d796634fff0008efbd16

Copy link

netlify bot commented Apr 8, 2025

Deploy Preview for label-studio-docs-new-theme canceled.

Name Link
🔨 Latest commit ae4da82
🔍 Latest deploy log https://app.netlify.com/sites/label-studio-docs-new-theme/deploys/67f9d7968c91280008360fd5

Copy link

netlify bot commented Apr 8, 2025

Deploy Preview for heartex-docs canceled.

Name Link
🔨 Latest commit ae4da82
🔍 Latest deploy log https://app.netlify.com/sites/heartex-docs/deploys/67f9d79618a2b90008b52894

@farioas
Copy link
Member

farioas commented Apr 9, 2025

/fm sync

Workflow run

@AndrejOros
Copy link
Contributor

AndrejOros commented Apr 11, 2025

/fm sync

Workflow run

Copy link
Contributor

@bmartel bmartel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! The one thing I think that remains is how we align the browsers ability to cache the requests. One part of this whole solution was a special ServiceWorker that caches the resolution of the URI's so that the browser can effectively cache the requests when able, like with images for example. This saves on bandwidth significantly in presigned url use cases with normal usage within DataManager -> QuickView. The ServiceWorker only acted on url's that had the word presign in it, but now I see the endpoint is resolve instead. This will mean that the previous behaviour will not work completely, so we will have to update the ServiceWorker accordingly as well. I can make that change on this PR.

Copy link

codecov bot commented Apr 11, 2025

Codecov Report

Attention: Patch coverage is 95.94017% with 19 lines in your changes missing coverage. Please review.

Project coverage is 77.74%. Comparing base (dbc5eb8) to head (ae4da82).

Files with missing lines Patch % Lines
label_studio/io_storages/all_api.py 8.33% 11 Missing ⚠️
label_studio/io_storages/base_models.py 30.00% 7 Missing ⚠️
label_studio/io_storages/proxy_api.py 98.95% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #7354      +/-   ##
===========================================
+ Coverage    77.26%   77.74%   +0.47%     
===========================================
  Files          190      193       +3     
  Lines        14682    15082     +400     
===========================================
+ Hits         11344    11725     +381     
- Misses        3338     3357      +19     
Flag Coverage Δ
pytests 77.74% <95.94%> (+0.47%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@makseq
Copy link
Member Author

makseq commented Apr 11, 2025

@bmartel great point! Thank you for the fix! /resolve now makes more sense. I kept /presign for backward compatibility.

# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants