Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Add an Azure storage backend that uses the Microsoft SDK #606

Closed
gaul opened this issue Feb 20, 2024 · 1 comment · Fixed by #696
Closed

Add an Azure storage backend that uses the Microsoft SDK #606

gaul opened this issue Feb 20, 2024 · 1 comment · Fixed by #696

Comments

@gaul
Copy link
Owner

gaul commented Feb 20, 2024

S3Proxy currently uses Apache jclouds for its Azure Blob integration. This uses the native REST API and has some advantages in terms of similar configuration and logging. However the Microsoft SDK has many more features including authentication that are popular user requests, e.g., #536. This might not be a lot of work to just create a BlobStore implementation that calls into the SDK.
@gaul gaul mentioned this issue Apr 13, 2024
Open
@gaul
Copy link
Owner Author

gaul commented Aug 4, 2024

The existing jclouds implementation:

https://github.com/apache/jclouds/blob/master/providers/azureblob/src/main/java/org/jclouds/azureblob/blobstore/AzureBlobStore.java

gaul added a commit that referenced this issue Oct 2, 2024
@gaul
Experiment with the Azure SDK
5adf4cb 
This may replace the jclouds azureblob provider.  This is incomplete;
notably missing multi-part uploads and many other metadata and
options.  References #606.
This was referenced Oct 2, 2024
Merged
Closed
gaul added a commit that referenced this issue Oct 3, 2024
@gaul
Experiment with the Azure SDK
9f8c69a 
This may replace the jclouds azureblob provider.  This is incomplete;
notably missing multi-part uploads and many other metadata and
options.  References #606.
gaul added a commit that referenced this issue Oct 4, 2024
@gaul
Experiment with the Azure SDK
9b886b7 
This may replace the jclouds azureblob provider.  The implementation
is incomplete, notably lacking multi-part uploads and other error
codes and options.  References #606.
gaul added a commit that referenced this issue Oct 5, 2024
@gaul
Experiment with the Azure SDK
c072d69 
This may replace the jclouds azureblob provider.  The implementation
is incomplete, notably lacking multi-part uploads and other error
codes and options.  References #606.
gaul added a commit that referenced this issue Oct 5, 2024
@gaul
Experiment with the Azure SDK
ab409ea 
This may replace the jclouds azureblob provider.  The implementation
is incomplete, notably lacking multi-part uploads and other error
codes and options.  References #606.
gaul added a commit that referenced this issue Oct 5, 2024
@gaul
Experiment with the Azure SDK
c1dceb4 
This may replace the jclouds azureblob provider.  The implementation
is incomplete, notably lacking multi-part uploads and other error
codes and options.  References #606.
gaul added a commit that referenced this issue Oct 5, 2024
@gaul
Experiment with the Azure SDK
f903d21 
This may replace the jclouds azureblob provider.  The implementation
is incomplete, lacking some error codes and options.  References #606.
gaul added a commit that referenced this issue Oct 5, 2024
@gaul
Experiment with the Azure SDK
0adb0bc 
This may replace the jclouds azureblob provider.  The implementation
lacks some error codes and options.  References #606.
gaul added a commit that referenced this issue Oct 5, 2024
@gaul
Experiment with the Azure SDK
3e08b57 
This may replace the jclouds azureblob provider.  The implementation
lacks some error codes and options.  References #606.
gaul added a commit that referenced this issue Oct 6, 2024
@gaul
Add azureblob-sdk provider backed by Azure SDK
1037bf5 
This may replace the jclouds azureblob provider.  The implementation
lacks some error codes and options.  Fixes #606.
@gaul gaul closed this as completed in #696 Oct 7, 2024
@gaul gaul closed this as completed in b33f3e2 Oct 7, 2024
This was referenced Oct 7, 2024
Closed
Open
Closed
Open
Closed
Closed
Open
gaul added a commit that referenced this issue Oct 27, 2024
@gaul
Add azureblob-sdk to Quirks
4f36e7a 
References #606.
gaul added a commit that referenced this issue Oct 27, 2024
@gaul
Add mapping for NO_SUCH_KEY for azureblob-sdk
7a52cfd 
References #606.
gaul added a commit that referenced this issue Oct 27, 2024
@gaul
Do not set Tier for RELATIVE_PATH in azureblob-sdk
39ca93b 
References #606.
gaul added a commit that referenced this issue Oct 27, 2024
@gaul
Ignore CONTAINER_NOT_FOUND when deleting a bucket
e40f040 
Reference #606.
gaul added a commit that referenced this issue Oct 27, 2024
@gaul
Handle BUCKET_ALREADY_OWNED_BY_YOU in azureblob-sdk
ecbb56c 
References #606.
gaul added a commit that referenced this issue Oct 27, 2024
@gaul
Always overwrite in uploadPart in azureblob-sdk
73b0047 
References #606.
gaul added a commit that referenced this issue Oct 27, 2024
@gaul
Opt into custom Azure logic with azureblob-sdk
11b8dbb 
References #606.
gaul added a commit that referenced this issue Oct 28, 2024
@gaul
Implement copyBlob for azureblob-sdk
fc7a135 
References #606.
gaul added a commit that referenced this issue Oct 28, 2024
@gaul
Skip copy blob tests on Azurite
8311d11 
References #606.
gaul added a commit that referenced this issue Oct 28, 2024
@gaul
Add workarounds for azureblob-sdk tests
c28f7db 
References #606.
gaul added a commit that referenced this issue Oct 28, 2024
@gaul
Support condition get object in azureblob-sdk
38d2881 
References #606.
gaul added a commit that referenced this issue Oct 28, 2024
@gaul
Translate Azure errors in AzureBlobStore
38cf4a9 
This addresses a layering violation and may make AzureBlobStore more
useful to non-S3Proxy users.  References #606.
gaul added a commit that referenced this issue Oct 28, 2024
@gaul
Handle listing a missing bucket in azureblob-sdk
c893578 
Found via s3-tests.  References #606.
gaul added a commit that referenced this issue Oct 28, 2024
@gaul
Handle missing buckets and keys in azureblob-sdk
ff8e12e 
Found via s3-tests.  References #606.
gaul added a commit that referenced this issue Oct 29, 2024
@gaul
Handle non-empty delete container in azureblob-sdk
8ebf6ce 
Found via s3-tests.  References #606.
gaul added a commit that referenced this issue Oct 30, 2024
@gaul
Add Last-Modified to azureblob-sdk getBlob
30b7f94 
References #606.
gaul added a commit that referenced this issue Oct 30, 2024
@gaul
Add Last-Modified to azureblob-sdk getBlob
c40237d 
Also ETag and creation date.  References #606.
gaul added a commit that referenced this issue Oct 30, 2024
@gaul
Include Content-MD5 in azureblob-sdk putBlob
18d2f0e 
References #606.  Fixes #576.
gaul added a commit that referenced this issue Nov 8, 2024
@gaul
Use marker when listing blobs in azureblob-sdk
07613b9 
References #606.
gaul added a commit that referenced this issue Nov 8, 2024
@gaul
Add support for getBlob ranges in azureblob-sdk
5da1178 
References #606.
gaul added a commit that referenced this issue Nov 10, 2024
@gaul
Handle invalid container names
2e05eee 
References #606.
gaul added a commit that referenced this issue Nov 10, 2024
@gaul
Handle invalid MD5 hashes
b02ce67 
References #606.
gaul added a commit that referenced this issue Nov 10, 2024
@gaul
Handle missing container in more places
5272c79 
References #606.
gaul added a commit that referenced this issue Nov 11, 2024
@gaul
Consolidate azureblob-sdk error handling
9e2c40a 
References #606.
gaul added a commit that referenced this issue Nov 11, 2024
@gaul
Handle deleting missing container in azureblob-sdk
2e66fb2 
References #606.
gaul added a commit that referenced this issue Nov 11, 2024
@gaul
Clarify state of azureblob-sdk
783b1f1 
References #606.
gaul added a commit that referenced this issue Dec 19, 2024
@gaul
Check for null BlobAccessType in azureblob-sdk
4b3806d 
References #606.
@gaul gaul mentioned this issue Dec 27, 2024
Open
@gaul gaul mentioned this issue Jan 4, 2025
Open
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add azureblob-sdk provider backed by Azure SDK gaul/s3proxy
1 participant
@gaul