Skip to content

Default to a RemoteStore for (some) string StoreLike #2195

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
TomAugspurger opened this issue Sep 17, 2024 · 1 comment · Fixed by #2198
Closed

Default to a RemoteStore for (some) string StoreLike #2195

TomAugspurger opened this issue Sep 17, 2024 · 1 comment · Fixed by #2198
Labels
bug Potential issues with the zarr-python library

Comments

@TomAugspurger
Copy link
Contributor

Zarr version

v3

Numcodecs version

n/a

Python Version

n/a

Operating System

n/a

Installation

n/a

Description

Zarr v2 users (either direct, or indirect through libraries like xarray) probably relied on the ability to pass an fsspec URI to zarr.open and have it work with a remote file system. Is there any tolerance for trying to interpret some string store arguments as an argument to RemoteStore rather than LocalFile? For example:

zarr.open_group("s3://hrrrzarr")

would be interpreted as

zarr.open_group(zarr.store.RemoteStore("s3://hrrrzarr"))

We'd also need to expose storage_options in all the API endpoints that create a Store / StorePath from user input, to allow customizing the filesystem that's created.

Steps to reproduce

import zarr

g = zarr.open_group(store="s3://hrrrzarr", storage_options={"anon": False})

Additional output

No response

@TomAugspurger TomAugspurger added the bug Potential issues with the zarr-python library label Sep 17, 2024
@d-v-b
Copy link
Contributor

d-v-b commented Sep 17, 2024

Is there any tolerance for trying to interpret some string store arguments as an argument to RemoteStore rather than LocalFile? For example:

Definitely! In the short term, we should copy what v2 was doing, modulo the storage classes. Longer term, I would like to consider a few things in this direction:

  • expose a URI attribute on stores, groups and arrays.
  • set up a scheme: store class registry

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Potential issues with the zarr-python library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants