Skip to content

Appending to re-opened zarr array does not respect write_empty_chunks #2931

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
ianhi opened this issue Mar 26, 2025 · 0 comments · May be fixed by #2933
Open

Appending to re-opened zarr array does not respect write_empty_chunks #2931

ianhi opened this issue Mar 26, 2025 · 0 comments · May be fixed by #2933
Labels
bug Potential issues with the zarr-python library

Comments

@ianhi
Copy link
Contributor

ianhi commented Mar 26, 2025

Zarr version

3.0.7.dev3+g9e8b50ae

Numcodecs version

0.15.1

Python Version

3.13.2

Operating System

mac

Installation

from source

Description

When you open an existing array in order to resize and write new values to it the write_empty_chunks config is no longer respected.

Steps to reproduce

import os
import zarr

z = zarr.create_array(
    store="appending.zarr",
    name="test",
    shape=(2,),
    dtype=int,
    chunks=(1,),
    config={"write_empty_chunks": True},
    overwrite=True,
    fill_value=0,
    zarr_format=3,
)

z[:] = 0

# if you comment out this re-opening then everything works as intended
z = zarr.open(
    "appending.zarr", path="test", config={"write_empty_chunks": True}, fill_value=0
)
print(os.listdir("appending.zarr/test/c/"))

z.resize((4,))

print(os.listdir("appending.zarr/test/c/"))
l = os.listdir("appending.zarr/test/c/")
print(l)
assert l == ['0', '1',]

# now put a value into the newly appended chunks
z[2:] = 0
l = os.listdir("appending.zarr/test/c/")
print(l)
# assert l == ['0', '1', '2', '3']


# putting in a value of zero makes all empty chunks be removed from disk
z[:] = 0

l = os.listdir("appending.zarr/test/c/")
print(l)
# assert l == ['0', '1', '2', '3']

gives

['0', '1']
['0', '1']
['0', '1']
['0', '1']
[]

not only are chunks 2 and 3 not written, but unexpectedly chunks 0 and 1 are removed from disk.

if you comment out the zarr.open... line then the output looks like:

['0', '1']
['0', '1']
['0', '1']
['0', '1', '3', '2']
['0', '1', '3', '2']

Additional output

No response

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Potential issues with the zarr-python library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant