Skip to content

adds codecs that numcodecs defines #2

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions codecs/numcodecs.adler32/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Adler32 codec

Defines a `bytes -> bytes` codec that adds a Adler32 checksum to the data.

## Codec name

The value of the `name` member in the codec object MUST be `numcodecs.adler32`.

## Configuration parameters

See https://numcodecs.readthedocs.io/en/stable/checksum32.html#adler32 for details about the configuration parameters.

- `location`

## Example

For example, the array metadata below specifies that the array contains chunks with appended checksum:

```json
{
"codecs": [{
"name": "bytes",
"configuration": { "endian": "little" }
}, {
"name": "numcodecs.adler32",
"configuration": {
"location": "end"
}
}],
}
```


## Format and algorithm

See https://numcodecs.readthedocs.io/en/stable/checksum32.html#adler32 for details about the encoding.

## Change log

No changes yet.

## Current maintainers

* [zarr-python core development team](https://github.com/orgs/zarr-developers/teams/python-core-devs)
26 changes: 26 additions & 0 deletions codecs/numcodecs.adler32/schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"oneOf": [
{
"type": "object",
"properties": {
"name": {
"const": "numcodecs.adler32"
},
"configuration": {
"type": "object",
"properties": {
"location": {
"type": "string",
"enum": ["start", "end"]
}
},
"additionalProperties": false
}
},
"required": ["name"],
"additionalProperties": false
},
{ "const": "numcodecs.adler32" }
]
}
42 changes: 42 additions & 0 deletions codecs/numcodecs.astype/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# AsType codec

Defines a `array -> array` codec that converts data between different types.

## Codec name

The value of the `name` member in the codec object MUST be `numcodecs.astype`.

## Configuration parameters

See https://numcodecs.readthedocs.io/en/stable/filter/astype.html for details about the configuration parameters.

- `encode_dtype`
- `decode_dtype` (optional)

## Example

For example, the array metadata below specifies that the array contains converted chunks:

```json
{
"codecs": [{
"name": "numcodecs.astype",
"configuration": {
"encode_dtype": "float32"
}
}, { "name": "bytes", "configuration": { "endian": "little" } }],
}
```


## Format and algorithm

See https://numcodecs.readthedocs.io/en/stable/filter/astype.html for details about the encoding.

## Change log

No changes yet.

## Current maintainers

* [zarr-python core development team](https://github.com/orgs/zarr-developers/teams/python-core-devs)
24 changes: 24 additions & 0 deletions codecs/numcodecs.astype/schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"name": {
"const": "numcodecs.astype"
},
"configuration": {
"type": "object",
"properties": {
"encode_dtype": {
"type": "string"
},
"decode_dtype": {
"type": "string"
}
},
"required": ["encode_dtype"],
"additionalProperties": false
}
},
"required": ["name", "configuration"],
"additionalProperties": false
}
41 changes: 41 additions & 0 deletions codecs/numcodecs.bitround/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# BitRound codec

Defines a `array -> array` codec to bit-round floating-point numbers.

## Codec name

The value of the `name` member in the codec object MUST be `numcodecs.bitround`.

## Configuration parameters

See https://numcodecs.readthedocs.io/en/stable/filter/bitround.html for details about the configuration parameters.

- `keepbits`

## Example

For example, the array metadata below specifies that the array contains bitrounded chunks:

```json
{
"codecs": [{
"name": "numcodecs.bitround",
"configuration": {
"keepbits": 5
}
}, { "name": "bytes", "configuration": { "endian": "little" } }],
}
```


## Format and algorithm

See https://numcodecs.readthedocs.io/en/stable/filter/bitround.html for details about the encoding.

## Change log

No changes yet.

## Current maintainers

* [zarr-python core development team](https://github.com/orgs/zarr-developers/teams/python-core-devs)
21 changes: 21 additions & 0 deletions codecs/numcodecs.bitround/schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"name": {
"const": "numcodecs.bitround"
},
"configuration": {
"type": "object",
"properties": {
"keepbits": {
"type": "integer"
}
},
"required": ["keepbits"],
"additionalProperties": false
}
},
"required": ["name", "configuration"],
"additionalProperties": false
}
47 changes: 47 additions & 0 deletions codecs/numcodecs.blosc/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Blosc codec

Defines a `bytes -> bytes` codec that compresses chunks using the blosc (version 1) algorithm.

## Codec name

The value of the `name` member in the codec object MUST be `numcodecs.blosc`.

## Configuration parameters

See https://numcodecs.readthedocs.io/en/stable/compression/blosc.html for details about the configuration parameters.

- `cname` (optional)
- `clevel` (optional)
- `shuffle` (optional)
- `blocksize`

## Example

For example, the array metadata below specifies that the array contains blosc compressed chunks:

```json
{
"codecs": [{ "name": "bytes" }, {
"name": "numcodecs.blosc",
"configuration": {
"cname": "zstd",
"clevel": 5,
"shuffle": 1,
"blocksize": 0
}
}],
}
```


## Format and algorithm

See https://numcodecs.readthedocs.io/en/stable/compression/blosc.html for details about the encoding.

## Change log

No changes yet.

## Current maintainers

* [zarr-python core development team](https://github.com/orgs/zarr-developers/teams/python-core-devs)
39 changes: 39 additions & 0 deletions codecs/numcodecs.blosc/schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"oneOf": [
{
"type": "object",
"properties": {
"name": {
"const": "numcodecs.blosc"
},
"configuration": {
"type": "object",
"properties": {
"cname": {
"type": "string",
"enum": ["zstd", "blosclz", "lz4", "lz4hc", "zlib", "snappy"]
},
"clevel": {
"type": "integer",
"minimum": 0,
"maximum": 12
},
"shuffle": {
"type": "integer",
"minimum": -1,
"maximum": 2
},
"blocksize": {
"type": "integer"
}
},
"additionalProperties": false
}
},
"required": ["name"],
"additionalProperties": false
},
{ "const": "numcodecs.blosc" }
]
}
41 changes: 41 additions & 0 deletions codecs/numcodecs.bz2/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# BZ2 codec

Defines a `bytes -> bytes` codec that compresses chunks using the bzip2 algorithm.

## Codec name

The value of the `name` member in the codec object MUST be `numcodecs.bz2`.

## Configuration parameters

See https://numcodecs.readthedocs.io/en/stable/compression/bz2.html for details about the configuration parameters.

- `level`

## Example

For example, the array metadata below specifies that the array contains bzip2 compressed chunks:

```json
{
"codecs": [{ "name": "bytes" }, {
"name": "numcodecs.bz2",
"configuration": {
"level": 1
}
}],
}
```


## Format and algorithm

See https://numcodecs.readthedocs.io/en/stable/compression/bz2.html for details about the encoding.

## Change log

No changes yet.

## Current maintainers

* [zarr-python core development team](https://github.com/orgs/zarr-developers/teams/python-core-devs)
28 changes: 28 additions & 0 deletions codecs/numcodecs.bz2/schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"oneOf": [
{
"type": "object",
"properties": {
"name": {
"const": "numcodecs.bz2"
},
"configuration": {
"type": "object",
"properties": {
"level": {
"type": "integer",
"minimum": 1,
"maximum": 9
}
},
"required": ["level"],
"additionalProperties": false
}
},
"required": ["name"],
"additionalProperties": false
},
{ "const": "numcodecs.bz2" }
]
}
Loading