deprecate blosc enums by d-v-b · Pull Request #3963 · zarr-developers/zarr-python

d-v-b · 2026-05-10T21:26:50Z

This PR deprecates the string enums used for the blosc codec. Literal strings are used instead. We only use enums in a few places -- Literal strings are much more abundant for representing constant string types -- and I think that's because enums in python are rather cumbersome compared to Literal[<str>].

If this PR has legs, I would follow up with similar deprecations for the other enums (used in the bytes and sharding codecs).

~~closes~~ partially addresses #3457

Design for steering BloscCodec users toward literal-string parameters, with the enum classes kept importable but deprecated on member access. Canonical: https://gist.github.com/d-v-b/9fd3fe92f82a24c929129f42a6f11f60 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Member access on BloscShuffle / BloscCname now emits DeprecationWarning and returns the equivalent string. BloscCodec stores cname/shuffle as literal strings; passing a real enum.Enum instance to __init__ warns. BloscShuffle.from_int returns a str. Internal call sites in migrate_v3 continue to work because BloscCodec accepts both forms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The BloscShuffle and BloscCname enums are deprecated; update doc examples to the recommended literal-string form. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The 0000 filename is a placeholder; rename to the PR number when the pull request is opened. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The design doc is published as a public gist linked from the PR description; the in-tree copy is no longer needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Replace Sphinx :class: roles and double-backticks with single backticks in the new docstrings added by the blosc enum deprecation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

codecov · 2026-05-10T21:38:45Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.27%. Comparing base (ad90884) to head (e90d2ba).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3963      +/-   ##
==========================================
+ Coverage   93.26%   93.27%   +0.01%     
==========================================
  Files          87       87              
  Lines       11721    11739      +18     
==========================================
+ Hits        10932    10950      +18     
  Misses        789      789

Files with missing lines	Coverage Δ
src/zarr/codecs/blosc.py	`95.77% <100.00%> (+0.61%)`	⬆️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add tests for the ValueError paths in _parse_cname / _parse_shuffle and the AttributeError path in the deprecated-enum metaclass, which codecov reported as uncovered. Drop a few "type: ignore" markers that mypy now flags as unused after the init signature widened. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…-python into deprecate-blosc-enums

mypy's per-file (pre-commit) and project-wide views disagree on whether the deliberately-wrong arguments need a type ignore. Using typing.cast keeps both views happy and is more explicit about what each test is asserting. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

ilan-gold

closes #3457

Don't think this Pr does that.

Not sure why we can't use Literal in the strings https://zarr--3963.org.readthedocs.build/en/3963/api/zarr/codecs/#zarr.codecs.BloscCodec for the docs.

Any reason we are not using https://github.com/tox-dev/sphinx-autodoc-typehints?

ilan-gold · 2026-05-11T09:51:53Z

 CName = Literal["lz4", "lz4hc", "blosclz", "snappy", "zlib", "zstd"]
 """The codec identifiers used in the blosc codec """

+CNAME: Final = ("lz4", "lz4hc", "blosclz", "snappy", "zlib", "zstd")


https://docs.python.org/3/library/typing.html#typing.get_args or?

get_args is only useful at runtime, it doesn't help you with type checking. for a small number of literal strings that won't change, writing them out explicitly as a final collection does help with runtime and type checking.

But this CNAME all-caps isn't used for anything except runtime checking unless I am missing something, no?

Like, why even define it? get_args(CName) + lru_cache if there is a performance concern

it's 6 strings that are defined in a spec. given the choice between get_args + lru_cache (two imports, produces something untyped) and just binding the 6 strings to a constant, the static constant is far simpler (no imports, gets typed correctly).

Anyone who wants to iterate over the values defined in the literal type needs a way of making those values. We should have been doing this in our tests, but we were not. I added some tests that exercise this.

ilan-gold · 2026-05-11T10:15:12Z



-class BloscShuffle(Enum):
+class _DeprecatedStrEnumMeta(type):


https://docs.python.org/3/library/enum.html#enum.EnumType or?

Also why not just warn on import instead of warn on usage? Just warn when people even import the class. It's not like it's used anywhere else.

Warning on use means fewer, more targeted warnings. I should see how many warnings we would get inside zarr-python if we warned on import.

So I guess the answer here is "too many?"

d-v-b · 2026-05-11T10:24:38Z

Any reason we are not using https://github.com/tox-dev/sphinx-autodoc-typehints?

we aren't using sphinx!

Rename Shuffle -> BloscShuffleLiteral and CName -> BloscCnameLiteral. The bare Shuffle name collided with numcodecs.Shuffle re-exported from zarr.codecs, which would have caused mkdocstrings cross-refs in the BloscCodec docstring to resolve to the wrong symbol. The Literal suffix also clearly distinguishes the type alias from the deprecated BloscShuffle / BloscCname enum classes. Update the BloscCodec docstring to reference the new names in the Attributes and Parameters sections (Convention A from cast_value.py), with literal values enumerated in prose. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

- Parametrize the BloscShuffle / BloscCname member-access warning tests into a single test_blosc_enum_member_access_warns. - Parametrize the cname / shuffle reject-unknown tests into a single test_blosc_codec_rejects_unknown driven by **kwargs. - Parametrize the AttributeError-on-unknown-member tests into a single test_blosc_enum_attribute_error_for_unknown_member. - Add a docstring to every new test explaining what behavior it verifies, so reviewers don't have to read the body to understand the intent. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

d-v-b · 2026-05-11T10:57:37Z

closes #3457

Don't think this Pr does that.

good catch, I updated the PR summary to say "partially addresses"

ilan-gold · 2026-05-11T10:58:28Z

we aren't using sphinx!

Oh right, mkdocs is completely separate - for a moment I was thinking it used sphinx as a backbone, but I guess not

…trip Backfill missing coverage: previously every test in the blosc suite used only "lz4" or "zstd" for cname and only "bitshuffle" or "shuffle" for shuffle. Add four parametrized tests driven by BLOSC_CNAME / BLOSC_SHUFFLE: - accepts_all_cnames / accepts_all_shuffles: every value in the runtime tuple is accepted by BloscCodec and round-trips on the stored attribute. Catches drift between the BloscCnameLiteral / BloscShuffleLiteral type aliases and their runtime BLOSC_* counterparts. - json_roundtrip_all_cnames / json_roundtrip_all_shuffles: BloscCodec to_dict / from_dict preserves every value. Codec fields are fully specified so the test doesn't trip over tunable-attribute state, which is not part of the JSON form. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

d-v-b · 2026-05-11T11:41:51Z

        The data type size in bytes used for shuffle filtering.
-    cname : BloscCname
-        The compression algorithm being used (lz4, lz4hc, blosclz, snappy, zlib, or zstd).
+    cname : BloscCnameLiteral


Once we fully deprecate the enums, the BloscCname identifier will be available, and I would bind it to the literal type.

d-v-b · 2026-05-11T14:00:22Z

contextual point I should have mentioned earlier: now that we have code in zarr-metadata dedicated to defining the types + constants that are relevant to the codecs defined in the core spec, we should aim to eventually remove all definitions of those types in zarr-python itself. that will first require the de-enumification we have here.

ilan-gold

Not sure what the status of warning on importing is, but it seems like it would be a bit less code and not change the BloscShuffle type, for example, but if the warnings would be too aggressive because of how things are being imported, I understand. I don't fully follow, though - what in the code base would import these values? Seems like they should be removed.

Other than that, looks good!

ilan-gold · 2026-05-11T14:10:20Z



-class BloscShuffle(Enum):
+class _DeprecatedStrEnumMeta(type):


So I guess the answer here is "too many?"

Replace the cname/shuffle JSON-roundtrip pair with a single parametrized test driven by [("cname", v) for v in BLOSC_CNAME] + [("shuffle", v) for v in BLOSC_SHUFFLE]. They asserted the same property (to_dict / from_dict preserves every literal) on two independent axes, so a single test covers both with clear per-case IDs (e.g. cname-lz4, shuffle-bitshuffle). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Stacked parametrize over BLOSC_CNAME and BLOSC_SHUFFLE so the JSON roundtrip exercises every (cname, shuffle) pair (18 cases instead of 9 in a disjoint union). Drops the **kwargs/dict[str, Any] indirection the disjoint form needed, since the cross-product form passes typed arguments directly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(deps): bump the actions group across 1 directory with 8 updates (#176) Bumps the actions group with 8 updates in the / directory: | Package | From | To | | --- | --- | --- | | [prefix-dev/setup-pixi](https://github.com/prefix-dev/setup-pixi) | `0.9.5` | `0.9.6` | | [codecov/codecov-action](https://github.com/codecov/codecov-action) | `6.0.0` | `6.0.1` | | [github/issue-metrics](https://github.com/github/issue-metrics) | `4.2.2` | `4.2.7` | | [j178/prek-action](https://github.com/j178/prek-action) | `2.0.3` | `2.0.4` | | [actions/upload-artifact](https://github.com/actions/upload-artifact) | `7.0.0` | `7.0.1` | | [actions/download-artifact](https://github.com/actions/download-artifact) | `7.0.0` | `8.0.1` | | [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) | `1.13.0` | `1.14.0` | | [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) | `0.5.3` | `0.5.6` | Updates `prefix-dev/setup-pixi` from 0.9.5 to 0.9.6 - [Release notes](https://github.com/prefix-dev/setup-pixi/releases) - [Commits](prefix-dev/setup-pixi@1b2de7f...5185adf) Updates `codecov/codecov-action` from 6.0.0 to 6.0.1 - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@57e3a13...e79a696) Updates `github/issue-metrics` from 4.2.2 to 4.2.7 - [Release notes](https://github.com/github/issue-metrics/releases) - [Commits](github-community-projects/issue-metrics@c9e9838...1e38d5e) Updates `j178/prek-action` from 2.0.3 to 2.0.4 - [Release notes](https://github.com/j178/prek-action/releases) - [Commits](j178/prek-action@6ad8027...bdca6f1) Updates `actions/upload-artifact` from 7.0.0 to 7.0.1 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v7...043fb46) Updates `actions/download-artifact` from 7.0.0 to 8.0.1 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v7...3e5f45b) Updates `pypa/gh-action-pypi-publish` from 1.13.0 to 1.14.0 - [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases) - [Commits](pypa/gh-action-pypi-publish@v1.13.0...cef2210) Updates `zizmorcore/zizmor-action` from 0.5.3 to 0.5.6 - [Release notes](https://github.com/zizmorcore/zizmor-action/releases) - [Commits](zizmorcore/zizmor-action@b1d7e1f...5f14fd0) --- updated-dependencies: - dependency-name: prefix-dev/setup-pixi dependency-version: 0.9.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: codecov/codecov-action dependency-version: 6.0.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: github/issue-metrics dependency-version: 4.2.7 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: j178/prek-action dependency-version: 2.0.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: actions/upload-artifact dependency-version: 7.0.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions - dependency-name: actions/download-artifact dependency-version: 8.0.1 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: pypa/gh-action-pypi-publish dependency-version: 1.14.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: actions - dependency-name: zizmorcore/zizmor-action dependency-version: 0.5.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: byte-order handling for structured dtypes in the bytes codec (#220) * fix: byte-order handling for structured dtypes in the bytes codec The bytes codec neither byte-swapped structured-dtype fields to its configured endian on encode (numpy reports byteorder '|' for void dtypes, so the top-level byteorder comparison never detected a mismatch) nor honored its endian when decoding, silently corrupting any structured data whose field byte order differed from the stored one (e.g. virtual references to external big-endian data). Encode now detects byte-order mismatches by comparing full dtypes via newbyteorder, and decode reinterprets raw bytes in the stored byte order before converting to the data type's declared byte order, so the stored layout (codec state) and the in-memory layout (array data type) are independent. Closes #4141 Assisted-by: ClaudeCode:claude-fable-5 * test: fold structured byte-order cases into existing bytes codec tests Extend test_endian's parametrization with structured dtypes and test_bytes_codec_sync_roundtrip with endian/dtype parametrization plus stored-layout and decoded-dtype assertions, instead of adding parallel test functions for the same properties. Assisted-by: ClaudeCode:claude-fable-5 * refactor: rename stored_dtype to view_dtype in BytesCodec decode The variable is the dtype used to view the raw chunk bytes (byte order from the codec's endian configuration), not a property of the stored data or of the returned buffer, which always carries the array's declared dtype. Assisted-by: ClaudeCode:claude-fable-5 * docs: note that the decode-side byte-order conversion copies the chunk Assisted-by: ClaudeCode:claude-fable-5 * docs: 3.3.0 release notes Add missing changelog fragments for #3955 (datetime64/timedelta64 V3 metadata types), #3966 (writes to 0-dimensional sharded arrays), and the public alias renames that accompanied the #3963/#3968 enum deprecations, then build the 3.3.0 release notes with towncrier, consuming all fragments accumulated since v3.2.1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

d-v-b and others added 7 commits May 10, 2026 13:03

docs: use literal strings for BloscCodec shuffle parameter

dfaec37

The BloscShuffle and BloscCname enums are deprecated; update doc examples to the recommended literal-string form. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

chore(changes): add changelog entry for blosc enum deprecation

d73cd52

The 0000 filename is a placeholder; rename to the PR number when the pull request is opened. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

docs: drop local copy of blosc enum deprecation spec

28d13c9

The design doc is published as a public gist linked from the PR description; the in-tree copy is no longer needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

style(codecs): use plain backticks in blosc docstrings

c7a676a

Replace Sphinx :class: roles and double-backticks with single backticks in the new docstrings added by the blosc enum deprecation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Merge branch 'main' into deprecate-blosc-enums

95bc44e

d-v-b requested a review from ilan-gold May 10, 2026 21:27

d-v-b and others added 4 commits May 11, 2026 00:03

Merge branch 'deprecate-blosc-enums' of https://github.com/d-v-b/zarr…

22ddd92

…-python into deprecate-blosc-enums

Merge branch 'main' into deprecate-blosc-enums

392f03d

d-v-b added the downstream Downstream libraries using zarr label May 11, 2026

ilan-gold reviewed May 11, 2026

View reviewed changes

d-v-b and others added 3 commits May 11, 2026 12:35

chore: rename blosc constants

acfa7a4

d-v-b commented May 11, 2026

View reviewed changes

Comment thread tests/test_codecs/test_blosc.py

d-v-b commented May 11, 2026

View reviewed changes

ilan-gold approved these changes May 11, 2026

View reviewed changes

d-v-b and others added 3 commits May 11, 2026 17:22

Rename 0000.removal.md to 3963.removal.md

e90d2ba

d-v-b enabled auto-merge (squash) May 11, 2026 15:51

d-v-b mentioned this pull request Jul 15, 2026

docs: 3.3.0 release notes #4148

Merged

d-v-b mentioned this pull request Jul 22, 2026

docs: amend 3.3.0 release notes with behavior-change and dependency details d-v-b/zarr-python#235

Open

maxrjones mentioned this pull request Jul 22, 2026

ODD PI 26.3 Objective 2: 🌳 Evolve the Zarr Ecosystem NASA-IMPACT/veda-odd#347

Closed

4 tasks

github-actions Bot mentioned this pull request Jul 29, 2026

Link Checker Report d-v-b/zarr-python#265

Open



		class BloscShuffle(Enum):
		class _DeprecatedStrEnumMeta(type):

Uh oh!

Uh oh!

Conversation

d-v-b commented May 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented May 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ilan-gold left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilan-gold May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

d-v-b commented May 11, 2026

Uh oh!

d-v-b commented May 11, 2026

Uh oh!

ilan-gold commented May 11, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

d-v-b commented May 11, 2026

Uh oh!

ilan-gold left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

d-v-b commented May 10, 2026 •

edited

Loading

codecov Bot commented May 10, 2026 •

edited

Loading

ilan-gold left a comment •

edited

Loading

ilan-gold May 11, 2026 •

edited

Loading