docs(zarr-metadata): standalone documentation site; add package justfile - #4208
Merged
d-v-b merged 8 commits intoJul 29, 2026
Merged
Conversation
Add mkdocstrings pages for every public zarr_metadata module (model, pydantic, v2, and v3 with its chunk_grid, chunk_key_encoding, codec, and data_type subpackages) under a new zarr-metadata group in the API Reference nav. griffe documents the package statically from packages/zarr-metadata/src, so the docs build environment does not need the package installed. Point the package's Documentation URL at the rendered reference instead of the README. Assisted-by: ClaudeCode:claude-fable-5
Recipes mirror the zarr-metadata CI jobs (pytest, ruff, pyright pinned to the version CI uses, on CI's python) plus changelog-draft and docs-serve conveniences. Recipes run from the package directory regardless of where just is invoked, and remain reachable from the repo root as 'just packages/zarr-metadata/<recipe>'; a future root justfile can namespace them with a 'mod' declaration. Assisted-by: ClaudeCode:claude-fable-5
With no argument, docs-serve now binds port 8000 if free and otherwise falls back to an ephemeral free port. An explicitly requested port is used as-is so a conflict fails loudly. Assisted-by: ClaudeCode:claude-fable-5
Print the zarr-metadata API reference URL once the server accepts connections, since mkdocs's own 'Serving on' line points at the zarr-python site root. Run the server in its own process group so stopping the recipe kills the whole uv->mkdocs tree instead of leaving an orphaned server holding the port. Assisted-by: ClaudeCode:claude-fable-5
Move the zarr-metadata API reference out of the zarr-python site into a self-contained mkdocs site under packages/zarr-metadata (own mkdocs.yml, landing page, and .readthedocs.yaml for a dedicated RTD project), so the package presents as a separate project with docs versioned by its own zarr_metadata-v* release tags rather than zarr-python's. The zarr-python API Reference nav now links out to the standalone site instead of embedding the pages. The package gains a pinned docs dependency group, a docs build job in its CI workflow, and docs-check / docs-serve justfile recipes targeting the package site. Assisted-by: ClaudeCode:claude-fable-5
The workflow duplicated every command the justfile defines; jobs now run 'just test/lint/typecheck/docs-check' so the justfile is the single source of truth for the package's verbs. CI keeps only its own concerns: the python matrix sync for pytest, and uv caching. The pyright job's python/sync steps are dropped because the typecheck recipe pins the interpreter and pyright version itself. Assisted-by: ClaudeCode:claude-fable-5
github-actions
Bot
added
the
needs release notes
d-v-b
marked this pull request as ready for review
July 29, 2026 17:16
Contributor
Author
|
i'm self-merging when tests pass because I'm the only person who cares about zarr-metadata right now, and this just adds dev tooling / docs |
This was referenced Jul 29, 2026
Closed
d-v-b
added a commit
to d-v-b/zarr-python
that referenced
this pull request
Jul 30, 2026
Mirror the treatment zarr-metadata received in zarr-developers#4208/zarr-developers#4210 onto zarr-indexing: a self-contained mkdocs site under the package (own mkdocs.yml, landing page, ndsel wire-format guide, mkdocstrings page per module, and .readthedocs.yaml for a dedicated RTD project), so the package presents as a separate project with docs versioned by its own zarr_indexing-v* release tags rather than zarr-python's. The zarr-python site's API Reference nav links out to it, and each RTD project now skips PR builds that do not touch its half of the repo. The package gains a pinned docs dependency group, a docs build job in its CI workflow, and a justfile with package-scoped dev recipes. Two recipes deviate from the zarr-metadata original by design: - `test` runs against the workspace-root environment (`uv run --project ../.. --all-packages --group test`), because the chunk-resolution tests exercise this package against zarr's chunk grids and `zarr` is deliberately not a dependency of this package. - `typecheck` uses plain `pyright`, unpinned and on the default interpreter, mirroring this package's own CI invocation. The zarr-metadata pin exists for a PEP 661 sentinel regression that zarr-indexing's sources do not hit. composition.py gains the module docstring the other modules already have, since mkdocstrings renders it as the page introduction. Assisted-by: ClaudeCode:claude-fable-5
d-v-b
added a commit
that referenced
this pull request
Jul 31, 2026
…ms, ndsel wire format) (#4196) * fix: byte-order handling for structured dtypes in the bytes codec (#220) * fix: byte-order handling for structured dtypes in the bytes codec The bytes codec neither byte-swapped structured-dtype fields to its configured endian on encode (numpy reports byteorder '|' for void dtypes, so the top-level byteorder comparison never detected a mismatch) nor honored its endian when decoding, silently corrupting any structured data whose field byte order differed from the stored one (e.g. virtual references to external big-endian data). Encode now detects byte-order mismatches by comparing full dtypes via newbyteorder, and decode reinterprets raw bytes in the stored byte order before converting to the data type's declared byte order, so the stored layout (codec state) and the in-memory layout (array data type) are independent. Closes #4141 Assisted-by: ClaudeCode:claude-fable-5 * test: fold structured byte-order cases into existing bytes codec tests Extend test_endian's parametrization with structured dtypes and test_bytes_codec_sync_roundtrip with endian/dtype parametrization plus stored-layout and decoded-dtype assertions, instead of adding parallel test functions for the same properties. Assisted-by: ClaudeCode:claude-fable-5 * refactor: rename stored_dtype to view_dtype in BytesCodec decode The variable is the dtype used to view the raw chunk bytes (byte order from the codec's endian configuration), not a property of the stored data or of the returned buffer, which always carries the array's declared dtype. Assisted-by: ClaudeCode:claude-fable-5 * docs: note that the decode-side byte-order conversion copies the chunk Assisted-by: ClaudeCode:claude-fable-5 * feat: add the zarr-indexing package (TensorStore-style index transforms) Standalone workspace package extracted from the lazy-indexing branch (#3906): composable, lazy coordinate transforms (IndexTransform / IndexDomain / output maps), dependency-aware chunk resolution against a DimensionGridLike protocol, and an ndsel-conformant JSON wire format validated against the vendored conformance corpus. zarr itself does not depend on zarr-indexing yet — the runtime wiring lands separately once 0.1.0 is published. The package is numpy-only; its tests exercise chunk resolution against zarr's concrete ChunkGrid, so they run from the workspace root (uv sync --all-packages). Assisted-by: ClaudeCode:claude-fable-5 * style: conventional submodule import in the chunk-resolution tests Assisted-by: ClaudeCode:claude-fable-5 * perf(zarr-indexing): joint chunk enumeration for correlated vindex maps Candidate-chunk enumeration took the cartesian product of each correlated ArrayMap's per-dimension distinct chunk ids and relied on intersect() to filter untouched combinations. For a diagonal selection of P scattered points that is P**2 intersect calls — quadratic in the number of selected points, the same workload shape as #4174 (400 points: ~2.6s; 10k points: ~30min). Group correlated maps jointly instead: broadcast their per-point chunk ids, take the distinct rows (np.unique(axis=0), O(P log P)), and enumerate exactly the touched combinations. Candidate slots now carry chunk-coordinate tuples covering one or more output dimensions; orthogonal/constant/slice dimensions keep their existing per-dimension candidates. 400-point diagonal resolution drops from 2628ms to 14ms and scales linearly. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): standalone documentation site; add package justfile Mirror the treatment zarr-metadata received in #4208/#4210 onto zarr-indexing: a self-contained mkdocs site under the package (own mkdocs.yml, landing page, ndsel wire-format guide, mkdocstrings page per module, and .readthedocs.yaml for a dedicated RTD project), so the package presents as a separate project with docs versioned by its own zarr_indexing-v* release tags rather than zarr-python's. The zarr-python site's API Reference nav links out to it, and each RTD project now skips PR builds that do not touch its half of the repo. The package gains a pinned docs dependency group, a docs build job in its CI workflow, and a justfile with package-scoped dev recipes. Two recipes deviate from the zarr-metadata original by design: - `test` runs against the workspace-root environment (`uv run --project ../.. --all-packages --group test`), because the chunk-resolution tests exercise this package against zarr's chunk grids and `zarr` is deliberately not a dependency of this package. - `typecheck` uses plain `pyright`, unpinned and on the default interpreter, mirroring this package's own CI invocation. The zarr-metadata pin exists for a PEP 661 sentinel regression that zarr-indexing's sources do not hit. composition.py gains the module docstring the other modules already have, since mkdocstrings renders it as the page introduction. Assisted-by: ClaudeCode:claude-fable-5 * chore: drop the already-released 4141 changelog fragment The bytes-codec byte-order fix this fragment describes shipped upstream and its entry is already in docs/release-notes.md; the fragment survived on this branch only as a rebase remnant, and would emit a duplicate entry in the next release. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): canonicalize ndsel references to zarr-developers/ndsel Also aligns the zarr-indexing workflow's setup-uv pin (v8.3.2) with the rest of the repo. The vendored-corpus sha is present upstream; the historical d-v-b/ndsel#1 PR reference stays as provenance. Assisted-by: ClaudeCode:claude-fable-5 * chore: drop the root uv-workspace wiring for zarr-indexing Per review: the root pyproject.toml should not change in this PR. The package now operates fully standalone (like zarr-metadata); the test invocations layer the package into the repo-root environment as an editable overlay instead (python -m pytest, since a base-env console script would not see the overlay). Assisted-by: ClaudeCode:claude-fable-5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds docs for zarr-metadata, and coordinates docs building (and other verbs) with a justfile.
claude wrote it, although I contributed some prose to
index.md. original PR here: d-v-b#263Author attestation
TODO
docs/user-guide/*.mdchanges/