Skip to content

feat(data): wait for asynchronous batch readers - #184

Draft
9prady9 wants to merge 1 commit into
NVIDIA:mainfrom
9prady9:reader_event
Draft

feat(data): wait for asynchronous batch readers#184
9prady9 wants to merge 1 commit into
NVIDIA:mainfrom
9prady9:reader_event

Conversation

@9prady9

@9prady9 9prady9 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add read_only_data_batch::record_reader_event(stream) so consumers can publish asynchronous GPU reads before releasing their shared lock
  • retain all outstanding reader completions in reusable, per-device CUDA event pools
  • make blocking mutable acquisition wait for every recorded reader while keeping try_to_mutable() non-blocking
  • preserve the requested no-op behavior for host and disk representations, with a destructor safety fallback for the final batch owner

Why

A reader can enqueue an asynchronous copy from a non-owning GPU view and release its read lock before the copy finishes. The batch can then be reclaimed and its memory reused, silently corrupting the in-flight copy. Writer events only order readers after writers; they do not order reclaim after readers.

This change records the inverse dependency on the batch. Any path that obtains mutable access—including replacement, conversion, rebind, and raw mutable access—crosses the same reader-event barrier before it can reuse the representation.

Correctness details

  • concurrent readers retain independent events; a later fast reader cannot overwrite an earlier slow dependency
  • event pools are keyed by the CUDA device that owns each reader stream
  • failed event registration synchronizes the already-enqueued reader stream before propagating, preventing an untracked read from escaping the lock lifetime
  • readonly_to_mutable() uses the same barrier, while try_to_mutable() queries events and returns nullopt when any remain pending

Test plan

  • pixi run pre-commit run --files include/cucascade/data/data_batch.hpp src/data/data_batch.cpp test/data/test_data_batch.cpp
  • warnings-as-errors build of cucascade_tests
  • warnings-as-errors build of cucascade_cudf_tests
  • regression covers two concurrent readers, both mutable transition paths, poison-on-reclaim ordering, completed-event reuse, and non-GPU no-op behavior
  • hidden multi-GPU regression covers event-pool reuse after representation replacement on another device
  • run cucascade_cudf_tests '[reader_event]' on a GPU host; this development host has no usable NVIDIA driver and returns cudaErrorNoDevice

@copy-pr-bot

copy-pr-bot Bot commented Aug 14, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant