Skip to content

[FEA] cuda.core: support copies with attributes (CUDA 13.2) #2365

Description

@leofang

Summary

CUDA 13.2 adds attribute-carrying variants of single copies:

  • cuMemcpyWithAttributesAsync(CUdeviceptr dst, CUdeviceptr src, size_t size, CUmemcpyAttributes* attr, CUstream)
    — flat 1D copy taking the same CUmemcpyAttributes (access-order and location hints)
    introduced with the 12.8 batched-copy machinery;
  • cuMemcpy3DWithAttributesAsync(CUDA_MEMCPY3D_BATCH_OP* op, unsigned long long flags, CUstream)
    — executes a single 3D batch-op descriptor.

cuda.core's copy APIs (Buffer.copy_to / copy_from) accept no attributes today. #1333
tracks the batched entry points (cuMemcpyBatchAsync — its P1 discard/prefetch half is
already implemented in
_memory/_managed_memory_ops.pyx);
the attribute plumbing is shared, so this issue should be implemented in coordination with
#1333.

Underlying C APIs to cover

cuMemcpyWithAttributesAsync, cuMemcpy3DWithAttributesAsync (13.2); reuses CUmemcpyAttributes
/ CUDA_MEMCPY3D_BATCH_OP (12.8).

Design sketch (draft — needs design-meeting review)

Important

Starting point only, not a settled design — review in the cuda.core design meeting,
together with #1333.

attrs = CopyAttributes(src_access_order="stream", ...)   # names TBD; mirrors CUmemcpyAttributes
buf.copy_to(dst, stream=s, attributes=attrs)

The same CopyAttributes object would feed #1333's batched API.

Open questions for the meeting:

  1. Dataclass shape mirroring CUmemcpyAttributes (StrEnum for access order; Device/Host
    objects for location hints).
  2. Whether/how to expose the 3D variant (StridedMemoryView-based copies?).
  3. Separate PR or folded into Support batched memory movement #1333's implementation.
  4. Version gating (13.2+) with actionable errors.

References

-- Leo's bot

Metadata

Metadata

Assignees

No one assigned

    Labels

    cuda.coreEverything related to the cuda.core modulefeatureNew feature or requesttriageNeeds the team's attention

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions