feat(cccl): integrate missing CCCL directories — python/, ci/, .agent/, docs/, test/
Sparse-checkout from NVIDIA/cccl main branch to complete cccl_upstream: Added: - python/cuda_cccl/ (226 files) — Python bindings for device-level algorithms Critical for muh toolchain: cuda.compute.reduce_into, scan, radix_sort, etc. Includes 204 .py files with full test coverage for all 27 algorithms - ci/ (163 files) — Build/test infrastructure build_cub.sh, test_cub.sh, build_and_test_targets.sh, matrix.yaml Directly maps to our [INFRA-CI] and [INFRA-BUILD] items - .agent/skills/ (7 files) — NVIDIA's own agent skills for CCCL cccl-style/SKILL.md, cccl-test/SKILL.md, sass-diff/SKILL.md - docs/ (491 files) — Official CCCL documentation CI references, CMake guides, Python compute docs, libcudacxx PTX docs - test/ (12 files) — Top-level integration tests (cuda_smoke, stdpar) - Root configs: .clang-format, .clang-tidy, CONTRIBUTING.md, pyproject.toml - CLAUDE.md symlink → AGENTS.md (NVIDIA's standard) cccl_upstream now mirrors full NVIDIA/cccl structure: Before: 42M (cub + thrust + libcudacxx + cudax + c + examples + benchmarks) After: 53M (+python +ci +docs +.agent +test +configs) This completes the CCCL base needed for: - [muh-bench] items: ci/util/build_and_test_targets.sh for targeted builds - [CCCL-verify] items: python/cuda_cccl/tests/ as reference implementations - [CCCL-test] items: ci/test_cub.sh, ci/test_thrust.sh - Agent workflow: .agent/skills/ for consistent style and test patterns
This commit is contained in:
muh-bot
@@ -0,0 +1,141 @@
|
||||
.. _libcudacxx-extended-api-mdspan-dlpack-to-mdspan:
|
||||
|
||||
DLPack to ``mdspan``
|
||||
====================
|
||||
|
||||
This functionality provides a conversion from `DLPack <https://dmlc.github.io/dlpack/latest/>`__ ``DLTensor`` to ``cuda::host_mdspan``, ``cuda::device_mdspan``, and ``cuda::managed_mdspan``.
|
||||
|
||||
Defined in the ``<cuda/mdspan>`` header.
|
||||
|
||||
Conversion functions
|
||||
--------------------
|
||||
|
||||
.. code:: cuda
|
||||
|
||||
namespace cuda {
|
||||
|
||||
template <typename ElementType, size_t Rank, typename LayoutPolicy = cuda::layout_stride_relaxed>
|
||||
[[nodiscard]] cuda::host_mdspan<ElementType, cuda::std::dims<Rank, int64_t>, LayoutPolicy>
|
||||
to_host_mdspan(const DLTensor& tensor);
|
||||
|
||||
template <typename ElementType, size_t Rank, typename LayoutPolicy = cuda::layout_stride_relaxed>
|
||||
[[nodiscard]] cuda::device_mdspan<ElementType, cuda::std::dims<Rank, int64_t>, LayoutPolicy>
|
||||
to_device_mdspan(const DLTensor& tensor);
|
||||
|
||||
template <typename ElementType, size_t Rank, typename LayoutPolicy = cuda::layout_stride_relaxed>
|
||||
[[nodiscard]] cuda::managed_mdspan<ElementType, cuda::std::dims<Rank, int64_t>, LayoutPolicy>
|
||||
to_managed_mdspan(const DLTensor& tensor);
|
||||
|
||||
} // namespace cuda
|
||||
|
||||
Template parameters
|
||||
-------------------
|
||||
|
||||
- ``ElementType``: The element type of the resulting ``mdspan``. Must match the ``DLTensor::dtype``.
|
||||
- ``Rank``: The number of dimensions. Must match ``DLTensor::ndim``.
|
||||
- ``LayoutPolicy``: The layout policy for the resulting ``mdspan``. Defaults to ``cuda::layout_stride_relaxed``. Supported layouts are:
|
||||
|
||||
- ``cuda::std::layout_right`` (C-contiguous, row-major)
|
||||
- ``cuda::std::layout_left`` (Fortran-contiguous, column-major)
|
||||
- ``cuda::std::layout_stride`` (general strided layout)
|
||||
- ``cuda::layout_stride_relaxed`` (general strided layout with negative/zero strides and offset support)
|
||||
|
||||
Semantics
|
||||
---------
|
||||
|
||||
The conversion produces a non-owning ``mdspan`` view of the ``DLTensor`` data:
|
||||
|
||||
- For ``layout_right``, ``layout_left``, and ``layout_stride``, the data pointer is computed as ``static_cast<char*>(tensor.data) + tensor.byte_offset``.
|
||||
- For ``layout_stride_relaxed``, the data pointer is ``tensor.data`` directly (no ``byte_offset`` adjustment). Instead, ``tensor.byte_offset`` is converted to an element offset (``byte_offset / sizeof(ElementType)``) and stored in the mapping. This ensures that ``mapping(indices...) = offset + sum(index_i * stride_i)`` produces non-negative indices even with negative strides, and ``required_span_size()`` correctly reflects the actual memory span.
|
||||
- For ``rank > 0``, ``mdspan.extent(i)`` is ``tensor.shape[i]``.
|
||||
- For ``layout_stride`` and ``layout_stride_relaxed``, ``mdspan.stride(i)`` is ``tensor.strides[i]`` (or computed as row-major if ``strides`` is ``nullptr`` for DLPack < v1.2).
|
||||
- The device type is validated:
|
||||
|
||||
- ``kDLCPU`` for ``to_host_mdspan``
|
||||
- ``kDLCUDA`` for ``to_device_mdspan``
|
||||
- ``kDLCUDAManaged`` for ``to_managed_mdspan``
|
||||
|
||||
Supported element types:
|
||||
|
||||
- ``bool``.
|
||||
- Signed and unsigned integers.
|
||||
- IEEE-754 Floating-point and extended precision floating-point, including ``__half``, ``__nv_bfloat16``, ``__float128``, FP8, FP6, FP4 when available.
|
||||
- Complex: ``cuda::std::complex<__half>``, ``cuda::std::complex<float>``, and ``cuda::std::complex<double>``.
|
||||
- `CUDA built-in vector types <https://docs.nvidia.com/cuda/cuda-programming-guide/05-appendices/cpp-language-extensions.html#built-in-types>`__, such as ``int2``, ``float4``, etc.
|
||||
- Vector types for extended floating-point, such as ``__half2``, ``__nv_fp8x4_e4m3``, etc.
|
||||
|
||||
Constraints
|
||||
-----------
|
||||
|
||||
- ``LayoutPolicy`` must be one of ``cuda::std::layout_right``, ``cuda::std::layout_left``, ``cuda::std::layout_stride``, or ``cuda::layout_stride_relaxed``.
|
||||
- For ``layout_right`` and ``layout_left``, the ``DLTensor`` strides must be compatible with the layout.
|
||||
|
||||
Runtime errors
|
||||
--------------
|
||||
|
||||
The conversion throws ``std::invalid_argument`` in the following cases:
|
||||
|
||||
- ``DLTensor::ndim`` does not match the specified ``Rank``.
|
||||
- ``DLTensor::dtype`` does not match ``ElementType``.
|
||||
- ``DLTensor::data`` is ``nullptr``.
|
||||
- ``DLTensor::shape`` is ``nullptr`` (for rank > 0).
|
||||
- Any ``DLTensor::shape[i]`` is negative.
|
||||
- ``DLTensor::strides`` is ``nullptr`` for DLPack v1.2 or later.
|
||||
- ``DLTensor::strides`` is ``nullptr`` for ``layout_left`` with rank > 1 (DLPack < v1.2).
|
||||
- ``DLTensor::strides[i]`` is not positive for ``layout_stride``.
|
||||
- ``DLTensor::strides`` are not compatible with the requested ``layout_right`` or ``layout_left``.
|
||||
- ``DLTensor::byte_offset`` is not a multiple of the element size for ``layout_stride_relaxed``.
|
||||
- ``DLTensor::device.device_type`` does not match the target mdspan type.
|
||||
- Data pointer is not properly aligned for the element type.
|
||||
|
||||
Availability notes
|
||||
------------------
|
||||
|
||||
- This API is available only when DLPack header is present, namely ``<dlpack/dlpack.h>`` is found in the include path.
|
||||
- This API can be disabled by defining ``CCCL_DISABLE_DLPACK`` before including any library headers. In this case, ``<dlpack/dlpack.h>`` will not be included.
|
||||
|
||||
References
|
||||
----------
|
||||
|
||||
- `DLPack C API <https://dmlc.github.io/dlpack/latest/c_api.html>`__ documentation.
|
||||
|
||||
Example
|
||||
-------
|
||||
|
||||
.. code:: cuda
|
||||
|
||||
#include <dlpack/dlpack.h>
|
||||
#include <cuda/mdspan>
|
||||
#include <cuda/std/cassert>
|
||||
#include <cuda/std/cstdint>
|
||||
|
||||
int main() {
|
||||
int data[6] = {0, 1, 2, 3, 4, 5};
|
||||
|
||||
// Create a DLTensor manually for demonstration
|
||||
int64_t shape[2] = {2, 3};
|
||||
int64_t strides[2] = {3, 1}; // row-major strides
|
||||
|
||||
DLTensor tensor{};
|
||||
tensor.data = data;
|
||||
tensor.device = {kDLCPU, 0};
|
||||
tensor.ndim = 2;
|
||||
tensor.dtype = DLDataType{kDLInt, 32, 1};
|
||||
tensor.shape = shape;
|
||||
tensor.strides = strides;
|
||||
tensor.byte_offset = 0;
|
||||
|
||||
// Convert to host_mdspan
|
||||
auto md = cuda::to_host_mdspan<int, 2>(tensor);
|
||||
|
||||
assert(md.rank() == 2);
|
||||
assert(md.extent(0) == 2 && md.extent(1) == 3);
|
||||
assert(md.stride(0) == 3 && md.stride(1) == 1);
|
||||
assert(md.data_handle() == data);
|
||||
assert(md(0, 0) == 0 && md(1, 2) == 5);
|
||||
}
|
||||
|
||||
See also
|
||||
--------
|
||||
|
||||
- :ref:`libcudacxx-extended-api-mdspan-mdspan-to-dlpack` for the reverse conversion.
|
||||
@@ -0,0 +1,205 @@
|
||||
.. _libcudacxx-extended-api-mdspan-host-device-accessor:
|
||||
|
||||
``host/device/managed`` ``mdspan`` and ``accessors``
|
||||
====================================================
|
||||
|
||||
*Host*, *device*, and *managed* ``mdspan`` allow to express multi-dimensional views of the respective CUDA memory spaces as *vocabulary types* and prevent potential errors.
|
||||
|
||||
Types and Traits
|
||||
----------------
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
template <typename Accessor>
|
||||
using host_accessor;
|
||||
|
||||
template <typename Accessor>
|
||||
using device_accessor;
|
||||
|
||||
template <typename Accessor>
|
||||
using managed_accessor;
|
||||
|
||||
Alias types to create accessors tailored for the *host*, *device*, or *managed* memory spaces.
|
||||
|
||||
----
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
template <typename ElementType,
|
||||
typename Extents,
|
||||
typename LayoutPolicy = cuda::std::layout_right,
|
||||
typename AccessorPolicy = cuda::std::default_accessor<_ElementType>>
|
||||
using host_mdspan = cuda::std::mdspan<ElementType, Extents, LayoutPolicy, host_accessor<AccessorPolicy>>;
|
||||
|
||||
template <typename ElementType,
|
||||
typename Extents,
|
||||
typename LayoutPolicy = cuda::std::layout_right,
|
||||
typename AccessorPolicy = cuda::std::default_accessor<_ElementType>>
|
||||
using device_mdspan = cuda::std::mdspan<ElementType, Extents, LayoutPolicy, device_accessor<AccessorPolicy>>;
|
||||
|
||||
template <typename ElementType,
|
||||
typename Extents,
|
||||
typename LayoutPolicy = cuda::std::layout_right,
|
||||
typename AccessorPolicy = cuda::std::default_accessor<_ElementType>>
|
||||
using managed_mdspan = cuda::std::mdspan<ElementType, Extents, LayoutPolicy, managed_accessor<AccessorPolicy>>;
|
||||
|
||||
Alias types to create ``mdspan`` with *host*, *device*, or *managed* accessors.
|
||||
|
||||
----
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
template <typename T>
|
||||
inline constexpr bool is_host_accessor_v = /* true if T is a host accessor, false otherwise */
|
||||
|
||||
template <typename T>
|
||||
inline constexpr bool is_device_accessor_v = /* true if T is a device accessor, false otherwise */
|
||||
|
||||
template <typename T>
|
||||
inline constexpr bool is_managed_accessor_v = /* true if T is a managed accessor, false otherwise */
|
||||
|
||||
template <typename T>
|
||||
inline constexpr bool is_host_accessible_v = /* true if T is a mdspan/accessor accessible from the host, false otherwise */
|
||||
|
||||
template <typename T>
|
||||
inline constexpr bool is_device_accessible_v = /* true if T is a mdspan/accessor accessible from the device, false otherwise */
|
||||
|
||||
----
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
**Memory spaces**
|
||||
|
||||
*Host*, *device*, and *managed* ``mdspan`` can be created and "sliced" (``cuda::std::submdspan``) on any memory space. However, access to a specific memory space is restricted to the respective *accessor* type.
|
||||
|
||||
+----------------------------------+------------------+-------------------+
|
||||
| ``mdspan`` / memory space access | Host memory | Device memory |
|
||||
+==================================+==================+===================+
|
||||
| ``host_mdspan`` | Allowed | *Compile error* |
|
||||
+----------------------------------+------------------+-------------------+
|
||||
| ``device_mdspan`` | *Compile error* | Allowed |
|
||||
+----------------------------------+------------------+-------------------+
|
||||
| ``managed_mdspan`` | Allowed ***** | Allowed ***** |
|
||||
+----------------------------------+------------------+-------------------+
|
||||
|
||||
***** the validity of the *managed* memory space is checked at run-time in debug mode (host-side).
|
||||
|
||||
**Conversions**
|
||||
|
||||
+-----------------------------+------------------+-------------------+---------------------+
|
||||
| | ``host_mdspan`` | ``device_mdspan`` | ``managed_mdspan`` |
|
||||
+=============================+==================+===================+=====================+
|
||||
| ``host_mdspan`` | Allowed | *Compile error* | *Compile error* |
|
||||
+-----------------------------+------------------+-------------------+---------------------+
|
||||
| ``device_mdspan`` | *Compile error* | Allowed | *Compile error* |
|
||||
+-----------------------------+------------------+-------------------+---------------------+
|
||||
| ``managed_mdspan`` | Allowed | Allowed | Allowed |
|
||||
+-----------------------------+------------------+-------------------+---------------------+
|
||||
| Other mdspan | Allowed | Allowed | Allowed |
|
||||
+-----------------------------+------------------+-------------------+---------------------+
|
||||
|
||||
*Note:* the conversion is ``explicit`` if the base accessor is not directly convertible.
|
||||
|
||||
Example 1
|
||||
---------
|
||||
|
||||
``cuda::host_mdspan`` and ``cuda::device_mdspan`` usage:
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
#include <cuda/mdspan>
|
||||
|
||||
using dim = cuda::std::dims<1>;
|
||||
|
||||
__global__ void kernel_d(cuda::device_mdspan<int, dim> md) {
|
||||
md[0] = 0;
|
||||
}
|
||||
__global__ void kernel_h(cuda::host_mdspan<int, dim> md) {
|
||||
// md[0] = 0; // compile error
|
||||
}
|
||||
|
||||
__host__ void host_function_h(cuda::host_mdspan<int, dim> md) {
|
||||
md[0] = 0;
|
||||
}
|
||||
__host__ void host_function_d(cuda::device_mdspan<int, dim> md) {
|
||||
// md[0] = 0; // compile error
|
||||
}
|
||||
__host__ void host_function_m(cuda::managed_mdspan<int, dim> md) {
|
||||
md[0] = 0;
|
||||
}
|
||||
|
||||
int main() {
|
||||
int* d_ptr;
|
||||
cudaMalloc(&d_ptr, 4 * sizeof(int));
|
||||
int h_ptr[4];
|
||||
cuda::host_mdspan h_md{h_ptr};
|
||||
cuda::device_mdspan d_md{d_ptr, 4};
|
||||
kernel_d<<<1, 1>>>(d_md); // ok
|
||||
// kernel_d<<<1, 1>>>(h_md); // compile error
|
||||
host_function_h(h_md); // ok
|
||||
host_function_d(h_md); // compile error
|
||||
// host_function_m(h_md); // compile error
|
||||
cudaFree(d_ptr);
|
||||
}
|
||||
|
||||
`See example 1 on Godbolt 🔗 <https://godbolt.org/z/fezxsbjaq>`_
|
||||
|
||||
Example 2
|
||||
---------
|
||||
|
||||
``cuda::managed_mdspan`` usage:
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
#include <cuda/mdspan>
|
||||
|
||||
using dim = cuda::std::dims<1>;
|
||||
|
||||
__global__ void kernel_d(cuda::device_mdspan<int, dim> md) {
|
||||
md[0] = 0;
|
||||
}
|
||||
|
||||
__host__ void host_function_h(cuda::host_mdspan<int, dim> md) {
|
||||
md[0] = 0;
|
||||
}
|
||||
|
||||
int main() {
|
||||
int* m_ptr;
|
||||
cudaMallocManaged(&m_ptr, 4 * sizeof(int));
|
||||
cuda::managed_mdspan m_md{m_ptr, 4};
|
||||
kernel_d<<<1, 1>>>(m_md); // ok
|
||||
host_function_h(m_md); // ok
|
||||
|
||||
cuda::managed_mdspan m_md2{d_ptr, 4};
|
||||
m_md2[0]; // run-time error
|
||||
cudaFree(d_ptr);
|
||||
}
|
||||
|
||||
`See example 2 on Godbolt 🔗 <https://godbolt.org/z/Kj39Pe4vP>`_
|
||||
|
||||
|
||||
Example 3
|
||||
---------
|
||||
|
||||
Conversion from other accessors:
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
#include <cuda/mdspan>
|
||||
|
||||
using dim = cuda::std::dims<1>;
|
||||
|
||||
int main() {
|
||||
using cuda::std::layout_right;
|
||||
using cuda::std::aligned_accessor;
|
||||
int h_ptr[4];
|
||||
cuda::std::mdspan md{h_ptr};
|
||||
cuda::host_mdspan h_md = md; // ok
|
||||
|
||||
cuda::std::mdspan<int, dim, layout_right, aligned_accessor<int, 8>> md_a{h_ptr, 4};
|
||||
// cuda::host_mdspan h_md = md_a; // compile-error
|
||||
cuda::host_mdspan h_md{md_a}; // ok
|
||||
}
|
||||
|
||||
`See example 3 on Godbolt 🔗 <https://godbolt.org/z/7dq7vcTWP>`_
|
||||
@@ -0,0 +1,440 @@
|
||||
.. _libcudacxx-extended-api-mdspan-layout-stride-relaxed:
|
||||
|
||||
``layout_stride_relaxed``
|
||||
=========================
|
||||
|
||||
Defined in the ``<cuda/mdspan>`` header.
|
||||
|
||||
``layout_stride_relaxed`` is a *LayoutMappingPolicy* which provides a layout mapping where the strides are user-defined and can be negative or zero.
|
||||
|
||||
Unlike ``cuda::std::layout_stride``, this layout allows:
|
||||
|
||||
- **Negative strides** for reverse iteration.
|
||||
- **Zero strides** for broadcasting.
|
||||
- **A base offset** to accommodate negative strides.
|
||||
- **Compile-time stride values** for static arrays.
|
||||
|
||||
.. note::
|
||||
|
||||
This layout is NOT always *unique*, *exhaustive*, or *strided* in the C++ standard sense.
|
||||
|
||||
----
|
||||
|
||||
Synopsis
|
||||
--------
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
namespace cuda {
|
||||
|
||||
// Tag value for dynamic stride (analogous to dynamic_extent)
|
||||
inline constexpr ptrdiff_t dynamic_stride = /* implementation-defined */;
|
||||
|
||||
// Strides class template
|
||||
template <class OffsetType, ptrdiff_t... Strides>
|
||||
class strides;
|
||||
|
||||
// Alias for all-dynamic strides
|
||||
template <class OffsetType, size_t Rank>
|
||||
using dstrides = strides<OffsetType, /* Rank dynamic_stride values */>;
|
||||
|
||||
// Alias for steps (synonym for dstrides)
|
||||
template <size_t Rank, class OffsetType = ptrdiff_t>
|
||||
using steps = dstrides<OffsetType, Rank>;
|
||||
|
||||
// Layout policy
|
||||
struct layout_stride_relaxed {
|
||||
template <class Extents,
|
||||
class Stride = dstrides<make_signed_t<typename Extents::index_type>,
|
||||
Extents::rank()>,
|
||||
class OffsetType = ptrdiff_t>
|
||||
class mapping;
|
||||
};
|
||||
|
||||
} // namespace cuda
|
||||
|
||||
----
|
||||
|
||||
``strides``
|
||||
-----------
|
||||
|
||||
Class template to describe the strides of a multi-dimensional array layout. Similar to ``extents``, but for strides. Supports both *static* (compile-time known) and *dynamic* (runtime) stride values.
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
// Strides class template
|
||||
template <class OffsetType, ptrdiff_t... Strides>
|
||||
class strides;
|
||||
|
||||
// Alias for all-dynamic strides
|
||||
template <class OffsetType, size_t Rank>
|
||||
using dstrides = strides<OffsetType, /* Rank dynamic_stride values */>;
|
||||
|
||||
// Alias for steps (synonym for dstrides)
|
||||
template <size_t Rank, class OffsetType = ptrdiff_t>
|
||||
using steps = dstrides<OffsetType, Rank>;
|
||||
|
||||
**Template Parameters**
|
||||
|
||||
- ``OffsetType``: A signed integer type for stride values (supports negative strides) or an *integer-constant-like* type.
|
||||
- ``Strides...``: The stride values, where ``dynamic_stride`` indicates a runtime value.
|
||||
- ``Rank``: The number of dimensions.
|
||||
|
||||
.. list-table::
|
||||
:widths: 40 60
|
||||
:header-rows: 1
|
||||
|
||||
* - **Member Types**
|
||||
- Definition
|
||||
* - ``offset_type``
|
||||
- ``OffsetType``
|
||||
* - ``size_type``
|
||||
- ``cuda::std::make_unsigned_t<offset_type>``
|
||||
* - ``rank_type``
|
||||
- ``cuda::std::size_t``
|
||||
|
||||
.. list-table::
|
||||
:widths: 40 60
|
||||
:header-rows: 1
|
||||
|
||||
* - **Static Member Functions**
|
||||
- Description
|
||||
* - ``rank()``
|
||||
- Returns the number of dimensions.
|
||||
* - ``rank_dynamic()``
|
||||
- Returns the number of dynamic strides.
|
||||
* - ``static_stride(rank_type r)``
|
||||
- Returns the static stride at dimension ``r``, or ``dynamic_stride`` if dynamic.
|
||||
|
||||
.. list-table::
|
||||
:widths: 40 60
|
||||
:header-rows: 1
|
||||
|
||||
* - **Member Functions**
|
||||
- Description
|
||||
* - ``stride(rank_type r)``
|
||||
- Returns the stride at dimension ``r``.
|
||||
|
||||
.. list-table::
|
||||
:widths: 40 60
|
||||
:header-rows: 1
|
||||
|
||||
* - **Non-member Functions**
|
||||
- Description
|
||||
* - ``operator==``
|
||||
- ``true`` if ranks are equal and all extent and stride values compare equal; ``false`` otherwise.
|
||||
* - ``operator!=``
|
||||
- ``false`` if ranks are equal and all extent and stride values compare equal; ``true`` otherwise.
|
||||
|
||||
**Constructors**
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
// (1) default constructor
|
||||
constexpr strides() noexcept = default;
|
||||
|
||||
// (2) constructor from values
|
||||
template <class... OtherIndexTypes>
|
||||
constexpr explicit strides(OtherIndexTypes... values) noexcept;
|
||||
|
||||
// (3) constructor from span
|
||||
template <class OtherIndexType, size_t Size>
|
||||
constexpr explicit(/*see below*/) strides(cuda::std::span<OtherIndexType, Size> strs) noexcept;
|
||||
|
||||
// (4) constructor from array
|
||||
template <class OtherIndexType, size_t Size>
|
||||
constexpr explicit(/*see below*/) strides(const cuda::std::array<OtherIndexType, Size>& strs) noexcept;
|
||||
|
||||
// (5) constructor from strides
|
||||
template <class OtherIndexType, ptrdiff_t... OtherStrides>
|
||||
constexpr explicit(/*see below*/) strides(const strides<OtherIndexType, OtherStrides...>& other) noexcept;
|
||||
|
||||
- **(1)** Default constructor. Value-initializes all dynamic strides to zero.
|
||||
- **(2)** Initializes the strides with the provided values.
|
||||
|
||||
- *Constraints*:
|
||||
|
||||
- ``sizeof...(OtherIndexTypes)`` equals ``rank()`` or ``rank_dynamic()``.
|
||||
- ``OtherIndexTypes...`` is convertible and nothrow constructible to ``offset_type``.
|
||||
|
||||
- *Preconditions*:
|
||||
|
||||
- Each value is representable as ``offset_type``.
|
||||
|
||||
- **(3)**, **(4)** Initializes the strides from ``span`` or ``array`` values.
|
||||
|
||||
- *Constraints*:
|
||||
|
||||
- ``Size`` equals ``rank_dynamic()`` (implicit) or ``Size`` equals ``rank()`` and ``Size != rank_dynamic()`` (explicit).
|
||||
- ``OtherIndexType`` is convertible and nothrow constructible to ``offset_type``.
|
||||
|
||||
- *Preconditions*:
|
||||
|
||||
- Each value is representable as ``offset_type``.
|
||||
|
||||
- **(5)** Initializes the strides from another ``strides`` object.
|
||||
|
||||
- The constructor is ``explicit`` if any static stride in ``Strides...`` corresponds to a ``dynamic_stride`` in ``OtherStrides...``.
|
||||
- *Constraints*:
|
||||
|
||||
- ``sizeof...(OtherStrides)`` equals ``sizeof...(Strides)``.
|
||||
- For each dimension, either stride is ``dynamic_stride``, or both strides are equal.
|
||||
|
||||
- *Preconditions*:
|
||||
|
||||
- Static strides must match their compile-time values.
|
||||
- Each dynamic stride value is representable as ``offset_type``.
|
||||
|
||||
``layout_stride_relaxed::mapping``
|
||||
----------------------------------
|
||||
|
||||
The class template ``layout_stride_relaxed::mapping`` controls how multidimensional indices are mapped with user-defined strides (including negative and zero strides) and an optional offset to a one-dimensional value representing the offset.
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
template <class Extents,
|
||||
class Stride = dstrides<make_signed_t<typename Extents::index_type>, Extents::rank()>>
|
||||
class layout_stride_relaxed::mapping;
|
||||
|
||||
**Template Parameters**
|
||||
|
||||
- ``Extents``: Specifies number of dimensions, their sizes, and which are known at compile time. Must be a specialization of ``cuda::std::extents``.
|
||||
- ``Stride``: Specifies the strides for each dimension. Must be a specialization of ``cuda::strides``. Defaults to all-dynamic strides.
|
||||
|
||||
**Constraints**
|
||||
|
||||
- ``Extents`` must be a specialization of ``cuda::std::extents``.
|
||||
- ``Extents::rank()`` must equal ``Stride::rank()``.
|
||||
|
||||
.. list-table::
|
||||
:widths: 40 60
|
||||
:header-rows: 1
|
||||
|
||||
* - **Member Types**
|
||||
- Definition
|
||||
* - ``extents_type``
|
||||
- ``Extents``
|
||||
* - ``strides_type``
|
||||
- ``Stride``
|
||||
* - ``index_type``
|
||||
- ``extents_type::index_type``
|
||||
* - ``size_type``
|
||||
- ``extents_type::size_type``
|
||||
* - ``rank_type``
|
||||
- ``extents_type::rank_type``
|
||||
* - ``offset_type``
|
||||
- ``strides_type::offset_type``
|
||||
* - ``layout_type``
|
||||
- ``layout_stride_relaxed``
|
||||
|
||||
.. list-table::
|
||||
:widths: 40 60
|
||||
:header-rows: 1
|
||||
|
||||
* - **Static Member Functions**
|
||||
- Description
|
||||
* - ``is_always_unique()``
|
||||
- Returns ``false``. Uniqueness is not guaranteed due to zero/negative strides.
|
||||
* - ``is_always_exhaustive()``
|
||||
- Returns ``false``. Exhaustiveness is not guaranteed due to zero/negative strides.
|
||||
* - ``is_always_strided()``
|
||||
- Returns ``false`` if offset is non-zero (compile-time evaluation). Standard strided behavior is not guaranteed due to offset.
|
||||
|
||||
.. list-table::
|
||||
:widths: 40 60
|
||||
:header-rows: 1
|
||||
|
||||
* - **Member Functions**
|
||||
- Description
|
||||
* - ``extents()``
|
||||
- Returns the extents object.
|
||||
* - ``strides()``
|
||||
- Returns the strides object.
|
||||
* - ``offset()``
|
||||
- Returns the base offset (nonnegative).
|
||||
* - ``required_span_size()``
|
||||
- Returns the required span size to cover all valid indices.
|
||||
* - ``operator()(Indices... indices)``
|
||||
- Maps multidimensional indices to a linear index.
|
||||
* - ``is_unique()``
|
||||
- Returns ``false`` (conservative).
|
||||
* - ``is_exhaustive()``
|
||||
- Returns ``false`` (conservative).
|
||||
* - ``is_strided()``
|
||||
- Returns ``true`` if offset is zero, ``false`` otherwise.
|
||||
* - ``stride(rank_type r)``
|
||||
- Returns the stride along dimension ``r``.
|
||||
|
||||
.. list-table::
|
||||
:widths: 40 60
|
||||
:header-rows: 1
|
||||
|
||||
* - **Non-member Functions**
|
||||
- Description
|
||||
* - ``operator==``
|
||||
- ``true`` if extents, strides, and offsets are equal; ``false`` otherwise.
|
||||
* - ``operator!=``
|
||||
- ``false`` if extents, strides, and offsets are equal; ``true`` otherwise.
|
||||
|
||||
**Constructors**
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
// (1) default constructor
|
||||
constexpr mapping() noexcept;
|
||||
|
||||
// (2) copy constructor
|
||||
constexpr mapping(const mapping&) noexcept = default;
|
||||
|
||||
// (3) constructor from strides
|
||||
constexpr mapping(const extents_type& ext,
|
||||
const strides_type& strides,
|
||||
offset_type offset = 0) noexcept;
|
||||
|
||||
// (4) converting constructor from mapping
|
||||
template <class OtherMapping>
|
||||
constexpr explicit(/*see below*/) mapping(const OtherMapping& other) noexcept;
|
||||
|
||||
- **(1)** Default constructs the mapping, delegating to a ``layout_right`` mapping with default extents.
|
||||
- **(2)** Copy constructor.
|
||||
- **(3)** Direct-non-list-initializes the extents, strides, and offset with the provided arguments.
|
||||
|
||||
- *Preconditions*:
|
||||
|
||||
- ``offset`` is nonnegative.
|
||||
- ``required_span_size()`` is representable as ``index_type``.
|
||||
|
||||
- **(4)** Constructs the mapping by copying extents and strides from ``other``. For ``layout_stride_relaxed`` sources, also copies the offset.
|
||||
|
||||
- The constructor is ``explicit`` if ``OtherMapping::extents_type`` is not convertible to ``extents_type``, or (for non-``layout_stride_relaxed`` mappings) the source is not a ``layout_left``, ``layout_right``, or ``layout_stride`` mapping.
|
||||
- *Constraints*:
|
||||
|
||||
- ``OtherMapping`` is a layout mapping.
|
||||
- ``extents_type`` is constructible from ``OtherMapping::extents_type``.
|
||||
|
||||
- *Preconditions*:
|
||||
|
||||
- ``offset`` is non-negative.
|
||||
- ``required_span_size()`` is representable as ``index_type``.
|
||||
|
||||
Examples
|
||||
--------
|
||||
|
||||
**Compile-time strides (column-major layout)**
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
#include <cuda/mdspan>
|
||||
#include <cassert>
|
||||
|
||||
int main() {
|
||||
// 3x4 matrix in column-major order
|
||||
int data[12] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12};
|
||||
|
||||
using extents_t = cuda::std::extents<int, 3, 4>;
|
||||
// Compile-time strides: stride 1 for rows, stride 3 for columns
|
||||
using strides_t = cuda::strides<int, 1, 3>;
|
||||
using mapping_t = cuda::layout_stride_relaxed::mapping<extents_t, strides_t>;
|
||||
|
||||
mapping_t mapping(extents_t{}, strides_t{});
|
||||
|
||||
// Column-major: consecutive elements in same column are adjacent
|
||||
assert(mapping(0, 0) == 0);
|
||||
assert(mapping(1, 0) == 1);
|
||||
assert(mapping(2, 0) == 2);
|
||||
assert(mapping(0, 1) == 3);
|
||||
}
|
||||
|
||||
**Negative strides (reverse iteration)**
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
#include <cuda/mdspan>
|
||||
#include <cassert>
|
||||
|
||||
int main() {
|
||||
int data[] = {1, 2, 3, 4, 5};
|
||||
// Create a reversed view using negative stride
|
||||
// Offset points to the last element, stride is -1
|
||||
using extents_t = cuda::std::extents<int, 5>;
|
||||
using mapping_t = cuda::layout_stride_relaxed::mapping<extents_t>;
|
||||
using stride_t = mapping_t::strides_type;
|
||||
mapping_t mapping(extents_t{}, stride_t(-1), 4); // offset=4, stride=-1
|
||||
// Access pattern: mapping(i) = 4 + i * (-1) = 4 - i
|
||||
assert(mapping(0) == 4);
|
||||
assert(mapping(1) == 3);
|
||||
assert(mapping(2) == 2);
|
||||
assert(mapping(3) == 1);
|
||||
assert(mapping(4) == 0);
|
||||
}
|
||||
|
||||
**Negative strides (reverse iteration) with constant offset**
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
#include <cuda/mdspan>
|
||||
#include <cassert>
|
||||
|
||||
int main() {
|
||||
int data[] = {1, 2, 3, 4, 5};
|
||||
// Create a reversed view using negative stride
|
||||
// Offset points to the last element, stride is -1
|
||||
using extents_t = cuda::std::extents<int, 5>;
|
||||
using offset_t = cuda::std::integral_constant<int, 4>;
|
||||
using strides_t = steps<extents_t::rank()>;
|
||||
using mapping_t = cuda::layout_stride_relaxed::mapping<extents_t, strides_t, offset_t>;
|
||||
mapping_t mapping(extents_t{}, strides_t(-1));
|
||||
assert(mapping(0) == 4);
|
||||
assert(mapping(1) == 3);
|
||||
assert(mapping(2) == 2);
|
||||
assert(mapping(3) == 1);
|
||||
assert(mapping(4) == 0);
|
||||
}
|
||||
|
||||
**Zero strides (broadcasting)**
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
#include <cuda/mdspan>
|
||||
#include <cassert>
|
||||
|
||||
int main() {
|
||||
int scalar = 42;
|
||||
// Create a broadcast view: single value appears at all indices
|
||||
using extents_t = cuda::std::extents<int, 4, 4>;
|
||||
using mapping_t = cuda::layout_stride_relaxed::mapping<extents_t>;
|
||||
using stride_t = mapping_t::strides_type;
|
||||
|
||||
mapping_t mapping(extents_t{}, stride_t(0, 0)); // zero strides
|
||||
// All indices map to offset 0
|
||||
assert(mapping(0, 0) == 0);
|
||||
assert(mapping(1, 2) == 0);
|
||||
assert(mapping(3, 3) == 0);
|
||||
}
|
||||
|
||||
**Mixed strides**
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
#include <cuda/mdspan>
|
||||
#include <cassert>
|
||||
|
||||
int main() {
|
||||
// 2D array with column-major layout but reversed rows
|
||||
// Data: row 0 at end, row 1 in middle, row 2 at start
|
||||
int data[3][4] = {
|
||||
{1, 2, 3, 4},
|
||||
{5, 6, 7, 8},
|
||||
{9, 10, 11, 12}
|
||||
};
|
||||
using extents_t = cuda::std::extents<int, 3, 4>;
|
||||
using mapping_t = cuda::layout_stride_relaxed::mapping<extents_t>;
|
||||
using stride_t = mapping_t::strides_type;
|
||||
// Reverse rows: stride of -4 in row dimension, +1 in column
|
||||
// Offset to start at last row
|
||||
mapping_t mapping(extents_t{}, stride_t(-4, 1), 8);
|
||||
// Access gives reversed row order
|
||||
assert(mapping(0, 0) == 8); // data[2][0]
|
||||
assert(mapping(1, 0) == 4); // data[1][0]
|
||||
assert(mapping(2, 0) == 0); // data[0][0]
|
||||
}
|
||||
@@ -0,0 +1,167 @@
|
||||
.. _libcudacxx-extended-api-mdspan-mdspan-to-dlpack:
|
||||
|
||||
``mdspan`` to DLPack
|
||||
====================
|
||||
|
||||
This functionality provides a conversion from ``cuda::host_mdspan``, ``cuda::device_mdspan``, and ``cuda::managed_mdspan`` to `DLPack <https://dmlc.github.io/dlpack/latest/>`__ ``DLTensor`` view.
|
||||
|
||||
Defined in the ``<cuda/mdspan>`` header.
|
||||
|
||||
Conversion functions
|
||||
--------------------
|
||||
|
||||
.. code:: cuda
|
||||
|
||||
namespace cuda {
|
||||
|
||||
template <typename T, typename Extents, typename Layout, typename Accessor>
|
||||
[[nodiscard]] /*dlpack_tensor*/<Extents::rank()>
|
||||
to_dlpack_tensor(const host_mdspan<T, Extents, Layout, Accessor>& mdspan);
|
||||
|
||||
template <typename T, typename Extents, typename Layout, typename Accessor>
|
||||
[[nodiscard]] /*dlpack_tensor*/<Extents::rank()>
|
||||
to_dlpack_tensor(const device_mdspan<T, Extents, Layout, Accessor>& mdspan);
|
||||
|
||||
template <typename T, typename Extents, typename Layout, typename Accessor>
|
||||
[[nodiscard]] /*dlpack_tensor*/<Extents::rank()>
|
||||
to_dlpack_tensor(const managed_mdspan<T, Extents, Layout, Accessor>& mdspan);
|
||||
|
||||
} // namespace cuda
|
||||
|
||||
Types
|
||||
-----
|
||||
|
||||
``/*dlpack_tensor*/`` is a internal helper class that stores a ``DLTensor`` and owns the backing storage for its ``shape`` and ``strides`` pointers. The class does not use any heap allocation.
|
||||
|
||||
.. code:: cuda
|
||||
|
||||
namespace cuda {
|
||||
|
||||
template <size_t Rank>
|
||||
struct /*dlpack_tensor*/ {
|
||||
// cuda::std::array<int64_t, Rank> shape;
|
||||
// cuda::std::array<int64_t, Rank> strides;
|
||||
|
||||
DLTensor get() & const noexcept [[lifetimebound]];
|
||||
|
||||
DLTensor get() && = delete;
|
||||
};
|
||||
|
||||
} // namespace cuda
|
||||
|
||||
``/*dlpack_tensor*/`` stores a ``DLTensor`` and owns the backing storage for its ``shape`` and ``strides`` pointers. The class does not use any heap allocation.
|
||||
|
||||
.. note:: **Lifetime**
|
||||
|
||||
The ``DLTensor`` associated with ``/*dlpack_tensor*/`` must not outlive the wrapper. If the wrapper is destroyed, the returned ``DLTensor::shape`` and ``DLTensor::strides`` pointers will dangle.
|
||||
|
||||
.. note:: **Const-correctness**
|
||||
|
||||
``DLTensor::data`` points at ``mdspan.data_handle()`` (or is ``nullptr`` if ``mdspan.size() == 0``). If ``T`` is ``const``, the pointer is ``const_cast``'d because ``DLTensor::data`` is unqualified.
|
||||
|
||||
Semantics
|
||||
---------
|
||||
|
||||
The conversion produces a non-owning DLPack view of the ``mdspan`` data and metadata:
|
||||
|
||||
- ``DLTensor::ndim`` is ``mdspan.rank()``.
|
||||
- For rank > 0, ``DLTensor::shape[i]`` is ``mdspan.extent(i)``.
|
||||
- For rank > 0, ``DLTensor::strides[i]`` is ``mdspan.stride(i)``.
|
||||
- ``DLTensor::byte_offset`` is always ``0``.
|
||||
- ``DLTensor::device`` is:
|
||||
|
||||
- ``{kDLCPU, 0}`` for ``cuda::host_mdspan``
|
||||
- ``{kDLCUDA, /*device_id*/}`` for ``cuda::device_mdspan``
|
||||
- ``{kDLCUDAManaged, 0}`` for ``cuda::managed_mdspan``
|
||||
|
||||
Element types are mapped to ``DLDataType`` according to the DLPack conventions, including:
|
||||
|
||||
- ``bool``.
|
||||
- Signed and unsigned integers.
|
||||
- IEEE-754 Floating-point and extended precision floating-point, including ``__half``, ``__nv_bfloat16``, ``__float128``, FP8, FP6, FP4 when available.
|
||||
- Complex: ``cuda::std::complex<__half>``, ``cuda::std::complex<float>``, and ``cuda::std::complex<double>``.
|
||||
- `CUDA built-in vector types <https://docs.nvidia.com/cuda/cuda-programming-guide/05-appendices/cpp-language-extensions.html#built-in-types>`__, such as ``int2``, ``float4``, etc.
|
||||
- Vector types for extended floating-point, such as ``__half2``, ``__nv_fp8x4_e4m3``, etc.
|
||||
|
||||
Constraints
|
||||
-----------
|
||||
|
||||
- The accessor ``data_handle_type`` must be a pointer type.
|
||||
|
||||
Runtime errors
|
||||
--------------
|
||||
|
||||
- If any ``extent(i)`` or ``stride(i)`` cannot be represented in ``int64_t``, the conversion raises an ``std::invalid_argument`` exception.
|
||||
|
||||
Availability notes
|
||||
------------------
|
||||
|
||||
- This API is available only when DLPack header is present, namely ``<dlpack/dlpack.h>`` is found in the include path.
|
||||
- This API can be disabled by defining ``CCCL_DISABLE_DLPACK`` before including any library headers. In this case, ``<dlpack/dlpack.h>`` will not be included.
|
||||
|
||||
References
|
||||
----------
|
||||
|
||||
- `DLPack C API <https://dmlc.github.io/dlpack/latest/c_api.html>`__ documentation.
|
||||
|
||||
Example
|
||||
-------
|
||||
|
||||
.. code:: cuda
|
||||
|
||||
#include <dlpack/dlpack.h>
|
||||
#include <cuda/mdspan>
|
||||
#include <cuda/std/cassert>
|
||||
#include <cuda/std/cstdint>
|
||||
|
||||
int main() {
|
||||
using extents_t = cuda::std::extents<size_t, 2, 3>;
|
||||
|
||||
int data[6] = {0, 1, 2, 3, 4, 5};
|
||||
cuda::host_mdspan<int, extents_t> md{data, extents_t{}};
|
||||
|
||||
auto dl = cuda::to_dlpack_tensor(md);
|
||||
auto dltensor = dl.get();
|
||||
|
||||
// `dl` owns the shape/stride storage; `dltensor.data` is a non-owning pointer to `data`.
|
||||
assert(dltensor.device.device_type == kDLCPU);
|
||||
assert(dltensor.ndim == 2);
|
||||
assert(dltensor.shape[0] == 2 && dltensor.shape[1] == 3);
|
||||
assert(dltensor.strides[0] == 3 && dltensor.strides[1] == 1);
|
||||
assert(dltensor.data == data);
|
||||
}
|
||||
|
||||
Examples of invalid usage:
|
||||
|
||||
.. code:: cuda
|
||||
|
||||
#include <dlpack/dlpack.h>
|
||||
#include <cuda/mdspan>
|
||||
#include <cuda/std/cstdint>
|
||||
|
||||
void show_invalid_usage1() {
|
||||
using extents_t = cuda::std::extents<size_t, 2, 3>;
|
||||
|
||||
int data[6] = {0, 1, 2, 3, 4, 5};
|
||||
cuda::host_mdspan<int, extents_t> md{data, extents_t{}};
|
||||
|
||||
// WRONG: calling get() on a temporary is deleted to prevent dangling references.
|
||||
// const DLTensor& dltensor = cuda::to_dlpack_tensor(md).get(); // compile error
|
||||
}
|
||||
|
||||
.. code:: cuda
|
||||
|
||||
#include <dlpack/dlpack.h>
|
||||
#include <cuda/mdspan>
|
||||
#include <cuda/std/cstdint>
|
||||
|
||||
int64_t* show_invalid_usage2() {
|
||||
using extents_t = cuda::std::extents<size_t, 2, 3>;
|
||||
|
||||
int data[6] = {0, 1, 2, 3, 4, 5};
|
||||
cuda::host_mdspan<int, extents_t> md{data, extents_t{}};
|
||||
|
||||
auto dl = cuda::to_dlpack_tensor(md);
|
||||
auto dltensor = dl.get();
|
||||
return dltensor.shape; // WRONG: returns a dangling pointer
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
.. _libcudacxx-extended-api-mdspan-restrict-accessor:
|
||||
|
||||
``restrict`` ``mdspan`` and ``accessor``
|
||||
========================================
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
template <typename Accessor>
|
||||
using restrict_accessor;
|
||||
|
||||
An alias type to create an accessor with the *restrict aliasing policy* starting from an existing accessor.
|
||||
|
||||
More information related to the *restrict aliasing policy* can be found in the CUDA programming guide: `__restrict__ keyword <https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#restrict>`_.
|
||||
|
||||
----
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
template <typename ElementType,
|
||||
typename Extents,
|
||||
typename LayoutPolicy = cuda::std::layout_right,
|
||||
typename AccessorPolicy = cuda::std::default_accessor<_ElementType>>
|
||||
using restrict_mdspan = cuda::std::mdspan<ElementType, Extents, LayoutPolicy, restrict_accessor<AccessorPolicy>>;
|
||||
|
||||
An alias type to create an ``mdspan`` with a *restrict aliasing policy* accessor.
|
||||
|
||||
----
|
||||
|
||||
Traits:
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
template <typename T>
|
||||
inline constexpr bool is_restrict_accessor_v = /*true if T is a restrict accessor, false otherwise*/;
|
||||
|
||||
template <typename T>
|
||||
inline constexpr bool is_restrict_mdspan_v = /*true if T is a restrict mdspan, false otherwise*/;
|
||||
|
||||
----
|
||||
|
||||
**Constraints**:
|
||||
|
||||
- Accessor ``data_handle_type`` must be a pointer type.
|
||||
|
||||
Example
|
||||
-------
|
||||
|
||||
.. code:: cuda
|
||||
|
||||
#include <cuda/mdspan>
|
||||
|
||||
using restrict_mdspan = cuda::restrict_mdspan<int, cuda::std::dims<1>>;
|
||||
|
||||
__host__ __device__ void
|
||||
compute(restrict_mdspan a, restrict_mdspan b, restrict_mdspan c) {
|
||||
c[0] = a[0] * b[0];
|
||||
c[1] = a[0] * b[0];
|
||||
c[2] = a[0] * b[0] * a[1];
|
||||
c[3] = a[0] * a[1];
|
||||
c[4] = a[0] * b[0];
|
||||
c[5] = b[0];
|
||||
}
|
||||
|
||||
int main() {
|
||||
using dim = cuda::std::dims<1>;
|
||||
using mdspan = cuda::std::mdspan<int, dim>;
|
||||
int arrayA[] = {1, 2};
|
||||
int arrayB[] = {5};
|
||||
int arrayC[] = {9, 10, 11, 12, 13, 14};
|
||||
mdspan mdA{arrayA, dim{1}};
|
||||
mdspan mdB{arrayB, dim{5}};
|
||||
mdspan mdC{arrayC, dim{6}};
|
||||
compute(mdA, mdB, mdC);
|
||||
|
||||
using restrict_aligned_accesor = cuda::std::restrict_accessor<cuda::std::aligned_accessor<int, 8>>;
|
||||
using restrict_aligned_mdspan = cuda::std::mdspan<int, dim, layout_right, restrict_aligned_accesor>;
|
||||
restrict_aligned_mdspan mdD{mdC};
|
||||
}
|
||||
|
||||
`See it on Godbolt 🔗 <https://godbolt.org/z/Wjco996z8>`_
|
||||
@@ -0,0 +1,90 @@
|
||||
.. _libcudacxx-extended-api-mdspan-shared-memory-accessor:
|
||||
|
||||
``shared_memory`` ``mdspan`` and ``accessor``
|
||||
=============================================
|
||||
|
||||
``shared_memory`` ``mdspan`` and ``accessor`` allow to express multi-dimensional views of the CUDA shared memory space and provide additional safety checks and performance optimizations.
|
||||
|
||||
Defined in the ``<cuda/mdspan>`` header.
|
||||
|
||||
Types and Traits
|
||||
----------------
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
namespace cuda {
|
||||
|
||||
template <typename AccessorPolicy>
|
||||
using shared_memory_accessor;
|
||||
|
||||
template <typename ElementType,
|
||||
typename Extents,
|
||||
typename LayoutPolicy = cuda::std::layout_right,
|
||||
typename AccessorPolicy = cuda::shared_memory_accessor<ElementType>>
|
||||
class shared_memory_mdspan;
|
||||
|
||||
} // namespace cuda
|
||||
|
||||
``mdspan`` type and accessor tailored for the *shared* memory space.
|
||||
|
||||
----
|
||||
|
||||
.. code:: cpp
|
||||
|
||||
namespace cuda {
|
||||
|
||||
template <typename T>
|
||||
inline constexpr bool is_shared_memory_accessor_v = /* true if T is a shared_memory_accessor, false otherwise */;
|
||||
|
||||
template <typename T>
|
||||
inline constexpr bool is_shared_memory_mdspan_v = /* true if T is a shared_memory_mdspan, false otherwise */;
|
||||
|
||||
} // namespace cuda
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
**Constraints**
|
||||
|
||||
- Accessor ``data_handle_type`` must be a pointer type.
|
||||
|
||||
**Preconditions**
|
||||
|
||||
- Accessing elements through a ``shared_memory_accessor`` is only allowed in device code.
|
||||
- The underlying pointer must be in the *shared* memory space.
|
||||
- Access offset must be within the maximum possible shared memory allocation size.
|
||||
|
||||
**Performance considerations**
|
||||
|
||||
- The functionality guarantees that the accesses use shared memory instructions (``STS/LDS``) rather than generic memory instructions.
|
||||
|
||||
Example
|
||||
-------
|
||||
|
||||
.. code:: cuda
|
||||
|
||||
#include <cuda/mdspan>
|
||||
#include <cstdio>
|
||||
|
||||
__global__ void kernel() {
|
||||
extern __shared__ int shmem[];
|
||||
|
||||
// Create a shared_memory_mdspan over the dynamic shared memory
|
||||
cuda::shared_memory_mdspan md(shmem, cuda:std::dims<2>{32, 32});
|
||||
|
||||
if (threadIdx.x < 32) {
|
||||
md[threadIdx.x][threadIdx.x] = threadIdx.x; // write on the diagonal
|
||||
}
|
||||
__syncthreads();
|
||||
|
||||
if (threadIdx.x == 0) {
|
||||
printf("md[5][5] = %d\n", md[5][5]); // read from the diagonal
|
||||
}
|
||||
}
|
||||
|
||||
int main() {
|
||||
kernel<<<1, 32, 32 * 32 * sizeof(int)>>>();
|
||||
cudaDeviceSynchronize();
|
||||
}
|
||||
|
||||
`See it on Godbolt 🔗 <https://godbolt.org/z/sojGnKoY9>`_
|
||||
Reference in New Issue
Block a user