feat(cccl): integrate missing CCCL directories — python/, ci/, .agent/, docs/, test/
Sparse-checkout from NVIDIA/cccl main branch to complete cccl_upstream: Added: - python/cuda_cccl/ (226 files) — Python bindings for device-level algorithms Critical for muh toolchain: cuda.compute.reduce_into, scan, radix_sort, etc. Includes 204 .py files with full test coverage for all 27 algorithms - ci/ (163 files) — Build/test infrastructure build_cub.sh, test_cub.sh, build_and_test_targets.sh, matrix.yaml Directly maps to our [INFRA-CI] and [INFRA-BUILD] items - .agent/skills/ (7 files) — NVIDIA's own agent skills for CCCL cccl-style/SKILL.md, cccl-test/SKILL.md, sass-diff/SKILL.md - docs/ (491 files) — Official CCCL documentation CI references, CMake guides, Python compute docs, libcudacxx PTX docs - test/ (12 files) — Top-level integration tests (cuda_smoke, stdpar) - Root configs: .clang-format, .clang-tidy, CONTRIBUTING.md, pyproject.toml - CLAUDE.md symlink → AGENTS.md (NVIDIA's standard) cccl_upstream now mirrors full NVIDIA/cccl structure: Before: 42M (cub + thrust + libcudacxx + cudax + c + examples + benchmarks) After: 53M (+python +ci +docs +.agent +test +configs) This completes the CCCL base needed for: - [muh-bench] items: ci/util/build_and_test_targets.sh for targeted builds - [CCCL-verify] items: python/cuda_cccl/tests/ as reference implementations - [CCCL-test] items: ci/test_cub.sh, ci/test_thrust.sh - Agent workflow: .agent/skills/ for consistent style and test patterns
This commit is contained in:
muh-bot
183
cccl_upstream/docs/thrust/developer/cmake_options.rst
Normal file
183
cccl_upstream/docs/thrust/developer/cmake_options.rst
Normal file
@@ -0,0 +1,183 @@
|
||||
.. _cmake-options:
|
||||
|
||||
Developer CMake Options
|
||||
=======================
|
||||
|
||||
.. important::
|
||||
|
||||
This document details the CMake options used for **developer builds**
|
||||
of the Thrust tests and examples included in the CCCL repository.
|
||||
The options presented here are not available when using Thrust as an
|
||||
end-user via our installed CMake packages, or when using
|
||||
``add_subdirectory`` / CPM to add CCCL/Thrust to your project's build system.
|
||||
For details and examples of using Thrust as an end-user, please refer
|
||||
to the `Thrust + CMake example <https://github.com/NVIDIA/cccl/tree/main/examples/thrust_flexible_device_system>`_
|
||||
|
||||
|
||||
A Thrust build is configured using CMake options. These may be passed to
|
||||
CMake using
|
||||
|
||||
.. code:: bash
|
||||
|
||||
cmake -D<option_name>=<value> /path/to/thrust/sources
|
||||
|
||||
or configured interactively with the ``ccmake`` or ``cmake-gui``
|
||||
interfaces.
|
||||
|
||||
Thrust supports two build modes. By default, a single configuration is
|
||||
built that targets a specific host system, device system, and C++
|
||||
dialect. When ``THRUST_ENABLE_MULTICONFIG`` is ``ON``, multiple
|
||||
configurations targeting a variety of systems and dialects are
|
||||
generated.
|
||||
|
||||
The CMake options are divided into these categories:
|
||||
|
||||
1. `Generic CMake Options <#generic-cmake-options>`__: Options
|
||||
applicable to all Thrust builds.
|
||||
2. `Single Config CMake Options <#single-config-cmake-options>`__
|
||||
Options applicable only when ``THRUST_ENABLE_MULTICONFIG`` is
|
||||
disabled.
|
||||
3. `Multi Config CMake Options <#multi-config-cmake-options>`__ Options
|
||||
applicable only when ``THRUST_ENABLE_MULTICONFIG`` is enabled.
|
||||
4. `CUDA Specific CMake Options <#cuda-specific-cmake-options>`__
|
||||
Options that control CUDA compilation. Only available when one or
|
||||
more configurations targets the CUDA system.
|
||||
5. `TBB Specific CMake Options <#tbb-specific-cmake-options>`__
|
||||
Options that control TBB compilation. Only available when one or
|
||||
more configurations targets the TBB system.
|
||||
|
||||
Generic CMake Options
|
||||
---------------------
|
||||
|
||||
- ``CMAKE_BUILD_TYPE={Release, Debug, RelWithDebInfo, MinSizeRel}``
|
||||
|
||||
- Standard CMake build option. Default: ``RelWithDebInfo``
|
||||
|
||||
- ``THRUST_ENABLE_HEADER_TESTING={ON, OFF}``
|
||||
|
||||
- Whether to test compile public headers. Default is ``ON``.
|
||||
|
||||
- ``THRUST_ENABLE_TESTING={ON, OFF}``
|
||||
|
||||
- Whether to build unit tests. Default is ``ON``.
|
||||
|
||||
- ``THRUST_ENABLE_EXAMPLES={ON, OFF}``
|
||||
|
||||
- Whether to build examples. Default is ``ON``.
|
||||
|
||||
- ``THRUST_ENABLE_MULTICONFIG={ON, OFF}``
|
||||
|
||||
- Toggles single-config and multi-config modes. Default is ``OFF``
|
||||
(single config).
|
||||
|
||||
- ``THRUST_ENABLE_EXAMPLE_FILECHECK={ON, OFF}``
|
||||
|
||||
- Enable validation of example outputs using the LLVM FileCheck
|
||||
utility. Default is ``OFF``.
|
||||
|
||||
- ``THRUST_ENABLE_INSTALL_RULES={ON, OFF}``
|
||||
|
||||
- If true, installation rules will be generated for thrust. Default
|
||||
is ``ON``.
|
||||
|
||||
- ``THRUST_DISPATCH_TYPE={Dynamic, Force32bit, Force64bit}``
|
||||
|
||||
- Allows the user to force Thrust to use a specific size for the offset type. Default
|
||||
is ``Dynamic``.
|
||||
|
||||
- ``Dynamic`` lets Thrust choose the index type based on input size, allowing
|
||||
large inputs and optimal performance at the cost of increased compile time and binary size,
|
||||
as Thrust will compile each kernel twice, once for 32 bit and once for 64 bit.
|
||||
- ``Force32bit`` forces Thrust to use a 32 bit offset type. This improves compile time and
|
||||
binary size but limits the input size.
|
||||
- ``Force64bit`` forces Thrust to use a 64 bit offset type. This improves compile time and
|
||||
binary size and allows large input sizes. However, it might degrade runtime performance.
|
||||
|
||||
Single Config CMake Options
|
||||
---------------------------
|
||||
|
||||
- ``THRUST_HOST_SYSTEM={CPP, TBB, OMP}``
|
||||
|
||||
- Selects the host system. Default: ``CPP``
|
||||
|
||||
- ``THRUST_DEVICE_SYSTEM={CUDA, TBB, OMP, CPP}``
|
||||
|
||||
- Selects the device system. Default: ``CUDA``
|
||||
|
||||
- ``THRUST_CPP_DIALECT={17, 20}``
|
||||
|
||||
- Selects the C++ standard dialect to use. Default is ``14``
|
||||
(C++14).
|
||||
|
||||
.. _cmake-multi-config-options:
|
||||
|
||||
Multi Config CMake Options
|
||||
--------------------------
|
||||
|
||||
- ``THRUST_MULTICONFIG_ENABLE_DIALECT_CPPXX={ON, OFF}``
|
||||
|
||||
- Toggle whether a specific C++ dialect will be targeted.
|
||||
- Possible values of ``XX`` are ``{17, 20}``.
|
||||
- By default, only C++14 is enabled.
|
||||
|
||||
- ``THRUST_MULTICONFIG_ENABLE_SYSTEM_XXXX={ON, OFF}``
|
||||
|
||||
- Toggle whether a specific system will be targeted.
|
||||
- Possible values of ``XXXX`` are ``{CPP, CUDA, TBB, OMP}``
|
||||
- By default, only ``CPP`` and ``CUDA`` are enabled.
|
||||
|
||||
- ``THRUST_MULTICONFIG_WORKLOAD={SMALL, MEDIUM, LARGE, FULL}``
|
||||
|
||||
- Restricts the host/device combinations that will be targeted.
|
||||
- By default, the ``SMALL`` workload is used.
|
||||
- The full cross product of ``host x device`` systems results in 12
|
||||
configurations, some of which are more important than others. This
|
||||
option can be used to prune some of the less important ones.
|
||||
- ``SMALL``: (3 configs) Minimal coverage and validation of each
|
||||
device system against the ``CPP`` host.
|
||||
- ``MEDIUM``: (6 configs) Cheap extended coverage.
|
||||
- ``LARGE``: (8 configs) Expensive extended coverage. Includes all
|
||||
useful build configurations.
|
||||
- ``FULL``: (12 configs) The complete cross product of all possible
|
||||
build configurations.
|
||||
|
||||
======== =========== ========== ========= ============================
|
||||
Config Workloads Value Expense Note
|
||||
======== =========== ========== ========= ============================
|
||||
CPP/CUDA ``F L M S`` Essential Expensive Validates CUDA against CPP
|
||||
CPP/OMP ``F L M S`` Essential Cheap Validates OMP against CPP
|
||||
CPP/TBB ``F L M S`` Essential Cheap Validates TBB against CPP
|
||||
CPP/CPP ``F L M`` Important Cheap Tests CPP as device
|
||||
OMP/OMP ``F L M`` Important Cheap Tests OMP as host
|
||||
TBB/TBB ``F L M`` Important Cheap Tests TBB as host
|
||||
TBB/CUDA ``F L`` Important Expensive Validates TBB/CUDA interop
|
||||
OMP/CUDA ``F L`` Important Expensive Validates OMP/CUDA interop
|
||||
TBB/OMP ``F`` Not useful Cheap Mixes CPU-parallel systems
|
||||
OMP/TBB ``F`` Not useful Cheap Mixes CPU-parallel systems
|
||||
TBB/CPP ``F`` Not Useful Cheap Parallel host, serial device
|
||||
OMP/CPP ``F`` Not Useful Cheap Parallel host, serial device
|
||||
======== =========== ========== ========= ============================
|
||||
|
||||
CUDA Specific CMake Options
|
||||
---------------------------
|
||||
|
||||
- ``THRUST_ENABLE_RDC_TESTS={ON, OFF}``
|
||||
|
||||
- Enable tests that require separable compilation.
|
||||
- Default is ``ON``.
|
||||
|
||||
- ``THRUST_FORCE_RDC={ON, OFF}``
|
||||
|
||||
- Enable separable compilation on all targets that are agnostic of
|
||||
RDC.
|
||||
- Targets that explicitly require RDC to be enabled or disabled will
|
||||
ignore this setting.
|
||||
- Default is ``OFF``.
|
||||
|
||||
TBB Specific CMake Options
|
||||
--------------------------
|
||||
|
||||
When using TBB as a host or device system, Thrust will automatically find and
|
||||
link against an installed Intel TBB. The TBB installation is discovered through
|
||||
CMake's standard ``find_package(TBB)`` mechanism. No additional Thrust-specific
|
||||
options are provided for TBB configuration.
|
||||
271
cccl_upstream/docs/thrust/developer/systems.rst
Normal file
271
cccl_upstream/docs/thrust/developer/systems.rst
Normal file
@@ -0,0 +1,271 @@
|
||||
.. _systems:
|
||||
|
||||
Thrust systems
|
||||
==============
|
||||
|
||||
Thrust offers a set of algorithms and APIs which can dispatch to various systems.
|
||||
A system is basically a backend and Thrust currently supports the following systems:
|
||||
|
||||
- cpp
|
||||
- cuda
|
||||
- omp
|
||||
- tbb
|
||||
- generic
|
||||
- sequential
|
||||
|
||||
The generic and sequential systems are implementation details.
|
||||
Users can define additional systems to add new backends.
|
||||
|
||||
Each system lives in a directory under ``thrust/system/[detail/]``.
|
||||
|
||||
|
||||
Execution policy base classes
|
||||
*****************************
|
||||
|
||||
Thrust defines common base classes for execution policies:
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
namespace thrust::detail {
|
||||
struct execution_policy_marker {};
|
||||
|
||||
template <typename DerivedPolicy>
|
||||
struct execution_policy_base : execution_policy_marker {};
|
||||
}
|
||||
namespace thrust {
|
||||
template <typename DerivedPolicy>
|
||||
struct execution_policy : thrust::detail::execution_policy_base<DerivedPolicy> {};
|
||||
}
|
||||
|
||||
There is an execution policy marker, which sits at the top of the inheritance chain.
|
||||
Then, we have an execution policy base and the actual execution policy,
|
||||
both are templated on the derived policy type (CRTP).
|
||||
|
||||
|
||||
System execution policy base classes and tags
|
||||
*********************************************
|
||||
|
||||
Inside each system directory is a header file ``execution_policy.h``
|
||||
which defines the execution policy and tag for that system,
|
||||
except for the generic system, which does not have a dedicated execution policy.
|
||||
The inheritance for a system, e.g. ``cpp``, looks like this:
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
namespace thrust::system::cpp {
|
||||
struct tag; // forward declaration
|
||||
|
||||
template <typename Derived>
|
||||
struct execution_policy; // forward declaration
|
||||
|
||||
template <>
|
||||
struct execution_policy<tag> : ... {};
|
||||
|
||||
struct tag : execution_policy<tag> {};
|
||||
|
||||
template <typename Derived>
|
||||
struct execution_policy : ... {
|
||||
using tag_type = tag;
|
||||
_CCCL_HOST_DEVICE operator tag() const { return {}; }
|
||||
};
|
||||
}
|
||||
|
||||
Each system has it's own execution policy, again templated on a further derived execution policy.
|
||||
The system's execution policy derives (directly or indirectly) from ``thrust::execution_policy``.
|
||||
System execution policies are templates and intended to be further derived from.
|
||||
Additionally, there is a tag for each system, without any template parameters,
|
||||
that derives from the system's execution policy.
|
||||
Tags are non-template class types.
|
||||
The system's execution policy is specialized for the tag type to have no members,
|
||||
otherwise it has an alias for the tag type and can convert to the tag type.
|
||||
Therefore, the execution policy can always be converted to a tag (either by downcasting or by a conversion).
|
||||
|
||||
Various systems now further extend this hierarchy of execution policies, or play other tricks.
|
||||
The ``cpp::execution_policy<Derived>`` inherits from ``sequential::execution_policy<Derived>`` for example
|
||||
(which then inherits from ``thrust::execution_policy``),
|
||||
so any dispatch to an algorithm in the cpp system may fall back to the sequential system.
|
||||
The cuda tag additionally inherits from ``allocator_aware_execution_policy`` to provide further functionality.
|
||||
|
||||
|
||||
Parallel and sequential policy
|
||||
******************************
|
||||
|
||||
Each system also defines an internal parallel policy ``thrust::system::*::detail::par_t``.
|
||||
The cpp system for example:
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
namespace thrust::system::cpp {
|
||||
namespace detail {
|
||||
struct par_t : execution_policy<par_t>, ... {};
|
||||
}
|
||||
inline constexpr detail::par_t par;
|
||||
}
|
||||
|
||||
These policies can be used by a user directly, to pick an execution order with a specific backend.
|
||||
Some systems also provide additional parallel execution policies,
|
||||
or member functions which can further configure a policy.
|
||||
The CUDA system for example also provides ``par_nosync_t``
|
||||
or can customize ``par_t`` by calling ``par.on(stream)``.
|
||||
In any case, the type passed to a Thrust algorithm will always be
|
||||
a class derived from the system's ``execution_policy`` class template.
|
||||
|
||||
Thrust further defines a single sequential policy, ``thrust::seq``:
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
namespace thrust {
|
||||
namespace detail {
|
||||
struct seq_t : system::detail::sequential::execution_policy<seq_t>, ... { ... };
|
||||
}
|
||||
inline constexpr detail::seq_t seq;
|
||||
}
|
||||
|
||||
which is a global constant of the execution policy to the sequential system.
|
||||
|
||||
|
||||
Host and device system policy
|
||||
*****************************
|
||||
|
||||
Thrust additionally defines an active host and device system,
|
||||
which are selected by the macros ``THRUST_HOST_SYSTEM`` and ``THRUST_DEVICE_SYSTEM``
|
||||
and alias to the corresponding system parallel policies:
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
namespace thrust {
|
||||
namespace detail {
|
||||
using host_t = thrust::__THRUST_HOST_SYSTEM_NAMESPACE::detail::par_t;
|
||||
using device_t = thrust::__THRUST_DEVICE_SYSTEM_NAMESPACE::detail::par_t;
|
||||
}
|
||||
inline constexpr detail::host_t host;
|
||||
inline constexpr detail::device_t device;
|
||||
}
|
||||
|
||||
Users most often use ``thrust::host`` and ``thrust::device`` to dispatch to the current host or device system.
|
||||
|
||||
|
||||
Algorithm dispatch
|
||||
******************
|
||||
|
||||
Each Thrust algorithm overload requires an execution policy to determine the backend to use.
|
||||
The policy can either be specified as a first argument by the user,
|
||||
or determined from the other arguments.
|
||||
We will focus on the first case for now.
|
||||
Let's take the public API entry point ``thrust::sort`` as an example.
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
namespace thrust {
|
||||
template <typename DerivedPolicy, typename RandomAccessIterator>
|
||||
void sort(const thrust::detail::execution_policy_base<DerivedPolicy>& exec,
|
||||
RandomAccessIterator first, RandomAccessIterator last);
|
||||
}
|
||||
|
||||
We can see that the first argument is a reference to ``execution_policy_base``,
|
||||
the highest base class in the execution policy hierarchy
|
||||
that still carries compile-time information on the most derived type.
|
||||
This ensures that this overload is only selected, when the user passes a valid execution policy.
|
||||
For comparison, C++17 parallel algorithms use a plain template parameter for the execution policy
|
||||
and apply a constraint (SFINAE or requires).
|
||||
The reference is also ``const`` so users can pass a temporary execution policy object,
|
||||
which was just created at the call site.
|
||||
|
||||
Let's have a look at the implementation of the public API entry point:
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
namespace thrust {
|
||||
template <typename DerivedPolicy, typename RandomAccessIterator>
|
||||
void sort(const thrust::detail::execution_policy_base<DerivedPolicy>& exec,
|
||||
RandomAccessIterator first, RandomAccessIterator last) {
|
||||
using thrust::system::detail::generic::sort;
|
||||
return sort(thrust::detail::derived_cast(thrust::detail::strip_const(exec)), first, last);
|
||||
}
|
||||
}
|
||||
|
||||
We first bring the generic sort implementation from the generic system into scope.
|
||||
Then, we strip away ``const`` and cast the reference to the execution policy to the most derived type,
|
||||
and perform an unqualified call to ``sort`` with the same arguments apart from the execution policy.
|
||||
|
||||
We have previously seen that execution policies form deeper inheritance chains,
|
||||
and some systems inherit the policy of other systems (e.g. the cpp system inherits the sequential system).
|
||||
The ``derived_cast`` makes sure we perform ADL (argument dependent lookup) using the most specialized execution policy
|
||||
when we try to find overloads of ``sort``.
|
||||
It's also necessary, because when an execution policy is passed to the public API,
|
||||
it binds to the reference of its base class ``execution_policy_base``,
|
||||
for which no backend system exists,
|
||||
so we have to bring the type down again the inheritance chain.
|
||||
|
||||
ADL will find a set of overloads for ``sort`` depending on the type of the execution policy.
|
||||
This set will at least include ``sort`` from the generic system and the API entry point itself.
|
||||
In case of the cpp system, it will also find ``sort`` from the sequential and cpp system.
|
||||
The compiler then ranks the overloads and the best match is the overload from the most specialized execution policy.
|
||||
|
||||
This is neat, because a system does not need to provide implementations of all algorithms.
|
||||
It can just fall back to a generic implementation (falling back to a different algorithm),
|
||||
or to an implementation from a different system.
|
||||
For example, ``thrust::count`` is not implemented in the cpp system,
|
||||
so it falls back to the generic implementation, which uses ``thrust::count_if``.
|
||||
That's also not implemented in the cpp system, so it falls back again to the generic system,
|
||||
which then implements it via ``thrust::transform_reduce``, and so on.
|
||||
As a different example, ``thrust::copy`` for the cpp system brings in the include of the sequential copy implementation,
|
||||
so ADL will find it and prefer it over the generic implementation.
|
||||
|
||||
Any generic algorithm is always outranked by a system specific implementation
|
||||
due to the inheritance chain of execution policies.
|
||||
Let's look at the generic sort implementation's interface:
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
namespace thrust::system::detail::generic {
|
||||
template <typename DerivedPolicy, typename RandomAccessIterator>
|
||||
void sort(thrust::execution_policy<DerivedPolicy>& exec,
|
||||
RandomAccessIterator first, RandomAccessIterator last);
|
||||
}
|
||||
|
||||
Notice that it takes the execution policy argument as ``thrust::execution_policy``,
|
||||
which is derived from ``thrust::detail::execution_policy_base``, which appears in the public API.
|
||||
This is why any overload in the generic system will always outrank the public API entry point.
|
||||
|
||||
|
||||
System selection
|
||||
****************
|
||||
|
||||
Thrust also provides overloads of most algorithms without an execution policy,
|
||||
in which case the execution policy is determined based on the remaining arguments, usually iterators.
|
||||
Let's look at ``thrust::adjacent_difference``:
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
namespace thrust {
|
||||
template <typename InputIterator, typename OutputIterator>
|
||||
OutputIterator adjacent_difference(InputIterator first, InputIterator last, OutputIterator result) {
|
||||
using system::detail::generic::select_system;
|
||||
using System1 = iterator_system_t<InputIterator>;
|
||||
using System2 = iterator_system_t<OutputIterator>;
|
||||
System1 system1;
|
||||
System2 system2;
|
||||
return thrust::adjacent_difference(select_system(system1, system2), first, last, result);
|
||||
}
|
||||
}
|
||||
|
||||
Such an API is implemented by first bringing ``select_system`` from the generic system into scope.
|
||||
Then, we determine the system types associated with all iterator types via ``thrust::iterator_system``,
|
||||
and instantiate these systems.
|
||||
We then select one of these systems and pass the it to the corresponding overload of ``adjacent_difference``,
|
||||
taking an execution policy as first argument.
|
||||
Notice that this call is qualified with ``thrust::``, so ADL is not used here.
|
||||
The dispatch to the correct system will be performed in the called overload of ``adjacent_difference``.
|
||||
|
||||
``select_system`` is implemented in the generic system, but no other Thrust system provides a different version of it.
|
||||
However, since users can define their own systems,
|
||||
they could also provide a different algorithm for selecting between multiple systems.
|
||||
The generic implementation will select the system to which all other systems are convertible
|
||||
(this is called the minimum system).
|
||||
If we remember how execution policies and tags are defined,
|
||||
they form inheritance hierarchies and tags have conversion operators,
|
||||
so those play a role here.
|
||||
``select_system`` may not find a minimum system,
|
||||
in which case it returns ``thrust::detail::unrelated_systems<System1, System2, ...>``,
|
||||
which usually fails to find an overload via ADL and lead to a compilation error.
|
||||
Reference in New Issue
Block a user