feat(cccl): integrate missing CCCL directories — python/, ci/, .agent/, docs/, test/

Sparse-checkout from NVIDIA/cccl main branch to complete cccl_upstream:

Added:
- python/cuda_cccl/ (226 files) — Python bindings for device-level algorithms
  Critical for muh toolchain: cuda.compute.reduce_into, scan, radix_sort, etc.
  Includes 204 .py files with full test coverage for all 27 algorithms
- ci/ (163 files) — Build/test infrastructure
  build_cub.sh, test_cub.sh, build_and_test_targets.sh, matrix.yaml
  Directly maps to our [INFRA-CI] and [INFRA-BUILD] items
- .agent/skills/ (7 files) — NVIDIA's own agent skills for CCCL
  cccl-style/SKILL.md, cccl-test/SKILL.md, sass-diff/SKILL.md
- docs/ (491 files) — Official CCCL documentation
  CI references, CMake guides, Python compute docs, libcudacxx PTX docs
- test/ (12 files) — Top-level integration tests (cuda_smoke, stdpar)
- Root configs: .clang-format, .clang-tidy, CONTRIBUTING.md, pyproject.toml
- CLAUDE.md symlink → AGENTS.md (NVIDIA's standard)

cccl_upstream now mirrors full NVIDIA/cccl structure:
  Before: 42M (cub + thrust + libcudacxx + cudax + c + examples + benchmarks)
  After:  53M (+python +ci +docs +.agent +test +configs)

This completes the CCCL base needed for:
- [muh-bench] items: ci/util/build_and_test_targets.sh for targeted builds
- [CCCL-verify] items: python/cuda_cccl/tests/ as reference implementations
- [CCCL-test] items: ci/test_cub.sh, ci/test_thrust.sh
- Agent workflow: .agent/skills/ for consistent style and test patterns

cccl_upstream/.agent/skills/cccl-style/SKILL.md

@@ -0,0 +1,14 @@
+---
+name: cccl-style
+description: Use when editing or reviewing CCCL code for style conventions; read common CCCL guidance and the path-specific references named by this skill.
+---
+
+# CCCL Style
+
+## Workflow
+
+1. Read `references/common.md` for guidance that applies across CCCL.
+2. For `libcudacxx/include/**/*`, also read `references/libcudacxx.md`.
+3. For `cudax/include/**/*`, also read `references/libcudacxx.md`.
+4. If no path-specific reference exists, follow nearby code and repository docs. Do not import rules from another subproject.
+5. Apply each reference only to its stated scope. Rules for one CCCL subproject do not automatically apply to another.

cccl_upstream/.agent/skills/cccl-style/references/common.md

@@ -0,0 +1,70 @@
+# Common CCCL Style Guidance
+
+Apply this guidance across CCCL unless a path-specific style reference says otherwise.
+
+## Naming Style
+
+- Macros: macro style, e.g. `MY_MACRO`.
+- Template parameters: PascalCase, e.g. `MyParameter`.
+- All other symbols: snake style, e.g. `my_variable`. The one exception is the CUB public API, which uses PascalCase.
+
+## Variables
+
+- All variables that are not modified must use `const`. This includes variables initialized by casts (`static_cast`, `reinterpret_cast`, `bit_cast`), function return values, and loop-invariant computations.
+- All variables that can be evaluated at compile-time must use `constexpr`.
+- All `constexpr` variables at namespace/global scope must use `inline`, including variable templates.
+- Consider using plural names for array, span, list, e.g. `int values[4]` instead of `int value[4]`.
+- Use uniform initialization for class constructors (not enforced to builtin types) and compile-time conversions, e.g. `constexpr auto x = int{sizeof(float)};`.
+
+## Headers
+
+- Files must include all headers related to the symbols that they are using.
+- Relying on transitive header inclusion is not allowed.
+- Unneeded headers must be removed.
+- All headers must have the correct license. This also applies to source files.
+- All header inclusions must use the syntax `<header>`.
+- Use forward declaration, namely `__fwd/header.h` or direct type declaration, when possible instead of including the implementation header.
+- Headers should be the most precise available, e.g. `#include <cuda/std/__type_traits/is_array.h>`.
+- Do not include headers in `cuda/std/__cccl/` directly; they are provided by `__config` or the prologue/epilogue mechanism.
+
+## Functions
+
+- Functions must be marked `_CCCL_HOST_API`, `_CCCL_DEVICE_API`, `_CCCL_HOST_DEVICE_API`, `_CCCL_TILE_API`, or `_CCCL_API`.
+- Non-template, non-`constexpr` functions must use `inline`.
+- Most functions with a non-void return type should use `[[nodiscard]]`; functions with known side effects may be exceptions.
+- Functions that do not throw exceptions must use `noexcept`.
+- Use `_CCCL_CONSTEVAL` when the function can only be evaluated at compile time.
+- Use C++20 concept macros instead of SFINAE, e.g. `_CCCL_TEMPLATE(...)` and `_CCCL_REQUIRES(...)`.
+
+## Function Calls And Types
+
+- In headers, apply global qualification where the subproject requires it:
+  - libcudacxx and cudax require free function calls to be fully qualified from the global namespace, e.g. `::cuda::ceil_div(...)`.
+  - CUB applies this rule only to calls to symbols under the `::cuda` namespace hierarchy; otherwise follow existing CUB qualification style.
+  - Thrust uses leading `::` for many symbols under the `::cuda` namespace hierarchy, but relies on ADL in many places and the blanket free-function qualification rule does not apply to those calls.
+- For covered calls, this includes calls to functions defined in the same namespace, e.g. inside `cuda::`, call `::cuda::ceil_div(...)`, not `ceil_div(...)`. This does not apply to (static) member functions of classes. The only exceptions for covered calls are functions that are supposed to be found through argument-dependent lookup (ADL), such as `::cuda::std::swap` and `::cuda::std::get`. Those functions can be called unqualified with a preceding `using ::cuda::std::get;`.
+- This global-qualification rule does not apply to source files such as tests and benchmarks.
+- In headers, apply type-name qualification where the subproject requires it:
+  - libcudacxx and cudax require type names to be fully qualified except when they are already declared in the current namespace or an enclosing one. Outside those namespaces, fully qualify `cuda::std` and standard integer type aliases such as `::cuda::std::size_t`.
+  - CUB applies this rule only to type names under the `::cuda` namespace hierarchy. Do not apply the libcudacxx/cudax blanket type-qualification rule to CUB namespaces or `detail` namespaces.
+  - Thrust does not apply the libcudacxx/cudax blanket type-qualification rule. It uses leading `::` for many `::cuda` and `::cuda::std` type names, but also uses Thrust namespace patterns; follow neighboring Thrust code.
+- A local `using` declaration, e.g. `using ::cuda::std::size_t;`, is acceptable to avoid repetition within a function body.
+- Static member functions of a class template inherit the class's namespace.
+
+## Comments
+
+- Commented code without a description is not allowed.
+
+## General Guidelines
+
+- The code must reuse `cuda/` or `cuda/std` functionalities as much as possible, including macros.
+- Try to use modern C++ as much as possible. The repository supports C++17 but many more recent functionalities have been backported with functions and macros.
+
+## Prevent Compiler Errors And Improve Compatibility
+
+- Remove unused code, variables, functions, types, template parameters, headers, etc.
+- Variables that are unsigned, or that can become unsigned after template instantiation, must not check for negative values directly. Use `cuda::std::is_unsigned_v<T> ? false : (var < 0)` instead.
+
+## Compiler Compatibility
+
+- Protect host-only code with `#if !_CCCL_COMPILER(NVRTC)`.

cccl_upstream/.agent/skills/cccl-style/references/libcudacxx.md

@@ -0,0 +1,55 @@
+# libcudacxx Style Guidance
+
+Use this reference for `libcudacxx/include/**/*` and `cudax/include/**/*`.
+
+## Naming Style
+
+All non-public symbols must be C++ reserved identifiers:
+
+- `_` for macros and template parameters, e.g. `_MY_MACRO`, `_MyParameter`.
+- `__` for all other symbols, e.g. `__my_variable`.
+- Never use reserved keywords, such as `__in`, `__out`, or `__inout` as variables, parameters, or function names.
+- Avoid single-letter template parameter names. Wrong: `_T`; correct: `_Tp`.
+
+## Class / Struct
+
+- Data member names have postfix `_`, e.g. `class __myclass { int __data_; };`.
+- Constructor parameter names should match class/struct data member names without the postfix `_`, e.g. `class __myclass { __myclass(int __data) : __data_(__data) {} };`.
+
+## Functions
+
+- Use `constexpr` for functions that do not depend on run-time features, such as pointers.
+- If the return type is not explicit (`auto`), then a trailing return type is strongly preferred.
+
+## Headers
+
+- Use the correct license:
+  - `libcudacxx/include/cuda/std` files ported from LLVM libc++ use the LLVM license.
+  - `libcudacxx/include/cuda/` files use Apache License v2.0 with LLVM Exceptions.
+- Headers use include guards with names derived from the uppercase full path and closing `#endif` comments repeating the guard name.
+- Right after the include guard, include:
+
+```cpp
+#include <cuda/std/detail/__config>
+
+#if defined(_CCCL_IMPLICIT_SYSTEM_HEADER_GCC)
+#  pragma GCC system_header
+#elif defined(_CCCL_IMPLICIT_SYSTEM_HEADER_CLANG)
+#  pragma clang system_header
+#elif defined(_CCCL_IMPLICIT_SYSTEM_HEADER_MSVC)
+#  pragma system_header
+#endif // no system header
+```
+
+- The last included header before code must be `<cuda/std/__cccl/prologue.h>`, and `<cuda/std/__cccl/epilogue.h>` must appear at the end of the file.
+
+## Comments
+
+- Use Doxygen-style `//! @brief` comments.
+- Documented functions must include `//! @brief`, `//! @param[in/out/in,out]` for every parameter, and `//! @return` for non-void functions.
+- The `@brief/@param/@return` description must accurately reflect the current functionality of the function.
+
+## Compiler Compatibility
+
+- Do not use lambda expressions in device-only or host-device code.
+- Do not rely on deduction guides for initialization; use explicit template arguments instead.

cccl_upstream/.agent/skills/cccl-test/SKILL.md

@@ -0,0 +1,13 @@
+---
+name: cccl-test
+description: Use when writing, updating, reviewing, or validating CCCL tests; read common CCCL test guidance and the path-specific references named by this skill.
+---
+
+# CCCL Test
+
+## Workflow
+
+1. Read `references/common.md` for guidance that applies across CCCL tests.
+2. For `libcudacxx/test/**/*`, also read `references/libcudacxx.md`.
+3. If no path-specific reference exists, follow nearby tests and repository docs. Do not import test rules from another subproject.
+4. Apply each reference only to its stated scope. Rules for one CCCL subproject do not automatically apply to another.

cccl_upstream/.agent/skills/cccl-test/references/common.md

@@ -0,0 +1,31 @@
+# Common CCCL Test Guidance
+
+Apply this guidance across CCCL tests unless a path-specific test reference says otherwise.
+
+## Local Consistency
+
+- Read nearby tests first and mirror their directory layout, file names, helper types, includes, assertion style, and local gating or skip mechanisms.
+
+## Coverage
+
+- Cover relevant edge cases.
+- Cover relevant input and output types.
+- Cover error behavior when applicable.
+- Cover runtime and compile-time behavior when applicable.
+- Cover device and host behavior when applicable.
+
+## Test Structure
+
+- All tests must have the correct license banner.
+- Use the local test harness assertions and helpers.
+- Use compile-time checks for compile-time guarantees and constexpr coverage when relevant.
+- Negative tests should check the intended diagnostic or failure mode when the local harness supports it.
+
+## Portability
+
+- Prefer project test macros and helpers for compiler, dialect, exception, host/device, and platform probes instead of spelling ad hoc checks directly.
+- If a test is unsupported, expected to fail, disabled, or skipped on a platform, motivate it with a comment.
+
+## Validation
+
+- Use targeted test runs for the project and files being changed.

cccl_upstream/.agent/skills/cccl-test/references/libcudacxx.md

@@ -0,0 +1,83 @@
+# libcudacxx Test
+
+## Organization
+
+- Put CUDA Standard Library tests under `libcudacxx/test/libcudacxx/std/...`.
+- Put CUDA-specific API tests under `libcudacxx/test/libcudacxx/cuda/...`, unless an adjacent `std/...` directory is clearly the established home for the functionality.
+- Read nearby tests first and mirror their directory layout, file names, helper types, includes, and lit gates.
+
+## Purpose
+
+- Validate libcudacxx functionality. It is fundamental to verify:
+
+  - Edge cases.
+  - Input and output types.
+  - Exception behavior.
+  - Runtime and constant-evaluation behavior.
+  - Device and host behavior.
+
+## Test kinds
+
+- `.pass.cpp`: compiles, links, runs, and returns 0.
+- `.compile.pass.cpp`: compiles correctly.
+- `.fail.cpp`: must fail compilation. Prefer precise `expected-error`, `expected-warning`, `expected-note`, or `expected-no-diagnostics` annotations when clang verify is supported.
+- `.runfail.cpp`: compiles and runs but must return non-zero.
+
+## Test structure
+
+- All tests must have the correct license banner.
+- Always include top level headers, never internal ones with `__` prefix.
+- Include support headers `"test_macros.h"`, `"test_iterators.h"`, `"test_comparisons.h"`, when needed.
+- Use `static_assert(...)` for compile-time guarantees and constexpr coverage.
+- Use `<cuda/std/cassert>` and `assert(...)` for runtime checks.
+- The `main` function must be present, dispatch runtime and static-evaluation tests, and return 0.
+
+## Style
+
+- Use `cuda::std` names, not `std::` names, unless the test is intentionally checking interoperability with host standard library types.
+- Do not fully qualify names in header includes unless the test is intentionally checking interoperability with host standard library types.
+- Mark helper functions that may run on host and device with `TEST_FUNC`; use `TEST_DEVICE_FUNC` for device-only helpers.
+  use `TEST_TILE_FUNC` for tile only helpers and `TEST_TILE_DEVICE_FUNC` for functions that can run on tile and device
+- `const`-qualification is discouraged.
+- Don't use `noexcept` for helper functions unless strictly necessary.
+- Do not use lambda expressions in host/device test code unless nearby tests already prove the pattern is supported.
+
+## Portability
+
+- Guard host-only or device-only behavior with `NV_IF_TARGET(NV_IS_HOST, (...))` and `NV_IF_TARGET(NV_IS_DEVICE, (...))` respectively.
+- Use `TEST_STD_VER`, `TEST_COMPILER`, `TEST_CUDA_COMPILER`, `TEST_HAS_EXCEPTIONS`, and `TEST_THROW` from `"test_macros.h"` instead of spelling compiler or dialect probes directly.
+- Unsupported platforms can be disabled with `UNSUPPORTED: <feature-name>` or `XFAIL: <feature-name>` lit directives. Some common feature names are `nvrtc`, `enable-tile`, `pre-sm-70`, `c++17`, `c++20`, `msvc`, `gcc-<version>`, or `clang-<version>`.
+
+  - Always motivate unsupported features with a comment.
+
+## Lit directives
+
+- Put lit directives near the top of the file before includes.
+- Common directives: `UNSUPPORTED:`, `XFAIL:`, `REQUIRES:`, `ADDITIONAL_COMPILE_DEFINITIONS:`, `ADDITIONAL_COMPILE_OPTIONS_HOST:`, `ADDITIONAL_COMPILE_OPTIONS_CUDA:`, `MODULES_DEFINES:`, and `CONSTEXPR_STEPS:`.
+- For diagnostics in `.fail.cpp`, annotate the exact line that should fail when possible:
+
+```cpp
+bad_expression(); // expected-error {{message fragment}}
+```
+
+- Prefer checking the intended diagnostic over accepting any compile failure.
+
+## Validation
+
+Use targeted libcudacxx lit runs. Paths passed to `--lit-precompile-tests` and `--lit-tests` are relative to `libcudacxx/test/libcudacxx/`.
+
+```bash
+ci/util/build_and_test_targets.sh \
+  --preset libcudacxx \
+  --lit-precompile-tests "std/algorithms/alg.nonmodifying/alg.any_of/any_of.pass.cpp" \
+  --lit-tests "std/algorithms/alg.nonmodifying/alg.any_of/any_of.pass.cpp"
+```
+
+If running lit directly, use the configured site file:
+
+```bash
+LIBCUDACXX_SITE_CONFIG=<path-to-cccl>/build/<preset>/libcudacxx/test/libcudacxx/lit.site.cfg \
+  lit -v libcudacxx/test/libcudacxx/<relative-test-path>
+```
+
+- Use `-Dexecutor=NoopExecutor()` for precompile-only validation when runtime execution is unavailable or GPU coverage is not required.

cccl_upstream/.agent/skills/sass-diff/SKILL.md

@@ -0,0 +1,50 @@
+---
+name: sass-diff
+description: Use when asked to check for SASS (or PTX) changes between commits, branches, or a local changeset; guides normalization, comparison, and reporting of CUDA disassembly diffs.
+---
+
+# SASS Diffs
+
+Use this when asked to check for SASS changes between commits, branches or a local changeset.
+
+## Goal
+
+Detect relevant changes in generated CUDA machine code (i.e. SASS) while filtering noise from addresses, symbols, metadata, etc.
+Any non-trivial change must be detected.
+
+## Inputs to establish
+
+* Compilation target under test
+* The CUDA SM architectures to compile for. Try to detect this from the code and offer the user a list of suggestions.
+  The user must confirm or provide this list.
+* Baseline source (e.g. the previous commit/branch or the current commit without the changes in the working copy).
+* Comparison source (e.g.  the current commit/branch or the current commit with the changes in the working copy).
+* Whether a SASS (default) or PTX diff is requested.
+
+## Disassembly listing generation
+
+* Compile both, the baseline and comparison source, with the same compiler flags and options.
+  When not specified otherwise, lookup the options from `compile_commands.json`
+  or the current build system (i.e. CMake files).
+  Make sure the CUDA SM architectures (`CMAKE_CUDA_ARCHITECTURES`) are set to the user-provided or approved list.
+* Dump the disassembly from the binaries produced in the previous set using `cuobjdump -sass` or `cuobjdump -ptx`.
+
+## Comparison rules (what matters)
+
+Ignore as trivial:
+
+* Register renaming with identical instruction sequence and operands.
+* Pure label renumbering or reordering of identical basic blocks.
+* Formatting-only differences or reordered symbol tables.
+* Changes to symbol names (global function names)
+
+## Reporting
+
+* If any non-trivial change was detected, report the top 5 regions where a non-trivial change was detected,
+  including the name of the kernel they appeared in.
+* Provide a short summary of the diff type,
+  including opcode changes, memory access size/cache policy changes, control-flow changes, register-count changes,
+  spills/local memory, shared memory, and occupancy-relevant resource deltas.
+* Explicitly state if only noise was detected.
+* If you are not sure if the differences are impactful, show it and ask the user for guidance.
+* Keep the disassembly dumps available and tell the user where they can find them.

cccl_upstream/.clang-format

@@ -0,0 +1,205 @@
+# Note that we don't specify the language in this file because some files are
+# detected as Cpp, but others are detected as ObjC and we want this formatting
+# to apply to all types of files.
+BasedOnStyle: LLVM
+AccessModifierOffset: -2
+AlignAfterOpenBracket: Align
+AlignConsecutiveAssignments: Consecutive
+AlignConsecutiveBitFields: Consecutive
+AlignConsecutiveMacros: Consecutive
+AlignEscapedNewlines: Left
+AlignOperands: AlignAfterOperator
+AllowAllArgumentsOnNextLine: true
+AlignTrailingComments:
+  Kind: Never
+AllowAllParametersOfDeclarationOnNextLine: true
+AllowAllConstructorInitializersOnNextLine: true
+AllowShortBlocksOnASingleLine: false
+AllowShortCaseLabelsOnASingleLine: false
+AllowShortFunctionsOnASingleLine: Empty
+AllowShortIfStatementsOnASingleLine: Never
+AllowShortLambdasOnASingleLine: Empty
+AllowShortLoopsOnASingleLine: false
+AlwaysBreakAfterReturnType: None
+AlwaysBreakTemplateDeclarations: Yes
+AttributeMacros: [
+                  '_CCCL_ALIGNAS_TYPE',
+                  '_CCCL_ALIGNAS',
+                  '_CCCL_CONSTEXPR_CXX20',
+                  '_CCCL_CONSTEXPR_CXX23',
+                  '_CCCL_DECLSPEC_EMPTY_BASES',
+                  '_CCCL_DEVICE',
+                  '_CCCL_FORCEINLINE',
+                  '_CCCL_HIDE_FROM_ABI',
+                  '_CCCL_HOST_DEVICE',
+                  '_CCCL_DEDUCTION_GUIDE_ATTRIBUTES',
+                  '_CCCL_HOST',
+                  '_CCCL_KERNEL_ATTRIBUTES',
+                  '_CCCL_NO_UNIQUE_ADDRESS',
+                  '_CCCL_TYPE_VISIBILITY_DEFAULT',
+                  '_CCCL_TYPE_VISIBILITY_HIDDEN',
+                  '_CCCL_VISIBILITY_HIDDEN',
+                  '_CCCL_LAUNCH_BOUNDS',
+                  '_CCCL_BLOCK_SIZE',
+                  'CUB_RUNTIME_FUNCTION',
+                  'THRUST_RUNTIME_FUNCTION',
+                  'CCCL_DEPRECATED',
+                  'CCCL_DEPRECATED_BECAUSE',
+                  '_CCCL_DEPRECATED_IN_CXX20',
+                  '_CCCL_DEPRECATED_IN_CXX23',
+                  '_CCCL_API',
+                  '_CCCL_HOST_API',
+                  '_CCCL_DEVICE_API',
+                  '_CCCL_NODEBUG_API',
+                  '_CCCL_NODEBUG_HOST_API',
+                  '_CCCL_NODEBUG_DEVICE_API',
+                  '_CCCL_TRIVIAL_API',
+                  '_CCCL_TRIVIAL_HOST_API',
+                  '_CCCL_TRIVIAL_DEVICE_API',
+                  '_CCCL_PUBLIC_API',
+                  '_CCCL_PUBLIC_HOST_API',
+                  '_CCCL_PUBLIC_DEVICE_API',
+                 ]
+BinPackArguments: false
+BinPackParameters: false
+BreakBeforeBraces: Custom
+BraceWrapping:
+  AfterCaseLabel: false
+  AfterClass: true
+  AfterControlStatement: true
+  AfterEnum: true
+  AfterFunction: true
+  AfterNamespace: true
+  AfterStruct: true
+  AfterUnion: true
+  BeforeCatch: true
+  BeforeElse: true
+  IndentBraces: false
+  SplitEmptyFunction: false
+  SplitEmptyRecord: false
+BreakBeforeConceptDeclarations: true
+BreakBeforeBinaryOperators: NonAssignment
+BreakBeforeTernaryOperators: true
+BreakConstructorInitializers: BeforeComma
+BreakInheritanceList: BeforeComma
+ColumnLimit: 120
+CompactNamespaces: false
+ContinuationIndentWidth: 2
+EmptyLineAfterAccessModifier: Never
+EmptyLineBeforeAccessModifier: Always
+FixNamespaceComments: true
+IfMacros: [
+  '_CUB_WEAKEN_IF_CONSTEXPR_IF_COMPILED_FOR_CCCL_C',
+  '_CCCL_CATCH'
+]
+IndentWrappedFunctionNames: false
+IncludeBlocks:   Regroup
+IncludeCategories:
+  - Regex:           '^<cuda/std/__cccl/prologue.h>'
+    Priority:            0x7FFFFFFF
+    SortPriority:        0x7FFFFFFF
+  - Regex:           '^<cuda/std/__cccl/epilogue.h>'
+    Priority:            -0x7FFFFFFF
+    SortPriority:        -0x7FFFFFFF
+  - Regex:           '^<cuda/experimental/__execution/prologue.cuh>'
+    Priority:            0x7FFFFFFF
+    SortPriority:        0x7FFFFFFF
+  - Regex:           '^"insert_nested_NVTX_range_guard.h"'
+    Priority:            -1
+    SortPriority:        -1
+  - Regex:           '^<(cuda/std/detail/__config|cub/config.cuh|thrust/detail/config.h|thrust/system/cuda/config.h)'
+    Priority:            0
+    SortPriority:        0
+  - Regex:           '^<cub/'
+    Priority:            2
+    SortPriority:        1
+  - Regex:           '^<thrust/'
+    Priority:            3
+    SortPriority:        2
+  - Regex:           '^<cuda/experimental'
+    Priority:            5
+    SortPriority:        4
+  - Regex:           '^<cuda/'
+    Priority:            4
+    SortPriority:        3
+  - Regex:           '^<nv/'
+    Priority:            6
+    SortPriority:        5
+  - Regex:           '^<[a-z_]*>$'
+    Priority:            7
+    SortPriority:        6
+  - Regex:           '^<[a-z_]*\.[a-z]+>$'
+    Priority:            8
+    SortPriority:        7
+  - Regex:           '^<cuda'
+    Priority:            0
+    SortPriority:        0
+InsertBraces: true
+IndentCaseLabels: true
+InsertNewlineAtEOF: true
+InsertTrailingCommas: Wrapped
+IndentRequires: true
+IndentPPDirectives: AfterHash
+IndentWidth: 2
+KeepEmptyLines:
+  AtEndOfFile: true
+  AtStartOfBlock: false
+  AtStartOfFile: false
+MaxEmptyLinesToKeep: 1
+Macros:
+- _CCCL_TEMPLATE(...)=template<...>
+- _CCCL_REQUIRES(...)=requires (...)
+- _CUDAX_SEMI_PRIVATE=private
+NamespaceIndentation: None
+PackConstructorInitializers: Never
+PenaltyBreakAssignment: 30
+PenaltyBreakBeforeFirstCallParameter: 50
+PenaltyBreakComment: 0
+PenaltyBreakFirstLessLess: 0
+PenaltyBreakString: 70
+PenaltyBreakTemplateDeclaration: 0
+PenaltyExcessCharacter: 100
+PenaltyReturnTypeOnItsOwnLine: 90
+PenaltyIndentedWhitespace: 2
+PointerAlignment: Left
+ReflowComments: true
+RemoveSemicolon: false
+SortIncludes: CaseInsensitive
+SpaceAfterCStyleCast: true
+SpaceAfterLogicalNot: false
+SpaceAfterTemplateKeyword: true
+SpaceBeforeAssignmentOperators: true
+SpaceBeforeCpp11BracedList: false
+SpaceBeforeCtorInitializerColon: true
+SpaceBeforeInheritanceColon: true
+SpaceBeforeParens: ControlStatements
+SpaceBeforeRangeBasedForLoopColon: true
+SpaceInEmptyParentheses: false
+SpacesBeforeTrailingComments: 1
+SpacesInAngles: Never
+SpacesInCStyleCastParentheses: false
+SpacesInParentheses: false
+SpacesInSquareBrackets: false
+Standard: c++20
+StatementMacros: [
+  '_CCCL_EXEC_CHECK_DISABLE',
+  'CUB_NAMESPACE_BEGIN',
+  'CUB_NAMESPACE_END',
+  'THRUST_NAMESPACE_BEGIN',
+  'THRUST_NAMESPACE_END',
+  '_CCCL_BEGIN_NAMESPACE_CUDA_STD',
+  '_CCCL_END_NAMESPACE_CUDA_STD',
+  '_CCCL_BEGIN_NAMESPACE_CUDA_STD_NOVERSION',
+  '_CCCL_END_NAMESPACE_CUDA_STD_NOVERSION',
+  '_CCCL_BEGIN_NAMESPACE_CUDA_STD_RANGES',
+  '_CCCL_END_NAMESPACE_CUDA_STD_RANGES',
+  '_CCCL_BEGIN_NAMESPACE_CUDA_STD_RANGES_ABI',
+  '_CCCL_END_NAMESPACE_CUDA_STD_RANGES_ABI',
+  '_CCCL_BEGIN_NAMESPACE_CUDA_STD_VIEWS',
+  '_CCCL_END_NAMESPACE_CUDA_STD_VIEWS',
+  '_CCCL_BEGIN_NAMESPACE_CPO',
+  '_CCCL_END_NAMESPACE_CPO',
+]
+TabWidth: 2
+UseTab: Never
+WrapNamespaceBodyWithEmptyLines: Never

cccl_upstream/.clang-tidy

@@ -0,0 +1,324 @@
+# SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES.
+#                         All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+---
+Checks:
+  # enable everything first
+  - 'performance-*'
+  - 'modernize-*'
+  - 'readability-*'
+  - 'clang-analyzer-*'
+  - 'clang-diagnostic-*'
+  - 'bugprone-*'
+  - 'misc-*'
+  - 'core-*'
+  - 'mpi-*'
+  - 'cert-*'
+  - 'portability-*'
+  - 'google-*'
+  - 'cppcoreguidelines-*'
+  - 'concurrency-*'
+  # TODO: BEGIN REMOVE ME
+  - '-*'
+  - 'performance-*'
+  - '-performance-enum-size'
+  - '-performance-unnecessary-value-param'
+  - 'modernize-*'
+  - '-modernize-use-integer-sign-comparison'
+  - '-modernize-use-nodiscard'
+  - '-modernize-return-braced-init-list'
+  - '-modernize-use-auto'
+  - 'clang-diagnostic-*'
+  - 'bugprone-*'
+  - '-bugprone-crtp-constructor-accessibility'
+  - '-bugprone-argument-comment'
+  # This is just noise
+  - '-bugprone-signed-char-misuse'
+  - '-bugprone-multi-level-implicit-pointer-conversion'
+  # END REMOVE ME
+  # TODO(jfaibussowit):
+  #
+  # Enable these once we move to C++20. Normally, clang-tidy does not suggest checks for
+  # cpp N+1 if you are compiling with -std=c++N. However some subprojects like c.parallel
+  # actually require cpp N+1 so clang-tidy emits diagnostics for any headers included by
+  # it.
+  - '-modernize-use-designated-initializers'
+  - '-modernize-use-constraints'
+  - '-modernize-use-ranges'
+  # Irrelevant for GPU code
+  - '-modernize-pass-by-value'
+  # HICPP is 99% aliased to other checks (mostly modernize-* and bugprone-*). We don't
+  # want to also enable it because then we need to duplicate the NOLINT. The only 2 checks
+  # that aren't aliases are:
+  #
+  # - hicpp-multiway-paths-covered. This check is useful for detecting degenerate if-else
+  #   branches, but the switch cases are IMO better handled by -Wswitch. To make -Wswitch
+  #   even stronger, we should ban default: cases entirely,
+  #
+  # - hicpp-signed-bitwise. This check is also covered by
+  #   clang.llvm.org/extra/clang-tidy/checks/clang-analyzer/core.BitwiseShift.html
+  #
+  # - 'hicpp-*'
+  #
+  # LLVM checks are extremely specific to the LLVM project and aren't suitable for
+  # downstream users.
+  #
+  # - 'llvm-*'
+  #
+  # then disable the stuff we don't want
+  - '-cert-dcl21-cpp'  # returning non-const from operator-- or operator++
+  - '-cert-dcl50-cpp'  # allow c-style variadics
+  # No reserved identifiers, both of these are aliased to bugprone-reserved-identifier,
+  # which we do enable. Leaving these enabled however, leads to needing to specify all
+  # three (bugprone-reserved-identifier, cert-dcl51-cpp, and cert-dcl37-c) in NOLINT lines
+  # which is a hassle. Since bugprone-reserved-identifier is enabled, the check still
+  # fires.
+  - '-cert-dcl51-cpp,-cert-dcl37-c,-cert-oop54-cpp'
+  # Covered by bugprone-throwing-static-initialization
+  - '-cert-err58-cpp'
+  - '-modernize-use-trailing-return-type'
+  - '-readability-function-cognitive-complexity'
+  - '-readability-implicit-bool-conversion'
+  - '-readability-braces-around-statements'
+  - '-readability-qualified-auto'
+  - '-readability-isolate-declaration'
+  - '-modernize-avoid-c-arrays'
+  - '-cppcoreguidelines-avoid-c-arrays'
+  - '-readability-named-parameter'
+  - '-readability-identifier-length'
+  - '-misc-non-private-member-variables-in-classes'
+  - '-bugprone-easily-swappable-parameters'
+  - '-bugprone-implicit-widening-of-multiplication-result'
+  - '-misc-include-cleaner'
+  - '-misc-header-include-cycle'
+  - '-modernize-macro-to-enum'
+  - '-cppcoreguidelines-macro-to-enum'
+  - '-misc-no-recursion'
+  - '-bugprone-dynamic-static-initializers'
+  - '-portability-avoid-pragma-once'
+  # Generally speaking if something is static then it has an express reason to be. 99% of
+  # the candidates identified by this check were because they just so happened not to
+  # touch any member variables, not because they are logically static. So we disable the
+  # check.
+  - '-readability-convert-member-functions-to-static'
+  # When running in a conda env, ignore options that may have been added by conda but are unused
+  - '-clang-diagnostic-unused-command-line-argument'
+  # Given
+  #
+  # std::make_pair<some_type, some_other_type>(...)
+  #
+  # Results in: error: for C++11-compatibility, use pair directly. But we don't care about
+  # C++11, and so we don't care about this warning.
+  - '-google-build-explicit-make-pair'
+  # This check is incredibly expensive for absolutely no reason, and since we a) use
+  # modern google-test and b) don't have clang-tidy enabled on our testing code, we don't
+  # need to enable it!
+  - '-google-upgrade-googletest-case'
+  # This warning will warn if you have redundant default-initializers for class
+  # members. For example:
+  #
+  # class Foo
+  # {
+  #   int x_{}; // WARNING: redundant initializer here
+  #
+  # public:
+  #   Foo(int x) : x_{x} { }
+  # };
+  #
+  # Since Foo can only ever be constructed with an explicit value for x_ via its constructor,
+  # the default initializer is technically redundant. However, if we change the definition
+  # of Foo to now allow a default ctor, then the initializer becomes non-redundant
+  # again. It is easier to just follow the rule of "always explicitly default initialize
+  # members" than to remember to change 2 places at once.
+  - '-readability-redundant-member-init'
+  # Alias for readability-enum-initial-value, disable this one because the readability-
+  # name is easier to understand, and we don't want to silence 2 things for the same
+  # warning.
+  - '-cert-int09-c'
+  # This one is potentially controversial. This check warns when iterating over unordered
+  # containers of pointers:
+  #
+  # {
+  #   int a = 1, b = 2;
+  #   std::unordered_set<int *> set = {&a, &b};
+  #
+  #   for (auto *i : set) { // iteration order not deterministic
+  #     f(i);
+  #   }
+  # }
+  #
+  # On the one hand, this is a clear case of non-determinism. But on the other hand, I
+  # feel it is obvious that a user does not care about the order of iteration
+  # because... they are using unordered containers!
+  - '-bugprone-nondeterministic-pointer-iteration-order'
+  - '-cppcoreguidelines-pro-type-reinterpret-cast'
+  # We don't use GSL
+  - '-cppcoreguidelines-owning-memory'
+  # Covered by modernize-use-override already
+  - '-cppcoreguidelines-explicit-virtual-functions'
+  # Covered by readability-magic-numbers
+  - '-cppcoreguidelines-avoid-magic-numbers'
+  # This check does more harm than good, as it appears to be very rudimentary. For
+  # example, it emits warnings saying that:
+  #
+  # #define DEFINE_OPERATOR(op)                            \
+  #   _CCCL_HOST_DEVICE custom_numeric operator op() const \
+  #   {                                                    \
+  #     return custom_numeric(op value[0]);                \
+  #   }
+  #
+  # Should be replaced by a template function, but this is not possible. We instead rely
+  # on best judgment of reviewers to catch macros that can be functions.
+  - '-cppcoreguidelines-macro-usage'
+  # This check does not understand more complex macro usage where brackets are not allowed
+  #
+  # error: macro replacement list should be enclosed in parentheses [bugprone-macro-parentheses]
+  # 36 | #define __THRUST_HOST_SYSTEM_INCLUDE(filename) <__THRUST_HOST_SYSTEM_ROOT/filename>
+  #    |                                                                           ^
+  #    |                                                                           (       )
+  #
+  # So better left disabled.
+  - '-bugprone-macro-parentheses'
+
+  # TODO: Enable for clang-tidy-22
+  - '-modernize-avoid-c-style-cast'
+  - '-bugprone-signed-bitwise'
+  - '-bugprone-std-namespace-modification'
+  - '-clang-diagnostic-deprecated-attributes'
+  - '-bugprone-throwing-static-initialization'
+  - '-bugprone-unchecked-string-to-number-conversion'
+  - '-bugprone-invalid-enum-default-initialization'
+  - '-bugprone-derived-method-shadowing-base-method'
+  - '-bugprone-unsafe-to-allow-exceptions'
+  - '-bugprone-sizeof-expression'
+  - '-modernize-use-nullptr'
+  - '-bugprone-std-exception-baseclass'
+  - '-modernize-avoid-variadic-functions'
+  - '-modernize-type-traits'
+  - '-bugprone-random-generator-seed'
+  - '-bugprone-command-processor'
+
+  # TODO: ironically enough, enable this someday
+  - '-google-readability-todo'
+  # Covered by modernize-avoid-c-style-cast, and also is buggier. It seems to fire on
+  # `_v`-style constexpr bools as well sometimes.
+  - '-google-readability-casting'
+  # Covered by performance-noexcept-move-constructor
+  - '-cppcoreguidelines-noexcept-move-operations'
+  # REVIEW ME: This warns about any usage of operator[], suggesting usage of .at() instead. Could
+  - '-cppcoreguidelines-pro-bounds-avoid-unchecked-container-access'
+  - '-cppcoreguidelines-narrowing-conversions'
+  # Covered by misc-unconventional-assign-operator
+  - '-cppcoreguidelines-c-copy-assignment-signature'
+  # Covered by performance-noexcept-swap
+  - '-cppcoreguidelines-noexcept-swap'
+  # Covered by modernize-use-default-member-init
+  - '-cppcoreguidelines-use-default-member-init'
+  # clang-tidy documentation says that for a given file, it matches the closest
+  # .clang-tidy up the directory stack and applies the configuration from it. However, by
+  # "it" `clang-tidy` means the closest `.clang-tidy` to the *source* file, not the file
+  # that was included by the source file.
+  #
+  # So `cuda/std/foo.h` included by `cccl/foo/bar/baz.cpp` will never "match" against
+  # `cccl/libcudacxx/.clang-tidy` because `clang-tidy` finds `cccl/.clang-tidy` instead.
+  #
+  # So we disable this for now. In the future we should add a custom check that bans
+  # reserved identifiers except symbols from libcu++.
+  - '-bugprone-reserved-identifier'
+WarningsAsErrors: '*'
+HeaderFileExtensions:
+  - ''
+  - h
+  - hh
+  - hpp
+  - hxx
+  - cuh
+  - inl
+  - ipp
+ImplementationFileExtensions:
+  - c
+  - cc
+  - cpp
+  - cxx
+  - cu
+SystemHeaders: false
+HeaderFilterRegex: '.*/(thrust|cub|cuda|nv|cccl|c2h|cudastf|test)/.*'
+ExtraArgsBefore:
+  # To match _CCCL_DOXYGEN_INVOKED
+  - '-D_CCCL_CLANG_TIDY_INVOKED=1'
+  - '-ftemplate-backtrace-limit=0'
+  - '-fmacro-backtrace-limit=0'
+ExtraArgs:
+  # These must come in ExtraArgs not ExtraArgsBefore, because they are overriding the
+  # effects of -Wall and -Werror, where the last flag "wins".
+  - '-Wno-unknown-warning-option'
+  - '-Wno-unknown-cuda-version'
+  - '-Wno-error=unused-command-line-argument'
+CheckOptions:
+  cert-dcl16-c.NewSuffixes: 'L;LL;LU;LLU'
+  cert-err33-c.CheckedFunctions: '::aligned_alloc;::asctime_s;::at_quick_exit;::atexit;::bsearch;::bsearch_s;::btowc;::c16rtomb;::c32rtomb;::calloc;::clock;::cnd_broadcast;::cnd_init;::cnd_signal;::cnd_timedwait;::cnd_wait;::ctime_s;::fclose;::fflush;::fgetc;::fgetpos;::fgets;::fgetwc;::fopen;::fopen_s;::fprintf;::fprintf_s;::fputc;::fputs;::fputwc;::fputws;::fread;::freopen;::freopen_s;::fscanf;::fscanf_s;::fseek;::fsetpos;::ftell;::fwprintf;::fwprintf_s;::fwrite;::fwscanf;::fwscanf_s;::getc;::getchar;::getenv;::getenv_s;::gets_s;::getwc;::getwchar;::gmtime;::gmtime_s;::localtime;::localtime_s;::malloc;::mbrtoc16;::mbrtoc32;::mbsrtowcs;::mbsrtowcs_s;::mbstowcs;::mbstowcs_s;::memchr;::mktime;::mtx_init;::mtx_lock;::mtx_timedlock;::mtx_trylock;::mtx_unlock;::printf_s;::putc;::putwc;::raise;::realloc;::remove;::rename;::scanf;::scanf_s;::setlocale;::setvbuf;::signal;::snprintf;::snprintf_s;::sprintf;::sprintf_s;::sscanf;::sscanf_s;::strchr;::strerror_s;::strftime;::strpbrk;::strrchr;::strstr;::strtod;::strtof;::strtoimax;::strtok;::strtok_s;::strtol;::strtold;::strtoll;::strtoul;::strtoull;::strtoumax;::strxfrm;::swprintf;::swprintf_s;::swscanf;::swscanf_s;::thrd_create;::thrd_detach;::thrd_join;::thrd_sleep;::time;::timespec_get;::tmpfile;::tmpfile_s;::tmpnam;::tmpnam_s;::tss_create;::tss_get;::tss_set;::ungetc;::ungetwc;::vfprintf;::vfprintf_s;::vfscanf;::vfscanf_s;::vfwprintf;::vfwprintf_s;::vfwscanf;::vfwscanf_s;::vprintf_s;::vscanf;::vscanf_s;::vsnprintf;::vsnprintf_s;::vsprintf;::vsprintf_s;::vsscanf;::vsscanf_s;::vswprintf;::vswprintf_s;::vswscanf;::vswscanf_s;::vwprintf_s;::vwscanf;::vwscanf_s;::wcrtomb;::wcschr;::wcsftime;::wcspbrk;::wcsrchr;::wcsrtombs;::wcsrtombs_s;::wcsstr;::wcstod;::wcstof;::wcstoimax;::wcstok;::wcstok_s;::wcstol;::wcstold;::wcstoll;::wcstombs;::wcstombs_s;::wcstoul;::wcstoull;::wcstoumax;::wcsxfrm;::wctob;::wctrans;::wctype;::wmemchr;::wprintf_s;::wscanf;::wscanf_s;'
+  llvm-else-after-return.WarnOnUnfixable: 'false'
+  cert-str34-c.DiagnoseSignedUnsignedCharComparisons: 'false'
+  cppcoreguidelines-non-private-member-variables-in-classes.IgnoreClassesWithAllMemberVariablesBeingPublic: 'true'
+  google-readability-braces-around-statements.ShortStatementLines: '1'
+  llvm-qualified-auto.AddConstToQualified: 'false'
+  llvm-else-after-return.WarnOnConditionVariables: 'false'
+  cert-oop54-cpp.WarnOnlyIfThisHasSuspiciousField: 'false'
+  performance-move-const-arg.CheckTriviallyCopyableMove: 'false'
+  performance-inefficient-string-concatenation.StrictMode: 'true'
+  readability-simplify-boolean-expr.ChainedConditionalReturn: 'true'
+  readability-simplify-boolean-expr.ChainedConditionalAssignment: 'true'
+  bugprone-dangling-handle.HandleClasses: '::cuda::std::span;::cuda::std::mdspan'
+  bugprone-unused-return-value.AllowCastToVoid: 'true'
+  readability-enum-initial-value.AllowExplicitZeroFirstInitialValue: 'false'
+  readability-enum-initial-value.AllowExplicitSequentialInitialValues: 'false'
+  readability-redundant-access-specifiers.CheckFirstDeclaration: 'true'
+  bugprone-lambda-function-name.IgnoreMacros: 'true'
+  # readability-identifier-naming.ClassCase: 'lower_case'
+  # readability-identifier-naming.UnionCase: 'lower_case'
+  # readability-identifier-naming.ClassConstantCase: 'UPPER_CASE'
+  # readability-identifier-naming.ClassIgnoredRegexp: 'tuple|has_.*|is_.*|as_.*|.*_tag|tag|.*_of'
+  # readability-identifier-naming.ConstantMemberCase: 'UPPER_CASE'
+  # readability-identifier-naming.ConstantMemberIgnoredRegexp: 'value'
+  # readability-identifier-naming.EnumCase: 'lower_case'
+  # readability-identifier-naming.EnumConstantCase: 'UPPER_CASE'
+  # readability-identifier-naming.FunctionCase: 'lower_case'
+  # readability-identifier-naming.GlobalConstantCase: 'UPPER_CASE'
+  # readability-identifier-naming.GlobalConstantIgnoredRegexp: '.*_v'
+  # readability-identifier-naming.LocalVariableCase: 'lower_case'
+  # # We want to allow constexpr auto MY_VAL1
+  # readability-identifier-naming.LocalVariableIgnoredRegexp: '[A-Z_0-9]+'
+  # readability-identifier-naming.MacroDefinitionCase: 'UPPER_CASE'
+  # # We want to allow MY_MACRO_PRIVATE_1_
+  # readability-identifier-naming.MacroDefinitionIgnoredRegexp: '[A-Z_0-9]+'
+  # readability-identifier-naming.NamespaceCase: 'lower_case'
+  # readability-identifier-naming.PrivateMemberCase: 'lower_case'
+  # readability-identifier-naming.PrivateMemberSuffix: '_'
+  # readability-identifier-naming.PrivateMethodCase: 'lower_case'
+  # readability-identifier-naming.PrivateMethodSuffix: '_'
+  # readability-identifier-naming.ProtectedMemberCase: 'lower_case'
+  # readability-identifier-naming.ProtectedMemberSuffix: '_'
+  # readability-identifier-naming.ProtectedMethodCase: 'lower_case'
+  # readability-identifier-naming.ProtectedMethodSuffix: '_'
+  # readability-identifier-naming.PublicMethodCase: 'lower_case'
+  # readability-identifier-naming.ScopedEnumCase: 'lower_case'
+  # readability-identifier-naming.ScopedEnumConstantCase: 'UPPER_CASE'
+  readability-magic-numbers.IgnoredIntegerValues: '0;1;2;3;4;5;6;7;8;9'
+  # There are some single-argument functions that we would not like to ignore
+  # (`to_string(bool provenance = false)` comes to mind), but setting this to false makes
+  # clang-tidy warn on a bunch of "obvious" calls, like `set_dim(1)` or
+  # `with_concurrent(true)`.
+  bugprone-argument-comment.IgnoreSingleArgument: true
+  bugprone-argument-comment.CommentBoolLiterals: true
+  bugprone-argument-comment.CommentIntegerLiterals: true
+  bugprone-argument-comment.CommentFloatLiterals: true
+  bugprone-argument-comment.CommentStringLiterals: true
+  bugprone-argument-comment.CommentCharacterLiterals: true
+  bugprone-argument-comment.CommentUserDefinedLiterals: true
+  bugprone-argument-comment.CommentNullPtrs: true
+  cppcoreguidelines-macro-usage.AllowedRegexp: '^DEBUG_.*|THRUST_PP_.*|_CCCL.*_PP_.*'
+  bugprone-reserved-identifier.AllowedIdentifiers: '^_CCCL.*'
+  performance-enum-size.EnumIgnoreList: '::cuda::std::.*'
+  performance-unnecessary-value-param.AllowedTypes: '.*(_ref(erence)?|_ptr|pointer|(_)?[Ii]t(erator)?|IteratorT|__half(2)?|(__nv_bfloat.*)|_view)$'
+  bugprone-exception-escape.CheckMain: false
+...

cccl_upstream/.pre-commit-config.yaml

@@ -0,0 +1,143 @@
+# Copyright (c) 2024, NVIDIA CORPORATION.
+ci:
+    autofix_commit_msg: |
+      [pre-commit.ci] auto code formatting
+    autofix_prs: false
+    autoupdate_branch: ''
+    autoupdate_commit_msg: '[pre-commit.ci] pre-commit autoupdate'
+    autoupdate_schedule: quarterly
+    skip: [mypy, secret-scan-trufflehog] # mypy fails on pre-commit.ci due to numpy stub incompatibility with Python 3.14
+    submodules: false
+
+repos:
+  # Runs first so a leaked credential blocks the commit before any formatter runs.
+  # Self-installing: the hook downloads a pinned, checksum-verified trufflehog on
+  # first use (no manual install). Skipped on pre-commit.ci; Pulse CI enforces server-side.
+  - repo: https://github.com/NVIDIA/security-workflows
+    rev: v0.2.0
+    hooks:
+      - id: secret-scan-trufflehog
+
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v5.0.0
+    hooks:
+      - id: end-of-file-fixer
+      - id: mixed-line-ending
+      - id: trailing-whitespace
+        # handled by clang-format
+        exclude_types: [c, c++, cuda]
+      - id: check-json
+      - id: check-toml
+      - id: pretty-format-json
+        args: ['--autofix', '--indent=2', '--no-sort-keys']
+      - id: check-symlinks
+      - id: check-executables-have-shebangs
+      - id: check-merge-conflict
+      - id: check-shebang-scripts-are-executable
+        # ruff checks this already
+        exclude_types: [python]
+      - id: check-yaml
+
+  - repo: https://github.com/shellcheck-py/shellcheck-py
+    rev: v0.11.0.1
+    hooks:
+      - id: shellcheck
+        exclude: |
+          (?x)^(
+            ^.*libcudacxx/cmake/config\.guess$
+          )
+
+  - repo: https://github.com/pre-commit/mirrors-clang-format
+    rev: v20.1.7
+    hooks:
+      - id: clang-format
+        types_or: [file]
+        files: |
+          (?x)^(
+            ^.*\.c$|
+            ^.*\.cpp$|
+            ^.*\.cu$|
+            ^.*\.cuh$|
+            ^.*\.cxx$|
+            ^.*\.h$|
+            ^.*\.hpp$|
+            ^.*\.inl$|
+            ^.*\.mm$|
+            ^libcudacxx/include/.*/[^.]*$
+          )
+        args: ["-fallback-style=none", "-style=file", "-i"]
+
+  # TODO/REMINDER: add the Ruff vscode extension to the devcontainers
+  # Ruff, the Python auto-correcting linter/formatter written in Rust
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.12.2
+    hooks:
+    - id: ruff  # linter
+    - id: ruff-format  # formatter
+
+  # CMake formatting
+  - repo: https://github.com/BlankSpruce/gersemi
+    rev: 0.23.1
+    hooks:
+    - id: gersemi
+
+  - repo: https://github.com/codespell-project/codespell
+    rev: v2.4.1
+    hooks:
+      - id: codespell
+        additional_dependencies: [tomli]
+        args: ["--toml", "pyproject.toml", "-I", ".codespell-ignore.txt"]
+        exclude: |
+          (?x)^(
+            build|
+            CITATION.md
+          )
+
+  - repo: https://github.com/pre-commit/mirrors-mypy
+    rev: 'v1.16.1'
+    hooks:
+      - id: mypy
+        # numpy is intentionally not installed here: numpy>=2.3 ships PEP 695
+        # `type` statements in its stubs that mypy rejects under
+        # python_version=3.10, and numpy<2.3 has no wheels for the Python that
+        # pre-commit.ci runs. numpy is instead ignored in the mypy config.
+        additional_dependencies: [types-cachetools]
+        args: ["--config-file=python/cuda_cccl/pyproject.toml",
+               "python/cuda_cccl/cuda/compute/"]
+        pass_filenames: false
+
+  - repo: https://github.com/sirosen/texthooks
+    rev: 0.7.1
+    hooks:
+      - id: fix-smartquotes
+      - id: fix-spaces
+      - id: forbid-bidi-controls
+      - id: fix-ligatures
+
+  - repo: local
+    hooks:
+      - id: check-shebang
+        name: check-shebang
+        entry: ci/util/pre-commit/check_shebang.py
+        language: python
+        types: [shell]
+        exclude: |
+          (?x)^(
+            ^.*libcudacxx/cmake/config\.guess$
+          )
+
+      - id: check-cub-test-macros
+        name: require CUB test memory classification
+        entry: ci/util/pre-commit/check_cub_test_macros.py
+        language: python
+        files: ^(cub/test/.*\.(cu|cuh|h)|ci/util/pre-commit/check_cub_test_macros\.py)$
+        types: [file]
+
+      - id: unprintable-unicode
+        name: unprintable-unicode
+        entry: ci/util/pre-commit/strip_unprintable.py
+        language: python
+        types: [text]
+
+default_language_version:
+  python: python3

cccl_upstream/CITATION.md

@@ -0,0 +1,12 @@
+# Citation Guide
+
+## To Cite CCCL
+If you use CCCL in a publication, please use citations in the following format (BibTeX entry for LaTeX):
+```tex
+@Manual{,
+  title = {{CCCL}: {CUDA} {C++} {C}ore {L}ibraries},
+  author = {{CCCL Development Team}},
+  year = {2023},
+  url = {https://github.com/NVIDIA/cccl},
+}
+```

cccl_upstream/CLAUDE.md

@@ -0,0 +1 @@
+AGENTS.md

cccl_upstream/CODE_OF_CONDUCT.md

@@ -0,0 +1,84 @@
+# Contributor Covenant Code of Conduct
+
+## Overview
+
+Define the code of conduct followed and enforced for CCCL.
+
+### Intended audience
+
+Community | Developers | Project Leads
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+  advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting GitHub_Conduct@nvidia.com. All complaints will be reviewed and
+investigated and will result in a response that is deemed necessary and appropriate
+to the circumstances. The project team is obligated to maintain confidentiality with
+regard to the reporter of an incident. Further details of specific enforcement policies
+may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see
+https://www.contributor-covenant.org/faq

cccl_upstream/CONTRIBUTING.md

@@ -0,0 +1,281 @@
+
+# Contributing to CCCL
+
+Thank you for your interest in contributing to the CUDA Core Compute Libraries (CCCL)!
+
+Looking for ideas for your first contribution? Check out: ![GitHub Issues or Pull Requests by label](https://img.shields.io/github/issues/nvidia/cccl/good%20first%20issue)
+
+
+## Getting Started
+
+1. **Fork & Clone the Repository**:
+
+   Fork the [CCCL GitHub Repository](https://github.com/nvidia/cccl) and clone the fork. For more information, check [GitHub's documentation on forking](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) and [cloning a repository](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/cloning-a-repository).
+
+2. **Set up Development Environment**:
+
+   CCCL uses Development Containers to provide a consistent development environment for both local development and CI. Contributors are strongly encouraged to use these containers as they simplify environment setup. See the [Dev Containers guide](.devcontainer/README.md) for instructions on how to quickly get up and running using dev containers with or without VSCode.
+
+## Making Changes
+
+1. **Create a New Branch**:
+
+   ```bash
+   git checkout -b your-feature-branch
+   ```
+
+2. **Make Changes**.
+
+3. **Build and Test**:
+
+   Ensure changes don't break existing functionality by building and running tests.
+
+   ```bash
+   ./ci/build_[thrust|cub|libcudacxx].sh -cxx <HOST_COMPILER> -std <CXX_STANDARD> -arch <GPU_ARCHS>
+
+   # test implies build
+   ./ci/test_[thrust|cub|libcudacxx].sh  -cxx <HOST_COMPILER> -std <CXX_STANDARD> -arch <GPU_ARCHS>
+   ```
+
+   For more details on building and testing, refer to the [Building and Testing](#building-and-testing) section below.
+
+4. **Commit Changes**:
+
+   ```bash
+   git commit -m "Brief description of the change"
+   ```
+
+### Developer Guides
+
+For more information about design and development practices for each CCCL component, refer to the following developer guides:
+
+#### CUB
+
+- [CUB Developer Guide](docs/cub/developer_overview.rst) - General overview of the design of CUB internals
+- [CUB Tests](docs/cub/test_overview.rst) - Overview of how to write CUB unit tests
+- [CUB Benchmarks](docs/cub/benchmarking.rst) - Overview of CUB's performance benchmarks
+- [CUB Tunings](docs/cub/tuning.rst) - Overview of CUB's performance tuning infrastructure
+
+#### Thrust
+
+Coming soon!
+
+#### libcudacxx
+
+Coming soon!
+
+## Building and Testing
+
+CCCL components are header-only libraries. This means there isn't a traditional build process for the library itself. However, before submitting contributions, it's a good idea to [build and run tests](#developer-guides).
+
+There are multiple options for building and running our tests. Which option you choose depends on your preferences and whether you are using [CCCL's DevContainers](.devcontainer/README.md) (highly recommended!).
+
+### Using Manual Build Scripts
+
+#### Building
+
+Use the build scripts provided in the `ci/` directory to build tests for each component. Building tests does not require a GPU.
+
+```bash
+   ci/build_[thrust|cub|libcudacxx].sh -cxx <HOST_COMPILER> -std <CXX_STANDARD> -arch <GPU_ARCHS>
+```
+- **HOST_COMPILER**: The desired host compiler (e.g., `g++`, `clang++`).
+- **CXX_STANDARD**: The C++ standard version (e.g., `17`, `20`).
+- **GPU_ARCHS**: A semicolon-separated list of CUDA GPU architectures (e.g., `"70;85;90"`). This uses the same syntax as CMake's [CUDA_ARCHITECTURES](https://cmake.org/cmake/help/latest/prop_tgt/CUDA_ARCHITECTURES.html#prop_tgt:CUDA_ARCHITECTURES):
+   - `70` - both PTX and SASS
+   - `70-real` - SASS only
+   - `70-virtual` - PTX only
+
+**Example:**
+```bash
+./ci/build_cub.sh -cxx g++ -std 17 -arch "70;75;80-virtual"
+```
+
+#### Testing
+
+Use the test scripts provided in the `ci/` directory to run tests for each component. These take the same arguments as the build scripts and will automatically build the tests if they haven't already been built. Running tests requires a GPU.
+
+```bash
+   ci/test_[thrust|cub|libcudacxx].sh -cxx <HOST_COMPILER> -std <CXX_STANDARD> -arch <GPU_ARCHS>
+```
+
+**Example:**
+
+```bash
+./ci/test_cub.sh -cxx g++ -std 17 -arch "70;75;80-virtual"
+```
+
+### Using CMake Presets
+
+[CMake Presets](https://cmake.org/cmake/help/latest/manual/cmake-presets.7.html) are a set of configurations defined in a JSON file that specify project-wide build details for CMake. They provide a standardized and sharable way to configure, build, and test projects across different platforms and development environments. Presets are available from CMake versions 3.19 and later.
+
+There are three kinds of Presets
+
+- Configure Presets: specify options for the `cmake` command,
+
+- Build Presets: specify options for the `cmake --build` command,
+
+- Test Presets: specify options for the `ctest` command.
+
+In CCCL we provide many presets to be used out of the box. You can find the complete list in our corresponding [CMakePresets.json](./CMakePresets.json) file.
+
+These commands can be used to get lists of the configure, build, and test presets.
+
+```bash
+cmake --list-presets # Configure presets
+cmake --build --list-presets # Build presets
+ctest --list-presets # Test presets
+```
+
+While there is a lot of overlap, there may be differences between the configure, build, and test presets to support various testing workflows.
+
+The `dev` presets are intended as a base for general development while the others are useful for replicating CI failures.
+
+#### Using CMake Presets via Command Line
+
+CMake automatically generates the preset build directories. You can configure, build and test for a specific preset (e.g. `thrust-cpp17`) via cmake from the root directory by appending `--preset=thrust-cpp17` to the corresponding commands. For example:
+
+```bash
+cmake --preset=thrust-cpp17
+cmake --build --preset=thrust-cpp17
+ctest --preset=thrust-cpp17
+```
+
+That will create `build/<optional devcontainer name>/thrust-cpp17/` and build everything in there. The devcontainer name is inserted automatically on devcontainer builds to keep build artifacts separate for the different toolchains.
+
+It's also worth mentioning that additional cmake options can still be passed in and will override the preset settings.
+
+As a common example, the presets are currently always `60;70;80` for `CMAKE_CUDA_ARCHITECTURES`, but this can be overridden at configure time with something like:
+
+```bash
+cmake --preset=thrust-cpp20 "-DCMAKE_CUDA_ARCHITECTURES=89"
+```
+
+> **Note**: Either using the `cmake` command from within the root directory or from within the build directory works, but will behave in slightly different ways. Building and running tests from the build directory will compile every target and run all of the tests configured in the configure step. Doing so from the root directory using the `--preset=<test_preset>` option will build and run a subset of configured targets and tests.
+
+#### Using CMake Presets via VS Code GUI extension (Recommended when using DevContainers)
+
+The recommended way to use CMake Presets is via the VS Code extension [CMake Tools](https://marketplace.visualstudio.com/items?itemName=ms-vscode.cmake-tools), already included in [CCCL's DevContainers](.devcontainer/README.md). As soon as you install the extension you would be able to see the sidebar menu below.
+
+   ![cmaketools sidebar](/.devcontainer/img/cmaketools_sidebar.png)
+
+You can specify the desired CMake Preset by clicking the "Select Configure Preset" button under the "Configure" node (see image below).
+
+   ![cmaketools presets](.devcontainer/img/cmaketools_presets.png)
+
+After that you can select the default build target from the "Build" node. As soon as you expand it, a list will appear with all the available targets that are included within the preset you selected. For example if you had selected the `all-dev` preset VS Code will display all the available targets we have in cccl.
+
+   ![cmaketools presets](.devcontainer/img/cmaketools_targets.png)
+
+You can build the selected target by pressing the gear button ![gear](.devcontainer/img/build_button.png) at the bottom of the VS Code window.
+
+Alternatively you can select the desired target from either the "Debug" or "Launch" drop down menu (for debugging or running correspondingly). <b>In that case after you select the target and either press "Run" ![run](.devcontainer/img/run.png) or "Debug" ![debug](.devcontainer/img/debug.png) the target will build on its own before running without the user having to build it explicitly from the gear button.</b>
+
+---
+
+We encourage users who want to debug device code to install the [Nsight Visual Studio Code Edition extension](https://marketplace.visualstudio.com/items?itemName=NVIDIA.nsight-vscode-edition) that enables the VS Code frontend for `cuda-gdb`. <u>To use it you should launch from the sidebar menu instead of pressing the "Debug" button from the bottom menu</u>.
+
+![nsight](.devcontainer/img/nsight.png)
+
+## Creating a Pull Request
+
+1. Push changes to your fork
+2. Create a pull request targeting the `main` branch of the original CCCL repository. Refer to [GitHub's documentation](https://docs.github.com/en/github/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests) for more information on creating a pull request.
+3. Describe the purpose and context of the changes in the pull request description.
+
+### Documentation Preview
+
+Documentation previews allow reviewers to see how changes will appear on the live documentation site before merging. Previews are automatically generated for all pull requests and updated with every commit. To skip building documentation for a PR, include `[skip-docs]` in your commit message.
+
+The preview URL will be posted as a comment on your PR and automatically cleaned up when the PR is closed.
+
+### Checking for Performance Regressions
+
+Performance stability is a key goal for CCCL, especially for `Device*` algorithms in CUB. When modifying any functionality that could impact these algorithms, contributors are encouraged to verify that no performance regressions occur.
+
+This verification is a two-step process:
+
+1. Determine whether your changes affect the generated SASS code (details on how to do this are provided below).
+2. If the generated SASS code changes, run the benchmarks (see [CUB Benchmarks](docs/cub/benchmarking.rst)) to quantify potential performance implications.
+
+**Steps to check whether your changes generate different SASS code:**
+
+1. Identify the `Device*` algorithm(s) that may be affected by the change. This isn't always straightforward, and you will need to confirm whether any of the CUB algorithms depend on components modified by your changes. If your changes affect only certain GPU architectures, make sure those architectures are included in the list of architectures used during compilation (for example, by specifying them with the `-arch` flag when using the build scripts, or with `-DCMAKE_CUDA_ARCHITECTURES` when building with CMake).
+2. Navigate to the build directory, compile the benchmarks for the specific `Device*` algorithm(s) identified in step 1, and dump the SASS code. For example: `ninja cub.bench.radix_sort.keys.base && cuobjdump -sass ./bin/cub.bench.radix_sort.keys.base |c++filt > ./radix_sort.keys_after.sass`.
+3. Check out the `main` branch to compare against the baseline SASS code: `git checkout $(git merge-base HEAD upstream/main)`
+4. Dump the SASS code emitted on the `main` branch. For example: `ninja cub.bench.radix_sort.keys.base && cuobjdump -sass ./bin/cub.bench.radix_sort.keys.base |c++filt > ./radix_sort.keys_before.sass`.
+5. Check whether there are differences in the generated SASS output: `git diff --text --no-index --word-diff radix_sort.keys_before.sass radix_sort.keys_after.sass`
+
+## Code Formatting (pre-commit hooks)
+
+CCCL uses [pre-commit](https://pre-commit.com/) to execute all code linters and formatters. These
+tools ensure a consistent coding style throughout the project. Using pre-commit ensures that linter
+versions and options are aligned for all developers. Additionally, there is a CI check in place to
+enforce that committed code follows our standards.
+
+The linters used by CCCL are listed in `.pre-commit-config.yaml`.
+For example, C++ and CUDA code is formatted with [`clang-format`](https://clang.llvm.org/docs/ClangFormat.html).
+
+To use `pre-commit`, install via `conda` or `pip`:
+
+```bash
+conda config --add channels conda-forge
+conda install pre-commit
+```
+
+```bash
+pip install pre-commit
+```
+
+Then run pre-commit hooks before committing code:
+
+```bash
+pre-commit run
+```
+
+By default, pre-commit runs on staged files (only changes and additions that will be committed).
+To run pre-commit checks on all files, execute:
+
+```bash
+pre-commit run --all-files
+```
+
+Optionally, you may set up the pre-commit hooks to run automatically when you make a git commit. This can be done by running:
+
+```bash
+pre-commit install
+```
+
+Now code linters and formatters will be run each time you commit changes.
+
+You can skip these checks with `git commit --no-verify` or with the short version `git commit -n`.
+
+## Secret Scanning
+
+The `secret-scan-trufflehog` pre-commit hook scans staged files and installs TruffleHog on first run (use Git Bash on Windows). If it flags a secret, remove it before committing, or contact a maintainer if it's a false positive. Secrets are also scanned server-side in CI on `main`.
+
+## Continuous Integration (CI)
+
+CCCL's CI pipeline tests across various CUDA versions, compilers, and GPU architectures.
+For external contributors, the CI pipeline will not begin until a maintainer leaves an `/ok to test` comment. For members of the NVIDIA GitHub enterprise, the CI pipeline will begin immediately.
+For a detailed overview of CCCL's CI, see [CI overview](docs/infrastructure/ci/references/ci_overview.rst).
+
+There is a CI check for pre-commit, called [pre-commit.ci](pre-commit.ci).
+This enforces that all linters (such as `clang-format`) pass.
+If pre-commit.ci is failing, you can comment `pre-commit.ci autofix` on a pull request to trigger the auto-fixer.
+The auto-fixer will push a commit to your pull request that applies changes made by pre-commit hooks.
+
+## Review Process
+
+Once submitted, maintainers will be automatically assigned to review the pull request. They might suggest changes or improvements. Constructive feedback is a part of the collaborative process, aimed at ensuring the highest quality code.
+
+For constructive feedback and effective communication during reviews, we recommend following [Conventional Comments](https://conventionalcomments.org/).
+
+Further recommended reading for successful PR reviews:
+
+- [How to Do Code Reviews Like a Human (Part One)](https://mtlynch.io/human-code-reviews-1/)
+- [How to Do Code Reviews Like a Human (Part Two)](https://mtlynch.io/human-code-reviews-2/)
+
+## Thank You
+
+Your contributions enhance CCCL for the entire community. We appreciate your effort and collaboration!

cccl_upstream/SECURITY.md

@@ -0,0 +1,14 @@
+## Security
+
+NVIDIA is dedicated to the security and trust of our software products and services, including all source code repositories managed through our organization.
+
+If you need to report a security issue, please use the appropriate contact points outlined below. **Please do not report security vulnerabilities through GitHub.**
+
+## Reporting Potential Security Vulnerability in an NVIDIA Product
+
+To report a potential security vulnerability in any NVIDIA product:
+- Web: [Security Vulnerability Submission Form](https://www.nvidia.com/object/submit-security-vulnerability.html)
+- E-Mail: psirt@nvidia.com
+    - We encourage you to use the following PGP key for secure email communication: [NVIDIA public PGP Key for communication](https://www.nvidia.com/en-us/security/pgp-key)
+    - Please include the following information:
+   	 - Product/Driver name and version/branch that contains the vulnerability

cccl_upstream/ci/CMakeLists.txt

@@ -0,0 +1 @@
+add_subdirectory("test")

cccl_upstream/ci/bench.template.yaml

@@ -0,0 +1,70 @@
+# # CCCL PR benchmark request config.
+#
+# ## Overview:
+#
+# This file is used to request benchmark comparisons in PR CI.
+#
+# This file must match ci/bench.template.yaml to merge.
+# CI branch protections will fail if they differ. Reset before merging.
+#
+# To update the defaults (e.g. new GPU pools), modify both this file and
+# ci/bench.template.yaml together in the same PR.
+#
+# !! Strongly consider appending the following to your **commit messages** while benchmarking. !!
+# This prevents wasteful non-benchmark CI jobs if they are not needed.
+#
+#     [bench-only]
+#
+# To skip compile-time benchmark telemetry on unrelated changes, use:
+#
+#     [skip-compile-time-bench]
+#
+# ## Quick start:
+#
+# 1. Add one or more benchmark regexes under benchmarks.filters.cub and/or
+#    benchmarks.filters.python.
+# 2. Enable at least one GPU by uncommenting or adding entries in benchmarks.gpus.
+# 3. Push and inspect the dispatched benchmark jobs/artifacts.
+# 4. Remove/reset benchmark-request edits before final merge.
+
+benchmarks:
+
+  # Benchmark filters grouped by project.
+  filters:
+    # CUB C++ benchmark filters (regex matched against ninja target names).
+    cub:
+      # Examples:
+      # - '^cub\.bench\.for_each\.base'
+      # - '^cub\.bench\.reduce\.(sum|min)\.'
+
+    # Python benchmark filters (regex matched against paths under benchmarks/).
+    python:
+      # Examples:
+      # - 'compute/reduce/sum\.py'
+      # - 'compute/transform/.*\.py'
+
+  # Select GPUs. These are limited and shared, be intentional and conservative.
+  gpus:
+    # - "t4"         # sm_75, 16 GB
+    # - "rtx2080"    # sm_75,  8 GB
+    # - "rtxa6000"   # sm_86, 48 GB
+    # - "l4"         # sm_89, 24 GB
+    # - "rtx4090"    # sm_89, 24 GB
+    # - "h100"       # sm_90, 80 GB
+    # - "rtxpro6000" # sm_120
+
+  # Extra .devcontainer/launch.sh -d args
+  # launch_args: "--cuda 13.3 --host gcc14"
+  launch_args: "" # Latest nvcc + gcc
+
+  # Advanced:
+  base_ref: "origin/main"
+  test_ref: "HEAD"
+  arch: "native"
+  nvbench_args: >-
+    --timeout 30
+    --skip-time 15e-6
+    --stopping-criterion entropy
+    --throttle-threshold 90
+    --throttle-recovery-delay 0.15
+  nvbench_compare_args: ""

cccl_upstream/ci/bench.yaml

@@ -0,0 +1,70 @@
+# # CCCL PR benchmark request config.
+#
+# ## Overview:
+#
+# This file is used to request benchmark comparisons in PR CI.
+#
+# This file must match ci/bench.template.yaml to merge.
+# CI branch protections will fail if they differ. Reset before merging.
+#
+# To update the defaults (e.g. new GPU pools), modify both this file and
+# ci/bench.template.yaml together in the same PR.
+#
+# !! Strongly consider appending the following to your **commit messages** while benchmarking. !!
+# This prevents wasteful non-benchmark CI jobs if they are not needed.
+#
+#     [bench-only]
+#
+# To skip compile-time benchmark telemetry on unrelated changes, use:
+#
+#     [skip-compile-time-bench]
+#
+# ## Quick start:
+#
+# 1. Add one or more benchmark regexes under benchmarks.filters.cub and/or
+#    benchmarks.filters.python.
+# 2. Enable at least one GPU by uncommenting or adding entries in benchmarks.gpus.
+# 3. Push and inspect the dispatched benchmark jobs/artifacts.
+# 4. Remove/reset benchmark-request edits before final merge.
+
+benchmarks:
+
+  # Benchmark filters grouped by project.
+  filters:
+    # CUB C++ benchmark filters (regex matched against ninja target names).
+    cub:
+      # Examples:
+      # - '^cub\.bench\.for_each\.base'
+      # - '^cub\.bench\.reduce\.(sum|min)\.'
+
+    # Python benchmark filters (regex matched against paths under benchmarks/).
+    python:
+      # Examples:
+      # - 'compute/reduce/sum\.py'
+      # - 'compute/transform/.*\.py'
+
+  # Select GPUs. These are limited and shared, be intentional and conservative.
+  gpus:
+    # - "t4"         # sm_75, 16 GB
+    # - "rtx2080"    # sm_75,  8 GB
+    # - "rtxa6000"   # sm_86, 48 GB
+    # - "l4"         # sm_89, 24 GB
+    # - "rtx4090"    # sm_89, 24 GB
+    # - "h100"       # sm_90, 80 GB
+    # - "rtxpro6000" # sm_120
+
+  # Extra .devcontainer/launch.sh -d args
+  # launch_args: "--cuda 13.3 --host gcc14"
+  launch_args: "" # Latest nvcc + gcc
+
+  # Advanced:
+  base_ref: "origin/main"
+  test_ref: "HEAD"
+  arch: "native"
+  nvbench_args: >-
+    --timeout 30
+    --skip-time 15e-6
+    --stopping-criterion entropy
+    --throttle-threshold 90
+    --throttle-recovery-delay 0.15
+  nvbench_compare_args: ""

cccl_upstream/ci/bench/README.md

@@ -0,0 +1,82 @@
+# Benchmark Compare Scripts
+
+This directory contains the scripts used by `.github/workflows/bench.yml` to compare benchmark results between two code states.
+
+## Scripts
+
+- `ci/bench/bench.sh`: CI-oriented wrapper that calls `ci/bench/compare_git_refs.sh`.
+- `ci/bench/compare_git_refs.sh`: checks out `<base-ref>` and `<test-ref>` in temporary worktrees, then forwards all remaining args to `ci/bench/compare_paths.sh`.
+- `ci/bench/compare_paths.sh`: configures/builds/runs CUB benchmarks and/or Python benchmarks in two source trees and runs comparison tools on produced JSON outputs.
+- `ci/bench/parse_bench_matrix.sh`: parses `ci/bench.yaml` and emits a dispatch matrix JSON object for `.github/workflows/bench.yml`.
+
+## Usage
+
+Compare CUB benchmarks between two refs:
+
+```bash
+"./ci/bench/bench.sh" "origin/main" "HEAD" \
+  --cub-filter "^cub\\.bench\\.copy\\.memcpy\\.base$"
+```
+
+Compare Python benchmarks between two refs:
+
+```bash
+"./ci/bench/bench.sh" "origin/main" "HEAD" \
+  --python-filter "compute/reduce/sum\\.py"
+```
+
+Run both CUB and Python benchmarks:
+
+```bash
+"./ci/bench/bench.sh" \
+  "origin/main" \
+  "HEAD" \
+  --arch "native" \
+  --nvbench-args "..." \
+  --cub-filter "^cub\\.bench\\.reduce\\..*$" \
+  --python-filter "compute/reduce/sum\\.py"
+```
+
+Compare already checked-out trees:
+
+```bash
+"./ci/bench/compare_paths.sh" \
+  "/path/to/base/cccl" \
+  "/path/to/test/cccl" \
+  --arch "native" \
+  --cub-filter "^cub\\.bench\\.copy\\.memcpy\\.base$" \
+  --python-filter "compute/transform/.*\\.py"
+```
+
+## Workflow Inputs
+
+In `.github/workflows/bench.yml`:
+
+- If `raw_args` is non-empty, it is parsed and passed directly to `ci/bench/bench.sh`.
+- Otherwise, args are assembled from `base_ref`, `test_ref`, `arch`, `cub_filters`, `python_filters`, `nvbench_args`, and `nvbench_compare_args`.
+- CUB filters are passed as `--cub-filter` flags. Python filters are passed as `--python-filter` flags.
+- Malformed quoted input (for example unmatched quotes) fails the workflow step.
+
+## Python Benchmarks
+
+Python benchmarks live under `python/cuda_cccl/benchmarks/` and use `cuda.bench` (the Python nvbench bindings). Each benchmark script outputs nvbench-compatible JSON.
+
+For Python benchmarks, `compare_paths.sh`:
+
+1. Creates isolated virtual environments for base and test trees.
+2. Installs `cuda-cccl[bench-cuXX]` (editable, from each worktree), which pulls in `cuda-bench`, `cupy`, and all other benchmark dependencies.
+3. Runs matching benchmark scripts in each venv.
+4. Compares results using `nvbench-compare` (installed with `cuda-bench`).
+
+Python filters are regex patterns matched against relative paths under `python/cuda_cccl/benchmarks/`, for example:
+- `compute/reduce/sum\.py` — single benchmark
+- `compute/transform/.*\.py` — all transform benchmarks
+
+## Artifacts
+
+`compare_paths.sh` writes a run directory under `${CCCL_BENCH_ARTIFACT_ROOT:-$(pwd)/bench-artifacts}` containing:
+
+- per-target JSON and markdown outputs for base/test runs,
+- grouped build logs (`build.base.log`, `build.test.log`), per-target run logs, and per-target compare logs (`compare.<target>.log`),
+- Python venv setup logs (`py.venv.base.log`, `py.venv.test.log`),
+- `summary.md` with run metadata and per-target collapsible full compare reports.

cccl_upstream/ci/bench/bench.sh

@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+die() {
+  local message="$1"
+  local code="${2:-2}"
+  echo "${message}" >&2
+  exit "${code}"
+}
+
+usage() {
+  cat <<EOF
+Usage: $0 <base-ref> <test-ref> [compare_paths args...]
+
+Wrapper for ci/bench/compare_git_refs.sh.
+EOF
+}
+
+if [[ "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then
+  usage
+  exit 0
+fi
+
+bench_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+"${bench_dir}/compare_git_refs.sh" "$@"

cccl_upstream/ci/bench/compare_git_refs.sh

@@ -0,0 +1,137 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+die() {
+  local message="$1"
+  local code="${2:-2}"
+  echo "${message}" >&2
+  exit "${code}"
+}
+
+usage() {
+  cat <<EOF
+Usage: $0 <base-ref> <test-ref> [compare_paths args...]
+
+Compare benchmark performance between two git refs from the current CCCL repo.
+Each ref is checked out in an isolated worktree and compared via compare_paths.sh.
+EOF
+}
+
+display_label_for_ref() {
+  local ref="$1"
+  local short_sha="$2"
+  if [[ "${ref}" =~ ^[0-9a-fA-F]{7,40}$ ]]; then
+    printf "%s" "${short_sha}"
+    return 0
+  fi
+  if [[ "${ref}" == "HEAD" ]]; then
+    printf "%s" "${short_sha}"
+    return 0
+  fi
+  printf "%s" "${ref}"
+}
+
+resolve_ref_to_commit() {
+  local repo_root="$1"
+  local ref="$2"
+  local remote=""
+  local branch=""
+  local alternate_ref=""
+
+  if git -C "${repo_root}" rev-parse --verify "${ref}^{commit}" >/dev/null 2>&1; then
+    git -C "${repo_root}" rev-parse --verify "${ref}^{commit}"
+    return 0
+  fi
+
+  if [[ "${ref}" =~ ^([^/]+)/(.+)$ ]] && git -C "${repo_root}" remote get-url "${BASH_REMATCH[1]}" >/dev/null 2>&1; then
+    remote="${BASH_REMATCH[1]}"
+    branch="${BASH_REMATCH[2]}"
+    git -C "${repo_root}" fetch --no-tags "${remote}" \
+      "+refs/heads/${branch}:refs/remotes/${remote}/${branch}" >/dev/null 2>&1 || true
+  elif [[ "${ref}" != refs/* ]]; then
+    # Try unqualified refs as origin branches/tags.
+    git -C "${repo_root}" fetch --no-tags origin \
+      "+refs/heads/${ref}:refs/remotes/origin/${ref}" >/dev/null 2>&1 || true
+    git -C "${repo_root}" fetch --no-tags origin \
+      "refs/tags/${ref}:refs/tags/${ref}" >/dev/null 2>&1 || true
+    alternate_ref="origin/${ref}"
+  fi
+
+  # Final best-effort fetch for raw refs (e.g. refs/pull/* or specific SHAs).
+  git -C "${repo_root}" fetch --no-tags origin "${ref}" >/dev/null 2>&1 || true
+
+  if git -C "${repo_root}" rev-parse --verify "${ref}^{commit}" >/dev/null 2>&1; then
+    git -C "${repo_root}" rev-parse --verify "${ref}^{commit}"
+    return 0
+  fi
+
+  if [[ -n "${alternate_ref}" ]] && git -C "${repo_root}" rev-parse --verify "${alternate_ref}^{commit}" >/dev/null 2>&1; then
+    git -C "${repo_root}" rev-parse --verify "${alternate_ref}^{commit}"
+    return 0
+  fi
+
+  return 1
+}
+
+if [[ "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then
+  usage
+  exit 0
+fi
+
+if [[ "$#" -lt 2 ]]; then
+  usage
+  exit 2
+fi
+
+base_ref="$1"
+test_ref="$2"
+shift 2
+compare_paths_args=("$@")
+
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+repo_root="$(cd "${ci_dir}/../.." && pwd)"
+
+if [[ ! -f "${repo_root}/cccl-version.json" ]]; then
+  die "This script must run from a CCCL checkout."
+fi
+
+if ! base_commit="$(resolve_ref_to_commit "${repo_root}" "${base_ref}")"; then
+  die "Unable to resolve base ref: ${base_ref}"
+fi
+if ! test_commit="$(resolve_ref_to_commit "${repo_root}" "${test_ref}")"; then
+  die "Unable to resolve test ref: ${test_ref}"
+fi
+
+base_short_sha="$(git -C "${repo_root}" rev-parse --short=12 "${base_commit}")"
+test_short_sha="$(git -C "${repo_root}" rev-parse --short=12 "${test_commit}")"
+base_label="$(display_label_for_ref "${base_ref}" "${base_short_sha}")"
+test_label="$(display_label_for_ref "${test_ref}" "${test_short_sha}")"
+
+worktree_root="$(mktemp -d "/tmp/cccl-bench-worktrees-XXXXXX")"
+base_path="${worktree_root}/base"
+test_path="${worktree_root}/test"
+
+cleanup() {
+  git -C "${repo_root}" worktree remove --force "${base_path}" >/dev/null 2>&1 || true
+  git -C "${repo_root}" worktree remove --force "${test_path}" >/dev/null 2>&1 || true
+  rm -rf "${worktree_root}"
+}
+trap cleanup EXIT
+
+echo "Creating worktree for base ref: ${base_ref}"
+git -C "${repo_root}" worktree add --detach "${base_path}" "${base_commit}" >/dev/null
+
+echo "Creating worktree for test ref: ${test_ref}"
+git -C "${repo_root}" worktree add --detach "${test_path}" "${test_commit}" >/dev/null
+
+compare_cmd=(
+  "${ci_dir}/compare_paths.sh"
+  "${base_path}"
+  "${test_path}"
+  "${compare_paths_args[@]}"
+)
+
+CCCL_BENCH_BASE_LABEL="${base_label}" \
+CCCL_BENCH_TEST_LABEL="${test_label}" \
+  "${compare_cmd[@]}"

cccl_upstream/ci/bench/parse_bench_matrix.sh

@@ -0,0 +1,80 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+die() {
+  local message="$1"
+  local code="${2:-2}"
+  echo "${message}" >&2
+  exit "${code}"
+}
+
+usage() {
+  cat <<EOF
+Usage: $0 [bench-yaml-path]
+
+Parse ci/bench.yaml and emit a GitHub Actions strategy matrix JSON object:
+  {"include":[...]}
+
+Each include entry maps one enabled GPU to a benchmark workflow invocation.
+EOF
+}
+
+if [[ "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then
+  usage
+  exit 0
+fi
+
+bench_yaml_path="${1:-ci/bench.yaml}"
+if [[ ! -f "${bench_yaml_path}" ]]; then
+  die "Benchmark config file not found: ${bench_yaml_path}"
+fi
+
+command -v "yq" >/dev/null 2>&1 || die "'yq' is required to parse ${bench_yaml_path}."
+command -v "jq" >/dev/null 2>&1 || die "'jq' is required to build the dispatch matrix."
+
+if ! bench_cfg_json="$(yq -o=json '.benchmarks // {}' "${bench_yaml_path}" 2>&1)"; then
+  die "Failed to parse ${bench_yaml_path} as YAML: ${bench_cfg_json}"
+fi
+
+# Extract CUB and Python filter arrays (default to empty arrays).
+cub_filters_json="$(jq -c '.filters.cub // []' <<<"${bench_cfg_json}")"
+python_filters_json="$(jq -c '.filters.python // []' <<<"${bench_cfg_json}")"
+
+has_cub_filters="$(jq -e 'type == "array" and length > 0 and all(.[]; type == "string")' <<<"${cub_filters_json}" >/dev/null 2>&1 && echo true || echo false)"
+has_python_filters="$(jq -e 'type == "array" and length > 0 and all(.[]; type == "string")' <<<"${python_filters_json}" >/dev/null 2>&1 && echo true || echo false)"
+
+if [[ "${has_cub_filters}" != "true" && "${has_python_filters}" != "true" ]]; then
+  die "${bench_yaml_path} must define at least one string entry in benchmarks.filters.cub or benchmarks.filters.python."
+fi
+
+cub_filters_arg=""
+if [[ "${has_cub_filters}" == "true" ]]; then
+  cub_filters_arg="$(jq -r '.filters.cub | map(@sh) | join(" ")' <<<"${bench_cfg_json}")"
+fi
+
+python_filters_arg=""
+if [[ "${has_python_filters}" == "true" ]]; then
+  python_filters_arg="$(jq -r '.filters.python | map(@sh) | join(" ")' <<<"${bench_cfg_json}")"
+fi
+
+jq -cn \
+  --argjson cfg "${bench_cfg_json}" \
+  --arg cub_filters "${cub_filters_arg}" \
+  --arg python_filters "${python_filters_arg}" \
+  '{
+    "include": [
+      ($cfg.gpus // [])[] as $gpu
+      | {
+          "gpu": $gpu,
+          "launch_args": ($cfg.launch_args // ""),
+          "arch": ($cfg.arch // "native"),
+          "base_ref": ($cfg.base_ref // "origin/main"),
+          "test_ref": ($cfg.test_ref // "HEAD"),
+          "cub_filters": $cub_filters,
+          "python_filters": $python_filters,
+          "nvbench_args": ($cfg.nvbench_args // ""),
+          "nvbench_compare_args": ($cfg.nvbench_compare_args // "")
+        }
+    ]
+  }'

cccl_upstream/ci/build_cccl_c_parallel.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+# shellcheck source=ci/build_common.sh
+source "$(dirname "${BASH_SOURCE[0]}")/build_common.sh"
+
+print_environment_details
+
+PRESET="cccl-c-parallel"
+
+CMAKE_OPTIONS=("-DCMAKE_CXX_STANDARD=${CXX_STANDARD}" "-DCMAKE_CUDA_STANDARD=${CXX_STANDARD}")
+
+configure_and_build_preset "CCCL C Parallel Library" "$PRESET" "${CMAKE_OPTIONS[@]}"
+
+print_time_summary

cccl_upstream/ci/build_cccl_c_stf.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+# shellcheck source=ci/build_common.sh
+source "$(dirname "${BASH_SOURCE[0]}")/build_common.sh"
+
+print_environment_details
+
+PRESET="cccl-c-stf"
+
+CMAKE_OPTIONS=("-DCMAKE_CXX_STANDARD=${CXX_STANDARD}" "-DCMAKE_CUDA_STANDARD=${CXX_STANDARD}")
+
+configure_and_build_preset "CCCL C CUDASTF Library" "$PRESET" "${CMAKE_OPTIONS[@]}"
+
+print_time_summary

cccl_upstream/ci/build_common.sh

@@ -0,0 +1,501 @@
+#!/usr/bin/env bash
+
+set -eo pipefail
+
+if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
+  echo "This script must be sourced, not executed directly." >&2
+  exit 1
+fi
+
+# Ensure the script is being executed in its containing directory
+cd "$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )";
+
+# Script defaults
+VERBOSE=${VERBOSE:-}
+HOST_COMPILER=${CXX:-g++} # $CXX if set, otherwise `g++`
+CXX_STANDARD=17
+CUDA_COMPILER=${CUDACXX:-nvcc} # $CUDACXX if set, otherwise `nvcc`
+CUDA_ARCHS= # Empty, use presets by default.
+GLOBAL_CMAKE_OPTIONS=()
+DISABLE_CUB_BENCHMARKS= # Enable to force-disable building CUB benchmarks.
+PEDANTIC=${PEDANTIC:-} # Enable strict warnings. Default: on in CI, off locally.
+CONFIGURE_ONLY=false
+CTEST_PARALLEL_LEVEL=1
+
+# Check if the correct number of arguments has been provided
+function usage {
+    echo "Usage: $0 [OPTIONS]"
+    echo
+    echo "The PARALLEL_LEVEL environment variable controls the amount of build parallelism. Default is the number of cores minus one."
+    echo
+    echo "Options:"
+    echo "  -v/-verbose: enable shell echo for debugging"
+    echo "  -configure: Only run cmake to configure, do not build or test."
+    echo "  -cuda: CUDA compiler (Defaults to \$CUDACXX if set, otherwise nvcc)"
+    echo "  -cxx: Host compiler (Defaults to \$CXX if set, otherwise g++)"
+    echo "  -std: CUDA/C++ standard (Defaults to 17)"
+    echo "  -arch: Target CUDA arches, e.g. \"60-real;70;80-virtual\" (Defaults to value in presets file)"
+    echo "  --test-par: CTest parallel level (Defaults to 1)"
+    echo "  -pedantic/--pedantic: Enable strict warnings-as-errors and expose CCCL header warnings (default in CI)"
+    echo "  -cmake-options: Additional options to pass to CMake"
+    echo
+    echo "Examples:"
+    echo "  $ PARALLEL_LEVEL=8 $0"
+    echo "  $ PARALLEL_LEVEL=8 $0 -cxx g++-9"
+    echo "  $ $0 -cxx clang++-8"
+    echo "  $ $0 -configure -arch 80"
+    echo "  $ $0 -cxx g++-8 -std 14 -arch 80-real -v -cuda /usr/local/bin/nvcc"
+    echo "  $ $0 -cmake-options \"-DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_FLAGS=-Wfatal-errors\""
+    exit 1
+}
+
+# Check for required dependencies
+function check_required_dependencies() {
+    local missing_deps=()
+
+    # Check for essential tools
+    local required_tools=("cmake" "git" "jq" "ninja" "nproc")
+    for tool in "${required_tools[@]}"; do
+        command -v "$tool" &>/dev/null || missing_deps+=("$tool")
+    done
+
+    if [[ ${#missing_deps[@]} -ne 0 ]]; then
+        echo "❌ Error: Missing required dependencies:" >&2
+        printf "   • %s\n" "${missing_deps[@]}" >&2
+        echo >&2
+        exit 1
+    fi
+}
+
+# Parse options
+
+# Copy the args into a temporary array, since we will modify them and
+# the parent script may still need them.
+args=("$@")
+while [[ "${#args[@]}" -ne 0 ]]; do
+    case "${args[0]}" in
+    -v | --verbose | -verbose) VERBOSE=1; args=("${args[@]:1}");;
+    -configure) CONFIGURE_ONLY=true;   args=("${args[@]:1}");;
+    -cxx)  HOST_COMPILER="${args[1]}"; args=("${args[@]:2}");;
+    -std)  CXX_STANDARD="${args[1]}";  args=("${args[@]:2}");;
+    -cuda) CUDA_COMPILER="${args[1]}"; args=("${args[@]:2}");;
+    -arch) CUDA_ARCHS="${args[1]}";    args=("${args[@]:2}");;
+    --test-par) CTEST_PARALLEL_LEVEL="${args[1]}"; args=("${args[@]:2}");;
+    -pedantic | --pedantic) PEDANTIC=1; args=("${args[@]:1}");;
+    -disable-benchmarks) export DISABLE_CUB_BENCHMARKS=1; args=("${args[@]:1}");;
+    -cmake-options)
+        if [[ -n "${args[1]}" ]]; then
+            IFS=' ' read -ra split_args <<< "${args[1]}"
+            GLOBAL_CMAKE_OPTIONS+=("${split_args[@]}")
+            args=("${args[@]:2}")
+        else
+            echo "Error: No arguments provided for -cmake-options"
+            usage
+            # usage will exit 1 for us, so below exit 1 is unreachable, but it does not
+            # hurt and guards against changes in usage.
+            # shellcheck disable=SC2317
+            exit 1
+        fi
+        ;;
+    -h | -help | --help) usage ;;
+    *) echo "Unrecognized option: ${args[0]}"; usage ;;
+    esac
+done
+
+# Convert to full paths and validate compilers exist:
+function validate_and_resolve_compiler() {
+    local compiler_name="$1"
+    local compiler_var="$2"
+    local compiler_path
+
+    compiler_path=$(command -v "${compiler_var}" 2>/dev/null)
+    if [[ -z "$compiler_path" ]]; then
+        echo "❌ Error: ${compiler_name} '${compiler_var}' not found in PATH" >&2
+        exit 1
+    fi
+
+    echo "$compiler_path"
+}
+
+HOST_COMPILER=$(validate_and_resolve_compiler "Host compiler" "${HOST_COMPILER}")
+CUDA_COMPILER=$(validate_and_resolve_compiler "CUDA compiler" "${CUDA_COMPILER}")
+
+if [[ "$(basename "$CUDA_COMPILER")" == nvcc* ]]; then
+    NVCC_VERSION=$("$CUDA_COMPILER" --version | grep "release" | sed 's/.*, V//')
+    # Verify that we have an X.Y.Z version in case the output format changes:
+    if ! [[ "$NVCC_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
+        echo "❌ Error: Detected nvcc version is not a valid X.Y.Z triple: '$NVCC_VERSION'" >&2
+        echo "$CUDA_COMPILER --version" >&2 || :
+        $CUDA_COMPILER --version >&2 || :
+        exit 1
+    fi
+fi
+
+
+if [[ -n "${CUDA_ARCHS}" ]]; then
+    GLOBAL_CMAKE_OPTIONS+=("-DCMAKE_CUDA_ARCHITECTURES=${CUDA_ARCHS}")
+fi
+
+# Default to pedantic mode in CI
+if [[ -z "${PEDANTIC}" && -n "${GITHUB_ACTIONS:-}" ]]; then
+    PEDANTIC=1
+fi
+
+if [[ -n "${PEDANTIC}" ]]; then
+    GLOBAL_CMAKE_OPTIONS+=("-DCCCL_ENABLE_WERROR=ON" "-DCCCL_ENABLE_PRAGMA_SYSTEM_HEADER=OFF")
+else
+    GLOBAL_CMAKE_OPTIONS+=("-DCCCL_ENABLE_WERROR=OFF" "-DCCCL_ENABLE_PRAGMA_SYSTEM_HEADER=ON")
+fi
+
+if [[ -n "$VERBOSE" ]]; then
+    set -x
+fi
+
+# Check for required dependencies
+check_required_dependencies
+
+# Begin processing unsets after option parsing
+set -u
+
+N_CPUS="$(nproc --all --ignore=1)"
+readonly N_CPUS
+readonly PARALLEL_LEVEL="${PARALLEL_LEVEL:=${N_CPUS}}"
+
+if [[ -z ${CCCL_BUILD_INFIX+x} ]]; then
+    CCCL_BUILD_INFIX=""
+fi
+
+mkdir -p ../build
+# Absolute path to cccl/build
+BUILD_ROOT=$(cd "../build" && pwd)
+
+# Absolute path to per-devcontainer build directory
+BUILD_DIR="$BUILD_ROOT/$CCCL_BUILD_INFIX"
+
+# The most recent devcontainer build dir will always be symlinked to cccl/build/latest
+mkdir -p "$BUILD_DIR"
+rm -f "$BUILD_ROOT"/latest
+ln -sf "$BUILD_DIR" "$BUILD_ROOT"/latest
+
+# The more recent preset build dir will always be symlinked to:
+# cccl/preset-latest
+function symlink_latest_preset {
+    local PRESET=$1
+    mkdir -p "$BUILD_DIR/$PRESET"
+    rm -f "$BUILD_ROOT/preset-latest"
+    ln -sf "$BUILD_DIR/$PRESET" "$BUILD_ROOT/preset-latest"
+}
+
+# Now that BUILD_DIR exists, use readlink to canonicalize the path:
+BUILD_DIR=$(readlink -f "${BUILD_DIR}")
+
+# Prepare environment for CMake:
+export CMAKE_BUILD_PARALLEL_LEVEL="$((PARALLEL_LEVEL > N_CPUS ? N_CPUS : PARALLEL_LEVEL))"
+
+export CTEST_PARALLEL_LEVEL
+export CXX="${HOST_COMPILER}"
+export CUDACXX="${CUDA_COMPILER}"
+export CUDAHOSTCXX="${HOST_COMPILER}"
+export CXX_STANDARD
+
+# shellcheck source=ci/pretty_printing.sh
+source ./pretty_printing.sh
+
+# Kill any build / test steps that exceed this time, otherwise CI jobs may be
+# killed by GHA before they can upload logs / artifacts needed to reproduce the timeout.
+# Only applies when running inside GitHub Actions.
+# Note that this is per-build/test limit, not a total timeout for the entire job.
+: "${CCCL_CI_COMMAND_TIMEOUT:=5.5h}"
+
+print_environment_details() {
+  begin_group "⚙️ Environment Details"
+
+  echo "free -h:"
+  free -h || :
+
+  echo "nproc=$(nproc || :)"
+
+  echo "pwd=$(pwd)"
+
+  print_var_values \
+      BUILD_DIR \
+      CXX_STANDARD \
+      CXX \
+      CUDACXX \
+      CUDAHOSTCXX \
+      NVCC_VERSION \
+      CMAKE_BUILD_PARALLEL_LEVEL \
+      CTEST_PARALLEL_LEVEL \
+      CCCL_CI_COMMAND_TIMEOUT \
+      CCCL_CUDA_EXTENDED \
+      CCCL_BUILD_INFIX \
+      PEDANTIC \
+      GLOBAL_CMAKE_OPTIONS \
+      TBB_ROOT
+
+  echo "Current commit is:"
+  git log -1 --format=short || echo "Not a repository"
+
+  if command -v nvidia-smi &> /dev/null; then
+    nvidia-smi
+  else
+    echo "nvidia-smi not found"
+  fi
+
+  if command -v sccache &> /dev/null; then
+    sccache --version
+  else
+    echo "sccache not found"
+  fi
+
+  if command -v cmake &> /dev/null; then
+    cmake --version
+  else
+    echo "cmake not found"
+  fi
+
+  if command -v ctest &> /dev/null; then
+    ctest --version
+  else
+    echo "ctest not found"
+  fi
+
+  end_group "⚙️ Environment Details"
+}
+
+run_ci_timed_command() {
+    local group_name="${1:-}"
+    shift
+    local -a command=("$@")
+
+    if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+        if [[ -n "${CCCL_CI_COMMAND_TIMEOUT}" && "${CCCL_CI_COMMAND_TIMEOUT}" != "0" ]]; then
+            if command -v timeout &> /dev/null; then
+                run_command "${group_name}" timeout "${CCCL_CI_COMMAND_TIMEOUT}" "${command[@]}"
+                return $?
+            fi
+            echo "Warning: timeout not found; running without CI timeout." >&2
+        fi
+    fi
+
+    run_command "${group_name}" "${command[@]}"
+}
+
+fail_if_no_gpu() {
+    if ! nvidia-smi &> /dev/null; then
+        echo "Error: No NVIDIA GPU detected. Please ensure you have an NVIDIA GPU installed and the drivers are properly configured." >&2
+        exit 1
+    fi
+}
+
+function cccl_configure_preset_for_test() {
+    local test_preset=$1
+    local presets_file="${BUILD_ROOT}/../CMakePresets.json"
+    local configure_preset
+
+    if [[ ! -f "${presets_file}" ]]; then
+        echo "Error: CMakePresets.json not found: ${presets_file}" >&2
+        return 1
+    fi
+
+    configure_preset=$(
+        jq -r --arg t "${test_preset}" '
+            .testPresets[] | select(.name == $t) | .configurePreset // empty
+        ' "${presets_file}"
+    )
+
+    if [[ -z "${configure_preset}" ]]; then
+        configure_preset="${test_preset}"
+    fi
+
+    echo "${configure_preset}"
+}
+
+function cccl_smoke_tests_enabled() {
+    local configure_preset=$1
+    local cache_file="${BUILD_DIR}/${configure_preset}/CMakeCache.txt"
+
+    [[ -f "${cache_file}" ]] \
+        && grep -q '^CCCL_ENABLE_CUDA_SMOKE_TESTS:BOOL=ON' "${cache_file}"
+}
+
+function run_cuda_smoke_test() {
+    local BUILD_NAME=$1
+    local test_preset=$2
+
+    local configure_preset
+    configure_preset="$(cccl_configure_preset_for_test "${test_preset}")"
+    local smoke_bin="${BUILD_DIR}/${configure_preset}/bin/cccl.test.cuda_runtime_smoke"
+
+    if [[ -x "${smoke_bin}" ]]; then
+        run_ci_timed_command "CUDA smoke ${BUILD_NAME}" "${smoke_bin}" || return $?
+    elif cccl_smoke_tests_enabled "${configure_preset}"; then
+        echo "Error: CCCL_ENABLE_CUDA_SMOKE_TESTS=ON but smoke binary not found: ${smoke_bin}" >&2
+        return 1
+    fi
+}
+
+function print_test_time_summary()
+{
+    ctest_log=${1}
+
+    if [[ -f "${ctest_log}" ]]; then
+        begin_group "⏱️ Longest Test Steps"
+        # Only print the full output in CI:
+        if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+            cmake -DLOGFILE="${ctest_log}" -P ../cmake/PrintCTestRunTimes.cmake
+        else
+            # `|| :` to avoid `set -o pipefail` from triggering when `head` closes the pipe before `cmake` finishes.
+            # Otherwise the script will exit early with status 141 (SIGPIPE).
+            cmake -DLOGFILE="${ctest_log}" -P ../cmake/PrintCTestRunTimes.cmake | head -n 15 || :
+        fi
+        end_group "⏱️ Longest Test Steps"
+    fi
+}
+
+function configure_preset()
+{
+    local BUILD_NAME=$1
+    local PRESET=$2
+    shift 2
+    local CMAKE_OPTIONS=("$@")
+
+    local GROUP_NAME="🛠️  CMake Configure ${BUILD_NAME}"
+
+    symlink_latest_preset "$PRESET"
+
+    pushd .. > /dev/null
+    if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+      # Retry 5 times with 30 seconds between attempts to try to WAR network issues during CPM fetch on CI runners:
+      export RUN_COMMAND_RETRY_PARAMS=(5 30)
+    fi
+    status=0
+    SCCACHE_NO_DIST_COMPILE=1 run_command "$GROUP_NAME" cmake --preset="$PRESET" --log-level=VERBOSE "${CMAKE_OPTIONS[@]}" "${GLOBAL_CMAKE_OPTIONS[@]}" || status=$?
+    if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+        unset RUN_COMMAND_RETRY_PARAMS
+    fi
+    popd > /dev/null
+
+    if $CONFIGURE_ONLY; then
+        echo "${BUILD_NAME} configuration complete:"
+        echo "  Exit code:       ${status}"
+        echo "  CMake Preset:    ${PRESET}"
+        echo "  CMake Options:   ${CMAKE_OPTIONS[*]}"
+        echo "  Build Directory: ${BUILD_DIR}/${PRESET}"
+        exit "$status"
+    fi
+
+    return "$status"
+}
+
+function build_preset() {
+    local BUILD_NAME=$1
+    local PRESET=$2
+    # shellcheck disable=SC2034
+    local green="1;32"
+    # shellcheck disable=SC2034
+    local red="1;31"
+    local GROUP_NAME="🏗️  Build ${BUILD_NAME}"
+    shift 2
+    local BUILD_COMMANDS=("$@")
+
+    symlink_latest_preset "$PRESET"
+
+    if $CONFIGURE_ONLY; then
+        return 0
+    fi
+
+    local preset_dir="${BUILD_DIR}/${PRESET}"
+    local sccache_json="${preset_dir}/sccache_stats.json"
+    local memmon_log="${preset_dir}/memmon.log"
+
+    sccache -z > /dev/null || :
+
+    # Track memory usage on CI:
+    if [[ -n "${GITHUB_ACTIONS:-}" || -n "${MEMMON:-}" ]]; then
+      util/memmon.sh --start \
+          --log-threshold "${MEMMON_LOG_THRESHOLD:-2}" \
+          --print-threshold "${MEMMON_PRINT_THRESHOLD:-5}" \
+          --log-file "$memmon_log" \
+          --poll "${MEMMON_POLL_INTERVAL:-5}" \
+          || :
+    fi
+
+    pushd .. > /dev/null
+    status=0
+    run_ci_timed_command "$GROUP_NAME" cmake --build --parallel "$PARALLEL_LEVEL" --preset="$PRESET" ${VERBOSE:+-v} "${BUILD_COMMANDS[@]}" || status=$?
+    popd > /dev/null
+
+    if [[ -n "${GITHUB_ACTIONS:-}" || -n "${MEMMON:-}" ]]; then
+      util/memmon.sh --stop || :
+      run_command "📝 Memory Monitor Log" head -n20 "$memmon_log" || :
+    fi
+
+    # Only print detailed stats in actions workflow
+    if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+        sccache --show-adv-stats --stats-format=json > "${sccache_json}" || :
+        run_command "📊 sccache stats" sccache --show-adv-stats || :
+
+        begin_group "🥷 ninja build times"
+        echo "The \"weighted\" time is the elapsed time of each build step divided by the number
+              of tasks that were running in parallel. This makes it an excellent approximation
+              of how \"important\" a slow step was. A link that is entirely or mostly serialized
+              will have a weighted time that is the same or similar to its elapsed time. A
+              compile that runs in parallel with 999 other compiles will have a weighted time
+              that is tiny."
+        ./ninja_summary.py -C "${BUILD_DIR}"/"${PRESET}" || echo "Warning: ninja_summary.py failed to execute properly."
+        end_group
+    else
+      sccache -s || :
+    fi
+
+    return "$status"
+}
+
+function test_preset()
+{
+    local BUILD_NAME=$1
+    local PRESET=$2
+    local GPU_REQUIRED=${3:-true}
+
+    symlink_latest_preset "$PRESET"
+
+    if $CONFIGURE_ONLY; then
+        return 0
+    fi
+
+    if $GPU_REQUIRED; then
+        fail_if_no_gpu
+        if [[ -z "${CCCL_SKIP_CI_SMOKE:-}" ]]; then
+            run_cuda_smoke_test "${BUILD_NAME}" "${PRESET}" || return $?
+        fi
+    fi
+
+    local GROUP_NAME="🚀  Test ${BUILD_NAME}"
+
+    local preset_dir="${BUILD_DIR}/${PRESET}"
+    local ctest_log="${preset_dir}/ctest.log"
+
+    pushd .. > /dev/null
+    status=0
+    run_ci_timed_command "$GROUP_NAME" ctest --output-on-failure --output-log "${ctest_log}" --preset="$PRESET" || status=$?
+    popd > /dev/null
+
+    print_test_time_summary "${ctest_log}"
+
+    return "$status"
+}
+
+function configure_and_build_preset()
+{
+    local BUILD_NAME=$1
+    local PRESET=$2
+    shift 2
+    local CMAKE_OPTIONS=("$@")
+
+    configure_preset "$BUILD_NAME" "$PRESET" "${CMAKE_OPTIONS[@]}"
+
+    if ! $CONFIGURE_ONLY; then
+        build_preset "$BUILD_NAME" "$PRESET"
+    fi
+}

cccl_upstream/ci/build_compile_time_bench.sh

@@ -0,0 +1,370 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+repo_root="$(cd "${ci_dir}/.." && pwd)"
+tool_dir="${repo_root}/ci/compile_time"
+
+default_preset="all-dev"
+preset="${default_preset}"
+skip_configure=0
+skip_build=0
+prepare_perfetto=1
+run_ctadvisor=0
+write_tu_csv=1
+explicit_tu_csv=0
+tu_csv=""
+perfetto_output_dir=""
+baseline_ref=""
+baseline_worktree=""
+max_detail_len=180
+cloc_processes=0
+
+declare -a build_targets=()
+declare -a event_args=()
+declare -a common_args=()
+declare -a default_build_targets=(
+  "cub.headers.base"
+  "thrust.cpp.cuda.headers.base"
+  "libcudacxx.test.public_headers"
+)
+declare -a bench_cmake_args=(
+  "-DCMAKE_EXPORT_COMPILE_COMMANDS=ON"
+  "-DCMAKE_CUDA_COMPILER_LAUNCHER="
+  "-DCMAKE_CXX_COMPILER_LAUNCHER="
+  "-DCCCL_ENABLE_TESTING=OFF"
+  "-DCCCL_ENABLE_EXAMPLES=OFF"
+  "-DCCCL_ENABLE_BENCHMARKS=OFF"
+  "-DCCCL_ENABLE_C_PARALLEL=OFF"
+  "-DCCCL_ENABLE_C_EXPERIMENTAL_STF=OFF"
+  "-DCUB_ENABLE_TESTING=OFF"
+  "-DCUB_ENABLE_EXAMPLES=OFF"
+  "-DTHRUST_ENABLE_TESTING=OFF"
+  "-DTHRUST_ENABLE_EXAMPLES=OFF"
+  "-DTHRUST_MULTICONFIG_WORKLOAD=SMALL"
+  "-DTHRUST_MULTICONFIG_ENABLE_SYSTEM_OMP=OFF"
+  "-DTHRUST_MULTICONFIG_ENABLE_SYSTEM_TBB=OFF"
+  "-Dcudax_ENABLE_TESTING=OFF"
+  "-Dcudax_ENABLE_EXAMPLES=OFF"
+  "-Dcudax_ENABLE_CUDASTF=OFF"
+  "-Dcudax_ENABLE_CUFILE=OFF"
+  "-DCCCL_COMPILE_TIME_SAVE_PREPROCESSED_TUS=ON"
+  "-DCCCL_COMPILE_TIME_GENERATE_DEVICE_TIME_TRACES=ON"
+)
+
+usage() {
+  cat <<'EOF'
+Usage: ci/build_compile_time_bench.sh [options] [-- <event summary args>]
+
+Build options:
+  -preset <name>              CMake configure preset (default: all-dev)
+  -cmake-options <args>       Extra CMake configure options handled by ci/build_common.sh
+  -target <name>              Build target; repeatable
+                              (default: public include-check target set)
+  -baseline-ref <commit-ish>  Build this commit-ish as a comparison baseline
+  -skip-configure             Do not run cmake configure
+  -skip-build                 Do not run cmake --build
+  -cuda, -cxx, -std, -arch    Common compiler/standard/arch options from ci/build_common.sh
+
+Summary options:
+  -tu-csv <path>              Generated-TU summary CSV
+                              (default: <preset-build-dir>/compile_time/tu_summary.csv)
+  -no-tu-csv                  Do not write the generated-TU summary CSV
+  -prepare-perfetto           Prepare Perfetto-friendly trace copies (default)
+  -no-prepare-perfetto        Skip Perfetto trace preparation
+  -perfetto-output <path>     Perfetto trace output directory
+                              (default: <preset-build-dir>/compile_time/perfetto_traces)
+  -max-detail-len <n>         Max promoted detail length for Perfetto traces (default: 180)
+  -cloc-processes <n>         cloc process count for generated-TU summary CSV
+  -ctadvisor                  Print ctadvisor report for raw traces
+
+Event summary args:
+  Arguments after '--' are forwarded to ci/compile_time/summarize_events.py
+  after the raw trace directory. If omitted, the default summary is:
+    -f file-processing -e -n 15
+  Pass --slices <json-file> after '--' to emit multiple event report slices
+  and an event_reports/summary.json manifest.
+  With -baseline-ref, comparison-only options such as --threshold <seconds>
+  may also be passed after '--'. Baseline raw traces are preserved under
+  <preset-build-dir>/compile_time/baseline_raw_traces.
+
+Examples:
+  ci/build_compile_time_bench.sh
+  ci/build_compile_time_bench.sh -target cudax.headers.basic.no_stf -- -f scanning-function-body -i -n 20
+  ci/build_compile_time_bench.sh -preset cub -target cub.headers.base -- -f template-instantiation -e -n 15
+  ci/build_compile_time_bench.sh -baseline-ref origin/main -- -f file-processing -e --threshold 0.001
+EOF
+}
+
+status() { echo "[compile-time-bench] $*" >&2; }
+
+require_command() {
+  local command_name="$1"
+  command -v "${command_name}" >/dev/null \
+    || { echo "error: ${command_name} not found" >&2; exit 1; }
+}
+
+cleanup_baseline_worktree() {
+  if [[ -n "${baseline_worktree}" && -d "${baseline_worktree}" ]]; then
+    git -C "${repo_root}" worktree remove --force "${baseline_worktree}" >/dev/null 2>&1 \
+      || rm -rf "${baseline_worktree}"
+  fi
+}
+
+cleanup_baseline_worktree_and_exit() {
+  local exit_code="$1"
+  trap - EXIT HUP INT TERM
+  cleanup_baseline_worktree
+  exit "${exit_code}"
+}
+
+install_baseline_cleanup_traps() {
+  trap cleanup_baseline_worktree EXIT
+  trap 'cleanup_baseline_worktree_and_exit 129' HUP
+  trap 'cleanup_baseline_worktree_and_exit 130' INT
+  trap 'cleanup_baseline_worktree_and_exit 143' TERM
+}
+
+overlay_current_bench_file() {
+  local rel_path="$1"
+  mkdir -p "$(dirname "${baseline_worktree}/${rel_path}")"
+  rm -rf "${baseline_worktree:?}/${rel_path}"
+  ln -s "${repo_root}/${rel_path}" "${baseline_worktree}/${rel_path}"
+}
+
+overlay_current_bench_logic() {
+  overlay_current_bench_file "ci"
+  overlay_current_bench_file "CMakePresets.json"
+  overlay_current_bench_file "cmake/CCCLGenerateHeaderTests.cmake"
+}
+
+for arg in "$@"; do
+  case "$arg" in
+    -h|-help|--help) usage; exit 0 ;;
+    *) ;;
+  esac
+done
+
+new_args="$("${ci_dir}/util/extract_switches.sh" \
+  -skip-configure \
+  -skip-build \
+  -no-tu-csv \
+  -prepare-perfetto \
+  -no-prepare-perfetto \
+  -ctadvisor \
+  -- "$@")"
+
+declare -a new_args="(${new_args})"
+set -- "${new_args[@]}"
+while true; do
+  case "$1" in
+    -skip-configure) skip_configure=1; shift ;;
+    -skip-build) skip_build=1; shift ;;
+    -no-tu-csv) write_tu_csv=0; shift ;;
+    -prepare-perfetto) prepare_perfetto=1; shift ;;
+    -no-prepare-perfetto) prepare_perfetto=0; shift ;;
+    -ctadvisor) run_ctadvisor=1; shift ;;
+    --) shift; break ;;
+    *) echo "Unknown argument: $1" >&2; usage; exit 1 ;;
+  esac
+done
+
+while (($#)); do
+  case "$1" in
+    -preset) preset="$2"; shift 2 ;;
+    -target) build_targets+=("$2"); shift 2 ;;
+    -baseline-ref) baseline_ref="$2"; shift 2 ;;
+    -tu-csv) explicit_tu_csv=1; write_tu_csv=1; tu_csv="$2"; shift 2 ;;
+    -perfetto-output) perfetto_output_dir="$2"; shift 2 ;;
+    -max-detail-len) max_detail_len="$2"; shift 2 ;;
+    -cloc-processes) cloc_processes="$2"; shift 2 ;;
+    --) shift; event_args+=("$@"); break ;;
+    *) common_args+=("$1"); shift ;;
+  esac
+done
+
+set -- "${common_args[@]}"
+# shellcheck source=ci/build_common.sh
+source "${ci_dir}/build_common.sh"
+
+current_build_root="${BUILD_ROOT}"
+current_build_dir="${BUILD_DIR}"
+
+build_root_for_source() {
+  local source_root="$1"
+  mkdir -p "${source_root}/build"
+  (cd "${source_root}/build" && pwd)
+}
+
+build_dir_for_source() {
+  local source_root="$1"
+  local build_root="$2"
+  local build_dir="${build_root}/${CCCL_BUILD_INFIX}"
+  mkdir -p "${build_dir}"
+  readlink -f "${build_dir}"
+}
+
+run_bench_build() {
+  local source_root="$1"
+  local build_name="$2"
+  local build_root="$3"
+  local build_dir="$4"
+
+  (
+    # build_common.sh keeps the active build tree in these globals; override
+    # them only inside this subshell so current-tree state is restored after
+    # each build.
+    # shellcheck disable=SC2030
+    BUILD_ROOT="${build_root}"
+    # shellcheck disable=SC2030
+    BUILD_DIR="${build_dir}"
+    cd "${source_root}/ci"
+
+    if (( ! skip_configure )); then
+      status "Configuring preset '${preset}' with compile-time bench instrumentation (${build_name})..."
+      configure_preset "${build_name}" "${preset}" "${bench_cmake_args[@]}"
+    fi
+
+    if (( ! skip_build )); then
+      status "Building target(s) (${build_name}): ${build_targets[*]}"
+      build_preset "${build_name}" "${preset}" --target "${build_targets[@]}"
+    fi
+  )
+}
+
+prepare_perfetto_traces() {
+  local input_dir="$1"
+  local output_dir="$2"
+  local trace_repo_root="$3"
+  local label="$4"
+
+  status "Preparing Perfetto trace copies (${label})..."
+  rm -rf "${output_dir}"
+  "${tool_dir}/prepare_traces.py" \
+    --input "${input_dir}" \
+    --output "${output_dir}" \
+    --repo-root "${trace_repo_root}" \
+    --max-detail-len "${max_detail_len}"
+  status "Perfetto traces (${label}): ${output_dir}"
+}
+
+if ((${#build_targets[@]} == 0)); then
+  build_targets=("${default_build_targets[@]}")
+fi
+
+preset_build_dir="${current_build_dir}/${preset}"
+baseline_trace_dir=""
+
+if [[ -n "${baseline_ref}" ]]; then
+  baseline_commit="$(git -C "${repo_root}" rev-parse --verify "${baseline_ref}^{commit}")"
+  baseline_worktree="$(mktemp -d "${TMPDIR:-/tmp}/cccl-compile-time-baseline.XXXXXX")"
+  rmdir "${baseline_worktree}"
+  install_baseline_cleanup_traps
+  status "Creating baseline worktree for ${baseline_ref} (${baseline_commit})..."
+  git -C "${repo_root}" worktree add --detach "${baseline_worktree}" "${baseline_commit}" >/dev/null
+  overlay_current_bench_logic
+
+  baseline_build_root="$(build_root_for_source "${baseline_worktree}")"
+  baseline_build_dir="$(build_dir_for_source "${baseline_worktree}" "${baseline_build_root}")"
+  baseline_trace_dir="${baseline_build_dir}/${preset}/compile_time/raw_traces"
+fi
+
+report_root="${preset_build_dir}/compile_time"
+trace_dir="${report_root}/raw_traces"
+baseline_artifact_trace_dir="${report_root}/baseline_raw_traces"
+event_output_dir="${report_root}/event_reports"
+tu_csv="${tu_csv:-${report_root}/tu_summary.csv}"
+perfetto_output_dir="${perfetto_output_dir:-${report_root}/perfetto_traces}"
+if [[ -n "${baseline_ref}" && "${explicit_tu_csv}" -eq 0 ]]; then
+  write_tu_csv=0
+fi
+
+require_command python3
+if (( write_tu_csv )); then
+  require_command cloc
+fi
+if (( run_ctadvisor )); then
+  require_command ctadvisor
+fi
+
+run_bench_build "${repo_root}" "Compile-time Bench (current)" "${current_build_root}" "${current_build_dir}"
+if $CONFIGURE_ONLY; then
+  exit 0
+fi
+
+if [[ -n "${baseline_ref}" ]]; then
+  run_bench_build \
+    "${baseline_worktree}" \
+    "Compile-time Bench (baseline)" \
+    "${baseline_build_root}" \
+    "${baseline_build_dir}"
+fi
+
+shopt -s nullglob globstar
+trace_paths=("${trace_dir}"/**/*.json)
+(( ${#trace_paths[@]} > 0 )) \
+  || { echo "error: no device-time-trace JSON files found under ${trace_dir}" >&2; exit 1; }
+if [[ -n "${baseline_ref}" ]]; then
+  baseline_trace_paths=("${baseline_trace_dir}"/**/*.json)
+  (( ${#baseline_trace_paths[@]} > 0 )) \
+    || { echo "error: no device-time-trace JSON files found under ${baseline_trace_dir}" >&2; exit 1; }
+
+  status "Copying baseline raw traces to ${baseline_artifact_trace_dir}..."
+  rm -rf "${baseline_artifact_trace_dir}"
+  mkdir -p "${baseline_artifact_trace_dir}"
+  cp -a "${baseline_trace_dir}/." "${baseline_artifact_trace_dir}/"
+fi
+
+if (( prepare_perfetto )); then
+  if [[ -n "${baseline_ref}" ]]; then
+    rm -rf "${perfetto_output_dir}"
+    prepare_perfetto_traces \
+      "${trace_dir}" \
+      "${perfetto_output_dir}/current" \
+      "${repo_root}" \
+      "current"
+    prepare_perfetto_traces \
+      "${baseline_trace_dir}" \
+      "${perfetto_output_dir}/baseline" \
+      "${baseline_worktree}" \
+      "baseline"
+  else
+    prepare_perfetto_traces \
+      "${trace_dir}" \
+      "${perfetto_output_dir}" \
+      "${repo_root}" \
+      "current"
+  fi
+fi
+
+if (( write_tu_csv )); then
+  status "Writing generated-TU summary CSV..."
+  "${tool_dir}/summarize_tus.py" \
+    --build-dir "${preset_build_dir}" \
+    --output-csv "${tu_csv}" \
+    --cloc-processes "${cloc_processes}"
+  status "Generated-TU summary CSV: ${tu_csv}"
+fi
+
+declare -a summary_args=("${event_args[@]}")
+if ((${#summary_args[@]} == 0)); then
+  summary_args=(-f file-processing -e -n 15)
+fi
+summary_args+=(-o "${event_output_dir}")
+if [[ -n "${baseline_ref}" ]]; then
+  summary_args+=(
+    --baseline-dir "${baseline_trace_dir}"
+    --baseline-repo-root "${baseline_worktree}"
+  )
+fi
+status "Writing event summary..."
+"${tool_dir}/summarize_events.py" "${trace_dir}" "${summary_args[@]}"
+
+if (( run_ctadvisor )); then
+  status "Running ctadvisor over ${#trace_paths[@]} trace(s)..."
+  ctadvisor \
+    --trace-file-path "${trace_dir}" \
+    --header-advisor-entries 20 \
+    --thread-number "$(nproc --all --ignore=2)"
+fi

cccl_upstream/ci/build_cub.sh

@@ -0,0 +1,111 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+NO_LID=false
+LID0=false
+LID1=false
+LID2=false
+ARTIFACT_TAGS=()
+
+ci_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+
+new_args="$("${ci_dir}/util/extract_switches.sh" \
+  -no-lid \
+  -lid0 \
+  -lid1 \
+  -lid2 \
+  -- "$@")"
+
+declare -a new_args="(${new_args})"
+set -- "${new_args[@]}"
+while true; do
+  case "$1" in
+  -no-lid)
+    ARTIFACT_TAGS+=("no_lid")
+    NO_LID=true
+    shift
+    ;;
+  -lid0)
+    ARTIFACT_TAGS+=("lid_0")
+    LID0=true
+    shift
+    ;;
+  -lid1)
+    ARTIFACT_TAGS+=("lid_1")
+    LID1=true
+    shift
+    ;;
+  -lid2)
+    ARTIFACT_TAGS+=("lid_2")
+    LID2=true
+    shift
+    ;;
+  --)
+    shift
+    break
+    ;;
+  *)
+    echo "Unknown argument: $1"
+    exit 1
+    ;;
+  esac
+done
+
+# shellcheck source=ci/build_common.sh
+source "${ci_dir}/build_common.sh"
+
+print_environment_details
+
+ENABLE_CCCL_BENCHMARKS="false"
+ENABLE_CUB_RDC="false"
+
+if [[ "$CUDA_COMPILER" == *nvcc* ]]; then
+    ENABLE_CUB_RDC="true"
+    NVCC_VERSION=$($CUDA_COMPILER --version | grep release | awk '{print $6}' | cut -c2-)
+    if [[ -n "${DISABLE_CUB_BENCHMARKS}" ]]; then
+        echo "Benchmarks have been forcefully disabled."
+    else
+        ENABLE_CCCL_BENCHMARKS="true"
+        echo "nvcc version is $NVCC_VERSION. Building CUB benchmarks."
+    fi
+else
+    echo "Not building with NVCC, disabling RDC and benchmarks."
+fi
+
+if [[ "$HOST_COMPILER" == *icpc* || "$HOST_COMPILER" == *nvhpc* ]]; then
+    ENABLE_CCCL_BENCHMARKS="false"
+fi
+
+PRESET="cub"
+if $NO_LID; then
+    PRESET="cub-nolid"
+elif $LID0; then
+    PRESET="cub-lid0"
+elif $LID1; then
+    PRESET="cub-lid1"
+elif $LID2; then
+    PRESET="cub-lid2"
+fi
+
+CMAKE_OPTIONS=(
+    "-DCMAKE_CXX_STANDARD=$CXX_STANDARD"
+    "-DCMAKE_CUDA_STANDARD=$CXX_STANDARD"
+    "-DCCCL_ENABLE_BENCHMARKS=$ENABLE_CCCL_BENCHMARKS"
+    "-DCUB_ENABLE_RDC_TESTS=$ENABLE_CUB_RDC"
+)
+
+configure_and_build_preset "CUB" "$PRESET" "${CMAKE_OPTIONS[@]}"
+
+# Create test artifacts:
+if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+    if [[ ${#ARTIFACT_TAGS[@]} -gt 0 ]]; then
+        run_command "📦  Packaging test artifacts" \
+            "${ci_dir}/upload_cub_test_artifacts.sh" \
+            "${ARTIFACT_TAGS[@]}"
+    else
+        run_command "📦  Packaging test artifacts" "${ci_dir}/upload_cub_test_artifacts.sh"
+    fi
+fi
+
+print_time_summary

cccl_upstream/ci/build_cuda_cccl_python.sh

@@ -0,0 +1,184 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+usage="Usage: $0 -py-version <python_version> [additional options...]"
+
+# shellcheck source=ci/util/python/common_arg_parser.sh
+source "$ci_dir/util/python/common_arg_parser.sh"
+parse_python_args "$@"
+
+# Check if py_version was provided (this script requires it)
+require_py_version "$usage" || exit 1
+
+echo "Docker socket: " "$(ls /var/run/docker.sock)"
+
+if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+  # Prepare mount points etc for getting artifacts in/out of the container.
+  # shellcheck source=ci/util/artifacts/common.sh
+  source "$ci_dir/util/artifacts/common.sh"
+  # Note that these mounts use the runner (not the devcontainer) filesystem for
+  # source directories because of docker-out-of-docker quirks.
+  # The workflow-job GH actions make sure that they exist before running any
+  # scripts.
+  action_mounts=(
+    --mount "type=bind,source=${ARTIFACT_ARCHIVES},target=${ARTIFACT_ARCHIVES}"
+    --mount "type=bind,source=${ARTIFACT_UPLOAD_STAGE},target=${ARTIFACT_UPLOAD_STAGE}"
+  )
+else
+  # If not running in GitHub Actions, we don't need to set up artifact mounts.
+  action_mounts=()
+fi
+
+# cuda_cccl must be built in a container that can produce manylinux wheels,
+# and has the CUDA toolkit installed. We use the rapidsai/ci-wheel image for this.
+# We build separate wheels using separate containers for each CUDA version,
+# then merge them into a single wheel.
+
+readonly cuda12_version=12.9.1
+readonly cuda13_version=13.1.1
+readonly devcontainer_version=26.04
+readonly devcontainer_distro=rockylinux8
+# Use a baseline Python tag for the rapidsai ci-wheel image. The requested
+# py_version is installed inside the container by setup_python_env (uv).
+# Pinning the image tag avoids relying on a per-py_version image being
+# published (e.g. py3.14 images may not yet exist).
+readonly devcontainer_python_version=3.10
+
+if [[ "$(uname -m)" == "aarch64" ]]; then
+  cuda12_image="rapidsai/ci-wheel:${devcontainer_version}-cuda${cuda12_version}-${devcontainer_distro}-py${devcontainer_python_version}-arm64"
+  cuda13_image="rapidsai/ci-wheel:${devcontainer_version}-cuda${cuda13_version}-${devcontainer_distro}-py${devcontainer_python_version}-arm64"
+else
+  cuda12_image="rapidsai/ci-wheel:${devcontainer_version}-cuda${cuda12_version}-${devcontainer_distro}-py${devcontainer_python_version}"
+  cuda13_image="rapidsai/ci-wheel:${devcontainer_version}-cuda${cuda13_version}-${devcontainer_distro}-py${devcontainer_python_version}"
+fi
+# shellcheck disable=SC2034
+readonly cuda12_image
+# shellcheck disable=SC2034
+readonly cuda13_image
+
+mkdir -p wheelhouse
+
+# Shared caches across the cu12 + cu13 wheel builds. Both jobs compile an
+# identical LLVM/clang tree (LLVM has no CUDA dep), so a shared ccache cuts
+# the second build's LLVM phase from ~10 min to under 2 min; a shared CPM
+# source cache skips the second LLVM git clone entirely.
+#
+# The `mkdir`s run inside the (dev)container where only the container-side
+# paths are visible. The docker bind-mount uses the host-side paths
+# (${HOST_WORKSPACE}) since the inner docker daemon is the host's.
+mkdir -p ./.ccache ./.cpm-cache
+host_ccache_dir="${HOST_WORKSPACE:?}/.ccache"
+host_cpm_cache_dir="${HOST_WORKSPACE:?}/.cpm-cache"
+
+for ctk in 12 13; do
+  image="cuda${ctk}_image"
+  image="${!image}"
+  echo "::group::⚒️ Building CUDA $ctk wheel on $image"
+  (
+    set -x
+    docker pull "$image"
+    docker run --rm -i \
+        --workdir /workspace/python/cuda_cccl \
+        --mount "type=bind,source=${HOST_WORKSPACE:?},target=/workspace/" \
+        --mount "type=bind,source=${host_ccache_dir},target=/root/.ccache" \
+        --mount "type=bind,source=${host_cpm_cache_dir},target=/root/.cpm-cache" \
+        "${action_mounts[@]}" \
+        --env "py_version=${py_version}" \
+        --env "GITHUB_ACTIONS=${GITHUB_ACTIONS:-}" \
+        --env "GITHUB_RUN_ID=${GITHUB_RUN_ID:-}" \
+        --env "JOB_ID=${JOB_ID:-}" \
+        --env "CCCL_PYTHON_USE_V2=${CCCL_PYTHON_USE_V2:-}" \
+        --env "CCCL_C_PARALLEL_SANITIZE_THREAD=${CCCL_C_PARALLEL_SANITIZE_THREAD:-}" \
+        --env "CCACHE_DIR=/root/.ccache" \
+        --env "CPM_SOURCE_CACHE=/root/.cpm-cache" \
+        "$image" \
+        /workspace/ci/build_cuda_cccl_wheel.sh
+    # Prevent GHA runners from exhausting available storage with leftover images:
+    if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+      docker rmi -f "$image"
+    fi
+  )
+  echo "::endgroup::"
+done
+
+echo "Merging CUDA wheels..."
+
+# Set up a Python environment for the merge/repair steps.
+source "$ci_dir/pyenv_helper.sh"
+setup_python_env "${py_version}"
+
+# Needed for unpacking and repacking wheels.
+python -m pip install wheel
+
+# Find the built wheels
+cu12_wheel=$(find wheelhouse -name "*cu12*.whl" | head -1)
+cu13_wheel=$(find wheelhouse -name "*cu13*.whl" | head -1)
+
+if [[ -z "$cu12_wheel" ]]; then
+  echo "Error: CUDA 12 wheel not found in wheelhouse/"
+  ls -la wheelhouse/
+  exit 1
+fi
+
+if [[ -z "$cu13_wheel" ]]; then
+  echo "Error: CUDA 13 wheel not found in wheelhouse/"
+  ls -la wheelhouse/
+  exit 1
+fi
+
+echo "Found CUDA 12 wheel: $cu12_wheel"
+echo "Found CUDA 13 wheel: $cu13_wheel"
+
+# Merge the wheels
+python python/cuda_cccl/merge_cuda_wheels.py "$cu12_wheel" "$cu13_wheel" --output-dir wheelhouse_merged
+
+# A ThreadSanitizer wheel links libtsan; keep it external (excluded) so it is
+# NOT bundled -- the TSan test lane LD_PRELOADs the runner's matching libtsan
+# instead. Harmless for normal builds (the .so has no libtsan dependency).
+tsan_exclude=()
+if [[ "${CCCL_C_PARALLEL_SANITIZE_THREAD:-}" =~ ^(1|true|TRUE|on|ON)$ ]]; then
+  tsan_exclude=(--exclude 'libtsan.so.2')
+fi
+
+# Install auditwheel and repair the merged wheel
+python -m pip install patchelf auditwheel
+for wheel in wheelhouse_merged/cuda_cccl-*.whl; do
+    echo "Repairing merged wheel: $wheel"
+    python -m auditwheel repair \
+        --exclude 'libnvrtc.so.12' \
+        --exclude 'libnvrtc.so.13' \
+        --exclude 'libnvJitLink.so.12' \
+        --exclude 'libnvJitLink.so.13' \
+        --exclude 'libcudart.so.12' \
+        --exclude 'libcudart.so.13' \
+        --exclude 'libcuda.so.1' \
+        "${tsan_exclude[@]}" \
+        "$wheel" \
+        --wheel-dir wheelhouse_final
+done
+
+# Clean up intermediate files and move only the final merged wheel to wheelhouse
+rm -rf wheelhouse/*  # Clean existing wheelhouse
+mkdir -p wheelhouse
+
+# Move only the final repaired merged wheel
+if ls wheelhouse_final/cuda_cccl-*.whl 1> /dev/null 2>&1; then
+    mv wheelhouse_final/cuda_cccl-*.whl wheelhouse/
+    echo "Final merged wheel moved to wheelhouse"
+else
+    echo "No final repaired wheel found, moving unrepaired merged wheel"
+    mv wheelhouse_merged/cuda_cccl-*.whl wheelhouse/
+fi
+
+# Clean up temporary directories
+rm -rf wheelhouse_merged wheelhouse_final
+
+echo "Final wheels in wheelhouse:"
+ls -la wheelhouse/
+
+if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+  wheel_artifact_name="$(ci/util/workflow/get_wheel_artifact_name.sh)"
+  ci/util/artifacts/upload.sh "$wheel_artifact_name" 'wheelhouse/.*'
+fi

cccl_upstream/ci/build_cuda_cccl_python_tsan.sh

@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+# Thin wrapper around build_cuda_cccl_python.sh that builds the cuda_cccl wheel
+# with ThreadSanitizer instrumentation on the c.parallel (v1) host code, for the
+# free-threaded (3.14t) TSan nightly lane. The shared build script honors
+# CCCL_C_PARALLEL_SANITIZE_THREAD (passes -DCCCL_C_PARALLEL_SANITIZE_THREAD=ON to
+# the wheel build and --exclude libtsan from the auditwheel repair).
+set -euo pipefail
+
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+export CCCL_C_PARALLEL_SANITIZE_THREAD=1
+"$ci_dir/build_cuda_cccl_python.sh" "$@"

cccl_upstream/ci/build_cuda_cccl_python_v2.sh

@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+# Thin wrapper around build_cuda_cccl_python.sh that builds the cuda_cccl
+# wheel against the HostJIT-based cccl.c.parallel.v2 library instead of the
+# legacy NVRTC v1 library. The shared build script honors CCCL_PYTHON_USE_V2.
+set -euo pipefail
+
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+export CCCL_PYTHON_USE_V2=1
+exec "$ci_dir/build_cuda_cccl_python.sh" "$@"

cccl_upstream/ci/build_cuda_cccl_wheel.sh

@@ -0,0 +1,130 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Target script for `docker run` command in build_cuda_cccl_python.sh
+# The /workspace pathnames are hard-wired here.
+
+# Toolchain version, pinned in one place. Every gcc-toolset package name and the
+# /opt/rh enable path below derive from it — the -fsanitize=thread libtsan must
+# come from the same toolset as the compiler, so these cannot drift apart.
+readonly gcc_toolset_version=13
+
+# Install the GCC toolset (needed for the build) and ccache (shared between
+# cu12 and cu13 builds via /root/.ccache bind-mount from the host).
+/workspace/ci/util/retry.sh 5 30 dnf -y install \
+  "gcc-toolset-${gcc_toolset_version}-gcc" "gcc-toolset-${gcc_toolset_version}-gcc-c++" ccache
+
+# ThreadSanitizer builds link libcccl.c.parallel.so with -fsanitize=thread, which
+# the linker resolves via the toolset's libtsan.so. That runtime lives in a
+# separate package not pulled in by -gcc-c++, so install it for the TSan lane.
+if [[ "${CCCL_C_PARALLEL_SANITIZE_THREAD:-}" =~ ^(1|true|TRUE|on|ON)$ ]]; then
+  /workspace/ci/util/retry.sh 5 30 dnf -y install "gcc-toolset-${gcc_toolset_version}-libtsan-devel"
+fi
+
+# When the caller bind-mounts a ccache dir, wire it through to CMake. This
+# transparently caches every compile, so the second wheel build (cu13 after
+# cu12, or vice versa) reuses the entire LLVM/clang object tree.
+if [[ -n "${CCACHE_DIR:-}" ]]; then
+  export CMAKE_C_COMPILER_LAUNCHER=ccache
+  export CMAKE_CXX_COMPILER_LAUNCHER=ccache
+  export CMAKE_CUDA_COMPILER_LAUNCHER=ccache
+  echo "ccache enabled: CCACHE_DIR=${CCACHE_DIR}"
+  ccache --version 2>&1 | head -1 || true
+  ccache --show-stats 2>&1 | head -5 || true
+fi
+echo -e "#!/usr/bin/env bash\nsource /opt/rh/gcc-toolset-${gcc_toolset_version}/enable" >/etc/profile.d/enable_devtools.sh
+# shellcheck disable=SC1091
+source /etc/profile.d/enable_devtools.sh
+
+# Check what's available
+command -v gcc
+gcc --version
+command -v nvcc
+nvcc --version
+
+# Set up Python environment
+# shellcheck source=ci/pyenv_helper.sh
+source /workspace/ci/pyenv_helper.sh
+# shellcheck disable=SC2154
+setup_python_env "${py_version}"
+command -v python
+python --version
+echo "Done setting up python env"
+
+# Figure out the version to use for the package, we need repo history
+if "$(git rev-parse --is-shallow-repository)"; then
+  git fetch --unshallow
+fi
+export PACKAGE_VERSION_PREFIX="0.1."
+package_version=$(/workspace/ci/generate_version.sh)
+echo "Using package version ${package_version}"
+# Override the version used by setuptools_scm to the custom version
+export SETUPTOOLS_SCM_PRETEND_VERSION_FOR_CUDA_CCCL="${package_version}"
+
+cd /workspace/python/cuda_cccl
+
+# Determine CUDA version from nvcc
+cuda_version=$(nvcc --version | grep -oP 'release \K[0-9]+\.[0-9]+' | cut -d. -f1)
+echo "Detected CUDA version: ${cuda_version}"
+
+# Configure compilers:
+CXX="$(command -v g++)"
+export CXX
+CUDACXX="$(command -v nvcc)"
+export CUDACXX
+CUDAHOSTCXX="$(command -v g++)"
+export CUDAHOSTCXX
+
+# When CCCL_PYTHON_USE_V2 is set (=1/true/on), build the wheel against the
+# HostJIT-based cccl.c.parallel.v2 library instead of the default v1.
+if [[ "${CCCL_PYTHON_USE_V2:-}" =~ ^(1|true|TRUE|on|ON)$ ]]; then
+  export CMAKE_ARGS="${CMAKE_ARGS:-} -DCCCL_PYTHON_USE_V2=ON"
+  echo "Building wheel with CCCL v2 backend: CMAKE_ARGS=${CMAKE_ARGS}"
+
+  # v2's hostjit links against libnvJitLink and libnvfatbin, which aren't in
+  # the base rapidsai/ci-wheel image. Install the matching CTK devel packages
+  # so CMake's FindCUDAToolkit picks them up. nvcc is on PATH; derive the
+  # version (e.g. "13-0") from it.
+  ctk_pkg_ver=$(nvcc --version 2>/dev/null \
+    | grep -oP 'release \K[0-9]+\.[0-9]+' | tr '.' '-')
+  if [[ -n "${ctk_pkg_ver}" ]]; then
+    echo "Installing libnvjitlink-devel-${ctk_pkg_ver} libnvfatbin-devel-${ctk_pkg_ver}..."
+    /workspace/ci/util/retry.sh 5 30 dnf -y install \
+      "libnvjitlink-devel-${ctk_pkg_ver}" \
+      "libnvfatbin-devel-${ctk_pkg_ver}"
+  else
+    echo "WARNING: could not derive CTK version from nvcc; skipping nvJitLink/nvfatbin install"
+  fi
+
+  # FindCUDAToolkit learned about CUDA::nvfatbin only in CMake 3.27. The base
+  # rapidsai/ci-wheel image ships an older CMake; install a newer one into
+  # the active venv so scikit-build-core picks it up over the system cmake.
+  echo "Pinning cmake>=3.27 for FindCUDAToolkit nvfatbin support..."
+  python -m pip install --upgrade 'cmake>=3.27'
+fi
+
+# When CCCL_C_PARALLEL_SANITIZE_THREAD is set (=1/true/on), instrument the
+# c.parallel (v1) host code with ThreadSanitizer for the free-threaded TSan
+# nightly lane. Host-only; the libtsan runtime stays external (the shared
+# build_cuda_cccl_python.sh --excludes it from auditwheel).
+if [[ "${CCCL_C_PARALLEL_SANITIZE_THREAD:-}" =~ ^(1|true|TRUE|on|ON)$ ]]; then
+  export CMAKE_ARGS="${CMAKE_ARGS:-} -DCCCL_C_PARALLEL_SANITIZE_THREAD=ON"
+  echo "Building wheel with ThreadSanitizer-instrumented c.parallel: CMAKE_ARGS=${CMAKE_ARGS}"
+fi
+
+# Build the wheel
+python -m pip wheel --no-deps --verbose --wheel-dir dist .
+
+# Rename wheel to include CUDA version suffix
+for wheel in dist/cuda_cccl-*.whl; do
+    if [[ -f "$wheel" ]]; then
+        base_name=$(basename "$wheel" .whl)
+        new_name="${base_name}.cu${cuda_version}.whl"
+        mv "$wheel" "dist/${new_name}"
+        echo "Renamed wheel to: ${new_name}"
+    fi
+done
+
+# Move wheel to output directory
+mkdir -p /workspace/wheelhouse
+mv dist/cuda_cccl-*.cu*.whl /workspace/wheelhouse/

cccl_upstream/ci/build_cudax.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+# shellcheck source=ci/build_common.sh
+source "$(dirname "${BASH_SOURCE[0]}")/build_common.sh"
+
+print_environment_details
+
+# If the cudax_ENABLE_CUFILE variable is specified, we don't modify it. Otherwise if we got an nvcc binary, check the
+# nvcc version and if it's less than 12.9, disable the cuFile support. NVHPC Toolkit doesn't come with cuFile, too, so
+# we don't enable cuFile support if the host compiler is nvc++.
+if [[ -z "${cudax_ENABLE_CUFILE:-}" ]]; then
+  cudax_ENABLE_CUFILE="false"
+  if [[ -n "${NVCC_VERSION:-}" ]] && [[ "$(basename "${HOST_COMPILER}")" != "nvc++" ]]; then
+    if util/version_compare.sh "${NVCC_VERSION}" ge 12.9; then
+      cudax_ENABLE_CUFILE="true"
+    fi
+  fi
+fi
+
+PRESET="cudax"
+
+CMAKE_OPTIONS=(
+  "-Dcudax_ENABLE_CUFILE=${cudax_ENABLE_CUFILE}"
+  "-DCMAKE_CXX_STANDARD=${CXX_STANDARD}"
+  "-DCMAKE_CUDA_STANDARD=${CXX_STANDARD}"
+)
+
+configure_and_build_preset "CUDA Experimental" "$PRESET" "${CMAKE_OPTIONS[@]}"
+
+print_time_summary

cccl_upstream/ci/build_libcudacxx.sh

@@ -0,0 +1,27 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+ci_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+
+# shellcheck source=ci/build_common.sh
+source "${ci_dir}/build_common.sh"
+
+print_environment_details
+
+PRESET="libcudacxx"
+CMAKE_OPTIONS=("-DCMAKE_CXX_STANDARD=${CXX_STANDARD}" "-DCMAKE_CUDA_STANDARD=${CXX_STANDARD}")
+
+upload_test_artifacts=false
+if [[ -n "${GITHUB_ACTIONS:-}" ]] && "${ci_dir}/util/workflow/has_consumers.sh"; then
+  upload_test_artifacts=true
+  export LIT_OPTS="${LIT_OPTS:+${LIT_OPTS} }-Dtest_executable_mode=build"
+fi
+
+configure_and_build_preset libcudacxx "$PRESET" "${CMAKE_OPTIONS[@]}"
+
+if $upload_test_artifacts; then
+  run_command "📦  Packaging test artifacts" "${ci_dir}/upload_libcudacxx_test_artifacts.sh"
+fi
+
+print_time_summary

cccl_upstream/ci/build_stdpar.sh

@@ -0,0 +1,37 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+# Ensure the script is being executed in the root cccl directory:
+cd "$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/..";
+
+# Get the current CCCL info:
+readonly cccl_repo="${PWD}"
+readonly workdir="${cccl_repo}/test/stdpar"
+
+CXX_STANDARD=17
+
+args=("$@")
+while [[ "${#args[@]}" -ne 0 ]]; do
+    case "${args[0]}" in
+    -std)  CXX_STANDARD="${args[1]}";  args=("${args[@]:2}");;
+    *) echo "Unrecognized option: ${args[0]}"; exit 1 ;;
+    esac
+done
+
+mkdir -p "${workdir}"
+cd "${workdir}"
+
+# Configure and build
+rm -rf build
+
+cmake -B build -S . -G Ninja \
+  -DCMAKE_CXX_STANDARD="${CXX_STANDARD}" \
+  `# Explicitly compile for hopper since the CI machine does not have a gpu:` \
+  -DCMAKE_CXX_FLAGS="-gpu=cc90"
+
+# Disabled because `cmake --build -j ""` is invalid, but so is
+# `cmake --build -j8`. CMake expects a space between `-j` and
+# the numeric argument, or no argument at all.
+# shellcheck disable=SC2086
+cmake --build build -j ${PARALLEL_LEVEL:-}

cccl_upstream/ci/build_thrust.sh

@@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+# shellcheck source=ci/build_common.sh
+source "$(dirname "${BASH_SOURCE[0]}")/build_common.sh"
+
+print_environment_details
+
+PRESET="thrust"
+
+CMAKE_OPTIONS=("-DCMAKE_CXX_STANDARD=$CXX_STANDARD" "-DCMAKE_CUDA_STANDARD=$CXX_STANDARD")
+
+configure_and_build_preset "Thrust" "$PRESET" "${CMAKE_OPTIONS[@]}"
+
+# Create test artifacts:
+if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+    run_command "📦  Packaging test artifacts" /home/coder/cccl/ci/upload_thrust_test_artifacts.sh
+fi
+
+print_time_summary

cccl_upstream/ci/build_tidy.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+source "$(dirname "${BASH_SOURCE[0]}")/build_common.sh"
+
+print_environment_details
+
+BUILD_NAME="clang-tidy"
+PRESET="all-tidy"
+CMAKE_OPTIONS=("-DCMAKE_CXX_STANDARD=${CXX_STANDARD}" "-DCMAKE_CUDA_STANDARD=${CXX_STANDARD}")
+# Clang does not understand -G, passed by all-dev-debug which all-tidy derives from
+CMAKE_OPTIONS+=("-DCMAKE_CUDA_FLAGS=")
+# TODO(jfaibussowit)
+#
+# STF seems to trip clang-cuda up pretty heavily. It's unclear whether this is because STF
+# hasn't been compiled against clang-cuda before or whether it's an issue with clang-cuda
+# itself.
+CMAKE_OPTIONS+=("-Dcudax_ENABLE_CUDASTF=OFF")
+CMAKE_OPTIONS+=("-Dcudax_ENABLE_PLACES=OFF")
+
+# todo(dabayer): Re-enable OpenMP thrust builds for clang-tidy.
+CMAKE_OPTIONS+=("-DTHRUST_MULTICONFIG_ENABLE_SYSTEM_OMP=OFF")
+
+# Cannot use configure_and_build_preset because that does not allow us to pass additional
+# arguments to the build command.
+configure_preset "${BUILD_NAME}" "${PRESET}" "${CMAKE_OPTIONS[@]}"
+# Keep going after errors, we want CI to unearth all clang-tidy errors in one go
+BUILD_OPTIONS=(-- -k 0)
+build_preset "${BUILD_NAME}" "${PRESET}" "${BUILD_OPTIONS[@]}"
+
+print_time_summary

cccl_upstream/ci/compile_time/README.md

@@ -0,0 +1,77 @@
+# Compile-time benchmark CI contracts
+
+The compile-time benchmark CI flow is configured from `ci/matrix.yaml` under
+`compile_time.pull_request`.
+
+## Matrix schema
+
+Each config is a GitHub Actions matrix entry:
+
+```yaml
+compile_time:
+  pull_request:
+    - id: public-headers-gcc13
+      name: Public headers compile-time bench
+      gpu: rtx2080
+      launch_args: "--cuda 13.3 --host gcc13"
+      baseline_ref: origin/main
+      preset: all-dev
+      targets:
+        - cub.headers.base
+      args: "-arch native"
+      slices:
+        - id: total-compilation
+          title: TU total compilation
+          filter: total-compilation
+          timing: inclusive
+          sort: total
+          top: 15
+          threshold: 0.001
+```
+
+Required config fields are `id`, `name`, `gpu`, `launch_args`,
+`baseline_ref`, `preset`, `targets`, and `slices`. `args`, `comment`, and
+`artifact_retention_days` are optional.
+
+Required slice fields are `id`, `title`, `filter`, `timing`, `sort`, `top`, and
+`threshold`. Slice `children` may be used to group nested report sections in the
+PR comment. Empty slice sections are omitted recursively by the renderer unless
+the summary manifest carries warnings for that slice.
+
+`ci/compile_time/parse_matrix.py ci/matrix.yaml --workflow pull_request` emits
+the GitHub Actions matrix JSON. Missing or empty `compile_time.pull_request`
+emits `{"include":[]}`.
+
+In baseline comparisons, `threshold` is measured against the total selected
+inclusive/exclusive impact across all matched traces. The per-side reports still
+use `sort` for their own top-N ordering; comparison worse/better tables always
+rank by total impact so a change repeated across many traces is not hidden by a
+larger single-trace movement.
+
+## Report contract
+
+`summarize_events.py --slices <json>` writes per-slice CSVs under
+`event_reports/<slice-id>/` and writes a normalized `event_reports/summary.json`
+manifest. The manifest is the renderer contract; CSVs are human artifacts.
+Configured slices that match no events, have no matching trace files, or have no
+comparable event keys record warnings in the manifest so reporting failures are
+not presented as ordinary no-regression results.
+
+In comparison mode, the wrapper preserves:
+
+- current raw traces: `compile_time/raw_traces`
+- baseline raw traces: `compile_time/baseline_raw_traces`
+- Perfetto copies: `compile_time/perfetto_traces/current` and
+  `compile_time/perfetto_traces/baseline`
+
+## PR comments
+
+`render_pr_comment.py` reads `summary.json`, config metadata, and an artifacts
+URL, then writes the sticky PR comment body. Regressions and improvements are
+rendered in separate `<details>` blocks and are never mixed in one table.
+Warnings are rendered separately and keep their slice visible even when there
+are no regression/improvement rows.
+
+The reusable workflow uses the sticky-comment header
+`compile-time-bench-<config-id>` with `hide_and_recreate: true`, so previous
+comments for the same config are archived as outdated.

cccl_upstream/ci/compile_time/analytics.ipynb

@@ -0,0 +1,333 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "id": "903704f7",
+      "metadata": {},
+      "source": [
+        "# Compile-Time Analytics\n",
+        "\n",
+        "This notebook helps you:\n",
+        "\n",
+        "1. Run `ci/build_compile_time_bench.sh` from the repo root.\n",
+        "2. Load and inspect an all-header processing CSV.\n",
+        "3. Explore high-impact headers by TU coverage and average processing time.\n",
+        "4. Build combined ranking scores to identify optimization targets.\n",
+        "\n",
+        "Expected CSV columns include:\n",
+        "- `header_path`\n",
+        "- `include_tu_count`\n",
+        "- `avg_process_time_s`\n",
+        "- `total_process_time_s`\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "ad58db28",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "%pip install pandas matplotlib plotly nbformat"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "16ba6808",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "import shlex\n",
+        "import sys\n",
+        "from pathlib import Path\n",
+        "\n",
+        "import pandas as pd\n",
+        "\n",
+        "pd.set_option(\"display.max_colwidth\", 160)\n",
+        "pd.set_option(\"display.width\", 200)\n",
+        "pd.set_option(\"display.max_columns\", 20)\n",
+        "\n",
+        "REPO_ROOT = Path.cwd()\n",
+        "while REPO_ROOT != REPO_ROOT.parent and not (REPO_ROOT / \".git\").exists():\n",
+        "    REPO_ROOT = REPO_ROOT.parent\n",
+        "\n",
+        "if not (REPO_ROOT / \".git\").exists():\n",
+        "    raise RuntimeError(\n",
+        "        \"Could not locate repo root (.git). Start notebook from inside the CCCL repo.\"\n",
+        "    )\n",
+        "\n",
+        "print(f\"Repo root: {REPO_ROOT}\")\n",
+        "print(f\"Python executable: {sys.executable}\")"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "373652c8",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# --- Run build_compile_time_bench.sh (file-processing mode) ---\n",
+        "import subprocess\n",
+        "from pathlib import Path\n",
+        "\n",
+        "if \"REPO_ROOT\" not in globals():\n",
+        "    REPO_ROOT = Path.cwd()\n",
+        "    while REPO_ROOT != REPO_ROOT.parent and not (REPO_ROOT / \".git\").exists():\n",
+        "        REPO_ROOT = REPO_ROOT.parent\n",
+        "\n",
+        "output_csv = Path(os.environ.get(\"COMPILE_TIME_CSV\", \"/tmp/compile_time.csv\"))\n",
+        "cmd = [\n",
+        "    \"bash\",\n",
+        "    str(REPO_ROOT / \"ci\" / \"build_compile_time_bench.sh\"),\n",
+        "    *shlex.split(os.environ.get(\"COMPILE_TIME_BUILD_ARGS\", \"\")),\n",
+        "    \"--\",\n",
+        "    \"-f\",\n",
+        "    \"file-processing\",\n",
+        "    \"-e\",\n",
+        "    \"-n\",\n",
+        "    os.environ.get(\"COMPILE_TIME_TOP_N\", \"5000\"),\n",
+        "    \"--sort\",\n",
+        "    \"total\",\n",
+        "    \"--output-csv\",\n",
+        "    str(output_csv),\n",
+        "]\n",
+        "\n",
+        "print(\"Command:\")\n",
+        "print(\"  \" + \" \".join(shlex.quote(x) for x in cmd))\n",
+        "\n",
+        "subprocess.run(cmd, cwd=REPO_ROOT, check=True)\n",
+        "print(f\"\\nWrote: {output_csv}\")"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "12ad4c5c",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# --- Load CSV ---\n",
+        "csv_path = Path(os.environ.get(\"COMPILE_TIME_CSV\", \"/tmp/compile_time.csv\"))\n",
+        "if not csv_path.exists():\n",
+        "    raise FileNotFoundError(f\"Missing CSV: {csv_path}. Run the script first.\")\n",
+        "\n",
+        "df = pd.read_csv(csv_path)\n",
+        "\n",
+        "event_summary_cols = {\n",
+        "    \"event_name\",\n",
+        "    \"event_key\",\n",
+        "    \"root_tu_count\",\n",
+        "    \"selected_avg_per_root_tu_s\",\n",
+        "    \"selected_total_s\",\n",
+        "}\n",
+        "required_cols = [\n",
+        "    \"header_path\",\n",
+        "    \"include_tu_count\",\n",
+        "    \"avg_process_time_s\",\n",
+        "    \"total_process_time_s\",\n",
+        "]\n",
+        "\n",
+        "if event_summary_cols.issubset(df.columns):\n",
+        "    df[\"header_path\"] = df[\"event_key\"]\n",
+        "    df[\"include_tu_count\"] = df[\"root_tu_count\"]\n",
+        "    if (\n",
+        "        \"avg_inclusive_per_root_tu_s\" in df.columns\n",
+        "        and \"total_inclusive_s\" in df.columns\n",
+        "    ):\n",
+        "        df[\"avg_process_time_s\"] = df[\"avg_inclusive_per_root_tu_s\"]\n",
+        "        df[\"total_process_time_s\"] = df[\"total_inclusive_s\"]\n",
+        "    else:\n",
+        "        df[\"avg_process_time_s\"] = df[\"selected_avg_per_root_tu_s\"]\n",
+        "        df[\"total_process_time_s\"] = df[\"selected_total_s\"]\n",
+        "\n",
+        "missing = [c for c in required_cols if c not in df.columns]\n",
+        "if missing:\n",
+        "    raise ValueError(\n",
+        "        f\"CSV is not all-header processing output. Missing columns: {missing}. \"\n",
+        "        \"Run the script cell above to regenerate /tmp/compile_time.csv.\"\n",
+        "    )\n",
+        "\n",
+        "for col in [\"include_tu_count\", \"avg_process_time_s\", \"total_process_time_s\"]:\n",
+        "    df[col] = pd.to_numeric(df[col], errors=\"coerce\").fillna(0)\n",
+        "\n",
+        "print(f\"Rows: {len(df):,}\")\n",
+        "print(f\"Columns: {list(df.columns)}\")\n",
+        "df.head(5)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "1a9b36a8",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# --- Quick top-N views ---\n",
+        "TOP_N = 5\n",
+        "\n",
+        "print(\"Top by avg_process_time_s\")\n",
+        "display(\n",
+        "    df.nlargest(TOP_N, \"avg_process_time_s\")[\n",
+        "        [\n",
+        "            \"header_path\",\n",
+        "            \"avg_process_time_s\",\n",
+        "            \"include_tu_count\",\n",
+        "            \"total_process_time_s\",\n",
+        "        ]\n",
+        "    ]\n",
+        ")\n",
+        "\n",
+        "print(\"\\nTop by include_tu_count\")\n",
+        "display(\n",
+        "    df.nlargest(TOP_N, \"include_tu_count\")[\n",
+        "        [\n",
+        "            \"header_path\",\n",
+        "            \"include_tu_count\",\n",
+        "            \"avg_process_time_s\",\n",
+        "            \"total_process_time_s\",\n",
+        "        ]\n",
+        "    ]\n",
+        ")\n",
+        "\n",
+        "print(\"\\nTop by impact score (include_tu_count * avg_process_time_s)\")\n",
+        "df_score = df.copy()\n",
+        "df_score[\"impact_score\"] = df_score[\"include_tu_count\"] * df_score[\"avg_process_time_s\"]\n",
+        "display(\n",
+        "    df_score.nlargest(TOP_N, \"impact_score\")[\n",
+        "        [\n",
+        "            \"header_path\",\n",
+        "            \"impact_score\",\n",
+        "            \"include_tu_count\",\n",
+        "            \"avg_process_time_s\",\n",
+        "            \"total_process_time_s\",\n",
+        "        ]\n",
+        "    ]\n",
+        ")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "174eabd4",
+      "metadata": {},
+      "source": [
+        "### How to read this plot\n",
+        "\n",
+        "- Each point is one header seen in all-mode profiling.\n",
+        "- **X axis (`include_tu_count`)**: how many generated public-header TUs include this header at least once.\n",
+        "- **Y axis (`avg_process_time_s`)**: average time spent processing that header per including TU.\n",
+        "- Headers near the **upper-right** are usually the best optimization candidates because they are both widespread and expensive per include."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "9ddc0725",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# --- Interactive scatter (hover shows header name) ---\n",
+        "\n",
+        "import plotly.express as px\n",
+        "\n",
+        "plot_df = df.copy()\n",
+        "\n",
+        "total_headers = len(plot_df)\n",
+        "total_public_headers = int(plot_df[\"include_tu_count\"].max())\n",
+        "\n",
+        "fig = px.scatter(\n",
+        "    plot_df,\n",
+        "    x=\"include_tu_count\",\n",
+        "    y=\"avg_process_time_s\",\n",
+        "    hover_name=\"header_path\",\n",
+        "    hover_data={\n",
+        "        \"include_tu_count\": True,\n",
+        "        \"avg_process_time_s\": \":.6f\",\n",
+        "        \"total_process_time_s\": \":.3f\",\n",
+        "        \"header_path\": False,\n",
+        "    },\n",
+        "    opacity=0.6,\n",
+        "    title=(\n",
+        "        f\"Header include count vs avg processing time ({total_headers:,} total headers; \"\n",
+        "        f\"coverage measured across {total_public_headers} public headers)\"\n",
+        "    ),\n",
+        ")\n",
+        "\n",
+        "fig.update_layout(\n",
+        "    xaxis_title=(\n",
+        "        \"TU coverage: number of public headers that include this header \"\n",
+        "        f\"(out of {total_public_headers})\"\n",
+        "    ),\n",
+        "    yaxis_title=\"Average processing time per including TU (seconds)\",\n",
+        "    height=900,\n",
+        ")\n",
+        "fig.show()"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "f429b934",
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# --- Optional: run ctadvisor and parse expensive headers ---\n",
+        "run_ctadvisor = os.environ.get(\"COMPILE_TIME_RUN_CTADVISOR\") == \"1\"\n",
+        "CTADVISOR_ENTRIES = 10\n",
+        "CTADVISOR_THREADS = os.cpu_count() or 8\n",
+        "\n",
+        "trace_root = (\n",
+        "    REPO_ROOT\n",
+        "    / \"build\"\n",
+        "    / os.environ.get(\"CCCL_BUILD_INFIX\", \"cuda13.1-gcc14\")\n",
+        "    / os.environ.get(\"CCCL_COMPILE_TIME_PRESET\", \"all-dev\")\n",
+        "    / \"compile_time\"\n",
+        "    / \"raw_traces\"\n",
+        ")\n",
+        "ctadvisor_cmd = [\n",
+        "    \"ctadvisor\",\n",
+        "    \"--trace-file-path\",\n",
+        "    str(trace_root),\n",
+        "    \"--header-advisor-entries\",\n",
+        "    str(CTADVISOR_ENTRIES),\n",
+        "    \"--thread-number\",\n",
+        "    str(CTADVISOR_THREADS),\n",
+        "]\n",
+        "\n",
+        "if run_ctadvisor:\n",
+        "    print(\"Command:\")\n",
+        "    print(\"  \" + \" \".join(shlex.quote(x) for x in ctadvisor_cmd))\n",
+        "\n",
+        "    result = subprocess.run(\n",
+        "        ctadvisor_cmd, cwd=REPO_ROOT, check=True, capture_output=True, text=True\n",
+        "    )\n",
+        "    print(result.stdout)\n",
+        "else:\n",
+        "    print(\"Skipping ctadvisor. Set COMPILE_TIME_RUN_CTADVISOR=1 to run it.\")"
+      ]
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "cccl",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.12.3"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 5
+}

cccl_upstream/ci/compile_time/parse_matrix.py

@@ -0,0 +1,272 @@
+#!/usr/bin/env python3
+
+import argparse
+import json
+import re
+import subprocess
+import sys
+from pathlib import Path
+from typing import Any
+
+try:
+    import yaml
+except ModuleNotFoundError:
+    yaml = None
+    YAML_ERROR_TYPES: tuple[type[BaseException], ...] = ()
+else:
+    YAML_ERROR_TYPES = (yaml.YAMLError,)
+
+ID_RE = re.compile(r"^[a-z0-9][a-z0-9_.-]*$")
+TIMINGS = {"inclusive", "exclusive"}
+SORTS = {"total", "avg", "avg-root-tu", "max"}
+
+
+def die(message: str) -> None:
+    print(f"error: {message}", file=sys.stderr)
+    raise SystemExit(2)
+
+
+def require_mapping(value: Any, where: str) -> dict[str, Any]:
+    if not isinstance(value, dict):
+        die(f"{where} must be a mapping")
+    return value
+
+
+def require_field(mapping: dict[str, Any], field: str, where: str) -> Any:
+    if field not in mapping:
+        die(f"{where} is missing required field '{field}'")
+    return mapping[field]
+
+
+def require_string(value: Any, where: str, *, nonempty: bool = True) -> str:
+    if not isinstance(value, str):
+        die(f"{where} must be a string")
+    if nonempty and not value:
+        die(f"{where} must be non-empty")
+    return value
+
+
+def require_id(value: Any, where: str) -> str:
+    text = require_string(value, where)
+    if not ID_RE.fullmatch(text):
+        die(f"{where} must match {ID_RE.pattern}")
+    return text
+
+
+def require_string_list(value: Any, where: str) -> list[str]:
+    if not isinstance(value, list) or not value:
+        die(f"{where} must be a non-empty list")
+    strings: list[str] = []
+    for index, item in enumerate(value):
+        strings.append(require_string(item, f"{where}[{index}]"))
+    return strings
+
+
+def require_bool(value: Any, where: str) -> bool:
+    if not isinstance(value, bool):
+        die(f"{where} must be a boolean")
+    return value
+
+
+def require_positive_int(value: Any, where: str) -> int:
+    if isinstance(value, bool) or not isinstance(value, int) or value <= 0:
+        die(f"{where} must be a positive integer")
+    return value
+
+
+def validate_slice(
+    slice_data: Any,
+    *,
+    where: str,
+    seen_ids: set[str],
+) -> dict[str, Any]:
+    data = require_mapping(slice_data, where)
+    slice_id = require_id(require_field(data, "id", where), f"{where}.id")
+    if slice_id in seen_ids:
+        die(f"duplicate slice id '{slice_id}' in {where}")
+    seen_ids.add(slice_id)
+
+    title = require_string(require_field(data, "title", where), f"{where}.title")
+    filter_name = require_string(
+        require_field(data, "filter", where), f"{where}.filter"
+    )
+    timing = require_string(require_field(data, "timing", where), f"{where}.timing")
+    if timing not in TIMINGS:
+        die(f"{where}.timing must be one of {sorted(TIMINGS)}")
+    sort = require_string(require_field(data, "sort", where), f"{where}.sort")
+    if sort not in SORTS:
+        die(f"{where}.sort must be one of {sorted(SORTS)}")
+
+    top = require_field(data, "top", where)
+    if isinstance(top, bool) or not isinstance(top, int) or top <= 0:
+        die(f"{where}.top must be a positive integer")
+    threshold = require_field(data, "threshold", where)
+    if (
+        isinstance(threshold, bool)
+        or not isinstance(threshold, (int, float))
+        or threshold < 0
+    ):
+        die(f"{where}.threshold must be a non-negative number")
+
+    result: dict[str, Any] = {
+        "id": slice_id,
+        "title": title,
+        "filter": filter_name,
+        "timing": timing,
+        "sort": sort,
+        "top": top,
+        "threshold": threshold,
+    }
+    for optional in ("scope_filter", "exclusive_scope"):
+        if optional in data:
+            result[optional] = require_string(
+                data[optional], f"{where}.{optional}", nonempty=False
+            )
+
+    children = data.get("children", [])
+    if not isinstance(children, list):
+        die(f"{where}.children must be a list")
+    if children:
+        result["children"] = [
+            validate_slice(
+                child,
+                where=f"{where}.children[{index}]",
+                seen_ids=seen_ids,
+            )
+            for index, child in enumerate(children)
+        ]
+    return result
+
+
+def validate_config(
+    config_data: Any, *, where: str, seen_ids: set[str]
+) -> dict[str, Any]:
+    data = require_mapping(config_data, where)
+    config_id = require_id(require_field(data, "id", where), f"{where}.id")
+    if config_id in seen_ids:
+        die(f"duplicate compile_time config id '{config_id}'")
+    seen_ids.add(config_id)
+
+    targets = require_string_list(
+        require_field(data, "targets", where), f"{where}.targets"
+    )
+    slices = require_field(data, "slices", where)
+    if not isinstance(slices, list) or not slices:
+        die(f"{where}.slices must be a non-empty list")
+
+    slice_ids: set[str] = set()
+    normalized_slices = [
+        validate_slice(
+            slice_data,
+            where=f"{where}.slices[{index}]",
+            seen_ids=slice_ids,
+        )
+        for index, slice_data in enumerate(slices)
+    ]
+
+    return {
+        "id": config_id,
+        "name": require_string(require_field(data, "name", where), f"{where}.name"),
+        "gpu": require_string(require_field(data, "gpu", where), f"{where}.gpu"),
+        "launch_args": require_string(
+            require_field(data, "launch_args", where), f"{where}.launch_args"
+        ),
+        "baseline_ref": require_string(
+            require_field(data, "baseline_ref", where), f"{where}.baseline_ref"
+        ),
+        "preset": require_string(
+            require_field(data, "preset", where), f"{where}.preset"
+        ),
+        "targets": targets,
+        "args": require_string(data.get("args", ""), f"{where}.args", nonempty=False),
+        "comment": require_bool(data.get("comment", True), f"{where}.comment"),
+        "artifact_retention_days": require_positive_int(
+            data.get("artifact_retention_days", 14),
+            f"{where}.artifact_retention_days",
+        ),
+        "slices": normalized_slices,
+    }
+
+
+def matrix_entry(config: dict[str, Any]) -> dict[str, Any]:
+    config_id = config["id"]
+    return {
+        "id": config_id,
+        "name": config["name"],
+        "gpu": config["gpu"],
+        "launch_args": config["launch_args"],
+        "baseline_ref": config["baseline_ref"],
+        "preset": config["preset"],
+        "targets_json": json.dumps(config["targets"], separators=(",", ":")),
+        "args": config["args"],
+        "slices_json": json.dumps({"slices": config["slices"]}, separators=(",", ":")),
+        "comment": str(config["comment"]).lower(),
+        "artifact_retention_days": config["artifact_retention_days"],
+        "comment_header": f"compile-time-bench-{config_id}",
+    }
+
+
+def parse_matrix(path: Path, workflow: str) -> dict[str, Any]:
+    try:
+        if yaml is not None:
+            with path.open(encoding="utf-8") as f:
+                matrix = yaml.safe_load(f) or {}
+        else:
+            completed = subprocess.run(
+                ["yq", "-o=json", ".", path.as_posix()],
+                check=True,
+                text=True,
+                stdout=subprocess.PIPE,
+                stderr=subprocess.PIPE,
+            )
+            matrix = json.loads(completed.stdout or "{}")
+    except OSError as e:
+        die(f"failed to read {path}: {e}")
+    except subprocess.CalledProcessError as e:
+        die(f"failed to parse {path} with yq: {e.stderr.strip()}")
+    except json.JSONDecodeError as e:
+        die(f"failed to decode {path} as JSON: {e}")
+    except YAML_ERROR_TYPES as e:
+        die(f"failed to parse {path}: {e}")
+
+    compile_time = matrix.get("compile_time")
+    if compile_time is None:
+        return {"include": []}
+    compile_time = require_mapping(compile_time, "compile_time")
+    configs = compile_time.get(workflow, [])
+    if configs is None:
+        configs = []
+    if not isinstance(configs, list):
+        die(f"compile_time.{workflow} must be a list")
+    if not configs:
+        return {"include": []}
+
+    seen_ids: set[str] = set()
+    return {
+        "include": [
+            matrix_entry(
+                validate_config(
+                    config,
+                    where=f"compile_time.{workflow}[{index}]",
+                    seen_ids=seen_ids,
+                )
+            )
+            for index, config in enumerate(configs)
+        ]
+    }
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Parse ci/matrix.yaml compile_time entries for GitHub Actions."
+    )
+    parser.add_argument("matrix_yaml", type=Path)
+    parser.add_argument("--workflow", default="pull_request")
+    args = parser.parse_args()
+
+    json.dump(parse_matrix(args.matrix_yaml, args.workflow), sys.stdout)
+    print()
+
+
+if __name__ == "__main__":
+    main()

cccl_upstream/ci/compile_time/prepare_traces.py

@@ -0,0 +1,167 @@
+#!/usr/bin/env python3
+
+import argparse
+import json
+from pathlib import Path
+
+DETAIL_EVENT_NAMES = {
+    "Code Generation Function",
+    "CodeGen Function",
+    "ExecuteCompiler",
+    "Frontend",
+    "Instantiating Template Class",
+    "Instantiating Template Function",
+    "InstantiateClass",
+    "InstantiateFunction",
+    "OptFunction",
+    "ParseClass",
+    "PerformPendingInstantiations",
+    "Processing Header File",
+    "RunPass",
+    "Scanning Function Body",
+    "Source",
+}
+
+DETAIL_PREFIXES_TO_STRIP = (
+    "libcudacxx/include/",
+    "cudax/include/",
+    "c/parallel/include/",
+)
+
+DETAIL_PREFIXES_TO_COLLAPSE = (
+    ("cub/cub/", "cub/"),
+    ("thrust/thrust/", "thrust/"),
+)
+
+
+def normalize_detail(detail: str, repo_root: Path) -> str:
+    detail_path = Path(detail)
+    if detail_path.is_absolute():
+        try:
+            rel = detail_path.resolve(strict=False).relative_to(repo_root)
+            detail = rel.as_posix()
+        except ValueError:
+            pass
+
+    for prefix in DETAIL_PREFIXES_TO_STRIP:
+        if detail.startswith(prefix):
+            detail = detail[len(prefix) :]
+            break
+
+    for prefix, replacement in DETAIL_PREFIXES_TO_COLLAPSE:
+        if detail.startswith(prefix):
+            detail = replacement + detail[len(prefix) :]
+            break
+
+    return detail
+
+
+def display_detail(detail: str, repo_root: Path, max_detail_len: int | None) -> str:
+    detail = normalize_detail(detail, repo_root)
+    if (
+        max_detail_len is not None
+        and max_detail_len > 0
+        and len(detail) > max_detail_len
+    ):
+        return detail[: max_detail_len - 1] + "..."
+    return detail
+
+
+def rewrite_event_name(
+    event: dict, repo_root: Path, max_detail_len: int | None
+) -> bool:
+    name = event.get("name")
+    if name not in DETAIL_EVENT_NAMES:
+        return False
+
+    args = event.get("args")
+    if not isinstance(args, dict):
+        return False
+
+    detail = args.get("detail")
+    if not detail:
+        return False
+
+    args.setdefault("original_name", name)
+    event["name"] = f"{name}: {display_detail(str(detail), repo_root, max_detail_len)}"
+    return True
+
+
+def prepare_trace(
+    input_path: Path, output_path: Path, repo_root: Path, max_detail_len: int | None
+) -> int:
+    with input_path.open(encoding="utf-8") as f:
+        trace = json.load(f)
+
+    rewritten = 0
+    for event in trace.get("traceEvents", []):
+        if rewrite_event_name(event, repo_root, max_detail_len):
+            rewritten += 1
+
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    with output_path.open("w", encoding="utf-8") as f:
+        json.dump(trace, f, separators=(",", ":"))
+
+    return rewritten
+
+
+def iter_input_traces(input_path: Path) -> list[Path]:
+    if input_path.is_file():
+        return [input_path]
+    return sorted(input_path.rglob("*.json"))
+
+
+def output_path_for(input_trace: Path, input_root: Path, output_path: Path) -> Path:
+    if input_root.is_file():
+        if output_path.is_dir() or not output_path.suffix:
+            return output_path / f"{input_trace.stem}.perfetto.json"
+        return output_path
+
+    rel = input_trace.relative_to(input_root)
+    return output_path / rel.parent / f"{rel.stem}.perfetto.json"
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Prepare NVCC device-time-trace JSON files for Perfetto by promoting args.detail into event names."
+    )
+    parser.add_argument(
+        "--input", required=True, type=Path, help="Input trace JSON file or directory"
+    )
+    parser.add_argument(
+        "--output", required=True, type=Path, help="Output trace JSON file or directory"
+    )
+    parser.add_argument(
+        "--repo-root", default=Path(__file__).resolve().parents[2], type=Path
+    )
+    parser.add_argument(
+        "--max-detail-len",
+        default=0,
+        type=int,
+        help="Truncate promoted detail text to this many characters; 0 keeps full details",
+    )
+    args = parser.parse_args()
+
+    input_path = args.input.resolve(strict=False)
+    output_path = args.output.resolve(strict=False)
+    repo_root = args.repo_root.resolve(strict=False)
+    max_detail_len = args.max_detail_len if args.max_detail_len > 0 else None
+
+    traces = iter_input_traces(input_path)
+    if not traces:
+        raise SystemExit(f"no JSON traces found under {args.input}")
+
+    total_rewritten = 0
+    for trace_path in traces:
+        total_rewritten += prepare_trace(
+            trace_path,
+            output_path_for(trace_path, input_path, output_path),
+            repo_root,
+            max_detail_len,
+        )
+
+    print(f"prepared {len(traces)} trace(s); renamed {total_rewritten} event(s)")
+
+
+if __name__ == "__main__":
+    main()

cccl_upstream/ci/compile_time/render_pr_comment.py

@@ -0,0 +1,246 @@
+#!/usr/bin/env python3
+
+import argparse
+import json
+from pathlib import Path
+from typing import Any
+
+
+def load_json(path: Path) -> dict[str, Any]:
+    with path.open(encoding="utf-8") as f:
+        payload = json.load(f)
+    if not isinstance(payload, dict):
+        raise SystemExit(f"{path} must contain a JSON object")
+    return payload
+
+
+def md_escape(value: object) -> str:
+    text = str(value)
+    return (
+        text.replace("&", "&")
+        .replace("<", "&lt;")
+        .replace(">", "&gt;")
+        .replace("|", "\\|")
+        .replace("\n", " ")
+    )
+
+
+def md_code_span(value: object) -> str:
+    text = str(value).replace("\n", " ")
+    max_backtick_run = 0
+    current_backtick_run = 0
+    for char in text:
+        if char == "`":
+            current_backtick_run += 1
+            max_backtick_run = max(max_backtick_run, current_backtick_run)
+        else:
+            current_backtick_run = 0
+    delimiter = "`" * (max_backtick_run + 1)
+    if text.startswith("`") or text.endswith("`"):
+        text = f" {text} "
+    return f"{delimiter}{text}{delimiter}"
+
+
+def render_event_name(row: dict[str, Any]) -> str:
+    event_name = row.get("event_name", "")
+    event_key = row.get("event_key", "")
+    if event_key:
+        return f"{md_escape(event_name)}: {md_code_span(event_key)}"
+    return md_escape(event_name)
+
+
+def render_rows(rows: list[dict[str, Any]], *, direction: str) -> str:
+    delta_heading = (
+        "Regression impact" if direction == "worse" else "Improvement impact"
+    )
+    lines = [
+        f"| Rank | {delta_heading} | Selected Δ | Baseline | Current | Event | Matched traces |",
+        "| ---: | ---: | ---: | ---: | ---: | --- | ---: |",
+    ]
+    for row in rows:
+        lines.append(
+            "| {rank} | `{impact}` | `{selected_delta}` | `{baseline}` | `{current}` | {event} | {traces} |".format(
+                rank=md_escape(row.get("rank", "")),
+                impact=md_escape(row.get("impact_magnitude_s", "")),
+                selected_delta=md_escape(row.get("selected_delta_s", "")),
+                baseline=md_escape(row.get("baseline_selected_s", "")),
+                current=md_escape(row.get("current_selected_s", "")),
+                event=render_event_name(row),
+                traces=md_escape(row.get("matched_trace_count", "")),
+            )
+        )
+    return "\n".join(lines)
+
+
+def render_direction_details(
+    slice_title: str,
+    direction: str,
+    rows: list[dict[str, Any]],
+) -> str:
+    if not rows:
+        return ""
+    label = "Regressions" if direction == "worse" else "Improvements"
+    icon = "🔴" if direction == "worse" else "🟢"
+    return "\n".join(
+        [
+            "<details>",
+            f"<summary><strong>{icon} {md_escape(slice_title)} — {label}</strong></summary>",
+            "",
+            render_rows(rows, direction=direction),
+            "",
+            "</details>",
+        ]
+    )
+
+
+def render_warning_details(slice_title: str, warnings: list[Any]) -> str:
+    if not warnings:
+        return ""
+    lines = [
+        "<details open>",
+        f"<summary><strong>⚠️ {md_escape(slice_title)} — Warnings</strong></summary>",
+        "",
+    ]
+    lines.extend(f"- {md_escape(warning)}" for warning in warnings)
+    lines.extend(["", "</details>"])
+    return "\n".join(lines)
+
+
+def render_slice(slice_data: dict[str, Any], *, level: int = 3) -> str:
+    comparison = slice_data.get("comparison", {})
+    worse_rows = comparison.get("worse", {}).get("rows", [])
+    better_rows = comparison.get("better", {}).get("rows", [])
+    warnings = slice_data.get("warnings", [])
+    child_sections = [
+        rendered
+        for child in slice_data.get("children", [])
+        if (rendered := render_slice(child, level=level + 1))
+    ]
+    direct_sections = [
+        section
+        for section in (
+            render_warning_details(slice_data.get("title", "Slice"), warnings),
+            render_direction_details(
+                slice_data.get("title", "Slice"), "worse", worse_rows
+            ),
+            render_direction_details(
+                slice_data.get("title", "Slice"), "better", better_rows
+            ),
+        )
+        if section
+    ]
+    if not direct_sections and not child_sections:
+        return ""
+
+    heading_prefix = "#" * min(level, 6)
+    subtitle = (
+        f"`-f {slice_data.get('filter', '')}` "
+        f"`{slice_data.get('timing', '')}` "
+        f"`--sort {slice_data.get('sort', '')}`"
+    )
+    lines = [
+        f"{heading_prefix} {md_escape(slice_data.get('title', 'Slice'))}",
+        "",
+        subtitle,
+        "",
+    ]
+    lines.extend(join_sections(direct_sections))
+    if child_sections:
+        lines.extend(["", *join_sections(child_sections)])
+    return "\n".join(lines).strip()
+
+
+def join_sections(sections: list[str]) -> list[str]:
+    lines: list[str] = []
+    for section in sections:
+        if lines:
+            lines.append("")
+        lines.append(section)
+    return lines
+
+
+def count_rows(slice_data: dict[str, Any], direction: str) -> int:
+    comparison = slice_data.get("comparison", {})
+    total = len(comparison.get(direction, {}).get("rows", []))
+    return total + sum(
+        count_rows(child, direction) for child in slice_data.get("children", [])
+    )
+
+
+def count_warnings(slice_data: dict[str, Any]) -> int:
+    return len(slice_data.get("warnings", [])) + sum(
+        count_warnings(child) for child in slice_data.get("children", [])
+    )
+
+
+def render_comment(
+    summary: dict[str, Any],
+    config: dict[str, Any],
+    *,
+    artifacts_url: str,
+) -> str:
+    config_id = str(config["id"])
+    slices = summary.get("slices", [])
+    sections = [
+        section for slice_data in slices if (section := render_slice(slice_data))
+    ]
+    worse_count = sum(count_rows(slice_data, "worse") for slice_data in slices)
+    better_count = sum(count_rows(slice_data, "better") for slice_data in slices)
+    warning_count = sum(count_warnings(slice_data) for slice_data in slices)
+    result = (
+        f"**Result:** {worse_count} regression row(s), "
+        f"{better_count} improvement row(s) above threshold."
+    )
+    if warning_count:
+        result += f" {warning_count} warning(s)."
+
+    lines = [
+        f"<!-- cccl-compile-time-bench: {md_escape(config_id)} -->",
+        f"## ⏱️ CCCL compile-time benchmark comparison: {md_escape(config.get('name', config_id))}",
+        "",
+        result,
+        "",
+        "| Run | Value |",
+        "| --- | --- |",
+        f"| Config | {md_code_span(config_id)} |",
+        f"| Baseline | {md_code_span(config.get('baseline_ref', ''))} |",
+        f"| Preset | {md_code_span(config.get('preset', ''))} |",
+        f"| Targets | {md_code_span(', '.join(config.get('targets', [])))} |",
+        f"| GPU / launch args | {md_code_span(config.get('gpu', ''))} / {md_code_span(config.get('launch_args', ''))} |",
+        "",
+        f"**Artifacts:** [reports and traces]({artifacts_url})",
+        "",
+    ]
+    if sections:
+        lines.extend(join_sections(sections))
+    else:
+        lines.append(
+            "No compile-time benchmark changes exceeded the configured thresholds."
+        )
+    return "\n".join(lines).rstrip() + "\n"
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Render a GitHub PR comment from compile-time report JSON."
+    )
+    parser.add_argument("--summary", type=Path, required=True)
+    parser.add_argument("--config", type=Path, required=True)
+    parser.add_argument("--artifacts-url", required=True)
+    parser.add_argument("-o", "--output", type=Path)
+    args = parser.parse_args()
+
+    comment = render_comment(
+        load_json(args.summary),
+        load_json(args.config),
+        artifacts_url=args.artifacts_url,
+    )
+    if args.output:
+        args.output.parent.mkdir(parents=True, exist_ok=True)
+        args.output.write_text(comment, encoding="utf-8")
+    else:
+        print(comment, end="")
+
+
+if __name__ == "__main__":
+    main()

cccl_upstream/ci/compile_time/summarize_tus.py

@@ -0,0 +1,147 @@
+#!/usr/bin/env python3
+
+import argparse
+import csv
+import subprocess
+from pathlib import Path
+
+GENERATED_TU_MARKER = "/headers/"
+GENERATED_TU_SOURCE_SUFFIXES = (".cu", ".cpp", ".cxx", ".cc", ".c")
+PREPROCESSED_TU_SUFFIX = ".cpp4.ii"
+PREPROCESSED_TU_SUFFIXES = (".cpp4.ii", ".ii")
+
+
+def strip_generated_tu_suffix(path_text: str) -> str:
+    for suffix in GENERATED_TU_SOURCE_SUFFIXES:
+        if path_text.endswith(suffix):
+            return path_text[: -len(suffix)]
+    return path_text
+
+
+def generated_tu_input(tu: Path) -> str:
+    parts = tu.as_posix().split(GENERATED_TU_MARKER, 1)
+    if len(parts) != 2:
+        return tu.as_posix()
+
+    rel = parts[1].split("/", 1)
+    if len(rel) != 2:
+        return tu.as_posix()
+
+    return strip_generated_tu_suffix(rel[1])
+
+
+def find_preprocessed_tus(build_dir: Path) -> list[Path]:
+    return sorted(
+        {
+            path
+            for suffix in PREPROCESSED_TU_SUFFIXES
+            for path in build_dir.glob(f"**/headers/**/*{suffix}")
+        }
+    )
+
+
+def tu_source_for_preprocessed_tu(pp_path: Path) -> Path:
+    pp_text = pp_path.as_posix()
+    for suffix in PREPROCESSED_TU_SUFFIXES:
+        if pp_text.endswith(suffix):
+            return Path(pp_text[: -len(suffix)])
+    return pp_path.with_suffix("")
+
+
+def run_cloc(preprocessed_tus: list[Path], processes: int) -> dict[str, int]:
+    if not preprocessed_tus:
+        return {}
+
+    command = [
+        "cloc",
+        "--csv",
+        "--by-file",
+        "--skip-uniqueness",
+        "--processes",
+        str(processes),
+        "--force-lang=C++,ii",
+        *[path.as_posix() for path in preprocessed_tus],
+    ]
+    result = subprocess.run(command, check=True, capture_output=True, text=True)
+
+    loc_by_file: dict[str, int] = {}
+    reader = csv.reader(result.stdout.splitlines())
+    for row in reader:
+        if len(row) < 5 or row[1] == "filename":
+            continue
+        try:
+            loc_by_file[row[1]] = int(row[4])
+        except ValueError:
+            continue
+    return loc_by_file
+
+
+def write_summary(
+    output_csv: Path,
+    preprocessed_tus: list[Path],
+    loc_by_file: dict[str, int],
+) -> None:
+    output_csv.parent.mkdir(parents=True, exist_ok=True)
+    with output_csv.open("w", newline="", encoding="utf-8") as f:
+        writer = csv.DictWriter(
+            f,
+            fieldnames=[
+                "tu_input",
+                "transitive_loc",
+                "tu_source",
+                "preprocessed_tu",
+            ],
+        )
+        writer.writeheader()
+        for pp_path in preprocessed_tus:
+            tu_path = tu_source_for_preprocessed_tu(pp_path)
+            writer.writerow(
+                {
+                    "tu_input": generated_tu_input(tu_path),
+                    "transitive_loc": loc_by_file.get(pp_path.as_posix(), 0),
+                    "tu_source": tu_path.as_posix(),
+                    "preprocessed_tu": pp_path.as_posix(),
+                }
+            )
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Summarize generated TU inputs and preprocessed LOC."
+    )
+    parser.add_argument("--build-dir", required=True, type=Path)
+    parser.add_argument("--output-csv", required=True, type=Path)
+    parser.add_argument(
+        "--cloc-processes",
+        type=int,
+        default=0,
+        help="cloc process count; 0 uses nproc --all --ignore=2 when available",
+    )
+    args = parser.parse_args()
+
+    build_dir = args.build_dir.resolve(strict=False)
+    preprocessed_tus = find_preprocessed_tus(build_dir)
+    if not preprocessed_tus:
+        raise SystemExit(f"no preprocessed generated TUs found under {build_dir}")
+
+    processes = args.cloc_processes
+    if processes <= 0:
+        try:
+            processes = int(
+                subprocess.check_output(
+                    ["nproc", "--all", "--ignore=2"], text=True
+                ).strip()
+            )
+        except (subprocess.SubprocessError, ValueError):
+            processes = 1
+
+    write_summary(
+        output_csv=args.output_csv,
+        preprocessed_tus=preprocessed_tus,
+        loc_by_file=run_cloc(preprocessed_tus, processes),
+    )
+    print(f"wrote {len(preprocessed_tus)} generated TU row(s) to {args.output_csv}")
+
+
+if __name__ == "__main__":
+    main()

cccl_upstream/ci/compute-sanitizer-suppressions.xml

@@ -0,0 +1,453 @@
+<?xml version="1.0" encoding="utf-8"?>
+<ComputeSanitizerOutput>
+  <!--
+  thrust::equal reduces using an accumulator type tuple<bool, OffsetT>.
+  The padding bytes inside the tuple are not initialized.
+  This causes issues during `cub::ThreadLoad`, which loads the memory as aliased
+  machine words that are cast to the tuple type.
+  -->
+  <record>
+    <kind>Initcheck</kind>
+    <what>
+      <text>Uninitialized __global__ memory read of size 2 bytes</text>
+      <size>2</size>
+    </what>
+    <where>
+      <func>ThreadLoad</func>
+    </where>
+    <deviceStack>
+      <frame>
+        <func>UnrolledThreadLoadImpl</func>
+      </frame>
+      <frame>
+        <func>UnrolledThreadLoad</func>
+      </frame>
+      <frame>
+        <func>ThreadLoad</func>
+      </frame>
+      <frame>
+        <func>ThreadLoad</func>
+      </frame>
+    </deviceStack>
+    <hostStack>
+      <frame>
+        <module>.*libcuda.so.*</module>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>cudaLaunchKernel</func>
+      </frame>
+      <frame>
+        <func>.*cub::.*::DeviceReduce.*.*thrust::.*find_if.*</func>
+      </frame>
+    </hostStack>
+  </record>
+  <!--
+  Similar to the above, thrust::equal copies a tuple<bool, OffsetT> from host -> device
+  with the result of the comparison. The padding bytes trigger host API initialization
+  errors during the cudaMemcpy.
+  Sadly, this is a very generic suppression that may hide real issues, but it's the best
+  we can do given the current tooling.
+  -->
+  <record>
+    <kind>InitcheckApiError</kind>
+    <level>Error</level>
+    <what>
+      <text>Host API uninitialized memory access</text>
+      <accessSize>16</accessSize>
+    </what>
+    <hostStack>
+      <saveLocation>error</saveLocation>
+      <frame>
+        <module>.*/libcuda.so.*</module>
+      </frame>
+      <frame>
+        <func>libcudart_static_.*</func>
+      </frame>
+      <frame>
+        <func>libcudart_static_.*</func>
+      </frame>
+      <frame>
+        <func>cudaMemcpyAsync</func>
+      </frame>
+      <frame>
+        <func>void C_A_T_C_H_T_E_M_P_L_A_T_E_T_E_S_T_F_U_N_C_.*</func>
+      </frame>
+    </hostStack>
+  </record>
+  <!-- Another variant of the above with a different catch2 dispatcher -->
+  <record>
+    <kind>InitcheckApiError</kind>
+    <level>Error</level>
+    <what>
+      <text>Host API uninitialized memory access</text>
+      <accessSize>16</accessSize>
+    </what>
+    <hostStack>
+      <saveLocation>error</saveLocation>
+      <frame>
+        <module>.*/libcuda.so.*</module>
+      </frame>
+      <frame>
+        <func>libcudart_static_.*</func>
+      </frame>
+      <frame>
+        <func>libcudart_static_.*</func>
+      </frame>
+      <frame>
+        <func>cudaMemcpyAsync</func>
+      </frame>
+      <frame>
+        <func>void CATCH2_INTERNAL_TEMPLATE_TEST.*</func>
+      </frame>
+    </hostStack>
+  </record>
+  <!-- Yet another instance of that tuple<Offset, bool> padding -->
+  <record>
+    <kind>InitcheckApiError</kind>
+    <level>Error</level>
+    <what>
+      <text>Host API uninitialized memory access</text>
+      <accessSize>16</accessSize>
+    </what>
+    <hostStack>
+      <saveLocation>error</saveLocation>
+      <frame>
+        <module>.*libcuda.so.*</module>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>cudaMemcpyAsync</func>
+      </frame>
+      <frame>
+        <func>.*thrust::.*(equal|operator==|mismatch|find_if).*</func>
+      </frame>
+    </hostStack>
+  </record>
+  <!-- Yet another instance of that tuple<Offset, bool> padding -->
+  <record>
+    <kind>InitcheckApiError</kind>
+    <level>Error</level>
+    <what>
+      <text>Host API uninitialized memory access</text>
+      <accessSize>16</accessSize>
+    </what>
+    <hostStack>
+      <saveLocation>error</saveLocation>
+      <frame>
+        <module>.*libcuda.so.*</module>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>cudaMemcpyAsync</func>
+      </frame>
+      <frame>
+        <func>bool binary_equal.*</func> <!-- Implementation detail of CUB's block radix sort tests, calls thrust::equal -->
+      </frame>
+    </hostStack>
+  </record>
+  <!-- Yet another instance of that tuple<Offset, bool> padding -->
+  <record>
+    <kind>InitcheckApiError</kind>
+    <level>Error</level>
+    <what>
+      <text>Host API uninitialized memory access</text>
+      <accessSize>16</accessSize>
+    </what>
+    <hostStack>
+      <saveLocation>error</saveLocation>
+      <frame>
+        <module>.*libcuda.so.*</module>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>cudaMemcpyAsync</func>
+      </frame>
+      <frame>
+        <!-- CUB's namespace_wrapped test has this issue inlined into main: -->
+        <func>main</func>
+        <module>.*cub.*test.namespace_wrapped</module>
+      </frame>
+    </hostStack>
+  </record>
+  <!-- Yet another instance of that tuple<Offset, bool> padding -->
+  <record>
+    <kind>InitcheckApiError</kind>
+    <what>
+      <text>Host API uninitialized memory access</text>
+      <accessSize>16</accessSize>
+    </what>
+    <hostStack>
+      <saveLocation>error</saveLocation>
+      <frame>
+        <module>.*libcuda.so.*</module>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>cudaMemcpyAsync</func>
+      </frame>
+      <frame>
+        <!-- The segmented sort test have several stacks that have padding bit issues. -->
+        <module>.*cub.*device_segmented_sort.*</module>
+      </frame>
+    </hostStack>
+  </record>
+  <!-- Yet another instance of that tuple<Offset, bool> padding -->
+  <record>
+    <kind>InitcheckApiError</kind>
+    <what>
+      <text>Host API uninitialized memory access</text>
+      <accessSize>16</accessSize>
+    </what>
+    <hostStack>
+      <saveLocation>error</saveLocation>
+      <frame>
+        <module>.*libcuda.so.*</module>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>cudaMemcpyAsync</func>
+      </frame>
+      <frame>
+        <!-- The RLE tests have several stacks that have padding bit issues. -->
+        <module>.*cub.*device_run_length_encode.*</module>
+      </frame>
+    </hostStack>
+  </record>
+  <!--
+  DeviceRunLengthEncode performs a WarpExchange::ScatterToStriped that 'discards'
+  elements by scattering them to the same (ignored) destination address. This
+  triggers a WAW race that we can safely ignore.
+  -->
+  <record>
+    <kind>Analysis</kind>
+    <level>Error</level>
+    <what>
+      <text>Race condition</text>
+      <source>
+        <direction>Write</direction>
+        <where>
+          <func>ScatterToStriped</func>
+        </where>
+      </source>
+      <destination>
+        <direction>Write</direction>
+        <where>
+          <func>ScatterToStriped</func>
+        </where>
+      </destination>
+      <destination>
+        <direction>Write</direction>
+        <where>
+          <func>ScatterToStriped</func>
+        </where>
+      </destination>
+      <destination>
+        <direction>Write</direction>
+        <where>
+          <func>ScatterToStriped</func>
+        </where>
+      </destination>
+    </what>
+  </record>
+  <!-- Another variation of the above -->
+  <record>
+    <kind>Analysis</kind>
+    <level>Error</level>
+    <what>
+      <text>Race condition</text>
+      <source>
+        <direction>Write</direction>
+        <where>
+          <func>ScatterToStriped</func>
+        </where>
+      </source>
+      <destination>
+        <direction>Write</direction>
+        <where>
+          <func>ScatterToStriped</func>
+        </where>
+      </destination>
+    </what>
+  </record>
+  <!-- Another variation of the above -->
+  <record>
+    <kind>Analysis</kind>
+    <level>Error</level>
+    <what>
+      <text>Race condition</text>
+      <source>
+        <direction>Write</direction>
+        <where>
+          <func>ScatterToStriped</func>
+        </where>
+      </source>
+      <destination>
+        <direction>Write</direction>
+        <where>
+          <func>ScatterToStriped</func>
+        </where>
+      </destination>
+      <destination>
+        <direction>Write</direction>
+        <where>
+          <func>ScatterToStriped</func>
+        </where>
+      </destination>
+    </what>
+  </record>
+  <!--
+  There are uninitialized padding bits inside cub::ConstantInputIterator,
+  which is basically a struct{ T value; ptrdiff_t offset; }.
+  -->
+  <record>
+    <kind>InitcheckApiError</kind>
+    <level>Error</level>
+    <what>
+      <text>Host API uninitialized memory access</text>
+      <accessSize>32</accessSize>
+    </what>
+    <hostStack>
+      <saveLocation>error</saveLocation>
+      <frame>
+        <module>.*libcuda.so.*</module>
+      </frame>
+      <frame>
+        <func>libcudart_static_.*</func>
+      </frame>
+      <frame>
+        <func>libcudart_static_.*</func>
+      </frame>
+      <frame>
+        <func>cudaMemcpyAsync</func>
+      </frame>
+      <frame>
+        <func>thrust.*vector_base.*ConstantInputIterator.*</func>
+      </frame>
+      <frame>
+        <func>void test_iterator.*ConstantInputIterator.*</func>
+      </frame>
+    </hostStack>
+  </record>
+  <!--
+  Similar to the above; cub::TransformInputIterator is a struct{TransformOp op; InputIterT iter;}.
+  In this case TransformOp is 1 byte, and InputIterT is an 8-byte pointer.
+  Padding bits strike again.
+  -->
+    <record>
+    <kind>InitcheckApiError</kind>
+    <level>Error</level>
+    <what>
+      <text>Host API uninitialized memory access</text>
+      <accessSize>32</accessSize>
+    </what>
+    <hostStack>
+      <saveLocation>error</saveLocation>
+      <frame>
+        <module>.*/libcuda.so.*</module>
+      </frame>
+      <frame>
+        <func>libcudart_static_.*</func>
+      </frame>
+      <frame>
+        <func>libcudart_static_.*</func>
+      </frame>
+      <frame>
+        <func>cudaMemcpyAsync</func>
+      </frame>
+      <frame>
+        <func>thrust.*vector_base.*TransformInputIterator.*</func>
+      </frame>
+      <frame>
+        <func>void test_iterator.*TransformInputIterator.*</func>
+      </frame>
+    </hostStack>
+  </record>
+  <!--
+  Same as above; this time InputIterT is a cub::TexObjInputIterator.
+  -->
+  <record>
+    <kind>InitcheckApiError</kind>
+    <level>Error</level>
+    <what>
+      <text>Host API uninitialized memory access</text>
+      <accessSize>64</accessSize>
+    </what>
+    <hostStack>
+      <saveLocation>error</saveLocation>
+      <frame>
+        <module>.*/libcuda.so.*</module>
+      </frame>
+      <frame>
+        <func>libcudart_static_.*</func>
+      </frame>
+      <frame>
+        <func>libcudart_static_.*</func>
+      </frame>
+      <frame>
+        <func>cudaMemcpyAsync</func>
+      </frame>
+      <frame>
+        <func>thrust.*vector_base.*TransformInputIterator.*</func>
+      </frame>
+      <frame>
+        <func>void test_iterator.*TransformInputIterator.*</func>
+      </frame>
+    </hostStack>
+  </record>
+  <!--
+  cub.test.device_reduce transfers cub::KeyValuePair<int, unwrap_value_t<...>> from
+  device -> host. This suppresses warnings about transferring padding bits inside
+  of the KeyValuePair.
+  -->
+  <record>
+    <kind>InitcheckApiError</kind>
+    <level>Error</level>
+    <what>
+      <text>Host API uninitialized memory access</text>
+    </what>
+    <hostStack>
+      <saveLocation>error</saveLocation>
+      <frame>
+        <module>.*libcuda.so.*</module>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>libcudart_static.*</func>
+      </frame>
+      <frame>
+        <func>cudaMemcpyAsync</func>
+        <module>.*cub.*device_(segmented_|)reduce.*</module>
+      </frame>
+    </hostStack>
+  </record>
+</ComputeSanitizerOutput>

cccl_upstream/ci/generate_version.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+
+# Generate a version number string using metadata from git or JSON.
+# Use the PyPi package versioning convention for pre-release or
+# post release patches.
+
+# Set some defaults for variables.
+CCCL_BRANCH="${CCCL_BRANCH:-dev}"
+PACKAGE_VERSION_PREFIX="${PACKAGE_VERSION_PREFIX:-}"
+
+GIT_DESCRIBE_TAG=$(git describe --tags --match "v[0-9]*" --abbrev=0)
+GIT_DESCRIBE_NUMBER=$(git rev-list "${GIT_DESCRIBE_TAG}"..HEAD --count)
+
+JSON_VERSION=$(jq -r .full /workspace/cccl-version.json)
+
+# Generate a suffix depending on release or dev branch.
+PACKAGE_VERSION_SUFFIX=""
+if [[ "${GIT_DESCRIBE_NUMBER}" != "0" ]]; then
+  if [[ ${CCCL_BRANCH} == "dev" ]]; then
+    PACKAGE_VERSION_SUFFIX=".dev${GIT_DESCRIBE_NUMBER}"
+  else
+    PACKAGE_VERSION_SUFFIX=".post${GIT_DESCRIBE_NUMBER}"
+  fi
+fi
+
+# If using Git metadata this could generate it from the last tag.
+# VERSION="${GIT_DESCRIBE_TAG#v}.dev${GIT_DESCRIBE_NUMBER}"
+
+# Generate the version using a combination of JSON and git commit number.
+VERSION="${PACKAGE_VERSION_PREFIX}${JSON_VERSION}${PACKAGE_VERSION_SUFFIX}"
+
+echo -n "${VERSION}"

cccl_upstream/ci/inspect_changes.py

@@ -0,0 +1,691 @@
+#!/usr/bin/env python3
+"""Identify dirty CCCL subprojects between two commits or explicit path lists."""
+
+from __future__ import annotations
+
+import argparse
+import os
+import re
+import subprocess
+import sys
+from collections import deque
+from dataclasses import dataclass, replace
+from pathlib import Path
+from typing import Dict, List, Optional, Sequence, Tuple
+
+import yaml
+
+REPO_ROOT = Path(__file__).resolve().parent.parent
+CONFIG_PATH = REPO_ROOT / "ci" / "project_files_and_dependencies.yaml"
+CORE_PROJECT_KEY = "core"
+
+
+class SummaryWriter:
+    """Utility for duplicating output to stdout and an optional summary file."""
+
+    def __init__(self, path: Optional[Path]):
+        self._handle = path.open("a", encoding="utf-8") if path else None
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, exc_type, exc, tb):
+        if self._handle:
+            self._handle.close()
+
+    def log(self, line: str = "") -> None:
+        print(line)
+        if self._handle:
+            self._handle.write(line + "\n")
+
+
+@dataclass(frozen=True)
+class ProjectConfig:
+    """Per-project configuration derived from the YAML file."""
+
+    key: str
+    name: str
+    matrix_project: Optional[str]
+    include_regexes: Tuple[str, ...]
+    exclude_regexes: Tuple[str, ...]
+    exclude_project_files: Tuple[str, ...]
+    lite_dependencies: Tuple[str, ...]
+    full_dependencies: Tuple[str, ...]
+    transitive_lite_dependencies: Tuple[str, ...] = ()
+
+
+@dataclass(frozen=True)
+class Config:
+    """Aggregated configuration for change detection."""
+
+    projects: Dict[str, ProjectConfig]
+    project_keys: Tuple[str, ...]
+    ignore_regexes: Tuple[str, ...]
+
+    def project(self, key: str) -> ProjectConfig:
+        return self.projects[key]
+
+
+def parse_args(argv: Optional[Sequence[str]] = None) -> argparse.Namespace:
+    """Parse CLI arguments for determining dirty files."""
+    parser = argparse.ArgumentParser(
+        description="Identify which CCCL projects require rebuilds between two commits."
+    )
+    group = parser.add_mutually_exclusive_group(required=True)
+    group.add_argument(
+        "--refs",
+        nargs=2,
+        metavar=("BASE", "HEAD"),
+        help="Compare two refs using 'git diff --name-only' to determine dirty files",
+    )
+    group.add_argument(
+        "--file",
+        metavar="PATH",
+        help="Read dirty file paths (one per line) from PATH",
+    )
+    group.add_argument(
+        "--stdin",
+        action="store_true",
+        help="Read dirty file paths (one per line) from stdin",
+    )
+    parser.add_argument(
+        "--summary",
+        metavar="PATH",
+        default=None,
+        help="Optional path to write a markdown summary table",
+    )
+    return parser.parse_args(argv)
+
+
+def load_config(path: Path) -> Config:
+    raw = yaml.safe_load(path.read_text(encoding="utf-8"))
+    projects_raw = raw.get("projects")
+    if not isinstance(projects_raw, dict) or not projects_raw:
+        raise SystemExit(f"No projects defined in {path}")
+
+    def to_tuple(value: Optional[Sequence[str] | str]) -> Tuple[str, ...]:
+        if value is None:
+            return tuple()
+        if isinstance(value, (list, tuple)):
+            return tuple(value)
+        return (value,)
+
+    project_keys: List[str] = list(projects_raw.keys())
+    projects: Dict[str, ProjectConfig] = {}
+    for key in project_keys:
+        entry = projects_raw.get(key) or {}
+        name = entry.get("name", key)
+        matrix_project = entry.get("matrix_project")
+        include_regexes = to_tuple(entry.get("include_regexes", []))
+        exclude_regexes = to_tuple(entry.get("exclude_regexes", []))
+        exclude_project_files = to_tuple(entry.get("exclude_project_files"))
+        lite_dependencies = to_tuple(entry.get("lite_dependencies"))
+        full_dependencies = to_tuple(entry.get("full_dependencies"))
+
+        if key != CORE_PROJECT_KEY and not include_regexes:
+            raise SystemExit(
+                f"Project '{key}' must define at least one include_regex in {path}"
+            )
+
+        projects[key] = ProjectConfig(
+            key=key,
+            name=name,
+            matrix_project=matrix_project,
+            include_regexes=include_regexes,
+            exclude_regexes=exclude_regexes,
+            exclude_project_files=exclude_project_files,
+            lite_dependencies=lite_dependencies,
+            full_dependencies=full_dependencies,
+            transitive_lite_dependencies=tuple(),
+        )
+
+    if CORE_PROJECT_KEY not in projects:
+        raise SystemExit(f"Project configuration must define '{CORE_PROJECT_KEY}'")
+
+    dependency_graph = build_dependency_graph_raw(projects)
+    transitive_map = compute_transitive_dependencies(
+        project_keys, projects, dependency_graph
+    )
+
+    for key, transitive in transitive_map.items():
+        projects[key] = replace(projects[key], transitive_lite_dependencies=transitive)
+
+    ignore_regexes = tuple(raw.get("ignore_regexes", ()))
+
+    return Config(
+        projects=projects,
+        project_keys=tuple(project_keys),
+        ignore_regexes=ignore_regexes,
+    )
+
+
+def run_git(args: Sequence[str], *, capture_output: bool = True) -> str:
+    """Run a git command in the repo and optionally capture stdout."""
+    result = subprocess.run(
+        ["git", *args],
+        cwd=REPO_ROOT,
+        check=True,
+        text=True,
+        stdout=subprocess.PIPE if capture_output else None,
+        stderr=subprocess.PIPE if capture_output else None,
+    )
+    if capture_output:
+        return result.stdout.strip()
+    return ""
+
+
+def try_rev_parse(ref: str) -> str:
+    try:
+        return run_git(["rev-parse", ref])
+    except subprocess.CalledProcessError:
+        return ref
+
+
+def ensure_fetched(ref: str) -> None:
+    """Fetch a ref from origin ensuring availability for merge-base."""
+    run_git(["fetch", "origin", ref, "-q"], capture_output=False)
+
+
+def repo_is_shallow() -> bool:
+    """Return True when the repository has a shallow history."""
+    output = run_git(["rev-parse", "--is-shallow-repository"])
+    return output.strip().lower() == "true"
+
+
+def indent(text: str, prefix: str) -> str:
+    """Indent multi-line text for readable logging blocks."""
+    return "\n".join(f"{prefix}{line}" for line in text.splitlines())
+
+
+def anchor_regex(pattern: str) -> re.Pattern[str]:
+    """Anchor a path regex to the repository root."""
+    anchored = pattern if pattern.startswith("^") else f"^{pattern}"
+    return re.compile(anchored)
+
+
+def compile_patterns(patterns: Sequence[str]) -> Tuple[re.Pattern[str], ...]:
+    """Compile a list of regex strings into anchored patterns."""
+    return tuple(anchor_regex(pattern) for pattern in patterns)
+
+
+def matches_any(patterns: Sequence[re.Pattern[str]], path: str) -> bool:
+    """Return True when any compiled regex matches the given path."""
+    return any(pattern.search(path) for pattern in patterns)
+
+
+def build_dependency_graph_raw(
+    projects: Dict[str, ProjectConfig],
+) -> Dict[str, List[Tuple[str, str]]]:
+    """Return mapping of project -> [(dependency, type)]."""
+    graph: Dict[str, List[Tuple[str, str]]] = {}
+    for key, project in projects.items():
+        edges: List[Tuple[str, str]] = []
+        edges.extend((dep, "full") for dep in project.full_dependencies)
+        edges.extend((dep, "lite") for dep in project.lite_dependencies)
+        graph[key] = edges
+    return graph
+
+
+def compute_transitive_dependencies(
+    project_keys: Sequence[str],
+    projects: Dict[str, ProjectConfig],
+    graph: Dict[str, List[Tuple[str, str]]],
+) -> Dict[str, Tuple[str, ...]]:
+    """Return a dictionary of transitive dependencies for each project."""
+    result: Dict[str, Tuple[str, ...]] = {}
+
+    for key in project_keys:
+        visited: set[str] = set()
+        queue: deque[str] = deque(dep for dep, _ in graph.get(key, []))
+
+        while queue:
+            dep = queue.popleft()
+            if dep == key or dep in visited:
+                continue
+            visited.add(dep)
+            queue.extend(child for child, _ in graph.get(dep, []))
+
+        project_cfg = projects[key]
+        direct_full = set(project_cfg.full_dependencies)
+        direct_lite = set(project_cfg.lite_dependencies)
+        transitive = [
+            dep
+            for dep in project_keys
+            if dep in visited and dep not in direct_full and dep not in direct_lite
+        ]
+
+        result[key] = tuple(transitive)
+
+    return result
+
+
+def project_dirty_files(
+    project: ProjectConfig, dirty_files: Sequence[str]
+) -> List[str]:
+    """Collect dirty files that belong to the given project. Does not apply project exclusions yet."""
+    include_patterns = compile_patterns(project.include_regexes)
+    exclude_patterns = compile_patterns(project.exclude_regexes)
+
+    if not include_patterns:
+        return []
+
+    included = [path for path in dirty_files if matches_any(include_patterns, path)]
+
+    if exclude_patterns:
+        included = [
+            path for path in included if not matches_any(exclude_patterns, path)
+        ]
+
+    return included
+
+
+def build_project_dirty_map(
+    config: Config, dirty_files: Sequence[str]
+) -> Dict[str, List[str]]:
+    """Compute per-project dirty file lists, including a residual `core` list."""
+    project_files: Dict[str, List[str]] = {}
+
+    # First gather matches for every non-core project. Files may belong to multiple projects.
+    for key in config.project_keys:
+        if key == CORE_PROJECT_KEY:
+            continue
+        files = project_dirty_files(config.project(key), dirty_files)
+        project_files[key] = files
+
+    # Remove files that are owned by other projects when requested.
+    for key in config.project_keys:
+        if key == CORE_PROJECT_KEY:
+            continue
+        project = config.project(key)
+        if not project.exclude_project_files:
+            continue
+        excluded_sets = [
+            set(project_files.get(other, [])) for other in project.exclude_project_files
+        ]
+        if not excluded_sets:
+            continue
+        exclusions = set().union(*excluded_sets)
+        project_files[key] = [
+            path for path in project_files[key] if path not in exclusions
+        ]
+
+    matched_paths: set[str] = set()
+    for key in config.project_keys:
+        if key == CORE_PROJECT_KEY:
+            continue
+        matched_paths.update(project_files.get(key, []))
+
+    core_files = [path for path in dirty_files if path not in matched_paths]
+    project_files[CORE_PROJECT_KEY] = core_files
+    return project_files
+
+
+def write_output(key: str, value: str) -> None:
+    """Emit a GitHub Actions output key/value pair."""
+    line = f"{key}={value}"
+    print(line)
+    output_path = os.environ.get("GITHUB_OUTPUT")
+    if output_path:
+        with open(output_path, "a", encoding="utf-8") as handle:
+            handle.write(f"{line}\n")
+
+
+def collect_dirty_files(
+    args: argparse.Namespace, config: Config
+) -> Tuple[List[str], List[str]]:
+    """Return normalized dirty file paths and the subset filtered as ignored."""
+    if args.refs:
+        if repo_is_shallow():
+            run_git(["fetch", "origin", "--unshallow", "-q"], capture_output=False)
+
+        base_ref = try_rev_parse(args.refs[0])
+        head_ref = try_rev_parse(args.refs[1])
+
+        ensure_fetched(base_ref)
+        ensure_fetched(head_ref)
+
+        base_sha = run_git(["merge-base", base_ref, head_ref])
+        head_sha = run_git(["rev-parse", head_ref])
+
+        print(f"Base SHA: {base_sha}")
+        base_log = run_git(["log", "--oneline", "-1", base_sha])
+        if base_log:
+            print(indent(base_log, "  "))
+        print(f"HEAD SHA: {head_sha}")
+        head_log = run_git(["log", "--oneline", "-1", head_sha])
+        if head_log:
+            print(indent(head_log, "  "))
+
+        dirty_files = run_git(["diff", "--name-only", base_sha, head_sha]).splitlines()
+    elif args.file:
+        print(f"Dirty files provided via file: {args.file}")
+        with open(args.file, "r", encoding="utf-8") as handle:
+            dirty_files = [line.strip() for line in handle if line.strip()]
+    else:  # args.stdin
+        print("Dirty files provided via stdin")
+        dirty_files = [line.strip() for line in sys.stdin if line.strip()]
+
+    print()
+
+    dirty_files = [path for path in dirty_files if path]
+
+    ignore_patterns = compile_patterns(config.ignore_regexes)
+    if ignore_patterns:
+        kept: List[str] = []
+        ignored: List[str] = []
+        for path in dirty_files:
+            if matches_any(ignore_patterns, path):
+                ignored.append(path)
+            else:
+                kept.append(path)
+        return kept, ignored
+
+    return dirty_files, []
+
+
+def format_bullet_list(lines: Sequence[str], indent: str = "  ") -> List[str]:
+    """Convert each string into a markdown bullet line with the given indent."""
+    return [f"{indent}- {line}" for line in lines]
+
+
+def build_reverse_dependency_graph(config: Config) -> Dict[str, List[Tuple[str, bool]]]:
+    """Return mapping of dependency -> (dependent, requires_full_rebuild)."""
+    reverse: Dict[str, List[Tuple[str, bool]]] = {}
+
+    for project in config.projects.values():
+        for dep in project.full_dependencies:
+            reverse.setdefault(dep, []).append((project.key, True))
+        for dep in project.lite_dependencies:
+            reverse.setdefault(dep, []).append((project.key, False))
+        for dep in project.transitive_lite_dependencies:
+            reverse.setdefault(dep, []).append((project.key, False))
+
+    return reverse
+
+
+def propagate_dirty_projects(
+    config: Config,
+    initial_full: Sequence[str],  # Projects with dirty files
+    reverse_graph: Dict[
+        str, List[Tuple[str, bool]]
+    ],  # dependency -> [(dependent, requires_full_rebuild)]
+) -> Tuple[set[str], set[str]]:  # (full_set, lite_set)
+    """Propagate rebuild requirements through the reverse dependency graph."""
+    full_set: set[str] = set(initial_full)
+    lite_set: set[str] = set()
+    # BFS queue of (project_key, depth)
+    queue: deque[Tuple[str, int]] = deque((key, 0) for key in initial_full)
+    seen: set[Tuple[str, int]] = set(queue)
+
+    while queue:
+        current, depth = queue.popleft()
+        for dependent, edge_full in reverse_graph.get(current, []):
+            propagate_full = edge_full and depth == 0
+
+            if propagate_full:
+                if dependent in full_set:
+                    continue
+                if dependent in lite_set:
+                    lite_set.remove(dependent)
+                full_set.add(dependent)
+                print(
+                    "- Upstream dependency change detected (full rebuild): "
+                    f"'{dependent}' ({config.project(dependent).name}) depends on dirty project '{current}'"
+                )
+            else:
+                if dependent in full_set or dependent in lite_set:
+                    continue
+                lite_set.add(dependent)
+                print(
+                    "- Upstream dependency change detected (lite rebuild): "
+                    f"'{dependent}' ({config.project(dependent).name}) depends on dirty project '{current}'"
+                )
+
+            key = (dependent, depth + 1)
+            if key not in seen:
+                seen.add(key)
+                queue.append(key)
+
+    return full_set, lite_set
+
+
+def log_dependency_overview(config: Config) -> None:
+    """Pretty-print the dependency data for each project."""
+    print("Project Dependency Overview:")
+    for key in config.project_keys:
+        project = config.project(key)
+        header = f"- {project.name} (key={key}"
+        if project.matrix_project:
+            header += f" matrix_project={project.matrix_project}"
+        header += ")"
+        print(header)
+
+        if project.full_dependencies:
+            print("  - direct full deps:")
+            for line in format_bullet_list(project.full_dependencies, indent="      "):
+                print(line)
+        if project.lite_dependencies:
+            print("  - direct lite deps:")
+            for line in format_bullet_list(project.lite_dependencies, indent="      "):
+                print(line)
+        if project.transitive_lite_dependencies:
+            print("  - transitive deps:")
+            for line in format_bullet_list(
+                project.transitive_lite_dependencies, indent="      "
+            ):
+                print(line)
+    print()
+
+
+def build_dirty_sections(
+    config: Config,
+    project_dirty_map: Dict[str, List[str]],
+    ignored_files: Sequence[str],
+) -> List[Tuple[str, List[str]]]:
+    """Create (heading, files) tuples for non-empty dirty buckets."""
+    sections: List[Tuple[str, List[str]]] = []
+    for key in config.project_keys:
+        if key == CORE_PROJECT_KEY:
+            continue
+        files = project_dirty_map.get(key, [])
+        if files:
+            sections.append((f"{config.project(key).name} ({key})", list(files)))
+
+    core_files = project_dirty_map.get(CORE_PROJECT_KEY, [])
+    if core_files:
+        sections.append(
+            (
+                f"{config.project(CORE_PROJECT_KEY).name} ({CORE_PROJECT_KEY})",
+                list(core_files),
+            )
+        )
+
+    if ignored_files:
+        sections.append(("Ignored Files", list(ignored_files)))
+
+    return sections
+
+
+def log_dirty_files(
+    combined_dirty: Sequence[str],
+    sections: Sequence[Tuple[str, Sequence[str]]],
+) -> None:
+    """Emit dirty-file information in markdown list form."""
+    if sections:
+        print("Dirty files by project:")
+        for idx, (heading, files) in enumerate(sections):
+            print(f"{heading}:")
+            for line in format_bullet_list(files):
+                print(line)
+            if idx != len(sections) - 1:
+                print()
+        print()
+
+    print("All dirty files:")
+    if combined_dirty:
+        for line in format_bullet_list(combined_dirty):
+            print(line)
+    else:
+        print("  - (none)")
+    print()
+
+
+def determine_rebuild_sets(
+    config: Config,
+    project_dirty_map: Dict[str, List[str]],  # key -> dirty files
+    reverse_graph: Dict[
+        str, List[Tuple[str, bool]]
+    ],  # dependency -> [(dependent, requires_full_rebuild)]
+) -> Tuple[
+    Dict[str, str], set[str], set[str]
+]:  # (project_statuses, full_set, lite_set)
+    """Return project statuses plus the full/lite rebuild sets."""
+    core_dirty_files = project_dirty_map[CORE_PROJECT_KEY]
+
+    if core_dirty_files:
+        project_statuses: Dict[str, str] = {key: "Dirty" for key in config.project_keys}
+        return project_statuses, set(config.project_keys), set()
+
+    initially_dirty = [
+        key
+        for key in config.project_keys
+        if key != CORE_PROJECT_KEY and project_dirty_map.get(key)
+    ]
+    for key in initially_dirty:
+        print(f"- Changes detected in subproject '{key}' ({config.project(key).name})")
+
+    full_set, lite_set = propagate_dirty_projects(
+        config,
+        initially_dirty,
+        reverse_graph,
+    )
+
+    project_statuses: Dict[str, str] = {}
+    for key in config.project_keys:
+        if key in full_set:
+            project_statuses[key] = "Dirty"
+        elif key in lite_set:
+            project_statuses[key] = "Dirty Deps"
+        else:
+            project_statuses[key] = "Clean"
+
+    return project_statuses, full_set, lite_set
+
+
+def compute_outputs(
+    config: Config,
+    full_set: set[str],
+    lite_set: set[str],
+) -> Tuple[List[str], List[str]]:  # (FULL_BUILD, LITE_BUILD) matrix_project lists
+    """Convert rebuild sets into ordered matrix project lists."""
+    full_output = [
+        config.project(key).matrix_project
+        for key in config.project_keys
+        if key in full_set and config.project(key).matrix_project
+    ]
+    lite_output = [
+        config.project(key).matrix_project
+        for key in config.project_keys
+        if key in lite_set and config.project(key).matrix_project
+    ]
+    return full_output, lite_output
+
+
+def emit_outputs(full_output: Sequence[str], lite_output: Sequence[str]) -> None:
+    """Write FULL_BUILD/LITE_BUILD strings to stdout and GitHub outputs."""
+    print("Github Action Outputs:")
+    write_output("FULL_BUILD", " ".join(full_output))
+    write_output("LITE_BUILD", " ".join(lite_output))
+    print()
+
+
+def write_project_summary(
+    config: Config,
+    project_statuses: Dict[str, str],
+    writer: SummaryWriter,
+) -> None:
+    """Render the status table inside the summary section."""
+    name_width = max(len(config.project(key).name) for key in config.project_keys)
+    writer.log(f"| {'Project':<{name_width}} | Status     |")
+    writer.log(f"|{'-' * (name_width + 2)}|------------|")
+    for key in config.project_keys:
+        project = config.project(key)
+        status = project_statuses.get(key, "Unknown?")
+        writer.log(f"| {project.name:<{name_width}} | {status:<10} |")
+    writer.log()
+
+
+def write_summary_dirty_sections(
+    combined_dirty: Sequence[str],
+    sections: Sequence[Tuple[str, Sequence[str]]],
+    writer: SummaryWriter,
+) -> None:
+    """Render dirty-file details inside the summary section."""
+    writer.log("<details><summary><h4>👉 Dirty Files</h4></summary>")
+    if sections:
+        for idx, (heading, files) in enumerate(sections):
+            writer.log()
+            writer.log(f"{heading}:")
+            for line in format_bullet_list(files):
+                writer.log(line)
+
+    writer.log()
+    if combined_dirty:
+        writer.log("All dirty files:")
+        for line in format_bullet_list(combined_dirty):
+            writer.log(line)
+
+    writer.log()
+    writer.log("</details>")
+
+
+def main(argv: Optional[Sequence[str]] = None) -> int:
+    """Entrypoint used by the GitHub Action wrapper."""
+    args = parse_args(argv)
+    config = load_config(CONFIG_PATH)
+
+    summary_path = Path(args.summary) if args.summary else None
+
+    dirty_files, ignored_files = collect_dirty_files(args, config)
+
+    project_dirty_map = build_project_dirty_map(config, dirty_files)
+    reverse_graph = build_reverse_dependency_graph(config)
+
+    print(f"Subprojects: {' '.join(config.project_keys)}")
+    print()
+
+    log_dependency_overview(config)
+
+    combined_dirty = dirty_files + ignored_files
+    sections = build_dirty_sections(config, project_dirty_map, ignored_files)
+    log_dirty_files(combined_dirty, sections)
+
+    print("Checking for changes...")
+
+    project_statuses, full_set, lite_set = determine_rebuild_sets(
+        config,
+        project_dirty_map,
+        reverse_graph,
+    )
+
+    full_output, lite_output = compute_outputs(config, full_set, lite_set)
+
+    print()
+    emit_outputs(full_output, lite_output)
+
+    with SummaryWriter(summary_path) as summary_writer:
+        print("::group::Project Change Summary")
+        summary_writer.log(
+            "<details><summary><h3>👃 Inspect Project Changes</h3></summary>"
+        )
+        summary_writer.log()
+        write_project_summary(config, project_statuses, summary_writer)
+        write_summary_dirty_sections(combined_dirty, sections, summary_writer)
+        summary_writer.log("</details>")
+        print("::endgroup::")
+
+    return 0
+
+
+if __name__ == "__main__":  # pragma: no cover
+    sys.exit(main())

cccl_upstream/ci/install_cccl.sh

@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+
+set -eo pipefail
+
+target_dir=$(realpath "$1")
+mkdir -p "$target_dir"
+
+# Move script to the root directory of the project
+cd "$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )";
+
+source ./pretty_printing.sh
+
+# Check if the correct number of arguments has been provided
+function usage {
+    echo "Usage: $0 [OPTIONS] dir"
+    echo
+    echo "Installs CCCL to the provided directory"
+
+    echo "Options:"
+    echo "  -v/-verbose: Enable shell echo for debugging"
+    echo
+    echo "Examples:"
+    echo "  $ $0 ~/my/prefix"
+    exit 1
+}
+
+while [[ "$#" -gt 0 ]]; do
+    case "$1" in
+        --verbose) VERBOSE=true; ;;
+        -v) VERBOSE=true; ;;
+        *) break ;;
+    esac
+    shift
+done
+
+if [[ -n "$VERBOSE" ]]; then
+    set -x
+fi
+
+# Move to cccl/ dir
+pushd ".." > /dev/null
+GROUP_NAME="🛠️  CMake Configure CCCL - Install"
+run_command "$GROUP_NAME" cmake -G "Unix Makefiles" --preset install -DCMAKE_INSTALL_PREFIX="${target_dir}"
+status=$?
+
+GROUP_NAME="🏗️  Install CCCL"
+run_command "$GROUP_NAME" cmake --build --preset install --target install
+status=$?
+popd > /dev/null

cccl_upstream/ci/install_packaging.sh

@@ -0,0 +1,117 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+# Time limit for installation steps (seconds)
+readonly install_time_limit=20
+
+ci_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+source "$ci_dir/pretty_printing.sh"
+
+cd "$ci_dir/.."
+
+# Prefixes for installations
+prefix_default=$(mktemp -d /tmp/cccl-default-XXXX)
+prefix_preset=$(mktemp -d /tmp/cccl-preset-XXXX)
+prefix_script=$(mktemp -d /tmp/cccl-script-XXXX)
+prefix_preset_unstable=$(mktemp -d /tmp/cccl-preset-unstable-XXXX)
+prefix_preset_unstable_only=$(mktemp -d /tmp/cccl-preset-unstable-only-XXXX)
+
+# Default configure + install
+default_start_time=$SECONDS
+run_command "Configure default" \
+  cmake -S . -B build/default -DCMAKE_INSTALL_PREFIX="$prefix_default"
+run_command "Install default" \
+  cmake --build build/default --target install
+default_time=$((SECONDS - default_start_time))
+
+# Preset configure + install
+preset_start_time=$SECONDS
+CCCL_BUILD_INFIX=preset \
+run_command "Configure preset" \
+  cmake --preset install -DCMAKE_INSTALL_PREFIX="$prefix_preset"
+CCCL_BUILD_INFIX=preset \
+run_command "Install preset" \
+  cmake --build --preset install --target install
+preset_time=$((SECONDS - preset_start_time))
+
+# Preset configure + install-unstable
+preset_unstable_start_time=$SECONDS
+CCCL_BUILD_INFIX=preset_unstable \
+run_command "Configure preset-unstable" \
+  cmake --preset install-unstable -DCMAKE_INSTALL_PREFIX="$prefix_preset_unstable"
+CCCL_BUILD_INFIX=preset_unstable \
+run_command "Install preset-unstable" \
+  cmake --build build/preset_unstable/install-unstable --target install
+preset_unstable_time=$((SECONDS - preset_unstable_start_time))
+
+# Preset configure + install-unstable-only
+preset_unstable_only_start_time=$SECONDS
+CCCL_BUILD_INFIX=preset_unstable_only \
+run_command "Configure preset-unstable-only" \
+  cmake --preset install-unstable-only -DCMAKE_INSTALL_PREFIX="$prefix_preset_unstable_only"
+CCCL_BUILD_INFIX=preset_unstable_only \
+run_command "Install preset-unstable-only" \
+  cmake --build build/preset_unstable_only/install-unstable-only \
+        --target install
+preset_unstable_only_time=$((SECONDS - preset_unstable_only_start_time))
+
+# Script install
+script_start_time=$SECONDS
+CCCL_BUILD_INFIX=script \
+run_command "Install script" \
+  ci/install_cccl.sh "$prefix_script"
+script_time=$((SECONDS - script_start_time))
+
+# Compare directories
+if ! diff -ruN "$prefix_default" "$prefix_preset" > /tmp/diff_default_preset.log; then
+  cat /tmp/diff_default_preset.log
+  echo -e "\e[0;31mDefault and preset installations differ\e[0m"
+  exit 1
+fi
+if ! diff -ruN "$prefix_default" "$prefix_script" > /tmp/diff_default_script.log; then
+  cat /tmp/diff_default_script.log
+  echo -e "\e[0;31mDefault and script installations differ\e[0m"
+  exit 1
+fi
+
+# Verify unstable-only contains only cudax headers and cmake config
+if find "$prefix_preset_unstable_only" -type f | \
+    sed "s|$prefix_preset_unstable_only/||" | \
+    grep -vE '^include/cuda/experimental/|^lib/cmake/+cudax/|^lib/cmake/+cccl/' | \
+    grep -q .; then
+  echo -e "\e[0;31minstall-unstable-only contained unexpected files\e[0m" >&2
+  exit 1
+fi
+
+# Verify install-unstable is union of install and install-unstable-only
+prefix_union=$(mktemp -d /tmp/cccl-union-XXXX)
+cp -a "$prefix_default/." "$prefix_union/"
+cp -a "$prefix_preset_unstable_only/." "$prefix_union/"
+if ! diff -ruN "$prefix_union" "$prefix_preset_unstable" > /tmp/diff_union_unstable.log; then
+  echo -e "\e[0;31mUnion of install and install-unstable-only did not match install-unstable\e[0m" >&2
+  cat /tmp/diff_union_unstable.log
+  exit 1
+fi
+
+print_time_summary
+
+# Verify times
+violations=()
+(( default_time > install_time_limit )) && violations+=("default_time=${default_time}s")
+(( preset_time > install_time_limit )) && violations+=("preset_time=${preset_time}s")
+(( preset_unstable_time > install_time_limit )) && violations+=("preset_unstable_time=${preset_unstable_time}s")
+(( preset_unstable_only_time > install_time_limit )) && violations+=("preset_unstable_only_time=${preset_unstable_only_time}s")
+(( script_time > install_time_limit )) && violations+=("script_time=${script_time}s")
+
+if (( ${#violations[@]} > 0 )); then
+  echo -e "\e[0;31mInstallation time limit (${install_time_limit}s) exceeded by: ${violations[*]}\e[0m" >&2
+  exit 1
+fi
+
+# Clean up
+rm -rf "$prefix_default" "$prefix_preset" "$prefix_script" \
+       "$prefix_preset_unstable" "$prefix_preset_unstable_only" \
+       "$prefix_union"
+
+echo -e "\e[0;32mAll installation tests passed successfully.\e[0m"

cccl_upstream/ci/matrix.yaml

@@ -0,0 +1,816 @@
+compile_time:
+  pull_request:
+    - id: public-headers-gcc13
+      name: Public headers compile-time bench
+      gpu: rtx2080
+      launch_args: "--cuda 13.3 --host gcc13"
+      baseline_ref: origin/main
+      preset: all-dev
+      targets:
+        - cub.headers.base
+        - thrust.cpp.cuda.headers.base
+        - libcudacxx.test.public_headers
+      args: "-arch native"
+      slices:
+        - id: total-compilation
+          title: TU total compilation
+          filter: total-compilation
+          timing: inclusive
+          sort: total
+          top: 15
+          threshold: 3.0
+        - id: file-processing
+          title: Direct file processing
+          filter: file-processing
+          timing: exclusive
+          sort: total
+          top: 15
+          threshold: 0.2
+        - id: function-bodies
+          title: Function-body parsing
+          filter: scanning-function-body
+          timing: inclusive
+          sort: total
+          top: 15
+          threshold: 0.25
+        - id: template-instantiation
+          title: Template instantiation
+          filter: template-instantiation
+          timing: inclusive
+          sort: total
+          top: 15
+          threshold: 0.2
+
+workflows:
+  # If any jobs appear here, they will be executed instead of `pull_request' for PRs.
+  # This is useful for limiting resource usage when a full matrix is not needed.
+  # The branch protection checks will fail when using this override workflow.
+  #
+  # Example:
+  # override:
+  #   # Full project build: slow, expensive
+  #   - { jobs: ['test'], project: 'thrust', std: 17, ctk: '12.X', cxx: ['gcc12', 'clang16'] }
+  #
+  #   # Build / run targeted tests: faster turnaround, less runner usage.
+  #   # Use project 'target'.
+  #   # args are passed to ci/util/build_and_test_targets.sh. See that script for available options.
+  #   - { jobs: ['run_cpu'], project: 'target', ctk: ['12.X', '13.X'], cxx: ['gcc', 'clang', 'msvc'],
+  #       args: '--preset cub-cpp20 --build-targets "cub.cpp20.test.iterator"' }
+  #   - { jobs: ['run_gpu'], project: 'target', ctk: ['12.X', '13.X'], cxx: ['gcc', 'clang'], gpu: 'rtxa6000',
+  #       args: '--preset cub-cpp20 --build-targets "cub.cpp20.test.iterator" --ctest-targets "cub.cpp20.test.iterator"' }
+  #   - { jobs: ['run_cpu'], project: 'target', ctk: ['12.X', '13.X'], cxx: ['gcc', 'clang', 'msvc'],
+  #       args: '--preset libcudacxx --lit-precompile-tests "cuda/utility/basic_any.pass.cpp"' }
+  #   - { jobs: ['run_gpu'], project: 'target', ctk: ['12.X', '13.X'], cxx: ['gcc', 'clang'], gpu: 'rtx2080',
+  #       args: '--preset libcudacxx --lit-tests "cuda/utility/basic_any.pass.cpp"' }
+  #
+  # IMPORTANT: Do NOT delete or remove the `override:` key below, even when it is empty.
+  override:
+
+  pull_request:
+    # Old CTK: Oldest/newest supported host compilers:
+    - {jobs: ['build'], std: 'minmax', ctk: '12.0', cxx: ['gcc12', 'clang14',            'msvc2019', 'msvc14.39']}
+    - {jobs: ['build'], project: ['libcudacxx', 'thrust'], std: 'minmax', ctk: '12.0', cxx: 'gcc7'}
+    - {jobs: ['build_nolid', 'build_lid1', 'build_lid2'], project: 'cub', std: 'minmax', ctk: '12.0', cxx: 'gcc7'}
+    # CTK12.0/GCC7 CUB host-launch builds are memory-heavy with benchmarks enabled; keep this shard below
+    # the 61 GiB linux-amd64-cpu16 runner limit.
+    - {jobs: ['build_lid0'], project: 'cub', std: 'minmax', ctk: '12.0', cxx: 'gcc7', environment: ['PARALLEL_LEVEL=8']}
+    - {jobs: ['build'], std: 'minmax', ctk: '12.X', cxx: ['gcc7',  'gcc14',   'clang14', 'clang19', 'msvc2019', 'msvc2022' ]}
+    - {jobs: ['build'], std: 'minmax', ctk: '13.0', cxx: ['gcc11', 'gcc15',   'clang15', 'clang20', 'msvc2019', 'msvc2022' ]}
+    # Old CTK: cudax has a different support matrix:
+    - {jobs: ['build'], project: 'cudax', ctk: '12.0', std: 'minmax', cxx: ['gcc9',  'gcc12', 'clang14',            'msvc14.39']}
+    - {jobs: ['build'], project: 'cudax', ctk: '12.X', std: 'minmax', cxx: ['gcc9',  'gcc14',   'clang14', 'clang19', 'msvc2022']}
+    - {jobs: ['build'], project: 'cudax', ctk: '13.0', std: 'minmax', cxx: ['gcc11', 'gcc15',   'clang15', 'clang20', 'msvc2022']}
+    # Current CTK build-only:
+    - {jobs: ['build'], std: 'minmax', cxx: ['gcc11', 'clang15', 'msvc2019'] } # Oldest
+    - {jobs: ['build'], std: 'max',    cxx: ['gcc12', 'gcc13', 'gcc14'] }
+    - {jobs: ['build'], std: 'max',    cxx: ['clang16', 'clang17', 'clang18', 'clang19', 'clang20'] }
+    - {jobs: ['build'], std: 'max',    cxx: ['msvc2022'] }
+    - {jobs: ['build'], std: 'all',    cxx: ['gcc', 'clang', 'msvc']} # Latest
+    # Current CTK build-only: cudax has a different support matrix:
+    - {jobs: ['build'], project: 'cudax', std: 'minmax', cxx: ['gcc11', 'clang15', 'msvc2022']} # Oldest
+    - {jobs: ['build'], project: 'cudax', std: 'max',    cxx: ['gcc12']}
+    - {jobs: ['build'], project: 'cudax', std: 'max',    cxx: ['clang16', 'clang17', 'clang18', 'clang19', 'clang20']}
+    - {jobs: ['build'], project: 'cudax', std: 'all',    cxx: ['gcc', 'clang', 'msvc']} # Newest
+    # Current CTK testing:
+    - {jobs: ['test'], project: 'thrust',     std: 'max', cxx: ['gcc', 'clang', 'msvc'], gpu: 'rtx4090'}
+    - {jobs: ['test'], project: ['libcudacxx', 'cudax'], std: 'max', cxx: ['gcc', 'clang', 'msvc'], gpu: 't4'}
+
+    - {jobs: ['test_nolid', 'test_lid0'], project: 'cub', std: 'max', cxx: ['gcc', 'clang', 'msvc'], gpu: 'rtxa6000'}
+    - {jobs: ['test_lid1',  'test_lid2'], project: 'cub', std: 'max', cxx: ['gcc'],                  gpu: 'rtxa6000'}
+    # H100 coverage:
+    - {jobs: ['test_nolid', 'test_lid0'], project: 'cub',                   std: 'max', gpu: 'h100' }
+    - {jobs: ['test_gpu'],                project: 'thrust',                std: 'max', gpu: 'h100' }
+    - {jobs: ['test'],                    project: ['libcudacxx', 'cudax'], std: 'max', gpu: 'h100' }
+    # Multi-GPU coverage:
+    - {jobs: ['test'], project: ['libcudacxx', 'cudax'], std: 'max', gpu: 'h100_2gpu', sm: 'gpu'}
+    # RTX PRO 6000 coverage (limited due to small number of runners):
+    - {jobs: ['test_lid0'], project: 'cub', std: 'max', cxx: 'gcc', gpu: 'rtxpro6000'}
+    # Misc:
+    - {jobs: ['build'], cpu: 'arm64', project: ['libcudacxx', 'cub', 'thrust', 'cudax'], std: 'max', cxx: ['gcc', 'clang']}
+    - {jobs: ['test_gpu'], project: 'thrust', cmake_options: '-DTHRUST_DISPATCH_TYPE=Force32bit', gpu: 'rtx4090'}
+    - {jobs: ['nvrtc'], project: 'libcudacxx', std: 'all', gpu: 'rtx2080', sm: 'gpu'}
+    - {jobs: ['verify_codegen'], project: 'libcudacxx'}
+    # c.parallel -- pinned to gcc13 / msvc2022 to match python
+    - {jobs: ['test'], project: 'cccl_c_parallel', ctk: '12.X', cxx: ['gcc13', 'msvc2022'], gpu: ['t4']}
+    - {jobs: ['test'], project: 'cccl_c_parallel', ctk: '13.X', cxx: ['gcc13', 'msvc2022'], gpu: ['rtx2080', 'l4', 'h100']}
+    # RTX PRO 6000 coverage (limited due to small number of runners):
+    - {jobs: ['test'], project: 'cccl_c_parallel', ctk: '13.X', cxx: ['gcc13'], gpu: ['rtxpro6000']}
+    # c.parallel v2 (HostJIT-based)
+    #
+    # For now, this is a separate job run for Linux/CUDA13.
+    # Eventually v2 will replace v1 as the default and run across the
+    # entire matrix. Currently blocked on libnvfatbin availability on
+    # Windows containers, and for CUDA <12.4.
+    - {jobs: ['test'], project: 'cccl_c_parallel_v2', ctk: '13.X', cxx: ['gcc13', 'msvc'], gpu: 'rtx2080'}
+    # Python against c.parallel v2 (HostJIT-based). Single point of coverage
+    # for the v2 Python path; the main `python` matrix continues to test
+    # against v1 until v2 replaces it.
+    - {jobs: ['test'], project: 'python_v2', ctk: '13.X', py_version: '3.14', gpu: 'l4', cxx: 'gcc13'}
+    - {jobs: ['test_py_compute_minimal'], project: 'python_v2', ctk: '13.X', py_version: '3.14t', gpu: 'l4', cxx: 'gcc13'}
+    # c.experimental.stf-- pinned to gcc13 to match python
+    - {jobs: ['test'], project: 'cccl_c_stf', ctk: ['12.X', '13.X'], cxx: 'gcc13', gpu: 't4'}
+    - {jobs: ['test'], project: 'cccl_c_stf', ctk: '13.X', cxx: 'gcc13', gpu: ['l4', 'h100']}
+    # Python -- pinned to gcc13 / msvc2022 for consistency across CTK images
+    - {jobs: ['test'], project: 'python', ctk: ['12.X',        '13.X'], py_version: ['3.10'], gpu: 'l4', cxx: ['gcc13', 'msvc2022']}
+    - {jobs: ['test'], project: 'python', ctk: ['12.0', '12.X','13.0', '13.X'], py_version: '3.14', gpu: 'l4', cxx: ['gcc13', 'msvc2022']}
+    - {jobs: ['test'], project: 'python', py_version: '3.14', gpu: 'h100', cxx: 'gcc13'}
+    - {jobs: ['test'], project: 'python', ctk: '13.X', py_version: '3.14', gpu: ['t4', 'rtxa6000', 'rtxpro6000'], cxx: 'gcc13'}
+    - {jobs: ['test_py_compute_minimal'], project: 'python', ctk: '13.X', py_version: '3.14', gpu: 'l4', cxx: 'gcc13'}
+    - {jobs: ['test_py_compute_minimal'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: '3.14t', gpu: 'l4', cxx: ['gcc13', 'msvc2022']}
+    - {jobs: ['test_py_compute_minimal'], project: 'python_tsan', ctk: '13.X', py_version: '3.14t', gpu: 'l4', cxx: 'gcc13'}
+    # Deliberately unpinned: py_ctk_mode 'latest' skips the CTK pin so pip
+    # resolves the latest minor -- catching breakage a plain `pip install
+    # cuda-cccl[cu12]/[cu13]` would hit before the container CTK bumps.
+    - {jobs: ['test'], project: 'python', ctk: ['12.X', '13.X'], py_version: '3.14', gpu: 'l4', cxx: ['gcc13', 'msvc2022'], py_ctk_mode: 'latest'}
+    # sysctk: install the sysctk extras (system-provided CTK, no pip cuda-toolkit)
+    # rather than cu*.
+    # Full test on 3.14, minimal (numba-free) on 3.14t; all CTKs, Linux+Windows.
+    - {jobs: ['test'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: '3.14', gpu: 'l4', cxx: ['gcc13', 'msvc2022'], py_ctk_mode: 'sysctk'}
+    - {jobs: ['test_py_compute_minimal'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: '3.14t', gpu: 'l4', cxx: ['gcc13', 'msvc2022'], py_ctk_mode: 'sysctk'}
+    # cuda.cccl.headers: CPU-only (gpu:false), arch-insensitive
+    - {jobs: ['test_headers'], project: 'python', ctk: ['12.X', '13.X'], py_version: '3.14', cxx: ['gcc13', 'msvc2022'], py_ctk_mode: ['pinned', 'latest', 'sysctk']}
+    # CCCL packaging:
+    - {jobs: ['test'], project: 'packaging', ctk: '12.0', cxx: ['gcc10', 'clang14'], gpu: 't4', args: '-min-cmake'}
+    - {jobs: ['test'], project: 'packaging', ctk: '12.X', cxx: ['gcc10', 'clang14'], gpu: 't4'}
+    - {jobs: ['test'], project: 'packaging', ctk: '13.0', cxx: ['gcc15', 'clang20'], gpu: 't4', args: '-min-cmake'}
+    - {jobs: ['test'], project: 'packaging', ctk: '13.X', cxx: ['gcc',   'clang'],   gpu: 't4'}
+    - {jobs: ['install'], project: 'packaging'}
+    # NVBench Helper testing:
+    - {jobs: ['test'], project: 'nvbench_helper', ctk: ['12.0', '12.X'], cxx: ['gcc10', 'clang14'], gpu: 't4'}
+    - {jobs: ['test'], project: 'nvbench_helper', ctk: '13.0', cxx: ['gcc15', 'clang20'], gpu: 't4'}
+    - {jobs: ['test'], project: 'nvbench_helper', ctk: '13.X', cxx: ['gcc',   'clang'],   gpu: 't4'}
+    # NVHPC build
+    - {jobs: ['build'], cxx: 'nvhpc', ctk: 'nvhpc', std: 'max', project: ['libcudacxx', 'thrust', 'stdpar'], cpu: 'amd64'}
+    - {jobs: ['build_nolid'], cxx: 'nvhpc', ctk: 'nvhpc', std: 'max', project: 'cub', cpu: 'amd64'}
+    # clang-cuda
+    - {jobs: ['build'], cudacxx: 'clang', ctk: 'clang-cuda', cxx: 'clang-cuda',  std: 'all', sm: '75;80;90;100;120'}
+    - {jobs: ['build'], project: 'libcudacxx', cudacxx: 'clang', ctk: 'clang-cuda', cxx: 'clang-cuda',  std: 23, sm: '75;80;90;100;120'}
+    # libc++
+    # - arm64 for now as it's closest to android.
+    # - {jobs: ['build'], cpu: 'arm64', project: 'libcudacxx', std: 'all', cudacxx: 'clang', ctk: 'clang-cuda', cxx: 'clang-cuda', cmake_options: '-DCCCL_USE_LIBCXX=ON', sm: '75;80;90;100;120'}
+    # clang-tidy
+    #
+    # The precise value of sm is not important, but it is required for cmake to identify
+    # clang as a CUDA compiler (see
+    # https://discourse.cmake.org/t/cmake-cuda-clang-fails/8657/5).
+    #
+    # Standard being exactly "min" is required. clang-tidy may emit additional diagnostics
+    # for later C++ versions (for example, warning that you should use designated
+    # initializers in C++20 or higher).
+    - { jobs: ['build'], project: 'tidy', std: 'min', cxx: ['clang'], cudacxx: ['clang'], ctk: 'clang-cuda', sm: '75' }
+
+  # Used when an upstream project changes to reduce time spent smoke testing dependencies.
+  pull_request_lite:
+    # libcudacxx - Specialized, testing default SM
+    - {project: 'libcudacxx', jobs: ['test'],  std: 'max', cxx: ['gcc', 'msvc'], gpu: 'rtx2080', sm: 'gpu'}
+    - {project: 'libcudacxx', jobs: ['build'], std: 'max', cxx: 'clang'}
+    - {project: 'libcudacxx', jobs: ['build'], std: 'max', ctk: 'nvhpc', cxx: 'nvhpc'}
+    - {project: 'libcudacxx', jobs: ['build'], std: 'max', cudacxx: 'clang', ctk: 'clang-cuda', cxx: 'clang-cuda', sm: '70;80;90;100;120'}
+    - {project: 'libcudacxx', jobs: ['nvrtc'], std: 'max', gpu: 't4', sm: 'gpu'}
+    - {project: 'libcudacxx', jobs: ['verify_codegen']}
+    # CUB - Specialized, testing default SM
+    - {project: 'cub', jobs: ['test_nolid', 'test_lid0'], std: 'max', cxx: ['gcc', 'msvc'], gpu: 'rtxa6000', sm: 'gpu'}
+    - {project: 'cub', jobs: ['build_nolid', 'build_lid0'], std: 'max', cxx: 'clang'}
+    - {project: 'cub', jobs: ['build_nolid', 'build_lid0'], std: 'max', ctk: 'nvhpc', cxx: 'nvhpc'}
+    - {project: 'cub', jobs: ['build_nolid', 'build_lid0'], std: 'max', cudacxx: 'clang', ctk: 'clang-cuda', cxx: 'clang-cuda', sm: '75;80;90;100;120'}
+    # Thrust - Keep number of sm small. Kernel coverage is in CUB. This just tests dispatch / glue in lite mode:
+    - {project: 'thrust', jobs: ['test'], std: 'max', cxx: ['gcc', 'msvc'], gpu: 'rtx4090', sm: 'gpu'}
+    - {project: 'thrust', jobs: ['build'], std: 'max', cxx: 'clang', sm: '75;120'}
+    - {project: 'thrust', jobs: ['build'], std: 'max', ctk: 'nvhpc', cxx: 'nvhpc', sm: '75;120'}
+    - {project: 'thrust', jobs: ['build'], std: 'max', cudacxx: 'clang', ctk: 'clang-cuda', cxx: 'clang-cuda', sm: '75;120'}
+    # cudax
+    - {project: 'cudax', jobs: ['test'], std: 'max', cxx: ['gcc', 'msvc'], gpu: 'rtx2080', sm: 'gpu'}
+    - {project: 'cudax', jobs: ['build'], std: 'max', cxx: 'clang', sm: '75;120'}
+    - {project: 'cudax', jobs: ['build'], std: 'max', ctk: 'nvhpc', cxx: 'nvhpc', sm: '75;120'}
+    # stdpar
+    - {project: 'stdpar', jobs: ['build'], std: 'max', ctk: 'nvhpc', cxx: 'nvhpc'}
+    # Python + support
+    - {project: 'cccl_c_parallel', jobs: ['test'], ctk: '13.X', cxx: ['gcc13', 'msvc2022'], gpu: 'rtx2080', sm: 'gpu'}
+    - {project: 'cccl_c_parallel', jobs: ['test'], ctk: '13.X', cxx: 'gcc13', gpu: 'rtxpro6000', sm: 'gpu'}
+    - {project: 'cccl_c_stf',      jobs: ['test'], ctk: '13.X', cxx: 'gcc13',               gpu: 't4', sm: 'gpu'}
+    - {project: 'python', jobs: ['test'], ctk: '13.X', py_version: '3.14', gpu: 'l4', cxx: ['gcc13', 'msvc2022']}
+    - {project: 'python', jobs: ['test_headers'], ctk: '13.X', py_version: '3.14', cxx: ['gcc13', 'msvc2022']}
+    # Packaging / install
+    - {project: 'packaging', jobs: ['test'], ctk: '13.X', cxx: ['gcc', 'clang'], gpu: 'rtx2080', sm: 'gpu'}
+    - {project: 'packaging', jobs: ['test'], args: '-min-cmake', gpu: 't4', sm: 'gpu'}
+    - {project: 'packaging', jobs: ['install']}
+    # NVBench Helper testing:
+    - {project: 'nvbench_helper', jobs: ['test'], ctk: '13.X', cxx: ['gcc', 'clang'], gpu: 'rtx2080'}
+    # c.parallel v2 (HostJIT-based)
+    - {jobs: ['test'], project: 'cccl_c_parallel_v2', ctk: '13.X', cxx: ['gcc13'], gpu: 'rtx2080'}
+    # Python against c.parallel v2 (HostJIT-based)
+    - {jobs: ['test'], project: 'python_v2', ctk: '13.X', py_version: '3.14', gpu: 'l4', cxx: 'gcc13'}
+
+  nightly:
+    # CTK 12.0 full matrix build: default projects
+    - {jobs: ['build'], std: 'all', ctk: '12.0', cxx: ['gcc7', 'gcc8', 'gcc9', 'gcc10', 'gcc11', 'gcc12']}
+    - {jobs: ['build'], std: 'all', ctk: '12.0', cxx: ['clang14']}
+    - {jobs: ['build'], std: 'all', ctk: '12.0', cxx: ['msvc2019', 'msvc14.39']}
+    # CTK 12.X full matrix build: default projects
+    - {jobs: ['build'], std: 'all', ctk: '12.X', cxx: ['gcc7', 'gcc8', 'gcc9', 'gcc10', 'gcc11', 'gcc12', 'gcc13', 'gcc14']}
+    - {jobs: ['build'], std: 'all', ctk: '12.X', cxx: ['clang14', 'clang15', 'clang16', 'clang17', 'clang18', 'clang19']}
+    - {jobs: ['build'], std: 'all', ctk: '12.X', cxx: ['msvc2019', 'msvc2022']}
+    # CTK 13.0 full matrix build: default projects
+    - {jobs: ['build'], std: 'all', ctk: '13.0', cxx: ['gcc11', 'gcc12', 'gcc13', 'gcc14', 'gcc15']}
+    - {jobs: ['build'], std: 'all', ctk: '13.0', cxx: ['clang15', 'clang16', 'clang17', 'clang18', 'clang19', 'clang20']}
+    - {jobs: ['build'], std: 'all', ctk: '13.0', cxx: ['msvc2019', 'msvc2022']}
+    # CTK '13.X' full matrix build: default projects
+    - {jobs: ['build'], std: 'all', ctk: '13.X', cxx: ['gcc11', 'gcc12', 'gcc13', 'gcc14', 'gcc15']}
+    - {jobs: ['build'], std: 'all', ctk: '13.X', cxx: ['clang15', 'clang16', 'clang17', 'clang18', 'clang19', 'clang20', 'clang21']}
+    - {jobs: ['build'], std: 'all', ctk: '13.X', cxx: ['msvc2019', 'msvc2022', 'msvc2026']}
+    # CTK 12.0 full matrix build: cudax
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '12.0', cxx: ['gcc9', 'gcc10', 'gcc11', 'gcc12']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '12.0', cxx: ['clang14']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '12.0', cxx: ['msvc14.39']}
+    # CTK 12.X full matrix build: cudax
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '12.X', cxx: ['gcc9', 'gcc10', 'gcc11', 'gcc12', 'gcc13', 'gcc14']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '12.X', cxx: ['clang14', 'clang15', 'clang16', 'clang17', 'clang18', 'clang19']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '12.X', cxx: ['msvc2022']}
+    # CTK 13.0 full matrix build: cudax
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '13.0', cxx: ['gcc11', 'gcc12', 'gcc13', 'gcc14', 'gcc15']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '13.0', cxx: ['clang15', 'clang16', 'clang17', 'clang18', 'clang19', 'clang20']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '13.0', cxx: ['msvc2022']}
+    # CTK '13.X' full matrix build: cudax
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '13.X', cxx: ['gcc11', 'gcc12', 'gcc13', 'gcc14', 'gcc15']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '13.X', cxx: ['clang15', 'clang16', 'clang17', 'clang18', 'clang19', 'clang20', 'clang21']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '13.X', cxx: ['msvc2022', 'msvc2026']}
+    # CTK 12.X testing:
+    - {jobs: ['test'], project: 'libcudacxx', ctk: '12.X', std: 'max', cxx: ['gcc14', 'clang19', 'msvc2022'], gpu: 't4'}
+    - {jobs: ['test'], project: 'cub',        ctk: '12.X', std: 'max', cxx: ['gcc14', 'clang19', 'msvc2022'], gpu: 'rtxa6000'}
+    - {jobs: ['test'], project: 'thrust',     ctk: '12.X', std: 'max', cxx: ['gcc14', 'clang19', 'msvc2022'], gpu: 'rtx4090'}
+    - {jobs: ['test'], project: 'cudax',      ctk: '12.X', std: 'max', cxx: ['gcc14', 'clang19', 'msvc2022'], gpu: 'rtx2080'}
+    - {jobs: ['test'], project: ['libcudacxx', 'cub', 'thrust', 'cudax'], ctk: '12.X', std: 'max', cxx: 'gcc14', gpu: 'h100' }
+    # CTK '13.X' testing:
+    - {jobs: ['test'], project: 'libcudacxx', ctk: '13.X', std: 'max', cxx: ['gcc', 'clang', 'msvc'], gpu: 'rtx2080'}
+    - {jobs: ['test'], project: 'cub',        ctk: '13.X', std: 'max', cxx: ['gcc', 'clang', 'msvc'], gpu: 'rtxa6000'}
+    - {jobs: ['test'], project: 'thrust',     ctk: '13.X', std: 'max', cxx: ['gcc', 'clang', 'msvc'], gpu: 'rtx4090'}
+    - {jobs: ['test'], project: 'cudax',      ctk: '13.X', std: 'max', cxx: ['gcc', 'clang', 'msvc'], gpu: 't4'}
+    - {jobs: ['test'], project: ['libcudacxx', 'cub', 'thrust', 'cudax'], ctk: '13.X', std: 'max', gpu: 'h100' }
+      # RTX PRO 6000 coverage (limited due to small number of runners):
+    - {jobs: ['test_nolid', 'test_lid0'], project: ['cub', 'thrust'], std: 'max', cxx: 'gcc', gpu: 'rtxpro6000'}
+    # Misc:
+    - {jobs: ['build'], cpu: 'arm64', project: ['libcudacxx', 'cub', 'thrust', 'cudax'], ctk: '12.X', std: 'all', cxx: ['gcc14', 'clang19']}
+    - {jobs: ['build'], cpu: 'arm64', project: ['libcudacxx', 'cub', 'thrust', 'cudax'], ctk: '13.X', std: 'all', cxx: ['gcc', 'clang']}
+    - {jobs: ['test_gpu'], project: 'thrust', cmake_options: '-DTHRUST_DISPATCH_TYPE=Force32bit', gpu: 'rtx4090'}
+    - {jobs: ['test_gpu'], project: 'thrust', cmake_options: '-DTHRUST_DISPATCH_TYPE=Force64bit', gpu: 'rtx4090'}
+    - {jobs: ['limited'], project: 'cub', std: 17, gpu: 'rtx2080'}
+    # NVRTC tests don't currently support 12.0:
+    - {jobs: ['nvrtc'],          project: 'libcudacxx', ctk: [        '12.X', '13.0', '13.X'], cxx: 'gcc12', std: 'all', gpu: 'rtx2080', sm: 'gpu'}
+    - {jobs: ['verify_codegen'], project: 'libcudacxx'}
+    # c.parallel -- pinned to gcc13 / msvc2022 to match python
+    - {jobs: ['test'],  project: ['cccl_c_parallel'], ctk: '12.X', cxx: ['gcc13', 'msvc2022'], gpu: ['t4']}
+    - {jobs: ['test'],  project: ['cccl_c_parallel'], ctk: '13.X', cxx: ['gcc13', 'msvc2022'], gpu: ['rtx2080', 'l4', 'h100']}
+    # RTX PRO 6000 coverage (limited due to small number of runners):
+    - {jobs: ['test'], project: 'cccl_c_parallel', ctk: '13.X', cxx: ['gcc13'], gpu: ['rtxpro6000']}
+    # c.experimental.stf -- pinned to gcc13 to match python
+    - {jobs: ['test'],  project: ['cccl_c_stf'], ctk: '12.X', cxx: 'gcc13', gpu: ['rtx2080']}
+    - {jobs: ['test'],  project: ['cccl_c_stf'], ctk: '13.X', cxx: 'gcc13', gpu: ['t4', 'l4', 'h100']}
+    # Python -- pinned to gcc13 / msvc2022 on Linux for consistency across CTK images
+    - {jobs: ['test'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: ['3.10', '3.11', '3.12', '3.13', '3.14'], gpu: 'l4', cxx: ['gcc13', 'msvc2022']}
+    - {jobs: ['test'], project: 'python', ctk: ['12.X', '13.X'], py_version: '3.14', gpu: 'h100', cxx: 'gcc13'}
+    - {jobs: ['test'], project: 'python', ctk: ['12.X', '13.X'], py_version: '3.14', gpu: ['t4', 'rtxa6000'], cxx: 'gcc13'}
+    - {jobs: ['test'], project: 'python', ctk: '13.X', py_version: '3.14', gpu: 'rtxpro6000', cxx: 'gcc13'}
+    # Python free-threaded (3.14t) minimal lanes -- mirrors the pull_request rows
+    # so FT regressions (e.g. from dependency bumps) surface between PRs.
+    - {jobs: ['test_py_compute_minimal'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: '3.14t', gpu: 'l4', cxx: ['gcc13', 'msvc2022']}
+    - {jobs: ['test_py_compute_minimal'], project: 'python_v2', ctk: '13.X', py_version: '3.14t', gpu: 'l4', cxx: 'gcc13'}
+    - {jobs: ['test_py_compute_minimal'], project: 'python_tsan', ctk: '13.X', py_version: '3.14t', gpu: 'l4', cxx: 'gcc13'}
+    - {jobs: ['test'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: '3.14', gpu: 'l4', cxx: ['gcc13', 'msvc2022'], py_ctk_mode: 'sysctk'}
+    - {jobs: ['test_py_compute_minimal'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: '3.14t', gpu: 'l4', cxx: ['gcc13', 'msvc2022'], py_ctk_mode: 'sysctk'}
+    # cuda.cccl.headers (CPU-only): all CTK x source x OS, py endpoints 3.10 + 3.14
+    - {jobs: ['test_headers'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: ['3.10', '3.14'], cxx: ['gcc13', 'msvc2022'], py_ctk_mode: ['pinned', 'latest', 'sysctk']}
+    # CCCL packaging:
+    - {jobs: ['test'], project: 'packaging', ctk: '12.0', cxx: ['gcc10', 'clang14'], gpu: 't4', args: '-min-cmake'}
+    - {jobs: ['test'], project: 'packaging', ctk: '12.X', cxx: ['gcc10', 'clang14'], gpu: 'rtx2080'}
+    - {jobs: ['test'], project: 'packaging', ctk: '13.0', cxx: ['gcc15', 'clang20'], gpu: 't4', args: '-min-cmake'}
+    - {jobs: ['test'], project: 'packaging', ctk: '13.X', cxx: ['gcc',   'clang'],   gpu: 'rtx2080'}
+    - {jobs: ['install'], project: 'packaging'}
+    # NVBench Helper testing:
+    - {jobs: ['test'], project: 'nvbench_helper', ctk: '12.0', cxx: ['gcc10', 'clang14'], gpu: 'rtx2080'}
+    - {jobs: ['test'], project: 'nvbench_helper', ctk: '12.X', cxx: ['gcc10', 'clang14'], gpu: 't4'}
+    - {jobs: ['test'], project: 'nvbench_helper', ctk: '13.0', cxx: ['gcc15', 'clang20'], gpu: 'rtx2080'}
+    - {jobs: ['test'], project: 'nvbench_helper', ctk: '13.X', cxx: ['gcc',   'clang'],   gpu: 't4'}
+    # NVHPC build
+    - {jobs: ['build'], cxx: 'nvhpc-prev', ctk: 'nvhpc-prev', std: 'all', project: ['libcudacxx', 'cub', 'thrust', 'cudax', 'stdpar'], cpu: ['amd64', 'arm64']}
+    - {jobs: ['build'], cxx: 'nvhpc',      ctk: 'nvhpc',      std: 'all', project: ['libcudacxx', 'cub', 'thrust', 'cudax', 'stdpar'], cpu: ['amd64', 'arm64']}
+    # clang-cuda
+    - {jobs: ['build'], cudacxx: 'clang', ctk: 'clang-cuda', cxx: 'clang-cuda', std: 'all', sm: '75;80;90;100;120'}
+    - {jobs: ['build'], cudacxx: 'clang', ctk: 'clang_preview-cuda', cxx: 'clang_preview-cuda', std: 'all', sm: '75;80;90;100;120'}
+    - {jobs: ['build'], project: 'libcudacxx', cudacxx: 'clang', ctk: 'clang-cuda', cxx: 'clang-cuda',  std: 23, sm: '75;80;90;100;120'}
+    - {jobs: ['build'], project: 'libcudacxx', cudacxx: 'clang', ctk: 'clang_preview-cuda', cxx: 'clang_preview-cuda', std: 23, sm: '75;80;90;100;120'}
+    # clang-tidy
+    - { jobs: ['build'], project: 'tidy', std: 'min', cxx: ['clang'], cudacxx: ['clang'], ctk: 'clang-cuda', sm: '75' }
+    # arch-specific and family-specific arch builds
+    - {jobs: ['build'], project: ['libcudacxx', 'cub', 'thrust', 'cudax'], std: 'all', sm: '90a;100a;103a;110a;120a;121a'}
+    - {jobs: ['build'], project: ['libcudacxx', 'cub', 'thrust', 'cudax'], std: 'all', sm: '100f;103f;110f;120f;121f'}
+
+  weekly:
+    # CTK 12.0 full matrix build: default projects
+    - {jobs: ['build'], std: 'all', ctk: '12.0', cxx: ['gcc7', 'gcc8', 'gcc9', 'gcc10', 'gcc11', 'gcc12']}
+    - {jobs: ['build'], std: 'all', ctk: '12.0', cxx: ['clang14']}
+    - {jobs: ['build'], std: 'all', ctk: '12.0', cxx: ['msvc2019', 'msvc14.39']}
+    # CTK 12.X full matrix build: default projects
+    - {jobs: ['build'], std: 'all', ctk: '12.X', cxx: ['gcc7', 'gcc8', 'gcc9', 'gcc10', 'gcc11', 'gcc12', 'gcc13', 'gcc14']}
+    - {jobs: ['build'], std: 'all', ctk: '12.X', cxx: ['clang14', 'clang15', 'clang16', 'clang17', 'clang18', 'clang19']}
+    - {jobs: ['build'], std: 'all', ctk: '12.X', cxx: ['msvc2019', 'msvc2022']}
+    # CTK 13.0 full matrix build: default projects
+    - {jobs: ['build'], std: 'all', ctk: '13.0', cxx: ['gcc11', 'gcc12', 'gcc13', 'gcc14', 'gcc15']}
+    - {jobs: ['build'], std: 'all', ctk: '13.0', cxx: ['clang15', 'clang16', 'clang17', 'clang18', 'clang19', 'clang20']}
+    - {jobs: ['build'], std: 'all', ctk: '13.0', cxx: ['msvc2019', 'msvc2022']}
+    # CTK '13.X' full matrix build: default projects
+    - {jobs: ['build'], std: 'all', ctk: '13.X', cxx: ['gcc11', 'gcc12', 'gcc13', 'gcc14', 'gcc15']}
+    - {jobs: ['build'], std: 'all', ctk: '13.X', cxx: ['clang15', 'clang16', 'clang17', 'clang18', 'clang19', 'clang20', 'clang21']}
+    - {jobs: ['build'], std: 'all', ctk: '13.X', cxx: ['msvc2019', 'msvc2022', 'msvc2026']}
+    # CTK 12.0 full matrix build: cudax
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '12.0', cxx: ['gcc9', 'gcc10', 'gcc11', 'gcc12']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '12.0', cxx: ['clang14']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '12.0', cxx: ['msvc14.39']}
+    # CTK 12.X full matrix build: cudax
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '12.X', cxx: ['gcc9', 'gcc10', 'gcc11', 'gcc12', 'gcc13', 'gcc14']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '12.X', cxx: ['clang14', 'clang15', 'clang16', 'clang17', 'clang18', 'clang19']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '12.X', cxx: ['msvc2022']}
+    # CTK 13.0 full matrix build: cudax
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '13.0', cxx: ['gcc11', 'gcc12', 'gcc13', 'gcc14', 'gcc15']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '13.0', cxx: ['clang15', 'clang16', 'clang17', 'clang18', 'clang19', 'clang20']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '13.0', cxx: ['msvc2022']}
+    # CTK '13.X' full matrix build: cudax
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '13.X', cxx: ['gcc11', 'gcc12', 'gcc13', 'gcc14', 'gcc15']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '13.X', cxx: ['clang15', 'clang16', 'clang17', 'clang18', 'clang19', 'clang20', 'clang21']}
+    - {jobs: ['build'], project: 'cudax', std: 'all', ctk: '13.X', cxx: ['msvc2022', 'msvc2026']}
+    # CTK 12.X testing:
+    - {jobs: ['test'], project: 'libcudacxx', ctk: '12.X', std: 'minmax', cxx: ['gcc14', 'clang19', 'msvc2022'], gpu: 'rtx2080'}
+    - {jobs: ['test'], project: 'cub',        ctk: '12.X', std: 'minmax', cxx: ['gcc14', 'clang19', 'msvc2022'], gpu: 'rtxa6000'}
+    - {jobs: ['test'], project: 'thrust',     ctk: '12.X', std: 'minmax', cxx: ['gcc14', 'clang19', 'msvc2022'], gpu: 'rtx4090'}
+    - {jobs: ['test'], project: 'cudax',      ctk: '12.X', std: 'minmax', cxx: ['gcc14', 'clang19', 'msvc2022'], gpu: 't4'}
+    - {jobs: ['test'], project: ['libcudacxx', 'cub', 'thrust', 'cudax'], ctk: '12.X', std: 'minmax', cxx: 'gcc14', gpu: 'h100' }
+    # CTK '13.X' testing:
+    - {jobs: ['test'], project: 'libcudacxx', ctk: '13.X', std: 'minmax', cxx: ['gcc', 'clang', 'msvc'], gpu: 't4'}
+    - {jobs: ['test'], project: 'cub',        ctk: '13.X', std: 'minmax', cxx: ['gcc', 'clang', 'msvc'], gpu: 'rtxa6000'}
+    - {jobs: ['test'], project: 'thrust',     ctk: '13.X', std: 'minmax', cxx: ['gcc', 'clang', 'msvc'], gpu: 'rtx4090'}
+    - {jobs: ['test'], project: 'cudax',      ctk: '13.X', std: 'minmax', cxx: ['gcc', 'clang', 'msvc'], gpu: 'rtx2080'}
+    - {jobs: ['test'], project: ['libcudacxx', 'cub', 'thrust', 'cudax'], ctk: '13.X', std: 'minmax', gpu: 'h100' }
+    # RTX PRO 6000 coverage (limited due to small number of runners):
+    - {jobs: ['test'], project: ['cub', 'thrust'], std: 'max', cxx: 'gcc', gpu: 'rtxpro6000'}
+    # Misc:
+    - {jobs: ['build'], cpu: 'arm64', project: ['libcudacxx', 'cub', 'thrust', 'cudax'], ctk: '12.X', std: 'all', cxx: ['gcc14', 'clang19']}
+    - {jobs: ['build'], cpu: 'arm64', project: ['libcudacxx', 'cub', 'thrust', 'cudax'], ctk: '13.X', std: 'all', cxx: ['gcc', 'clang']}
+    - {jobs: ['test_gpu'], project: 'thrust', cmake_options: '-DTHRUST_DISPATCH_TYPE=Force32bit', gpu: 'rtx4090'}
+    - {jobs: ['test_gpu'], project: 'thrust', cmake_options: '-DTHRUST_DISPATCH_TYPE=Force64bit', gpu: 'rtx4090'}
+    - {jobs: ['limited'], project: 'cub', std: 17, gpu: 'rtx2080'}
+    # sm: all-cccl:
+    - {jobs: ['build'], project: ['thrust', 'libcudacxx', 'cudax'], std: 'max', sm: 'all-cccl' }
+    - {jobs: ['build_nolid', 'build_lid0'], project: ['cub'], std: 'max', sm: 'all-cccl'}
+    # NVRTC tests don't currently support 12.0:
+    - {jobs: ['nvrtc'],          project: 'libcudacxx', ctk: [        '12.X', '13.0', '13.X'], cxx: 'gcc12', std: 'all', gpu: 'rtx2080', sm: 'gpu'}
+    - {jobs: ['verify_codegen'], project: 'libcudacxx'}
+    # c.parallel -- pinned to gcc13 / msvc2022 to match python
+    - {jobs: ['test'],  project: ['cccl_c_parallel'], ctk: '12.X', cxx: ['gcc13', 'msvc2022'], gpu: ['t4']}
+    - {jobs: ['test'],  project: ['cccl_c_parallel'], ctk: '13.X', cxx: ['gcc13', 'msvc2022'], gpu: ['rtx2080', 'l4', 'h100']}
+    # RTX PRO 6000 coverage (limited due to small number of runners):
+    - {jobs: ['test'], project: 'cccl_c_parallel', ctk: '13.X', cxx: ['gcc13'], gpu: ['rtxpro6000']}
+    # c.experimental.stf -- pinned to gcc13 to match python
+    - {jobs: ['test'],  project: ['cccl_c_stf'], ctk: '12.X', cxx: 'gcc13', gpu: ['rtx2080']}
+    - {jobs: ['test'],  project: ['cccl_c_stf'], ctk: '13.X', cxx: 'gcc13', gpu: ['t4', 'l4', 'h100']}
+    # Python -- pinned to gcc13 / msvc2022 for consistency across CTK images
+    - {jobs: ['test'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: ['3.10', '3.11', '3.12', '3.13', '3.14'], gpu: 'l4', cxx: ['gcc13', 'msvc2022']}
+    - {jobs: ['test'], project: 'python', ctk: ['12.X', '13.X'], py_version: '3.14', gpu: 'h100', cxx: ['gcc13', 'msvc2022']}
+    - {jobs: ['test'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: ['3.10', '3.14'], gpu: 't4', cxx: 'gcc13'}
+    - {jobs: ['test'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: '3.14', gpu: 'rtxa6000', cxx: 'gcc13'}
+    - {jobs: ['test'], project: 'python', ctk: '13.X', py_version: '3.14', gpu: 'rtxpro6000', cxx: 'gcc13'}
+    - {jobs: ['test'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: '3.14', gpu: 'l4', cxx: ['gcc13', 'msvc2022'], py_ctk_mode: 'sysctk'}
+    - {jobs: ['test_py_compute_minimal'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: '3.14t', gpu: 'l4', cxx: ['gcc13', 'msvc2022'], py_ctk_mode: 'sysctk'}
+    # cuda.cccl.headers (CPU-only): all CTK x source x OS, py endpoints 3.10 + 3.14
+    - {jobs: ['test_headers'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: ['3.10', '3.14'], cxx: ['gcc13', 'msvc2022'], py_ctk_mode: ['pinned', 'latest', 'sysctk']}
+    # CCCL packaging:
+    - {jobs: ['test'], project: 'packaging', ctk: '12.0', cxx: ['gcc10', 'clang14'], gpu: 't4', args: '-min-cmake'}
+    - {jobs: ['test'], project: 'packaging', ctk: '12.X', cxx: ['gcc10', 'clang14'], gpu: 'rtx2080'}
+    - {jobs: ['test'], project: 'packaging', ctk: '13.0', cxx: ['gcc15', 'clang20'],   gpu: 't4', args: '-min-cmake'}
+    - {jobs: ['test'], project: 'packaging', ctk: '13.X', cxx: ['gcc',   'clang'],   gpu: 'rtx2080'}
+    - {jobs: ['install'], project: 'packaging'}
+    # NVBench Helper:
+    - {jobs: ['test'], project: 'nvbench_helper', ctk: '12.0', cxx: ['gcc10', 'clang14'], gpu: 'rtx2080'}
+    - {jobs: ['test'], project: 'nvbench_helper', ctk: '12.X', cxx: ['gcc10', 'clang14'], gpu: 't4'}
+    - {jobs: ['test'], project: 'nvbench_helper', ctk: '13.0', cxx: ['gcc15', 'clang20'], gpu: 'rtx2080'}
+    - {jobs: ['test'], project: 'nvbench_helper', ctk: '13.X', cxx: ['gcc',   'clang'],   gpu: 't4'}
+    # NVHPC build
+    - {jobs: ['build'], cxx: 'nvhpc-prev', ctk: 'nvhpc-prev', std: 'all', project: ['libcudacxx', 'cub', 'thrust', 'cudax', 'stdpar'], cpu: ['amd64', 'arm64']}
+    - {jobs: ['build'], cxx: 'nvhpc',      ctk: 'nvhpc',      std: 'all', project: ['libcudacxx', 'cub', 'thrust', 'cudax', 'stdpar'], cpu: ['amd64', 'arm64']}
+    # clang-cuda
+    - {jobs: ['build'], cudacxx: 'clang', ctk: 'clang-cuda', cxx: 'clang-cuda', std: 'all', sm: '75;80;90;100;120'}
+    - {jobs: ['build'], cudacxx: 'clang', ctk: 'clang_preview-cuda', cxx: 'clang_preview-cuda', std: 'all', sm: '75;80;90;100;120'}
+    - {jobs: ['build'], project: 'libcudacxx', cudacxx: 'clang', ctk: 'clang-cuda', cxx: 'clang-cuda',  std: 23, sm: '75;80;90;100;120'}
+    - {jobs: ['build'], project: 'libcudacxx', cudacxx: 'clang', ctk: 'clang_preview-cuda', cxx: 'clang_preview-cuda', std: 23, sm: '75;80;90;100;120'}
+    # compute-sanitizer
+    - {jobs: ['compute_sanitizer'], project: 'cub', std: 'max', gpu: 'rtxa6000', sm: 'gpu', cmake_options: '-DCMAKE_CUDA_FLAGS=-lineinfo'}
+    # clang-tidy
+    - { jobs: ['build'], project: 'tidy', std: 'min', cxx: ['clang'], cudacxx: ['clang'], ctk: 'clang-cuda', sm: '75' }
+    # arch-specific and family-specific arch builds
+    - {jobs: ['build'], project: ['libcudacxx', 'cub', 'thrust', 'cudax'], std: 'all', sm: '90a;100a;103a;110a;120a;121a'}
+    - {jobs: ['build'], project: ['libcudacxx', 'cub', 'thrust', 'cudax'], std: 'all', sm: '100f;103f;110f;120f;121f'}
+
+  python-wheels:
+    - {jobs: ['test'], project: 'python', ctk: ['12.0', '12.X', '13.0', '13.X'], py_version: ['3.10', '3.11', '3.12', '3.13', '3.14'], gpu: 'l4', cxx: ['gcc13', 'msvc2022']}
+    - {jobs: ['test'], project: 'python', ctk: ['12.X', '13.X'], py_version: '3.14', gpu: 'h100', cxx: ['gcc13', 'msvc2022']}
+    - {jobs: ['test'], project: 'python', cpu: 'arm64', ctk: ['12.X', '13.X'], py_version: ['3.10', '3.11', '3.12', '3.13', '3.14'], gpu: 'l4', cxx: 'gcc13'}
+
+
+  # This is just used to ensure that we generate devcontainers for all images we build.
+  # These do not map to any actual jobs.
+  devcontainers:
+    - {jobs: ['dc'],     ctk: ['12.0', '12.X'                ], cxx: ['clang14']}
+    - {jobs: ['dc'],     ctk: ['12.0', '12.X'                ], cxx: ['gcc7', 'gcc8', 'gcc9', 'gcc10']}
+    - {jobs: ['dc'],     ctk: ['12.0', '12.X', '13.0', '13.X'], cxx: ['gcc11', 'gcc12']}
+    - {jobs: ['dc'],     ctk: [        '12.X', '13.0', '13.X'], cxx: ['gcc13', 'gcc14']}
+    - {jobs: ['dc'],     ctk: [                '13.0', '13.X'], cxx: ['gcc15']}
+    - {jobs: ['dc'],     ctk: [        '12.X', '13.0', '13.X'], cxx: ['clang15', 'clang16', 'clang17', 'clang18', 'clang19']}
+    - {jobs: ['dc_ext'], ctk: [        '12.X', '13.0', '13.X'], cxx: ['gcc14', 'clang20']}
+    # Clang21+CTK12.9 is currently only used for cuda-clang testing. nvcc 12.9 doesn't support clang21.
+    - {jobs: ['dc_ext'], ctk: [        '12.X',         '13.X'], cxx: ['clang21']}
+    - {jobs: ['dc_ext'], ctk: [                        '13.X'], cxx: ['gcc15']}
+    # Clang22+CTK12.9 is currently only used for clang-cuda testing. nvcc 12.9 doesn't support clang22.
+    - {jobs: ['dc_ext'], ctk: [        '12.X'                ], cxx: ['clang_preview22']}
+    # 12.0 python image, pinned at gcc13 for simplicity. CTK 12.0 doesn't really play nice with gcc13, but
+    # that doesn't matter for running python tests.
+    - {jobs: ['dc'], ctk: ['12.0'], cxx: 'gcc13'}
+    # NVHPC
+    - {jobs: ['dc'], cxx: 'nvhpc-prev', ctk: 'nvhpc-prev'}
+    - {jobs: ['dc'], cxx: 'nvhpc',      ctk: 'nvhpc'}
+
+  # Any generated jobs that match the entries in `exclude` will be removed from the final matrix for all workflows.
+  exclude:
+    # GPU runners are not available on Windows.
+    - {jobs: ['test', 'test_gpu', 'test_nolid', 'test_lid0', 'test_lid1', 'test_lid2'], cxx: ['msvc2019', 'msvc14.39', 'msvc2022', 'msvc2026']}
+    # cudax doesn't support C++17 on msvc:
+    - {project: 'cudax', std: 17, cxx: ['msvc2019', 'msvc14.39', 'msvc2022', 'msvc2026']}
+
+
+#############################################################################################
+
+
+# The version of the devcontainer images to use from https://hub.docker.com/r/rapidsai/devcontainers
+devcontainer_version: '26.10'
+
+# Compiler versions used for the cuda99.X internal builds:
+cuda99_gcc_version: 15
+cuda99_clang_version: 21
+
+# All supported C++ standards:
+all_stds: [17, 20]
+
+# Aliases:
+# - 12.X: Newest CTK 12.X version.
+# - 13.X: Newest CTK 13.X version.
+# - nvhpc: CTK shipped in newest NVHPC
+# - nvhpc-prev: CTK shipped in previous NVHPC
+# - pybuild: Selects image to use for python wheel builds' outer docker instance
+ctk_versions:
+  12.0: { stds: [17, 20] }
+  12.9: { stds: [17, 20], alias: ['12.X', 'pybuild', 'clang-cuda', 'clang_preview-cuda'] }
+  13.0: { stds: [17, 20] }
+  13.1: { stds: [17, 20], alias: ['nvhpc-prev']}
+  13.2: { stds: [17, 20], alias: ['nvhpc']}
+  13.3: { stds: [17, 20], alias: ['13.X'] }
+
+device_compilers:
+  nvcc: # Version / stds are taken from CTK
+    name: 'nvcc'
+    exe: 'nvcc'
+  clang: # Requires cxx=clang. Version / stds are taken from cxx compiler.
+    name: "ClangCUDA"
+    exe: 'clang++'
+
+host_compilers:
+  gcc:
+    name: 'GCC'
+    container_tag: 'gcc'
+    exe: 'g++'
+    versions:
+      7:  { stds: [17,   ] }
+      8:  { stds: [17,   ] }
+      9:  { stds: [17,   ] }
+      10: { stds: [17, 20] }
+      11: { stds: [17, 20] }
+      12: { stds: [17, 20] }
+      13: { stds: [17, 20] }
+      14: { stds: [17, 20] }
+      15: { stds: [17, 20] }
+  clang:
+    name: 'Clang'
+    container_tag: 'llvm'
+    exe: 'clang++'
+    versions:
+      14: { stds: [17, 20] }
+      15: { stds: [17, 20] }
+      16: { stds: [17, 20] }
+      17: { stds: [17, 20] }
+      18: { stds: [17, 20] }
+      19: { stds: [17, 20] }
+      20: { stds: [17, 20] }
+      21: { stds: [17, 20], alias: 'cuda' }
+  clang_preview:
+    name: 'Clang'
+    container_tag: 'llvm'
+    exe: 'clang++'
+    versions:
+      22: { stds: [17, 20], alias: 'cuda' }
+  msvc:
+    name: 'MSVC'
+    container_tag: 'cl'
+    exe: cl
+    versions:
+      '14.29': { stds: [17,   ], alias: '2019' }
+      '14.39': { stds: [17, 20] } # CTK 12.0 doesn't recognize >14.39 as MSVC 2022.
+      '14.44': { stds: [17, 20], alias: '2022' }
+      '14.50': { stds: [17, 20], alias: '2026' }
+
+  nvhpc:
+    name: 'NVHPC'
+    container_tag: 'nvhpc'
+    exe: nvc++
+    versions:
+      # !! Update the ctk_versions 'nvhpc*' aliases when updating NVHPC versions:
+      26.3: { stds: [17, 20], alias: 'prev' }
+      26.5: { stds: [17, 20] }
+
+# Jobs support the following properties:
+#
+# - name: The human-readable name of the job. Default is the capitalized job key.
+# - needs:
+#   - A list of jobs that must be completed before this job can run. Default is an empty list.
+#   - These jobs are automatically added if needed:
+#     - Eg. "jobs: ['test']" in the workflow def will also create the required 'build' jobs.
+# - gpu: Whether the job requires a GPU runner. Default is false.
+# - cuda_ext: Whether the job requires a devcontainer with extra CUDA libraries. Default is false.
+# - invoke:
+#   - Map the job type to the script invocation spec:
+#     - prefix: The script invocation prefix. Default is the job name.
+#     - args: Additional arguments to pass to the script. Default is no args.
+#   - The script is invoked either:
+#     linux:   `ci/windows/<spec[prefix]>_<project>.ps1 <spec[args]>`
+#     windows: `ci/<spec[prefix]>_<project>.sh <spec[args]>`
+# - force_producer_ctk:
+#   - If set, force the auto-generated producers for this job to use a specific CTK version.
+#   - By default, the autogenerated job's CTK version is determined by the consumer's `ctk` tag.
+#   - This is useful for testing the cross-testing major version compat.
+#   - E.g. "force_producer_ctk: '12.0'" on a test step will force the generated build step to use CTK 12.0.
+
+jobs:
+  # General:
+  build:        { gpu: false }
+  test:         { gpu: true, needs: 'build' }
+  install:      { gpu: false }
+
+  test_nobuild: { gpu: true, name: 'Test', invoke: { prefix: 'test' } }
+
+  compute_sanitizer: { gpu: true, name: 'ComputeSanitizer', needs: 'build', invoke: { prefix: 'test', args: '-compute-sanitizer' } }
+
+  # libcudacxx:
+  nvrtc: { gpu: true, name: 'NVRTC' }
+  verify_codegen: { gpu: false, name: 'VerifyCodegen' }
+
+  # CUB:
+  build_nolid: { name: 'BuildNoLaunch',     gpu: false, invoke: { prefix: 'build', args: '-no-lid'} }
+  build_lid0:  { name: 'BuildHostLaunch',   gpu: false, invoke: { prefix: 'build', args: '-lid0'} }
+  build_lid1:  { name: 'BuildDeviceLaunch', gpu: false, invoke: { prefix: 'build', args: '-lid1'} }
+  build_lid2:  { name: 'BuildGraphCapture', gpu: false, invoke: { prefix: 'build', args: '-lid2'} }
+  # NoLid -> The string `lid_X` doesn't appear in the test name. Mostly warp/block tests, old device tests, and examples.
+  test_nolid: { name: 'TestNoLaunch',      gpu: true, needs: 'build_nolid', invoke: { prefix: 'test', args: '-no-lid --test-par 8'} }
+  # CUB uses `lid` to indicate launch strategies: whether CUB algorithms are:
+  # - launched from the host (lid0):
+  test_lid0:  { name: 'HostLaunch',   gpu: true, needs: 'build_lid0', invoke: { prefix: 'test', args: '-lid0 --test-par 8'} }
+  # - launched from the device (lid1):
+  test_lid1:  { name: 'DeviceLaunch', gpu: true, needs: 'build_lid1', invoke: { prefix: 'test', args: '-lid1 --test-par 8'} }
+  # - captured in a CUDA graph for deferred launch (lid2):
+  test_lid2:  { name: 'GraphCapture', gpu: true, needs: 'build_lid2', invoke: { prefix: 'test', args: '-lid2 --test-par 8'} }
+  # Limited build reduces the number of runtime test cases, available device memory, etc, and may be used
+  # to reduce test runtime in limited environments.
+  limited:    { name: "SmallGMem",   gpu: true, needs: 'build', invoke: { prefix: 'test', args: '-limited'} }
+  # Compute sanitizer jobs:
+  compute_mem_nolid:  { name: 'CSMem-TestGPU',       gpu: true, needs: 'build_nolid', invoke: { prefix: 'test', args: '-compute-sanitizer-memcheck  -no-lid'} }
+  compute_mem_lid0:   { name: 'CSMem-HostLaunch',    gpu: true, needs: 'build_lid0',  invoke: { prefix: 'test', args: '-compute-sanitizer-memcheck  -lid0'} }
+  compute_race_nolid: { name: 'CSRace-TestGPU',      gpu: true, needs: 'build_nolid', invoke: { prefix: 'test', args: '-compute-sanitizer-racecheck -no-lid'} }
+  compute_race_lid0:  { name: 'CSRace-HostLaunch',   gpu: true, needs: 'build_lid0',  invoke: { prefix: 'test', args: '-compute-sanitizer-racecheck -lid0'} }
+  compute_init_nolid: { name: 'CSInit-TestGPU',      gpu: true, needs: 'build_nolid', invoke: { prefix: 'test', args: '-compute-sanitizer-initcheck -no-lid'} }
+  compute_init_lid0:  { name: 'CSInit-HostLaunch',   gpu: true, needs: 'build_lid0',  invoke: { prefix: 'test', args: '-compute-sanitizer-initcheck -lid0'} }
+  compute_sync_nolid: { name: 'CSSync-TestGPU',      gpu: true, needs: 'build_nolid', invoke: { prefix: 'test', args: '-compute-sanitizer-synccheck -no-lid'} }
+  compute_sync_lid0:  { name: 'CSSync-HostLaunch',   gpu: true, needs: 'build_lid0',  invoke: { prefix: 'test', args: '-compute-sanitizer-synccheck -lid0'} }
+
+  # Thrust:
+  test_cpu: { name: 'TestCPU', gpu: false, needs: 'build', invoke: { prefix: 'test', args: '-cpu-only'} }
+  test_gpu: { name: 'TestGPU', gpu: true,  needs: 'build', invoke: { prefix: 'test', args: '-gpu-only'} }
+
+  # Python:
+  build_py_wheel:   { name: "Build cuda.cccl",             gpu: false, invoke: { prefix: 'build_cuda_cccl'} }
+  test_headers:     { name: "Test cuda.cccl.headers",      gpu: false, needs: 'build_py_wheel', force_producer_ctk: "pybuild", invoke: { prefix: 'test_cuda_cccl_headers'} }
+  test_py_par:      { name: "Test cuda.compute",     gpu: true,  needs: 'build_py_wheel', force_producer_ctk: "pybuild", invoke: { prefix: 'test_cuda_compute'} }
+  test_py_compute_minimal: { name: "Test cuda.compute minimal", gpu: true, needs: 'build_py_wheel', force_producer_ctk: "pybuild", invoke: { prefix: 'test_cuda_compute_minimal'} }
+  test_py_examples: { name: "Test cuda.cccl.examples",     gpu: true,  needs: 'build_py_wheel', force_producer_ctk: "pybuild", invoke: { prefix: 'test_cuda_cccl_examples'} }
+
+  # Run jobs for 'target' project (ci/util/build_and_test_targets.sh):
+  run_cpu: { gpu: false }
+  run_gpu: { gpu: true  }
+
+  # Only used for generating devcontainers. No scripts actually exist for these:
+  dc:     { gpu: false }
+  dc_ext: { gpu: false, cuda_ext: true }
+
+# Projects have the following properties:
+#
+# Keys are project subdirectories names. These will also be used in script names.
+#
+# - stds: A list of C++ standards to test. Required.
+# - name: The human-readable name of the project. Default is the project key.
+# - job_map: Map general jobs to arrays of project-specific jobs.
+#            Useful for things like splitting cpu/gpu testing for a project.
+#            E.g. "job_map: { test: ['test_cpu', 'test_gpu'] }" replaces
+#            the "test" job with distinct "test_cpu" and "test_gpu" jobs.
+projects:
+  packaging:
+    name: 'CCCL Packaging'
+    stds: [17, 20]
+    job_map:
+      build: []
+      test: ['test_nobuild']
+  libcudacxx:
+    name: 'libcu++'
+    stds: [17, 20]
+  cub:
+    name: 'CUB'
+    stds: [17, 20]
+    job_map:
+      build: ['build_nolid', 'build_lid0', 'build_lid1', 'build_lid2']
+      test:  ['test_nolid',  'test_lid0',  'test_lid1',  'test_lid2']
+      compute_sanitizer:
+        - compute_mem_nolid
+        - compute_mem_lid0
+        - compute_race_nolid
+        - compute_race_lid0
+        - compute_init_nolid
+        - compute_init_lid0
+        - compute_sync_nolid
+        - compute_sync_lid0
+  thrust:
+    name: 'Thrust'
+    stds: [17, 20]
+    job_map: { test: ['test_cpu', 'test_gpu'] }
+  cudax:
+    stds: [17, 20]
+  stdpar:
+    name: 'NVHPC stdpar'
+    stds: [17, 20]
+  python:
+    name: "Python"
+    job_map:
+      build: ['build_py_wheel']
+      test:  ['test_py_par', 'test_py_examples']
+  python_v2:
+    name: "Python (cuda.compute on v2/HostJIT)"
+    # Only cuda.compute differs between v1 and v2; cuda.cccl.headers does not
+    # need separate coverage. Run the examples because they exercise
+    # cuda.compute against the v2 backend.
+    job_map:
+      build: ['build_py_wheel']
+      test:  ['test_py_par', 'test_py_examples']
+  python_tsan:
+    name: "Python (cuda.compute free-threaded ThreadSanitizer)"
+    # Runs per-PR (and nightly). The producer build_py_wheel runs
+    # build_cuda_cccl_python_tsan.sh (c.parallel v1 host code built with
+    # -fsanitize=thread); test_py_compute_minimal runs
+    # test_cuda_compute_minimal_python_tsan.sh (the FT stress + sweep under the
+    # TSan runtime). Deliberately absent from the `python-wheels` publish
+    # workflow, so these instrumented wheels never reach PyPI.
+    job_map:
+      build: ['build_py_wheel']
+      test:  ['test_py_compute_minimal']
+  cccl_c_parallel:
+    name: 'CCCL C Parallel'
+    stds: [20]
+  cccl_c_parallel_v2:
+    name: 'CCCL C Parallel v2 (HostJIT)'
+    stds: [20]
+    # test_cccl_c_parallel_v2.sh builds inline (no separate build script),
+    # so suppress the default test→build dependency. test_nobuild invokes
+    # test_<project>.sh directly without a producer build job.
+    job_map:
+      build: []
+      test: ['test_nobuild']
+  cccl_c_stf:
+    name: 'CCCL C CUDASTF'
+    stds: [20]
+  nvbench_helper:
+    name: 'NVBench Helper'
+    stds: [17] # Only builds on oldest arch for max compat.
+    job_map:
+      build: []
+      test: ['test_nobuild']
+
+  # Run specific build_and_test_targets.sh invocations across the CI matrix.
+  # Use the override workflow and supply arguments via the `args` tag.
+  # Example:
+  #   override:
+  #     - { jobs: ['run'], project: 'target', ctk: ['12.X', '13.X'], cxx: 'gcc', gpu: 'rtx2080',
+  #         args: '--preset cub-cpp20 --build-targets "cub.cpp20.test.iterator" --ctest-targets "cub.cpp20.test.iterator"' }
+  target:
+    name: 'Target'
+    stds: [17, 20]
+  bisect:
+    name: 'Bisect'
+    stds: [17, 20]
+  tidy:
+    name: 'clang-tidy'
+    stds: [17]
+
+# name -> Display name for generated job labels.
+# runner -> GPU/driver/count segment of the GHA runner label.
+# testing -> Runner with GPU is in a nv-gh-runners testing pool.
+gpus:
+  t4:         { name: 'T4',          sm: 75,  runner: 't4-latest-1'         } # 16 GB,  10 runners
+  rtx2080:    { name: 'RTX2080',     sm: 75,  runner: 'rtx2080-latest-1'    } #  8 GB,  12 runners
+  rtxa6000:   { name: 'RTXA6000',    sm: 86,  runner: 'rtxa6000-latest-1'   } # 48 GB,  12 runners
+  l4:         { name: 'L4',          sm: 89,  runner: 'l4-latest-1'         } # 24 GB,  48 runners
+  rtx4090:    { name: 'RTX4090',     sm: 89,  runner: 'rtx4090-latest-1'    } # 24 GB,  10 runners
+  h100:       { name: 'H100',        sm: 90,  runner: 'h100-latest-1'       } # 80 GB,  16 runners
+  h100_2gpu:  { name: 'H100 2-GPU',  sm: 90,  runner: 'h100-latest-2'       } # 2 x 80 GB
+  # Very small number of runners on loan from cuda-python while we wait for our order to arrive.
+  # Limit jobs on these:
+  rtxpro6000: { name: 'RTXPRO6000',  sm: 120, runner: 'rtxpro6000-latest-1' }
+
+# Tags are used to define a `matrix job` in the workflow section.
+#
+# Tags have the following options:
+#  - required: Whether the tag is required. Default is false.
+#  - default: The default value for the tag. Default is null.
+tags:
+   # An array of jobs (e.g. 'build', 'test', 'nvrtc', 'infra', 'verify_codegen', ...)
+   # See the `jobs` map.
+  jobs: { required: true }
+  # CUDA ToolKit version
+  # See the `ctks` map.
+  ctk: { default: '13.X' }
+  # CPU architecture
+  cpu: { default: 'amd64' }
+  # GPU model
+  gpu: { default: 'rtx2080' }
+  # Host compiler {name, version, exe}
+  # See the `host_compilers` map.
+  cxx: { default: 'gcc' }
+  # Device compiler.
+  # See the `device_compilers` map.
+  cudacxx: { default: 'nvcc' }
+  # Project name (e.g. libcudacxx, cub, thrust, cccl)
+  # See the `projects` map.
+  project: { default: ['libcudacxx', 'cub', 'thrust'] }
+  # Python version for Python builds/tests
+  py_version: { required: false }
+  # Python CTK-source mode for test lanes (Python-only):
+  #   'pinned' (default) -- pin cuda-toolkit to the container CTK minor
+  #   'latest'           -- unpinned; pip resolves the newest minor
+  #   'sysctk'           -- use the system toolkit; install the sysctk extras
+  # If set, passed to the script with `-ctk-mode <mode>`. Exploded if an array,
+  # e.g. `['pinned','sysctk']`.
+  py_ctk_mode: { required: false }
+  # C++ standard
+  # If set to 'all', all stds supported by the ctk/compilers/project are used.
+  # If set to 'min', 'max', or 'minmax', the minimum, maximum, or both stds are used.
+  # If set, will be passed to script with `-std <std>`.
+  std: { required: false }
+  # GPU architecture
+  # - If set, passed to script with `-arch <sm>`.
+  # - Format is the same as `CMAKE_CUDA_ARCHITECTURES`:
+  #   - PTX only: 70-virtual
+  #   - SASS only: 70-real
+  #   - Both: 70
+  # - Can pass multiple architectures via "60;70-real;80-virtual"
+  # - Defaults to use the settings in the CMakePresets.json file.
+  # - Will be exploded if an array, e.g. `sm: ['60;70;80;90', '90a']` creates two jobs.
+  # - Set to 'gpu' to only target the GPU in the `gpu` tag.
+  sm: { required: false }
+  # Additional CMake options to pass to the build.
+  # If set, passed to script with `-cmake_options "<cmake_options>"`.
+  cmake_options: { required: false }
+  # Environment variables to add
+  environment: { required: false, default: [] }
+  # Additional arguments appended to the generated command.
+  # Typically used with the `target` project to forward options to
+  # ci/util/build_and_test_targets.sh, but works with all CI jobs.
+  args: { required: false, default: "" }

cccl_upstream/ci/matx/build_matx.sh

@@ -0,0 +1,113 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+readonly matx_repo=https://github.com/NVIDIA/MatX.git
+readonly matx_branch=main
+
+# Ensure the script is being executed in the root cccl directory:
+cd "$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/../..";
+
+log_vars() {
+  for var in "$@"; do
+    echo "${var}=${!var}"
+  done
+}
+
+# Take two version strings and return the greater of the two:
+version_max() {
+  local v1="${1}"
+  local v2="${2}"
+  if ci/util/version_compare.sh "$v1" ge "$v2"; then
+    echo "$v1"
+  else
+    echo "$v2"
+  fi
+}
+
+# Get the current CCCL info:
+readonly cccl_repo="${PWD}"
+
+# Define CCCL_TAG to override the default CCCL SHA. Otherwise the current HEAD of the local checkout is used.
+echo "CCCL_TAG (override): ${CCCL_TAG-}";
+if test -n "${CCCL_TAG-}"; then
+    # If CCCL_TAG is defined, fetch it to the local checkout
+    git -C "${cccl_repo}" fetch origin "${CCCL_TAG}";
+    cccl_sha="$(git -C "${cccl_repo}" rev-parse FETCH_HEAD)";
+else
+    cccl_sha="$(git -C "${cccl_repo}" rev-parse HEAD)";
+fi
+
+cccl_repo_version="$(git -C "${cccl_repo}" describe "${cccl_sha}"| grep -Eo '[0-9]+\.[0-9]+\.[0-9]+')"
+readonly cccl_repo_version
+
+# Define CCCL_VERSION to override the version used by rapids-cmake to patch CCCL.
+echo "CCCL_VERSION (override): ${CCCL_VERSION-}";
+if test -n "${CCCL_VERSION-}"; then
+  readonly cccl_rapids_cmake_version="${CCCL_VERSION}"
+else
+  cccl_rapids_cmake_version="${cccl_repo_version}"
+  # shellcheck disable=SC2034
+  readonly cccl_rapids_cmake_version
+fi
+
+# If the current version is less than 2.8.0, use 2.8.0 for the rapids-cmake version.
+# This is to allow rapids-cmake to correctly patch the CCCL install rules on current `main`.
+cccl_version=$(version_max "${cccl_repo_version}" "2.8.0")
+readonly cccl_version
+
+readonly workdir="${cccl_repo}/build/${CCCL_BUILD_INFIX:-}/matx"
+readonly version_file="${workdir}/MatX/cmake/versions.json"
+readonly version_override_file="${workdir}/versions-override.json"
+
+log_vars \
+  matx_repo matx_branch \
+  cccl_repo cccl_sha cccl_repo_version cccl_rapids_cmake_version \
+  workdir \
+  version_file version_override_file
+
+mkdir -p "${workdir}"
+cd "${workdir}"
+
+# Python deps:
+pip install numpy
+
+# Clone MatX
+rm -rf MatX
+git clone "${matx_repo}" -b "${matx_branch}"
+
+cd MatX
+echo "MatX HEAD:"
+git log -1 --format=short
+cd ..
+
+# Write out version override file
+jq -r ".packages.CCCL *=
+  {
+    \"git_url\": \"${cccl_repo}\",
+    \"git_tag\": \"${cccl_sha}\",
+    \"version\": \"${cccl_version}\",
+    \"always_download\": true
+  }" \
+  "${version_file}" > "${version_override_file}"
+
+echo "Overriding MatX versions.json file:"
+cat "$version_override_file"
+
+# Configure and build
+rm -rf build
+
+SCCACHE_NO_DIST_COMPILE=1 cmake \
+  -B build -S MatX -G Ninja \
+  "-DCMAKE_CUDA_ARCHITECTURES=75;120" \
+  "-DRAPIDS_CMAKE_CPM_OVERRIDE_VERSION_FILE=${version_override_file}" \
+  -DMATX_BUILD_TESTS=ON \
+  -DMATX_BUILD_EXAMPLES=ON \
+  -DMATX_BUILD_BENCHMARKS=ON \
+  -DMATX_EN_CUTENSOR=ON
+
+# Disabled because `cmake --build -j ""` is invalid, but so is
+# `cmake --build -j8`. CMake expects a space between `-j` and
+# the numeric argument, or no argument at all.
+# shellcheck disable=SC2086
+time cmake --build build -j ${PARALLEL_LEVEL:-}

cccl_upstream/ci/ninja_summary.py

@@ -0,0 +1,390 @@
+#!/usr/bin/env python3
+# Copyright (c) 2018 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+r"""Summarize the last ninja build, invoked with ninja's -C syntax.
+
+This script is designed to be automatically run after each ninja build in
+order to summarize the build's performance. Making build performance information
+more visible should make it easier to notice anomalies and opportunities. To use
+this script on Windows just set NINJA_SUMMARIZE_BUILD=1 and run autoninja.bat.
+
+On Linux you can get autoninja to invoke this script using this syntax:
+
+$ NINJA_SUMMARIZE_BUILD=1 autoninja -C out/Default/ chrome
+
+You can also call this script directly using ninja's syntax to specify the
+output directory of interest:
+
+> python3 post_build_ninja_summary.py -C out/Default
+
+Typical output looks like this:
+
+>ninja -C out\debug_component base
+ninja.exe -C out\debug_component base -j 960 -l 48  -d keeprsp
+ninja: Entering directory `out\debug_component'
+[1 processes, 1/1 @ 0.3/s : 3.092s ] Regenerating ninja files
+Longest build steps:
+       0.1 weighted s to build obj/base/base/trace_log.obj (6.7 s elapsed time)
+       0.2 weighted s to build nasm.exe, nasm.exe.pdb (0.2 s elapsed time)
+       0.3 weighted s to build obj/base/base/win_util.obj (12.4 s elapsed time)
+       1.2 weighted s to build base.dll, base.dll.lib (1.2 s elapsed time)
+Time by build-step type:
+       0.0 s weighted time to generate 6 .lib files (0.3 s elapsed time sum)
+       0.1 s weighted time to generate 25 .stamp files (1.2 s elapsed time sum)
+       0.2 s weighted time to generate 20 .o files (2.8 s elapsed time sum)
+       1.7 s weighted time to generate 4 PEFile (linking) files (2.0 s elapsed
+time sum)
+      23.9 s weighted time to generate 770 .obj files (974.8 s elapsed time sum)
+26.1 s weighted time (982.9 s elapsed time sum, 37.7x parallelism)
+839 build steps completed, average of 32.17/s
+
+If no gn clean has been done then results will be for the last non-NULL
+invocation of ninja. Ideas for future statistics, and implementations are
+appreciated.
+
+The "weighted" time is the elapsed time of each build step divided by the number
+of tasks that were running in parallel. This makes it an excellent approximation
+of how "important" a slow step was. A link that is entirely or mostly serialized
+will have a weighted time that is the same or similar to its elapsed time. A
+compile that runs in parallel with 999 other compiles will have a weighted time
+that is tiny."""
+
+import argparse
+import errno
+import fnmatch
+import os
+import subprocess
+import sys
+
+# The number of long build times to report:
+long_count = 10
+# The number of long times by extension to report
+long_ext_count = 10
+
+
+class Target:
+    """Represents a single line read for a .ninja_log file."""
+
+    def __init__(self, start, end):
+        """Creates a target object by passing in the start/end times in seconds
+        as a float."""
+        self.start = start
+        self.end = end
+        # A list of targets, appended to by the owner of this object.
+        self.targets = []
+        self.weighted_duration = 0.0
+
+    def Duration(self):
+        """Returns the task duration in seconds as a float."""
+        return self.end - self.start
+
+    def SetWeightedDuration(self, weighted_duration):
+        """Sets the duration, in seconds, passed in as a float."""
+        self.weighted_duration = weighted_duration
+
+    def WeightedDuration(self):
+        """Returns the task's weighted duration in seconds as a float.
+
+        Weighted_duration takes the elapsed time of the task and divides it
+        by how many other tasks were running at the same time. Thus, it
+        represents the approximate impact of this task on the total build time,
+        with serialized or serializing steps typically ending up with much
+        longer weighted durations.
+        weighted_duration should always be the same or shorter than duration.
+        """
+        # Allow for modest floating-point errors
+        epsilon = 0.000002
+        if self.weighted_duration > self.Duration() + epsilon:
+            print("%s > %s?" % (self.weighted_duration, self.Duration()))
+        assert self.weighted_duration <= self.Duration() + epsilon
+        return self.weighted_duration
+
+    def DescribeTargets(self):
+        """Returns a printable string that summarizes the targets."""
+        # Some build steps generate dozens of outputs - handle them sanely.
+        # The max_length was chosen so that it can fit most of the long
+        # single-target names, while minimizing word wrapping.
+        result = ", ".join(self.targets)
+        max_length = 65
+        if len(result) > max_length:
+            result = result[:max_length] + "..."
+        return result
+
+
+# Copied with some modifications from ninjatracing
+def ReadTargets(log, show_all):
+    """Reads all targets from .ninja_log file |log_file|, sorted by duration.
+
+    The result is a list of Target objects."""
+    header = log.readline()
+    # Handle empty ninja_log gracefully by silently returning an empty list of
+    # targets.
+    if not header:
+        return []
+    assert header == "# ninja log v6\n", "unrecognized ninja log version %r" % header
+    targets_dict = {}
+    last_end_seen = 0.0
+    for line in log:
+        parts = line.strip().split("\t")
+        if len(parts) != 5:
+            # If ninja.exe is rudely halted then the .ninja_log file may be
+            # corrupt. Silently continue.
+            continue
+        start, end, _, name, cmdhash = parts  # Ignore restat.
+        # Convert from integral milliseconds to float seconds.
+        start = int(start) / 1000.0
+        end = int(end) / 1000.0
+        if not show_all and end < last_end_seen:
+            # An earlier time stamp means that this step is the first in a new
+            # build, possibly an incremental build. Throw away the previous
+            # data so that this new build will be displayed independently.
+            # This has to be done by comparing end times because records are
+            # written to the .ninja_log file when commands complete, so end
+            # times are guaranteed to be in order, but start times are not.
+            targets_dict = {}
+        target = None
+        if cmdhash in targets_dict:
+            target = targets_dict[cmdhash]
+            if not show_all and (target.start != start or target.end != end):
+                # If several builds in a row just run one or two build steps
+                # then the end times may not go backwards so the last build may
+                # not be detected as such. However in many cases there will be a
+                # build step repeated in the two builds and the changed
+                # start/stop points for that command, identified by the hash,
+                # can be used to detect and reset the target dictionary.
+                targets_dict = {}
+                target = None
+        if not target:
+            targets_dict[cmdhash] = target = Target(start, end)
+        last_end_seen = end
+        target.targets.append(name)
+    return list(targets_dict.values())
+
+
+def GetExtension(target, extra_patterns):
+    """Return the file extension that best represents a target.
+
+    For targets that generate multiple outputs it is important to return a
+    consistent 'canonical' extension. Ultimately the goal is to group build steps
+    by type."""
+    for output in target.targets:
+        if extra_patterns:
+            for fn_pattern in extra_patterns.split(";"):
+                if fnmatch.fnmatch(output, "*" + fn_pattern + "*"):
+                    return fn_pattern
+        # Not a true extension, but a good grouping.
+        if output.endswith("type_mappings"):
+            extension = "type_mappings"
+            break
+
+        # Capture two extensions if present. For example: file.javac.jar should
+        # be distinguished from file.interface.jar.
+        root, ext1 = os.path.splitext(output)
+        _, ext2 = os.path.splitext(root)
+        extension = ext2 + ext1  # Preserve the order in the file name.
+
+        if len(extension) == 0:
+            extension = "(no extension found)"
+
+        if ext1 in [".pdb", ".dll", ".exe"]:
+            extension = "PEFile (linking)"
+            # Make sure that .dll and .exe are grouped together and that the
+            # .dll.lib files don't cause these to be listed as libraries
+            break
+        if ext1 in [".so", ".TOC"]:
+            extension = ".so (linking)"
+            # Attempt to identify linking, avoid identifying as '.TOC'
+            break
+        # Make sure .obj files don't get categorized as mojo files
+        if ext1 in [".obj", ".o"]:
+            break
+        # Jars are the canonical output of java targets.
+        if ext1 == ".jar":
+            break
+        # Normalize all mojo related outputs to 'mojo'.
+        if output.count(".mojom") > 0:
+            extension = "mojo"
+            break
+    return extension
+
+
+def SummarizeEntries(entries, extra_step_types, elapsed_time_sorting):
+    """Print a summary of the passed in list of Target objects."""
+
+    # Create a list that is in order by time stamp and has entries for the
+    # beginning and ending of each build step (one time stamp may have multiple
+    # entries due to multiple steps starting/stopping at exactly the same time).
+    # Iterate through this list, keeping track of which tasks are running at all
+    # times. At each time step calculate a running total for weighted time so
+    # that when each task ends its own weighted time can easily be calculated.
+    task_start_stop_times = []
+
+    earliest = -1
+    latest = 0
+    total_cpu_time = 0
+    for target in entries:
+        if earliest < 0 or target.start < earliest:
+            earliest = target.start
+        if target.end > latest:
+            latest = target.end
+        total_cpu_time += target.Duration()
+        task_start_stop_times.append((target.start, "start", target))
+        task_start_stop_times.append((target.end, "stop", target))
+    length = latest - earliest
+    weighted_total = 0.0
+
+    # Sort by the time/type records and ignore |target|
+    task_start_stop_times.sort(key=lambda times: times[:2])
+    # Now we have all task start/stop times sorted by when they happen. If a
+    # task starts and stops on the same time stamp then the start will come
+    # first because of the alphabet, which is important for making this work
+    # correctly.
+    # Track the tasks which are currently running.
+    running_tasks = {}
+    # Record the time we have processed up to so we know how to calculate time
+    # deltas.
+    last_time = task_start_stop_times[0][0]
+    # Track the accumulated weighted time so that it can efficiently be added
+    # to individual tasks.
+    last_weighted_time = 0.0
+    # Scan all start/stop events.
+    for event in task_start_stop_times:
+        time, action_name, target = event
+        # Accumulate weighted time up to now.
+        num_running = len(running_tasks)
+        if num_running > 0:
+            # Update the total weighted time up to this moment.
+            last_weighted_time += (time - last_time) / float(num_running)
+        if action_name == "start":
+            # Record the total weighted task time when this task starts.
+            running_tasks[target] = last_weighted_time
+        if action_name == "stop":
+            # Record the change in the total weighted task time while this task
+            # ran.
+            weighted_duration = last_weighted_time - running_tasks[target]
+            target.SetWeightedDuration(weighted_duration)
+            weighted_total += weighted_duration
+            del running_tasks[target]
+        last_time = time
+    assert len(running_tasks) == 0
+
+    # Warn if the sum of weighted times is off by more than half a second.
+    if abs(length - weighted_total) > 500:
+        print(
+            "Warning: Possible corrupt ninja log, results may be "
+            "untrustworthy. Length = %.3f, weighted total = %.3f"
+            % (length, weighted_total)
+        )
+
+    # Print the slowest build steps:
+    print("    Longest build steps:")
+    if elapsed_time_sorting:
+        entries.sort(key=lambda x: x.Duration())
+    else:
+        entries.sort(key=lambda x: x.WeightedDuration())
+    for target in entries[-long_count:]:
+        print(
+            "      %8.1f weighted s to build %s (%.1f s elapsed time)"
+            % (target.WeightedDuration(), target.DescribeTargets(), target.Duration())
+        )
+
+    # Sum up the time by file extension/type of the output file
+    count_by_ext = {}
+    time_by_ext = {}
+    weighted_time_by_ext = {}
+    # Scan through all of the targets to build up per-extension statistics.
+    for target in entries:
+        extension = GetExtension(target, extra_step_types)
+        time_by_ext[extension] = time_by_ext.get(extension, 0) + target.Duration()
+        weighted_time_by_ext[extension] = (
+            weighted_time_by_ext.get(extension, 0) + target.WeightedDuration()
+        )
+        count_by_ext[extension] = count_by_ext.get(extension, 0) + 1
+
+    print("    Time by build-step type:")
+    # Copy to a list with extension name and total time swapped, to (time, ext)
+    if elapsed_time_sorting:
+        weighted_time_by_ext_sorted = sorted((y, x) for (x, y) in time_by_ext.items())
+    else:
+        weighted_time_by_ext_sorted = sorted(
+            (y, x) for (x, y) in weighted_time_by_ext.items()
+        )
+    # Print the slowest build target types:
+    for time, extension in weighted_time_by_ext_sorted[-long_ext_count:]:
+        print(
+            "      %8.1f s weighted time to generate %d %s files "
+            "(%1.1f s elapsed time sum)"
+            % (time, count_by_ext[extension], extension, time_by_ext[extension])
+        )
+
+    print(
+        "    %.1f s weighted time (%.1f s elapsed time sum, %1.1fx "
+        "parallelism)" % (length, total_cpu_time, total_cpu_time * 1.0 / length)
+    )
+    print(
+        "    %d build steps completed, average of %1.2f/s"
+        % (len(entries), len(entries) / (length))
+    )
+
+
+def main():
+    log_file = ".ninja_log"
+    metrics_file = "siso_metrics.json"
+    parser = argparse.ArgumentParser()
+    parser.add_argument("-C", dest="build_directory", help="Build directory.")
+    parser.add_argument(
+        "-s",
+        "--step-types",
+        help="semicolon separated fnmatch patterns for build-step grouping",
+    )
+    parser.add_argument(
+        "-e",
+        "--elapsed_time_sorting",
+        default=False,
+        action="store_true",
+        help="Sort output by elapsed time instead of weighted time",
+    )
+    parser.add_argument("--log-file", help="specific ninja log file to analyze.")
+    args, _extra_args = parser.parse_known_args()
+    if args.build_directory:
+        log_file = os.path.join(args.build_directory, log_file)
+        metrics_file = os.path.join(args.build_directory, metrics_file)
+    if args.log_file:
+        log_file = args.log_file
+    if not args.step_types:
+        # Offer a convenient way to add extra step types automatically,
+        # including when this script is run by autoninja. get() returns None if
+        # the variable isn't set.
+        args.step_types = os.environ.get("chromium_step_types")
+    if args.step_types:
+        # Make room for the extra build types.
+        global long_ext_count
+        long_ext_count += len(args.step_types.split(";"))
+
+    if os.path.exists(metrics_file):
+        # Automatically handle summarizing siso builds.
+        cmd = ["siso.bat" if "win32" in sys.platform else "siso"]
+        cmd.extend(["metrics", "summary"])
+        if args.build_directory:
+            cmd.extend(["-C", args.build_directory])
+        if args.step_types:
+            cmd.extend(["--step_types", args.step_types])
+        if args.elapsed_time_sorting:
+            cmd.append("--elapsed_time_sorting")
+        subprocess.run(cmd)
+    else:
+        try:
+            with open(log_file, "r") as log:
+                entries = ReadTargets(log, False)
+                if entries:
+                    SummarizeEntries(
+                        entries, args.step_types, args.elapsed_time_sorting
+                    )
+        except IOError:
+            print("Log file %r not found, no build summary created." % log_file)
+            return errno.ENOENT
+
+
+if __name__ == "__main__":
+    sys.exit(main())

cccl_upstream/ci/nvrtc_libcudacxx.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+
+# shellcheck source=ci/build_common.sh
+source "$(dirname "${BASH_SOURCE[0]}")/build_common.sh"
+
+print_environment_details
+
+
+PRESET="libcudacxx-nvrtc"
+CMAKE_OPTIONS=("-DCMAKE_CXX_STANDARD=${CXX_STANDARD}" "-DCMAKE_CUDA_STANDARD=${CXX_STANDARD}")
+
+configure_and_build_preset "libcudacxx NVRTC" "$PRESET" "${CMAKE_OPTIONS[@]}"
+
+sccache -z > /dev/null || :
+test_preset "libcudacxx NVRTC" "${PRESET}"
+sccache --show-adv-stats || :

cccl_upstream/ci/pretty_printing.sh

@@ -0,0 +1,122 @@
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+# Print "ARG=${ARG}" for all args.
+function print_var_values() {
+    # Iterate through the arguments
+    for var_name in "$@"; do
+        if [[ -z "$var_name" ]]; then
+            echo "Usage: print_var_values <variable_name1> <variable_name2> ..."
+            return 1
+        fi
+
+        # Dereference the variable and print the result
+        echo "$var_name=${!var_name:-(undefined)}"
+    done
+}
+
+# begin_group: Start a named section of log output, possibly with color.
+# Usage: begin_group "Group Name" [Color]
+#   Group Name: A string specifying the name of the group.
+#   Color (optional): ANSI color code to set text color. Default is blue (1;34).
+function begin_group() {
+    # See options for colors here: https://gist.github.com/JBlond/2fea43a3049b38287e5e9cefc87b2124
+    local blue="34"
+    local name="${1:-}"
+    local color="${2:-$blue}"
+
+    if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+        echo -e "::group::\e[${color}m${name}\e[0m"
+    else
+        echo -e "\e[${color}m================== ${name} ======================\e[0m"
+    fi
+}
+
+# end_group: End a named section of log output and print status based on exit status.
+# Usage: end_group "Group Name" [Exit Status]
+#   Group Name: A string specifying the name of the group.
+#   Exit Status (optional): The exit status of the command run within the group. Default is 0.
+function end_group() {
+    local name="${1:-}"
+    local build_status="${2:-0}"
+    local duration="${3:-}"
+    local red="31"
+    local blue="34"
+
+    if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+        echo "::endgroup::"
+
+        if [[ "$build_status" -ne 0 ]]; then
+            echo -e "::error::\e[${red}m ${name} - Failed (⬆️ click above for full log ⬆️)\e[0m"
+        fi
+    else
+        if [[ "$build_status" -ne 0 ]]; then
+            echo -e "\e[${red}m================== End ${name} - Failed${duration:+ - Duration: ${duration}s} ==================\e[0m"
+        else
+            echo -e "\e[${blue}m================== End ${name} - Success${duration:+ - Duration: ${duration}s} ==================\n\e[0m"
+        fi
+    fi
+}
+
+declare -A command_durations
+
+# Runs a command within a named group, handles the exit status, and prints appropriate messages based on the result.
+# Usage: run_command "Group Name" command [arguments...]
+function run_command() {
+    local group_name="${1:-}"
+    shift
+    local command=("$@")
+    local status
+
+    begin_group "$group_name"
+    echo "Working directory: $(pwd)"
+    echo "Running command: ${command[*]}"
+    set +e
+    local start_time
+    start_time=$(date +%s)
+    # If RUN_COMMAND_RETRY_PARAMS is set to "<retries> <sleep_time>", use retry.sh to run the command:
+    if [[ -v RUN_COMMAND_RETRY_PARAMS && "${#RUN_COMMAND_RETRY_PARAMS[@]}" -eq 2 ]]; then
+        status=0
+        "$ci_dir/util/retry.sh" "${RUN_COMMAND_RETRY_PARAMS[@]}" "${command[@]}" || status=$?
+    else
+        status=0
+        "${command[@]}" || status=$?
+    fi
+    local end_time
+    end_time=$(date +%s)
+    set -e
+    local duration=$((end_time - start_time))
+    end_group "$group_name" "$status" "$duration"
+    command_durations["$group_name"]=$duration
+    return "$status"
+}
+
+function string_width() {
+    local str="$1"
+    echo "$str" | awk '{print length}'
+}
+
+function print_time_summary() {
+    local max_length=0
+    local group
+
+    # Find the longest group name for formatting
+    for group in "${!command_durations[@]}"; do
+        local group_length
+        group_length=$(echo "$group" | awk '{print length}')
+        if [[ "$group_length" -gt "$max_length" ]]; then
+            max_length=$group_length
+        fi
+    done
+
+    if [[ "$max_length" -eq 0 ]]; then
+        return
+    fi
+
+    echo "Time Summary:"
+    for group in "${!command_durations[@]}"; do
+        printf "%-${max_length}s : %s seconds\n" "$group" "${command_durations[$group]}"
+    done
+
+    # Clear the array of timing info
+    declare -gA command_durations=()
+}

cccl_upstream/ci/project_files_and_dependencies.yaml

@@ -0,0 +1,287 @@
+# Configuration for CCCL project change detection.
+# Projects are declared in this file along with their human-friendly names, path
+# matching patterns, and dependency relationships. Every path pattern is
+# evaluated as a regular expression anchored to the repository root.
+#
+# Changed files that satisfy a project's include_regexes, exclude_regexes, and
+# exclude_project_files checks will mark that project as "dirty". The current project
+# will be added to the FULL_BUILD if it has dirty files or if any of its
+# full_dependencies are dirty.
+#
+# Additionally, if any of a project's lite_dependencies are dirty, the current project
+# will be added to the LITE_BUILD.
+#
+# If any transitive dependencies are dirty (eg. a full or lite dependency has
+# dirty dependencies of its own), the current project will also be added to the LITE_BUILD.
+#
+# Projects with large dependency trees are split into "public" and "internal" sub-projects.
+# The "public" sub-project contains only the public API files, while the "internal"
+# sub-project contains tests, examples, and other non-public files. This allows changes to
+# internal files to avoid triggering rebuilds of dependent projects when the public API
+# is not changed.
+#
+# Layout:
+#   project_key:          mapping from project key to configuration
+#     name:               display name used in logs/reports
+#     matrix_project:     project name to use for matrix generation. If not provided,
+#                           the project is considered internal-only and will not be listed
+#                           in the FULL_BUILD or LITE_BUILD outputs.
+#     lite_dependencies:  Add the current project to the `lite` matrix if any of these
+#                           dependencies are dirty.
+#     full_dependencies:  Add the current project to the `full` matrix if any of these
+#                           dependencies are dirty.
+#     include_regexes:    regexes that match files for this project
+#     exclude_regexes:    regexes that exclude files from the included set
+#     exclude_project_files: list of other project keys whose files should be
+#                           excluded from this project
+
+projects:
+  # `core` collects any dirty files that are not matched by other projects. It
+  # never appears in dependency lists because the script treats it specially,
+  # triggering a full rebuild of all projects when any core file is dirty.
+  core:
+    name: "CCCL Infrastructure"
+
+  libcudacxx_public:
+    name: "libcu++ Public API"
+    lite_dependencies: ['thrust_public', 'cub_public']
+    full_dependencies: []
+    include_regexes: ["libcudacxx/include/"]
+
+  libcudacxx_internal:
+    name: "libcu++ Tests/Infra"
+    matrix_project: "libcudacxx"
+    lite_dependencies: [c2h]
+    full_dependencies: [libcudacxx_public]
+    include_regexes: ["libcudacxx/"]
+    exclude_project_files: [libcudacxx_public]
+
+  cub_public:
+    name: "CUB Public API"
+    lite_dependencies: [libcudacxx_public, thrust_public]
+    full_dependencies: []
+    include_regexes: ["cub/cub/"]
+
+  cub_internal:
+    name: "CUB Tests/Infra"
+    matrix_project: "cub"
+    lite_dependencies: [c2h, nvbench_helper]
+    full_dependencies: [cub_public]
+    include_regexes: ["cub/"]
+    exclude_project_files: [cub_public]
+
+  thrust_public:
+    name: "Thrust Public API"
+    lite_dependencies: [libcudacxx_public, cub_public]
+    full_dependencies: []
+    include_regexes: ["thrust/thrust/"]
+
+  thrust_internal:
+    name: "Thrust Tests/Infra"
+    matrix_project: "thrust"
+    lite_dependencies: [nvbench_helper]
+    full_dependencies: [thrust_public]
+    include_regexes: ["thrust/"]
+    exclude_project_files: [thrust_public]
+
+  cudax_public:
+    name: "CUDA Experimental Public API"
+    lite_dependencies: [libcudacxx_public, thrust_public, cub_public]
+    full_dependencies: []
+    include_regexes: ["cudax/include/"]
+
+  cudax_internal:
+    name: "CUDA Experimental Tests/Infra"
+    matrix_project: "cudax"
+    lite_dependencies: [c2h, nvbench_helper]
+    full_dependencies: [cudax_public]
+    include_regexes: ["cudax/"]
+    exclude_project_files: [cudax_public]
+
+  cccl_c_parallel_public:
+    name: "CCCL C Parallel Library Public API"
+    lite_dependencies: [libcudacxx_public, cub_public, thrust_public]
+    full_dependencies: []
+    include_regexes:
+      - "c/parallel/include/"
+      - "c/parallel/src/"
+
+  cccl_c_parallel_internal:
+    name: "CCCL C Parallel Library Tests/Infra"
+    matrix_project: "cccl_c_parallel"
+    lite_dependencies: [c2h]
+    full_dependencies: [cccl_c_parallel_public]
+    include_regexes: ["c/parallel/"]
+    exclude_project_files: [cccl_c_parallel_public]
+
+  cccl_c_parallel_v2:
+    name: "CCCL C Parallel Library v2 (HostJIT)"
+    matrix_project: "cccl_c_parallel_v2"
+    # v2 depends on libcudacxx, cub, and thrust headers (it JIT-compiles
+    # CUB's host+device code via HostJIT). Any change to those should trigger
+    # v2 to run.
+    lite_dependencies: [libcudacxx_public, cub_public, thrust_public, c2h]
+    full_dependencies: []
+    include_regexes:
+      - "c/parallel\\.v2/"
+
+  python_v2:
+    name: "Python (cuda.compute on v2/HostJIT)"
+    matrix_project: "python_v2"
+    # cccl_c_parallel_v2 already pulls in libcudacxx/cub/thrust, so listing
+    # it here transitively triggers python_v2 on any of those upstream
+    # changes too. Direct includes catch Python-only edits.
+    lite_dependencies: [cccl_c_parallel_v2]
+    full_dependencies: []
+    include_regexes:
+      - "python/cuda_cccl/"
+      - "pyproject\\.toml"
+
+  python_tsan:
+    name: "Python (cuda.compute free-threaded ThreadSanitizer)"
+    matrix_project: "python_tsan"
+    # Same v1 sources as `python`, rebuilt with ThreadSanitizer. Must be listed
+    # here or the per-PR python_tsan job is pruned as never-dirty and never runs.
+    # cccl_c_parallel_public covers c/parallel/{src,include}; c/parallel/CMakeLists.txt
+    # is listed directly because the TSan build config (the -fsanitize=thread option)
+    # lives there and is owned by cccl_c_parallel_internal, which this lane otherwise
+    # does not depend on -- without it a change to the TSan flags would skip this lane.
+    lite_dependencies: [cccl_c_parallel_public]
+    full_dependencies: []
+    include_regexes:
+      - "python/cuda_cccl/"
+      - "pyproject\\.toml"
+      - "c/parallel/CMakeLists\\.txt"
+
+  cccl_c_stf:
+    name: "CCCL C CUDASTF Library"
+    matrix_project: "cccl_c_stf"
+    lite_dependencies: [libcudacxx_public, cudax_public, c2h]
+    full_dependencies: []
+    include_regexes:  ["c/experimental/stf/"]
+    exclude_regexes: []
+
+  python:
+    name: "Python"
+    matrix_project: "python"
+    lite_dependencies: [cccl_c_parallel_public]
+    full_dependencies: []
+    include_regexes:
+    - "python/"
+    - "pyproject.toml"
+
+  packaging:
+    name: "CCCL Packaging"
+    matrix_project: "packaging"
+    # Anything that affects CMake packages / install rules:
+    lite_dependencies:
+      - libcudacxx_internal
+      - libcudacxx_public
+      - cub_internal
+      - cub_public
+      - thrust_internal
+      - thrust_public
+      - cudax_internal
+      - cudax_public
+    full_dependencies: []
+    include_regexes:
+      - "examples/"
+      - "test/cmake/"
+      - "ci/test/"
+
+  stdpar:
+    name: "stdpar"
+    matrix_project: "stdpar"
+    lite_dependencies: [thrust_public]
+    full_dependencies: []
+    include_regexes: ["test/stdpar/"]
+
+  c2h:
+    name: "Catch2Helper"
+    lite_dependencies: [libcudacxx_public, cub_public, thrust_public]
+    full_dependencies: []
+    include_regexes: ["c2h/"]
+
+  nvbench_helper:
+    name: "NVBench Helper"
+    matrix_project: "nvbench_helper"
+    lite_dependencies: [libcudacxx_public, cub_public, thrust_public]
+    full_dependencies: []
+    include_regexes:
+      - "nvbench_helper/"
+      # CCCLBenchmarkRegistry.cmake is effectively part of nvbench_helper:
+      - "benchmarks/cmake/"
+
+  nvrtcc:
+    name: "nvrtcc"
+    matrix_project: "nvrtcc"
+    lite_dependencies: []
+    full_dependencies: []
+    include_regexes: ["nvrtcc/"]
+
+  # This is a dummy project, and only really serves to encode dependencies. Any time any
+  # of the C/C++ projects are modified, we want to launch one (and only one) clang-tidy
+  # job for the whole of CCCL.
+  tidy:
+    name: "clang-tidy"
+    matrix_project: "tidy"
+    full_dependencies:
+      - libcudacxx_public
+      - libcudacxx_internal
+      - cub_public
+      - cub_internal
+      - thrust_public
+      - thrust_internal
+      - cudax_public
+      - cudax_internal
+      - cccl_c_parallel_public
+      - cccl_c_parallel_internal
+      - cccl_c_parallel_v2
+      - cccl_c_stf
+      - stdpar
+      - c2h
+      - nvrtcc
+    include_regexes: [".clang-tidy"]
+
+# Files matching any of these regexes will be ignored globally.
+ignore_regexes:
+  - '.+\.md$'
+  - '\.branch_notes/'
+  - '\.coderabbit\.yaml'
+  - '\.clang-format'
+  - '\.clangd'
+  # Do not add .clang-tidy to this list. If we modify .clang-tidy to either add or remove
+  # a check we still want CI to run the clang-tidy job even if it results in no C/C++ code
+  # changes.
+  #
+  # - '\.clang-tidy'
+  - '\.devcontainer/img'
+  - '\.git-blame-ignore-revs'
+  - '\.github/actions/docs-build'
+  - '\.github/CODEOWNERS'
+  - '\.github/copy-pr-bot\.yaml'
+  - '\.github/ISSUE_TEMPLATE/'
+  - '\.github/problem-matchers/problem-matcher\.json'
+  - '\.github/workflows/backport-prs\.yml'
+  - '\.github/workflows/bench.*\.yml'
+  - '\.github/workflows/compile-time-bench\.yml'
+  - '\.github/workflows/build-docs\.yml'
+  - '\.github/workflows/build-matx\.yml'
+  - '\.github/workflows/build-pytorch\.yml'
+  - '\.github/workflows/build-rapids\.yml'
+  - '\.github/workflows/git-bisect\.yml'
+  - '\.github/workflows/project_automation.*\.yml'
+  - '\.github/workflows/release.*\.yml'
+  - '\.github/workflows/triage_rotation\.yml'
+  - '\.github/workflows/update_branch_version\.yml'
+  - '\.github/workflows/verify-devcontainers\.yml'
+  - '\.gitignore'
+  - "benchmarks/scripts/" # Python+bash scripts for running benchmarks, not used in tests.
+  - 'cccl-version.json'
+  - 'ci/bench/'
+  - 'ci/bench.+yaml'
+  - 'ci/matx/'
+  - 'ci/pytorch/'
+  - 'ci/rapids/'
+  - 'docs/'
+  - 'LICENSE'

cccl_upstream/ci/pyenv_helper.sh

@@ -0,0 +1,77 @@
+setup_python_env() {
+    local py_version=$1
+
+    # Source pretty_printing.sh for begin_group/end_group helpers
+    local script_dir
+    script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+    # shellcheck source=ci/pretty_printing.sh
+    source "${script_dir}/pretty_printing.sh"
+
+    begin_group "🐍 Setting up Python ${py_version} (uv)"
+
+    # Install uv if not present
+    if ! command -v uv &> /dev/null; then
+        curl -LsSf https://astral.sh/uv/install.sh | sh
+        export PATH="$HOME/.local/bin:$PATH"
+    fi
+
+    # Create a venv with the requested Python version.
+    # uv downloads a pre-built CPython binary automatically — no compilation needed.
+    uv venv --seed --python "${py_version}" "${HOME}/.cccl-venv"
+
+    # Windows venvs use Scripts/, Linux/macOS use bin/
+    if [[ -f "${HOME}/.cccl-venv/Scripts/activate" ]]; then
+        #shellcheck disable=SC1091
+        source "${HOME}/.cccl-venv/Scripts/activate"
+    else
+        #shellcheck disable=SC1091
+        source "${HOME}/.cccl-venv/bin/activate"
+    fi
+
+    end_group "🐍 Setting up Python ${py_version} (uv)"
+}
+
+# Pin the cuda-toolkit wheels to the container's CTK major.minor (read from nvcc)
+# via PIP_CONSTRAINT when the mode ($1) is "pinned" (the default; empty also means
+# pinned). "latest" and "sysctk" leave it unpinned; any other value is a hard
+# error. This is the lane's mode gate -- it runs before ctk_extra_flavor in every
+# script, so ctk_extra_flavor can assume the mode is already valid. Also sets and
+# exports cuda_version / cuda_major_version; the caller uses cuda_major_version in
+# the pip-extra name (e.g. minimal-cu${cuda_major_version}).
+pin_cuda_toolkit() {
+    cuda_version=$(nvcc --version | grep release | awk '{print $6}' | tr -d ',' | cut -d '.' -f 1-2 | cut -d 'V' -f 2)
+    cuda_major_version=$(echo "$cuda_version" | cut -d '.' -f 1)
+    export cuda_version cuda_major_version
+
+    local mode="${1:-pinned}"
+    case "${mode,,}" in
+        pinned)
+            export PIP_CONSTRAINT="${TMPDIR:-/tmp}/ctk-constraint.txt"
+            echo "cuda-toolkit==${cuda_version}.*" > "${PIP_CONSTRAINT}"
+            ;;
+        latest | sysctk)
+            # No pin. Clear any inherited constraint so it cannot affect the
+            # resolve (latest tests the newest minor; sysctk installs no
+            # cuda-toolkit wheel at all).
+            unset PIP_CONSTRAINT
+            ;;
+        *)
+            echo "ERROR: invalid ctk mode '${mode}' (expected pinned|latest|sysctk)" >&2
+            return 1
+            ;;
+    esac
+}
+
+# Echoes the pip-extra toolkit "flavor" for the mode ($1): "sysctk" when the mode
+# is sysctk (rely on the system-provided CUDA toolkit) or "cu" otherwise
+# (pip-installed toolkit). The mode is validated by pin_cuda_toolkit, which every
+# lane calls first. Combine with the CUDA major, e.g.
+# "minimal-$(ctk_extra_flavor "${ctk_mode}")${cuda_major_version}" -> minimal-sysctk12.
+ctk_extra_flavor() {
+    local mode="${1:-}"
+    if [[ "${mode,,}" == "sysctk" ]]; then
+        echo "sysctk"
+    else
+        echo "cu"
+    fi
+}

cccl_upstream/ci/pytorch/build_pytorch.sh

@@ -0,0 +1,130 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+readonly pytorch_repo=https://github.com/pytorch/pytorch.git
+readonly pytorch_branch=main
+
+# Ensure the script is being executed in the root cccl directory:
+cd "$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/../..";
+readonly cccl_repo="${PWD}"
+
+log_vars() {
+  for var in "$@"; do
+    echo "${var}=${!var}"
+  done
+}
+
+# Define CCCL_TAG to override the default CCCL SHA. Otherwise the current HEAD of the local checkout is used.
+echo "CCCL_TAG (override): ${CCCL_TAG-}";
+if test -n "${CCCL_TAG-}"; then
+    # If CCCL_TAG is defined, fetch it to the local checkout
+    git -C "${cccl_repo}" fetch origin "${CCCL_TAG}";
+    cccl_sha="$(git -C "${cccl_repo}" rev-parse FETCH_HEAD)";
+else
+    cccl_sha="$(git -C "${cccl_repo}" rev-parse HEAD)";
+fi
+
+readonly workdir="${cccl_repo}/build/${CCCL_BUILD_INFIX:-}/pytorch"
+
+log_vars \
+  pytorch_repo pytorch_branch \
+  cccl_repo cccl_sha \
+  workdir
+
+mkdir -p "${workdir}"
+cd "${workdir}"
+echo "Working in ${workdir}"
+
+echo "::group::Cloning CCCL..."
+rm -rf cccl
+git clone "${cccl_repo}"
+git -C cccl checkout "${cccl_sha}"
+echo "CCCL HEAD:"
+git -C cccl log -1 --format=short
+echo "::endgroup::"
+
+# Setup a CUDA environment with the requested CCCL.
+# Use a local directory to avoid modifying the actual CUDA install:
+echo "::group::Setting up clone of CUDA environment with custom CCCL..."
+(
+  set -x
+  rm -rf ./cuda
+  cp -Hr /usr/local/cuda ./cuda
+  rm -rf ./cuda/include/cccl/*
+  cccl/ci/install_cccl.sh ./cccl-install > /dev/null
+  cp -r ./cccl-install/include/* ./cuda/include/cccl
+)
+export PATH="$PWD/cuda/bin:$PATH"
+export CUDA_HOME="$PWD/cuda"
+export CUDA_PATH="$PWD/cuda"
+command -v nvcc
+nvcc --version
+echo "::endgroup::"
+
+echo "::group::Cloning PyTorch..."
+rm -rf pytorch
+git clone "${pytorch_repo}" -b "${pytorch_branch}" --recursive --depth 1
+echo "PyTorch HEAD:"
+git -C pytorch log -1 --format=short
+echo "::endgroup::"
+
+echo "::group::Installing PyTorch build dependencies..."
+pytorch_root="$PWD/pytorch"
+export PYTHONPATH="${pytorch_root}:${pytorch_root}/tools:${PYTHONPATH:-}"
+pip install -r "${pytorch_root}/requirements-build.txt"
+echo "::endgroup::"
+
+echo "::group::Configuring PyTorch..."
+rm -rf build
+mkdir build
+declare -a cmake_args=(
+  "-DUSE_NCCL=OFF"
+  # Need to define this explicitly, torch's FindCUDA logic adds ancient arches if left undefined:
+  "-DTORCH_CUDA_ARCH_LIST=7.5;8.0;9.0;10.0;12.0"
+)
+SCCACHE_NO_DIST_COMPILE=1 cmake -S ./pytorch -B ./build -G Ninja "${cmake_args[@]}"
+echo "::endgroup::"
+
+# Verify that the configured build is using the custom CUDA dir for CTK and nvcc:
+if ! grep -q "CUDA_TOOLKIT_ROOT_DIR:PATH=$PWD/cuda" ./build/CMakeCache.txt; then
+    echo "Error: CUDA_TOOLKIT_ROOT_DIR does not point to the custom CUDA";
+    exit 1;
+fi
+if ! grep -q "CUDA_NVCC_EXECUTABLE:FILEPATH=$PWD/cuda/bin/nvcc" ./build/CMakeCache.txt; then
+    echo "Error: CUDA_NVCC_EXECUTABLE does not point to the custom CUDA";
+    exit 1;
+fi
+
+# This builds a bunch of unnecessary targets. Leaving here to use as a fallback if the
+# ninja target extraction below starts failing:
+# echo "::group::Building torch_cuda target..."
+# cmake --build ./build/ --target torch_cuda
+# echo "::endgroup::"
+
+# This cuts the number of built targets roughly in half:
+echo "::group::Extracting cuda targets from build.ninja..."
+# Query ninja for all object files built from CUDA source files
+# that are part of the torch_cuda library:
+ninja -C ./build -t query lib/libtorch_cuda.so |
+  grep -E "torch_cuda\\.dir/.*\\.cu\\.o$" |
+  sort | uniq | tee build/cuda_targets.txt
+# At the time this script was written, there were 311 cuda targets.
+# Check that there are at least 100 detected targets, otherwise fail.
+num_targets=$(wc -l < build/cuda_targets.txt)
+if test "$num_targets" -lt 100; then
+    echo "Error: extracted cuda targets count is less than 100! ($num_targets)";
+    echo "This likely indicates a failure to extract the targets from ninja.";
+    exit 1;
+fi
+echo "::endgroup::"
+
+echo "::group::Building $num_targets pytorch CUDA targets with custom CCCL..."
+torch_cuda_targets="$(xargs -a build/cuda_targets.txt)"
+declare -a torch_cuda_targets="($torch_cuda_targets)"
+
+time ninja -C ./build "${torch_cuda_targets[@]}" "-j${PARALLEL_LEVEL:-}"
+
+echo "::endgroup::"
+
+echo "PyTorch CUDA targets built successfully with custom CCCL."

cccl_upstream/ci/rapids/cuda13.3-conda/devcontainer.json

@@ -0,0 +1,124 @@
+{
+  "image": "rapidsai/devcontainers:26.10-cpp-mambaforge",
+  "runArgs": [
+    "--init",
+    "--rm",
+    "--name",
+    "${localEnv:USER:anon}-${localWorkspaceFolderBasename}-rapids-26.10-cuda13.3-conda",
+    "--ulimit",
+    "nofile=500000"
+  ],
+  "hostRequirements": {
+    "gpu": "optional"
+  },
+  "containerEnv": {
+    "AWS_ROLE_ARN": "arn:aws:iam::279114543810:role/nv-gha-token-sccache-devs",
+    "CI": "${localEnv:CI}",
+    "CUDA_VERSION": "13.3",
+    "CUDAARCHS": "75-real",
+    "DEFAULT_CONDA_ENV": "rapids",
+    "DEVCONTAINER_UTILS_ENABLE_SCCACHE_DIST": "true",
+    "HISTFILE": "/home/coder/.cache/._bash_history",
+    "INFER_NUM_DEVICE_ARCHITECTURES": "1",
+    "LIBCUDF_KERNEL_CACHE_PATH": "/home/coder/cudf/cpp/build/latest/jitify_cache",
+    "MAX_DEVICE_OBJ_TO_COMPILE_IN_PARALLEL": "20",
+    "PYTHON_PACKAGE_MANAGER": "conda",
+    "PYTHONDONTWRITEBYTECODE": "1",
+    "PYTHONSAFEPATH": "1",
+    "PYTHONUNBUFFERED": "1",
+    "RAPIDS_cmake_GIT_REPO": "${localEnv:RAPIDS_cmake_GIT_REPO}",
+    "RAPIDS_cudf_GIT_REPO": "${localEnv:RAPIDS_cudf_GIT_REPO}",
+    "RAPIDS_cugraph_GIT_REPO": "${localEnv:RAPIDS_cugraph_GIT_REPO}",
+    "RAPIDS_cugraph_gnn_GIT_REPO": "${localEnv:RAPIDS_cugraph_gnn_GIT_REPO}",
+    "RAPIDS_cuml_GIT_REPO": "${localEnv:RAPIDS_cuml_GIT_REPO}",
+    "RAPIDS_cuopt_GIT_REPO": "${localEnv:RAPIDS_cuopt_GIT_REPO}",
+    "RAPIDS_cuvs_GIT_REPO": "${localEnv:RAPIDS_cuvs_GIT_REPO}",
+    "RAPIDS_kvikio_GIT_REPO": "${localEnv:RAPIDS_kvikio_GIT_REPO}",
+    "RAPIDS_LIBS": "${localEnv:RAPIDS_LIBS}",
+    "RAPIDS_nvforest_GIT_REPO": "${localEnv:RAPIDS_nvforest_GIT_REPO}",
+    "RAPIDS_raft_GIT_REPO": "${localEnv:RAPIDS_raft_GIT_REPO}",
+    "RAPIDS_rmm_GIT_REPO": "${localEnv:RAPIDS_rmm_GIT_REPO}",
+    "RAPIDS_ucxx_GIT_REPO": "${localEnv:RAPIDS_ucxx_GIT_REPO}",
+    "SCCACHE_BUCKET": "rapids-sccache-devs",
+    "SCCACHE_DIST_FALLBACK_TO_LOCAL_COMPILE": "${localEnv:SCCACHE_DIST_FALLBACK_TO_LOCAL_COMPILE:true}",
+    "SCCACHE_DIST_MAX_RETRIES": "${localEnv:SCCACHE_DIST_MAX_RETRIES:4}",
+    "SCCACHE_DIST_REQUEST_TIMEOUT": "${localEnv:SCCACHE_DIST_REQUEST_TIMEOUT:7140}",
+    "SCCACHE_IDLE_TIMEOUT": "${localEnv:SCCACHE_IDLE_TIMEOUT:0}",
+    "SCCACHE_REGION": "us-east-2",
+    "SCCACHE_S3_USE_PREPROCESSOR_CACHE_MODE": "true",
+    "SCCACHE_SERVER_LOG": "${localEnv:SCCACHE_SERVER_LOG:sccache=debug}"
+  },
+  "initializeCommand": [
+    "/bin/bash",
+    "-c",
+    "mkdir -m 0755 -p ${localWorkspaceFolder}/.{aws,cache,config,local/state} ${localWorkspaceFolder}/ci/rapids/.{conda,log/devcontainer-utils} ${localWorkspaceFolder}/ci/rapids/.repos/{rmm,kvikio,ucxx,cudf,rapidsmpf,raft,cuvs,nvforest,cuml,cugraph,cugraph-gnn,cuopt}"
+  ],
+  "postCreateCommand": [
+    "/bin/bash",
+    "-c",
+    "if [ ${CI:-false} = 'false' ]; then . /home/coder/cccl/ci/rapids/post-create-command.sh; fi; if test -z \"${DISABLE_SCCACHE:+x}\"; then echo \"export SCCACHE_DIST_URL='https://$(uname -m | sed -e 's/x86_/amd/' -e 's/aarch/arm/').linux.sccache.rapids.nvidia.com'\" >> /home/coder/.bashrc; fi"
+  ],
+  "postAttachCommand": [
+    "/bin/bash",
+    "-c",
+    "if [ ${CODESPACES:-false} = 'true' ]; then . devcontainer-utils-post-attach-command; fi"
+  ],
+  "workspaceFolder": "/home/coder/cccl",
+  "workspaceMount": "source=${localWorkspaceFolder},target=/home/coder/cccl,type=bind",
+  "mounts": [
+    "source=/etc/timezone,target=/etc/timezone,type=bind",
+    "source=/etc/localtime,target=/etc/localtime,type=bind",
+    "source=${localWorkspaceFolder}/.aws,target=/home/coder/.aws,type=bind",
+    "source=${localWorkspaceFolder}/.cache,target=/home/coder/.cache,type=bind",
+    "source=${localWorkspaceFolder}/.config,target=/home/coder/.config,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.repos/rmm,target=/home/coder/rmm,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.repos/kvikio,target=/home/coder/kvikio,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.repos/ucxx,target=/home/coder/ucxx,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.repos/cudf,target=/home/coder/cudf,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.repos/rapidsmpf,target=/home/coder/rapidsmpf,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.repos/raft,target=/home/coder/raft,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.repos/cuvs,target=/home/coder/cuvs,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.repos/nvforest,target=/home/coder/nvforest,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.repos/cuml,target=/home/coder/cuml,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.repos/cugraph,target=/home/coder/cugraph,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.repos/cugraph-gnn,target=/home/coder/cugraph-gnn,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.repos/cuopt,target=/home/coder/cuopt,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.conda,target=/home/coder/.conda,type=bind",
+    "source=${localWorkspaceFolder}/ci/rapids/.log/devcontainer-utils,target=/var/log/devcontainer-utils,type=bind"
+  ],
+  "features": {
+    "ghcr.io/rapidsai/devcontainers/features/rapids-build-utils:26.10": {}
+  },
+  "overrideFeatureInstallOrder": [
+    "ghcr.io/rapidsai/devcontainers/features/rapids-build-utils"
+  ],
+  "customizations": {
+    "vscode": {
+      "extensions": [
+        "augustocdias.tasks-shell-input",
+        "ms-python.flake8",
+        "nvidia.nsight-vscode-edition"
+      ],
+      "files.watcherExclude": {
+        "**/build/**": true,
+        "**/_skbuild/**": true,
+        "**/target/**": true,
+        "/home/coder/.aws/**/*": true,
+        "/home/coder/.cache/**/*": true,
+        "/home/coder/.conda/**/*": true,
+        "/home/coder/.local/share/**/*": true,
+        "/home/coder/.vscode-server/**/*": true
+      },
+      "search.exclude": {
+        "**/build/**": true,
+        "**/_skbuild/**": true,
+        "**/*.code-search": true,
+        "/home/coder/.aws/**/*": true,
+        "/home/coder/.cache/**/*": true,
+        "/home/coder/.conda/**/*": true,
+        "/home/coder/.local/share/**/*": true,
+        "/home/coder/.vscode-server/**/*": true
+      }
+    }
+  }
+}

cccl_upstream/ci/rapids/post-create-command.sh

@@ -0,0 +1,146 @@
+#!/usr/bin/env bash
+
+PROJECT_MANIFEST_YML="${PROJECT_MANIFEST_YML:-"/opt/rapids-build-utils/manifest.yaml"}"
+
+_restore_original_manifest() {
+    if ! test -f "/tmp/manifest.yaml.orig"; then
+        cp "${PROJECT_MANIFEST_YML}" /tmp/manifest.yaml.orig;
+    fi
+    sudo cp /tmp/manifest.yaml.orig "${PROJECT_MANIFEST_YML}";
+}
+
+_apply_manifest_modifications() {
+    # Restore the original manifest.yaml
+    _restore_original_manifest;
+    # Remove unnecessary libs from manifest.yaml
+    _prune_libs_from_manifest;
+    # Update manifest.yaml repo git info
+    _update_repo_git_info;
+    # Create rapids-cmake override JSON file and update default CMake arguments in manifest.yaml
+    _create_rapids_cmake_override_json;
+    # Print the entire manifest.yaml after modifications
+    cat "${PROJECT_MANIFEST_YML}";
+    # Regenerate the RAPIDS build scripts from the new manifest.yaml
+    rapids-generate-scripts;
+}
+
+_prune_libs_from_manifest() {
+    local -;
+    set -euo pipefail;
+    if test -n "${RAPIDS_LIBS-}"; then
+        local -a filters="(${RAPIDS_LIBS})";
+        # prefix each element
+        filters=("${filters[@]/#/'"'}");
+        # suffix each element
+        filters=("${filters[@]/%/'",'}");
+        # Remove trailing comma
+        local -r filters_str="$(cut -d',' -f1-${#filters[@]} <<< "${filters[*]}")";
+        sudo yq -i ".repos |= filter(.cpp[].name | contains(${filters_str}))" "${PROJECT_MANIFEST_YML}";
+    fi
+}
+
+# shellcheck disable=SC2016
+_update_repo_git_info() {
+    local -;
+    set -euo pipefail;
+    yq '.repos[].name' "${PROJECT_MANIFEST_YML}" \
+  | xargs -r -n1 bash -c '
+    var="RAPIDS_${1//-/_}_GIT_REPO";
+    if test -v "${var}" && test -n "${!var}"; then
+        sudo yq -i "(.repos[] | select(.name == \"$1\") | .git) *= ${!var}" "${0}";
+    fi' "${PROJECT_MANIFEST_YML}";
+}
+
+_create_rapids_cmake_override_json() {
+    local -;
+    set -euo pipefail;
+    local rapids_cmake_tag=;
+    local rapids_cmake_upstream=;
+    if test -n "${RAPIDS_cmake_GIT_REPO-}"; then
+        rapids_cmake_tag="$(jq -r '.tag' <<< "${RAPIDS_cmake_GIT_REPO}")";
+        rapids_cmake_upstream="$(jq -r '.upstream' <<< "${RAPIDS_cmake_GIT_REPO}")";
+    else
+        rapids_cmake_tag="$(yq '.x-git-defaults.tag' /opt/rapids-build-utils/manifest.yaml)";
+        rapids_cmake_upstream="$(yq '.x-git-defaults.upstream' /opt/rapids-build-utils/manifest.yaml)";
+    fi
+
+    # Define CCCL_TAG to override the default CCCL SHA. Otherwise the current HEAD of the local checkout is used.
+    if test -n "${CCCL_TAG-}"; then
+        # If CCCL_TAG is defined, fetch it to the local checkout
+        git -C "${HOME}/cccl" fetch origin "${CCCL_TAG}";
+        cccl_sha="$(git -C "${HOME}/cccl" rev-parse FETCH_HEAD)";
+    else
+        cccl_sha="$(git -C "${HOME}/cccl" rev-parse HEAD)";
+    fi
+    echo "CCCL_VERSION: ${CCCL_VERSION-}";
+    echo "CCCL_TAG: ${CCCL_TAG-}";
+    echo "cccl_sha: ${cccl_sha}";
+
+    echo
+    echo "Replacing CCCL repo information in rapids-cmake versions.json:";
+    curl -fsSL -o- "https://raw.githubusercontent.com/${rapids_cmake_upstream}/rapids-cmake/${rapids_cmake_tag}/rapids-cmake/cpm/versions.json" \
+  | jq -r ".packages.CCCL *= {\"git_url\": \"${HOME}/cccl\", \"git_tag\": \"${cccl_sha}\", \"always_download\": true}" \
+  | jq -r "del(.packages.CCCL.url) | del(.packages.CCCL.url_hash)" \
+  > ~/rapids-cmake-override-versions-cccl-repo.json;
+
+    if test -n "${CCCL_VERSION-}"; then
+        echo "Patching CCCL_VERSION in rapids-cmake versions.json:";
+        jq -r ".packages.CCCL.version = \"${CCCL_VERSION}\"" ~/rapids-cmake-override-versions-cccl-repo.json \
+      > ~/rapids-cmake-override-versions.json;
+    else
+        echo "Using the default CCCL version in rapids-cmake versions.json:";
+        mv ~/rapids-cmake-override-versions-cccl-repo.json ~/rapids-cmake-override-versions.json;
+    fi
+
+    echo
+    echo "Final rapids-cmake-override-versions.json:";
+    cat ~/rapids-cmake-override-versions.json;
+
+    # Define default CMake args for each repo
+    local -a cmake_args=(BUILD_TESTS BUILD_BENCHMARKS BUILD_PRIMS_BENCH BUILD_CUGRAPH_MG_TESTS);
+    # Enable tests
+    cmake_args=("${cmake_args[@]/#/"-D"}");
+    cmake_args=("${cmake_args[@]/%/"=${RAPIDS_ENABLE_TESTS:-ON}"}");
+
+    # Always build RAFT shared lib
+    cmake_args+=("-DBUILD_SHARED_LIBS=ON");
+    cmake_args+=("-DRAFT_COMPILE_LIBRARY=ON");
+
+    # Tell rapids-cmake to use custom CCCL and cuCollections forks
+    cmake_args+=("-Drapids-cmake-branch=${rapids_cmake_tag}");
+    cmake_args+=("-Drapids-cmake-repo=${rapids_cmake_upstream}/rapids-cmake");
+    cmake_args+=("-DRAPIDS_CMAKE_CPM_DEFAULT_VERSION_FILE=${HOME}/rapids-cmake-override-versions.json");
+
+    # Inject the CMake args into manifest.yaml
+    sudo yq -i "(.repos[] | .cpp[] | .args.cmake) += \" ${cmake_args[*]}\"" "${PROJECT_MANIFEST_YML}";
+    sudo yq -i "(.repos[] | .python[] | .args.cmake) += \" ${cmake_args[*]}\"" "${PROJECT_MANIFEST_YML}";
+}
+
+_run_post_create_command() {
+    local -;
+    set -e;
+
+    # Install `rapids-build-utils` feature if it's not installed (i.e. if running locally via `.devcontainer/launch.sh -d`)
+    if ! test -f "${PROJECT_MANIFEST_YML}"; then
+        git clone --depth 1 --filter=blob:none --sparse https://github.com/rapidsai/devcontainers.git /tmp/rapidsai-devcontainers;
+        git -C /tmp/rapidsai-devcontainers sparse-checkout set features/src/rapids-build-utils;
+        (
+            cd /tmp/rapidsai-devcontainers/features/src/rapids-build-utils;
+            sudo bash ./install.sh;
+        )
+        rm -rf /tmp/rapidsai-devcontainers;
+    fi
+
+    # Modify manifest.yaml based on envvars
+    _apply_manifest_modifications;
+
+    # Clone all the repos
+    gh config set git_protocol https;
+    gh config set git_protocol https --host github.com;
+
+    clone-all -j "$(nproc --all)" -v -q --clone-upstream --single-branch --shallow-submodules --no-update-env;
+}
+
+if [[ "$(basename "${BASH_SOURCE[${#BASH_SOURCE[@]}-1]}")" = post-create-command.sh ]]; then
+    _run_post_create_command;
+fi

cccl_upstream/ci/rapids/rapids-entrypoint.sh

@@ -0,0 +1,22 @@
+#!/usr/bin/env bash
+
+# shellcheck disable=SC1091
+
+set -e;
+
+if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+    echo "::group::Cloning RAPIDS..."
+fi
+
+ci/rapids/post-create-command.sh;
+rapids-post-start-command -f;
+
+if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+    echo "::endgroup::"
+fi
+
+if test $# -gt 0; then
+    exec "$@";
+else
+    exec /bin/bash -li;
+fi

cccl_upstream/ci/run_cpu_bisect.sh

@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+
+# CI wrapper for the `bisect` project build job.
+# Forwards all arguments to ci/util/git_bisect.sh to run a bisection
+# using the provided configuration/build/test options.
+
+set -euo pipefail
+
+ci_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+repo_dir=$(cd "${ci_dir}/.." && pwd)
+
+user_args=("$@")
+set --
+source "${ci_dir}/build_common.sh"
+set -- "${user_args[@]}"
+
+cd "${repo_dir}"
+cmd=("${ci_dir}/util/git_bisect.sh" "$@")
+printf '\033[34m%s\033[0m\n' "${cmd[*]}"
+"${cmd[@]}"

cccl_upstream/ci/run_cpu_target.sh

@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+
+# CI wrapper for the `target` project build job.
+# Forwards all arguments to ci/util/build_and_test_targets.sh to configure
+# and build selected targets on a CPU runner.
+
+set -euo pipefail
+
+ci_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+repo_dir=$(cd "${ci_dir}/.." && pwd)
+
+user_args=("$@")
+set --
+source "${ci_dir}/build_common.sh"
+set -- "${user_args[@]}"
+
+cd "${repo_dir}"
+cmd=("${ci_dir}/util/build_and_test_targets.sh" "$@")
+printf '\033[34m%s\033[0m\n' "${cmd[*]}"
+"${cmd[@]}"

cccl_upstream/ci/run_gpu_bisect.sh

@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+
+# CI wrapper for the `bisect` project test job.
+# Invokes ci/util/git_bisect.sh with the provided arguments to
+# run a bisection on a GPU runner.
+
+set -euo pipefail
+
+ci_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+repo_dir=$(cd "${ci_dir}/.." && pwd)
+
+user_args=("$@")
+set --
+source "${ci_dir}/build_common.sh"
+set -- "${user_args[@]}"
+
+cd "${repo_dir}"
+cmd=("${ci_dir}/util/git_bisect.sh" "$@")
+printf '\033[34m%s\033[0m\n' "${cmd[*]}"
+"${cmd[@]}"

cccl_upstream/ci/run_gpu_target.sh

@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+
+# CI wrapper for the `target` project test job.
+# Invokes ci/util/build_and_test_targets.sh with the provided arguments to
+# build and test selected targets on a GPU runner.
+
+set -euo pipefail
+
+ci_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+repo_dir=$(cd "${ci_dir}/.." && pwd)
+
+user_args=("$@")
+set --
+source "${ci_dir}/build_common.sh"
+set -- "${user_args[@]}"
+
+cd "${repo_dir}"
+cmd=("${ci_dir}/util/build_and_test_targets.sh" "$@")
+printf '\033[34m%s\033[0m\n' "${cmd[*]}"
+"${cmd[@]}"

cccl_upstream/ci/test/CMakeLists.txt

@@ -0,0 +1 @@
+add_subdirectory("inspect_changes")

cccl_upstream/ci/test/inspect_changes/CMakeLists.txt

@@ -0,0 +1,22 @@
+find_package(Python3 REQUIRED COMPONENTS Interpreter)
+
+file(GLOB output_files "${CMAKE_CURRENT_SOURCE_DIR}/*.output")
+foreach (output_file IN LISTS output_files)
+  get_filename_component(test_name "${output_file}" NAME_WE)
+  set(dirty_file "${CMAKE_CURRENT_SOURCE_DIR}/${test_name}.dirty_files")
+  if (NOT EXISTS "${dirty_file}")
+    message(FATAL_ERROR "Missing dirty file for ${test_name}")
+  endif()
+
+  add_test(
+    NAME cccl.ci.test.inspect_changes.${test_name}
+    # gersemi: off
+    COMMAND
+      "${Python3_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/run_inspect_changes_test.py"
+        "--python" "${Python3_EXECUTABLE}"
+        "--script" "${PROJECT_SOURCE_DIR}/ci/inspect_changes.py"
+        "--dirty" "${dirty_file}"
+        "--expected" "${output_file}"
+    # gersemi: on
+  )
+endforeach()

cccl_upstream/ci/test/inspect_changes/c2h_dependency.dirty_files

@@ -0,0 +1 @@
+c2h/catch2_runner.cu

cccl_upstream/ci/test/inspect_changes/c2h_dependency.output

@@ -0,0 +1,2 @@
+FULL_BUILD=tidy
+LITE_BUILD=libcudacxx cub cudax cccl_c_parallel cccl_c_parallel_v2 python_v2 cccl_c_stf packaging

cccl_upstream/ci/test/inspect_changes/core_dirty.dirty_files

@@ -0,0 +1 @@
+CMakePresets.json

cccl_upstream/ci/test/inspect_changes/core_dirty.output

@@ -0,0 +1,2 @@
+FULL_BUILD=libcudacxx cub thrust cudax cccl_c_parallel cccl_c_parallel_v2 python_v2 python_tsan cccl_c_stf python packaging stdpar nvbench_helper nvrtcc tidy
+LITE_BUILD=

cccl_upstream/ci/test/inspect_changes/ignored_only.dirty_files

@@ -0,0 +1 @@
+docs/index.rst

cccl_upstream/ci/test/inspect_changes/ignored_only.output

@@ -0,0 +1,2 @@
+FULL_BUILD=
+LITE_BUILD=

cccl_upstream/ci/test/inspect_changes/libcudacxx_both.dirty_files

@@ -0,0 +1,2 @@
+libcudacxx/CMakeLists.txt
+libcudacxx/include/cuda/__device/device_ref.h

cccl_upstream/ci/test/inspect_changes/libcudacxx_both.output

@@ -0,0 +1,2 @@
+FULL_BUILD=libcudacxx tidy
+LITE_BUILD=cub thrust cudax cccl_c_parallel cccl_c_parallel_v2 python_v2 python_tsan cccl_c_stf python packaging stdpar nvbench_helper

cccl_upstream/ci/test/inspect_changes/libcudacxx_internal_only.dirty_files

@@ -0,0 +1 @@
+libcudacxx/CMakeLists.txt

cccl_upstream/ci/test/inspect_changes/libcudacxx_internal_only.output

@@ -0,0 +1,2 @@
+FULL_BUILD=libcudacxx tidy
+LITE_BUILD=packaging

cccl_upstream/ci/test/inspect_changes/libcudacxx_public_only.dirty_files

@@ -0,0 +1 @@
+libcudacxx/include/cuda/__device/device_ref.h

cccl_upstream/ci/test/inspect_changes/libcudacxx_public_only.output

@@ -0,0 +1,2 @@
+FULL_BUILD=libcudacxx tidy
+LITE_BUILD=cub thrust cudax cccl_c_parallel cccl_c_parallel_v2 python_v2 python_tsan cccl_c_stf python packaging stdpar nvbench_helper

cccl_upstream/ci/test/inspect_changes/libcudacxx_thrust.dirty_files

@@ -0,0 +1,3 @@
+libcudacxx/include/cuda/__device/device_ref.h
+thrust/thrust/version.h
+README.md

cccl_upstream/ci/test/inspect_changes/libcudacxx_thrust.output

@@ -0,0 +1,2 @@
+FULL_BUILD=libcudacxx thrust tidy
+LITE_BUILD=cub cudax cccl_c_parallel cccl_c_parallel_v2 python_v2 python_tsan cccl_c_stf python packaging stdpar nvbench_helper

cccl_upstream/ci/test/inspect_changes/multiple_projects.dirty_files

@@ -0,0 +1,2 @@
+python/cuda_cccl/pyproject.toml
+examples/basic/CMakeLists.txt

cccl_upstream/ci/test/inspect_changes/multiple_projects.output

@@ -0,0 +1,2 @@
+FULL_BUILD=python_v2 python_tsan python packaging
+LITE_BUILD=

cccl_upstream/ci/test/inspect_changes/no_changes.output

@@ -0,0 +1,2 @@
+FULL_BUILD=
+LITE_BUILD=

cccl_upstream/ci/test/inspect_changes/packaging_only.dirty_files

@@ -0,0 +1 @@
+examples/CMakeLists.txt

cccl_upstream/ci/test/inspect_changes/packaging_only.output

@@ -0,0 +1,2 @@
+FULL_BUILD=packaging
+LITE_BUILD=

cccl_upstream/ci/test/inspect_changes/regenerate_outputs.sh

@@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+project_root=$(cd "${script_dir}/../../.." && pwd)
+inspect_changes="${project_root}/ci/inspect_changes.py"
+
+if [[ ! -x "${inspect_changes}" ]]; then
+  echo "Error: ${inspect_changes} not found or not executable." >&2
+  exit 1
+fi
+
+cd "${script_dir}"
+
+for dirty_file in *.dirty_files; do
+  test_name=${dirty_file%.dirty_files}
+  output_file="${test_name}.output"
+  echo "Regenerating ${output_file}"
+  python "${inspect_changes}" --file "${dirty_file}" \
+    | awk '/^FULL_BUILD=/{print}/^LITE_BUILD=/{print}' > "${output_file}"
+done

cccl_upstream/ci/test/inspect_changes/run_inspect_changes_test.py

@@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+"""Test harness for ci/inspect_changes.py."""
+
+from __future__ import annotations
+
+import argparse
+import difflib
+import subprocess
+import sys
+from pathlib import Path
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description="Run inspect_changes.py and compare output"
+    )
+    parser.add_argument(
+        "--script", required=True, type=Path, help="Path to inspect_changes.py"
+    )
+    parser.add_argument(
+        "--dirty", required=True, type=Path, help="File listing dirty files"
+    )
+    parser.add_argument(
+        "--expected", required=True, type=Path, help="Expected stdout contents"
+    )
+    parser.add_argument(
+        "--python", default=sys.executable, help="Python interpreter to use"
+    )
+    return parser.parse_args()
+
+
+def build_command(python: str, script: Path, dirty_file: Path) -> list[str]:
+    cmd = [python, str(script), "--file", str(dirty_file)]
+    return cmd
+
+
+def main() -> int:
+    args = parse_args()
+    cmd = build_command(args.python, args.script, args.dirty)
+
+    sys.stdout.write(f"COMMAND: {' '.join(cmd)}\n")
+    result = subprocess.run(cmd, capture_output=True, text=True)
+    sys.stdout.write("OUTPUT:\n")
+    sys.stdout.write(result.stdout)
+    if result.stdout and not result.stdout.endswith("\n"):
+        sys.stdout.write("\n")
+    if result.returncode != 0:
+        sys.stderr.write(result.stderr)
+        sys.stderr.write("\nCommand failed: {}\n".format(" ".join(cmd)))
+        return result.returncode
+
+    actual_lines = result.stdout.splitlines()
+    expected_lines = [
+        line.strip()
+        for line in args.expected.read_text(encoding="utf-8").splitlines()
+        if line.strip()
+    ]
+
+    missing = [line for line in expected_lines if line not in actual_lines]
+    if missing:
+        diff = "\n".join(
+            difflib.unified_diff(
+                expected_lines,
+                actual_lines,
+                fromfile="expected(subset)",
+                tofile="actual",
+                lineterm="",
+            )
+        )
+        if diff:
+            sys.stderr.write(diff + "\n\n")
+        sys.stderr.write("\nExpected lines missing from output:\n")
+        for line in missing:
+            sys.stderr.write(f"  {line}\n")
+        return 1
+
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

cccl_upstream/ci/test_cccl_c_parallel.sh

@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+
+source "$(dirname "${BASH_SOURCE[0]}")/build_common.sh"
+
+print_environment_details
+
+./build_cccl_c_parallel.sh "$@"
+
+PRESET="cccl-c-parallel"
+
+test_preset "CCCL C Parallel Library" "${PRESET}"
+
+print_time_summary

cccl_upstream/ci/test_cccl_c_parallel_v2.sh

@@ -0,0 +1,34 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+source "$(dirname "${BASH_SOURCE[0]}")/build_common.sh"
+
+print_environment_details
+
+# libnvfatbin is required by hostjit but is not included in the base rapidsai devcontainer
+# image. Detect the installed CTK version and install the matching package if missing.
+if [[ "$(uname -s)" == "Linux" ]] && ! ldconfig -p 2>/dev/null | grep -q libnvfatbin; then
+  CTK_DEB_VER=$(nvcc --version 2>/dev/null \
+    | grep -oP 'release \K[0-9]+\.[0-9]+' | tr '.' '-')
+  if [[ -n "$CTK_DEB_VER" ]]; then
+    echo "Installing libnvfatbin-dev-${CTK_DEB_VER}..."
+    sudo apt-get update -y
+    sudo apt-get install -y --no-install-recommends "libnvfatbin-dev-${CTK_DEB_VER}"
+  else
+    echo "WARNING: could not determine CTK version; skipping libnvfatbin install"
+  fi
+fi
+
+PRESET="cccl-c-parallel-v2"
+
+CMAKE_OPTIONS=()
+if test -n "${CXX_STANDARD:+x}"; then
+    CMAKE_OPTIONS+=("-DCMAKE_CXX_STANDARD=${CXX_STANDARD}" "-DCMAKE_CUDA_STANDARD=${CXX_STANDARD}")
+fi
+
+configure_and_build_preset "CCCL C Parallel Library v2 (HostJIT)" "$PRESET" "${CMAKE_OPTIONS[@]}"
+
+test_preset "CCCL C Parallel Library v2 (HostJIT)" "$PRESET"
+
+print_time_summary

cccl_upstream/ci/test_cccl_c_stf.sh

@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+
+source "$(dirname "${BASH_SOURCE[0]}")/build_common.sh"
+
+print_environment_details
+
+./build_cccl_c_stf.sh "$@"
+
+PRESET="cccl-c-stf"
+
+test_preset "CCCL C Parallel Library" "${PRESET}"
+
+print_time_summary

cccl_upstream/ci/test_cub.sh

@@ -0,0 +1,142 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+NO_LID=false
+LID0=false
+LID1=false
+LID2=false
+LIMITED=false
+COMPUTE_SANITIZER=false
+ARTIFACT_TAGS=()
+
+ci_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+
+new_args="$("${ci_dir}/util/extract_switches.sh" \
+  -no-lid \
+  -lid0 \
+  -lid1 \
+  -lid2 \
+  -limited \
+  -compute-sanitizer-memcheck \
+  -compute-sanitizer-racecheck \
+  -compute-sanitizer-initcheck \
+  -compute-sanitizer-synccheck \
+  -- "$@")"
+
+declare -a new_args="(${new_args})"
+set -- "${new_args[@]}"
+while true; do
+  case "$1" in
+  -no-lid)
+    ARTIFACT_TAGS+=("no_lid")
+    NO_LID=true
+    shift
+    ;;
+  -lid0)
+    ARTIFACT_TAGS+=("lid_0")
+    LID0=true
+    shift
+    ;;
+  -lid1)
+    ARTIFACT_TAGS+=("lid_1")
+    LID1=true
+    shift
+    ;;
+  -lid2)
+    ARTIFACT_TAGS+=("lid_2")
+    LID2=true
+    shift
+    ;;
+  -limited)
+    # Pull all artifacts:
+    ARTIFACT_TAGS+=("no_lid" "lid_0" "lid_1" "lid_2")
+    LIMITED=true
+    shift
+    ;;
+  -compute-sanitizer-memcheck)
+    COMPUTE_SANITIZER=true
+    TOOL=memcheck
+    shift
+    ;;
+  -compute-sanitizer-racecheck)
+    COMPUTE_SANITIZER=true
+    TOOL=racecheck
+    shift
+    ;;
+  -compute-sanitizer-initcheck)
+    COMPUTE_SANITIZER=true
+    TOOL=initcheck
+    shift
+    ;;
+  -compute-sanitizer-synccheck)
+    COMPUTE_SANITIZER=true
+    TOOL=synccheck
+    shift
+    ;;
+  --)
+    shift
+    break
+    ;;
+  *)
+    echo "Unknown argument: $1"
+    exit 1
+    ;;
+  esac
+done
+
+if $LIMITED; then
+
+  export C2H_SEED_COUNT_OVERRIDE=1
+  readonly device_mem_GiB=8
+  export C2H_DEVICE_MEMORY_LIMIT=$((device_mem_GiB * 1024 * 1024 * 1024))
+  export C2H_DEBUG_CHECKED_ALLOC_FAILURES=1
+
+  echo "Configuring limited environment:"
+  echo "  C2H_SEED_COUNT_OVERRIDE=${C2H_SEED_COUNT_OVERRIDE}"
+  echo "  C2H_DEVICE_MEMORY_LIMIT=${C2H_DEVICE_MEMORY_LIMIT} (${device_mem_GiB} GiB)"
+  echo "  C2H_DEBUG_CHECKED_ALLOC_FAILURES=${C2H_DEBUG_CHECKED_ALLOC_FAILURES}"
+  echo
+fi
+
+# shellcheck source=ci/build_common.sh
+source "${ci_dir}/build_common.sh"
+
+print_environment_details
+
+
+if [[ -z "${GITHUB_ACTIONS:-}" ]]; then
+  ./build_cub.sh "$@"
+else
+  producer_id=$(util/workflow/get_producer_id.sh)
+  for tag in "${ARTIFACT_TAGS[@]}"; do
+    artifact="z_cub-test-artifacts-${DEVCONTAINER_NAME:?}-$producer_id-$tag"
+    run_command "📦  Unpacking artifact '$artifact'" \
+      "${ci_dir}/util/artifacts/download_packed.sh" "$artifact" /home/coder/cccl
+  done
+fi
+
+if $NO_LID; then
+  PRESETS=("cub-nolid")
+elif $LID0; then
+  PRESETS=("cub-lid0")
+elif $LID1; then
+  PRESETS=("cub-lid1")
+elif $LID2; then
+  PRESETS=("cub-lid2")
+else
+  PRESETS=("cub")
+fi
+
+if $COMPUTE_SANITIZER; then
+  echo "Setting CCCL_TEST_MODE=compute-sanitizer-${TOOL}"
+  export CCCL_TEST_MODE=compute-sanitizer-${TOOL}
+  echo "Setting C2H_SEED_COUNT_OVERRIDE=1"
+  export C2H_SEED_COUNT_OVERRIDE=1
+fi
+
+for PRESET in "${PRESETS[@]}"; do
+  test_preset "CUB (${PRESET})" "${PRESET}"
+done
+
+print_time_summary

cccl_upstream/ci/test_cuda_cccl_examples_python.sh

@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "$ci_dir/pyenv_helper.sh"
+
+# Parse common arguments
+source "$ci_dir/util/python/common_arg_parser.sh"
+parse_python_args "$@"
+# Pin cuda-toolkit to the container's CTK minor and set cuda_version /
+# cuda_major_version (-ctk-mode latest opts out). See pyenv_helper.sh.
+pin_cuda_toolkit "${ctk_mode}"
+
+# Setup Python environment
+setup_python_env "${py_version}"
+
+# Fetch or build the cuda_cccl wheel:
+if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+  wheel_artifact_name=$("$ci_dir/util/workflow/get_wheel_artifact_name.sh")
+  "$ci_dir/util/artifacts/download.sh" "${wheel_artifact_name}" /home/coder/cccl/
+else
+  "$ci_dir/build_cuda_cccl_python.sh" -py-version "${py_version}"
+fi
+
+# Install cuda_cccl, plus CuPy which the cuda.compute examples require, plus
+# pytest-benchmark for the host-overhead benchmark smoke test below. (cuda-bench,
+# for the throughput smoke, is installed best-effort further down since it does
+# not always ship a wheel for the newest Python.)
+CUDA_CCCL_WHEEL_PATH="$(ls /home/coder/cccl/wheelhouse/cuda_cccl-*.whl)"
+ctk_flavor="$(ctk_extra_flavor "${ctk_mode}")"
+python -m pip install "${CUDA_CCCL_WHEEL_PATH}[test-${ctk_flavor}${cuda_major_version}]" "cupy-cuda${cuda_major_version}x" pytest-benchmark
+
+# Run tests for parallel module
+cd "/home/coder/cccl/python/cuda_cccl/tests/"
+python -m pytest -n 6 test_examples.py
+
+# Smoke-test the host-overhead benchmark harness: run every benchmark case
+# exactly once (pass/fail only, no timing) so harness rot fails CI here instead
+# of silently surviving until someone runs the perf suite.
+cd "/home/coder/cccl/python/cuda_cccl/benchmarks/compute/host/"
+python -m pytest -v --benchmark-disable .
+
+# Smoke-test the throughput (nvbench) benchmarks the same way. --profile runs
+# each configuration once (no sampling); --quick uses the reduced quick_configs
+# axes (one dtype, smallest size) so every benchmark harness still imports,
+# registers, launches, and completes. cuda-bench does not always ship a wheel for
+# the newest Python, so skip the throughput smoke ONLY for that known no-wheel
+# case. Note pip prints "No matching distribution"/"Could not find a version"
+# even when the index is unreachable, so check for fetch/network failures first
+# and fail on those; any other install error fails the lane too rather than
+# silently passing. tee streams pip's output live while capturing it for the
+# grep checks below (pipefail keeps pip's exit status, not tee's).
+install_log="$(mktemp)"
+if python -m pip install "cuda-bench[cu${cuda_major_version}]" pyyaml 2>&1 | tee "${install_log}"; then
+  cd "/home/coder/cccl/python/cuda_cccl/benchmarks/compute/"
+  python run_benchmarks.py --py --profile --quick
+elif grep -qiE "Could not fetch URL|Retrying \(Retry|connection broken|Failed to establish a new connection|Name or service not known|timed out|SSLError|certificate verify failed|ProxyError" "${install_log}"; then
+  echo "::error::cuda-bench install failed because pip could not reach the package index (network/DNS/TLS/auth); not skipping." >&2
+  exit 1
+elif grep -qiE "No matching distribution found for cuda-bench|Could not find a version that satisfies the requirement cuda-bench" "${install_log}"; then
+  echo "::warning::cuda-bench has no wheel for Python ${py_version}; skipping the throughput benchmark smoke test."
+else
+  echo "::error::cuda-bench install failed for an unrecognized reason." >&2
+  exit 1
+fi
+rm -f "${install_log}"

cccl_upstream/ci/test_cuda_cccl_examples_python_v2.sh

@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+# Run the cuda.compute examples against a wheel built with the v2 (HostJIT)
+# backend. Mirrors test_cuda_cccl_examples_python.sh; the only difference is
+# exporting CCCL_PYTHON_USE_V2 so the wheel build uses cccl.c.parallel.v2.
+set -euo pipefail
+
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+export CCCL_PYTHON_USE_V2=1
+exec "$ci_dir/test_cuda_cccl_examples_python.sh" "$@"

cccl_upstream/ci/test_cuda_cccl_headers_python.sh

@@ -0,0 +1,33 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+source "$ci_dir/pyenv_helper.sh"
+
+# Parse common arguments
+source "$ci_dir/util/python/common_arg_parser.sh"
+parse_python_args "$@"
+# Pin cuda-toolkit to the container's CTK minor and set cuda_version /
+# cuda_major_version (-ctk-mode latest opts out). See pyenv_helper.sh.
+pin_cuda_toolkit "${ctk_mode}"
+
+# Setup Python environment
+setup_python_env "${py_version}"
+
+# Fetch or build the cuda_cccl wheel:
+if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+  wheel_artifact_name=$("$ci_dir/util/workflow/get_wheel_artifact_name.sh")
+  "$ci_dir/util/artifacts/download.sh" "${wheel_artifact_name}" /home/coder/cccl/
+else
+  "$ci_dir/build_cuda_cccl_python.sh" -py-version "${py_version}"
+fi
+
+# Install cuda_cccl
+CUDA_CCCL_WHEEL_PATH="$(ls /home/coder/cccl/wheelhouse/cuda_cccl-*.whl)"
+ctk_flavor="$(ctk_extra_flavor "${ctk_mode}")"
+python -m pip install "${CUDA_CCCL_WHEEL_PATH}[test-${ctk_flavor}${cuda_major_version}]"
+
+# Run tests for core package
+cd "/home/coder/cccl/python/cuda_cccl/tests/"
+python -m pytest -n auto -v headers/

cccl_upstream/ci/test_cuda_compute_minimal_python.sh

@@ -0,0 +1,75 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+repo_root="$(cd "$ci_dir/.." && pwd)"
+source "$ci_dir/pyenv_helper.sh"
+
+# Parse common arguments
+source "$ci_dir/util/python/common_arg_parser.sh"
+parse_python_args "$@"
+require_py_version "Usage: $0 -py-version <python_version>"
+
+# Pin cuda-toolkit to the container's CTK minor and set cuda_version /
+# cuda_major_version (-ctk-mode latest opts out). See pyenv_helper.sh.
+pin_cuda_toolkit "${ctk_mode}"
+
+# Setup Python environment
+setup_python_env "${py_version}"
+
+# Fetch or build the cuda_cccl wheel:
+if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+  wheel_artifact_name=$("$ci_dir/util/workflow/get_wheel_artifact_name.sh")
+  "$ci_dir/util/artifacts/download.sh" "${wheel_artifact_name}" "${repo_root}/"
+  wheelhouse_dir="${repo_root}/wheelhouse"
+else
+  "$ci_dir/build_cuda_cccl_python.sh" -py-version "${py_version}"
+  wheelhouse_dir="${repo_root}/wheelhouse"
+fi
+
+# Install cuda_cccl with the minimal CUDA extra. This intentionally avoids the
+# full cu*/sysctk* extras because those pull in numba/numba-cuda. The flavor is
+# "cu" (pip toolkit) or "sysctk" (system toolkit) per the -ctk-mode arg.
+CUDA_CCCL_WHEEL_PATH="$(ls "${wheelhouse_dir}"/cuda_cccl-*.whl)"
+ctk_flavor="$(ctk_extra_flavor "${ctk_mode}")"
+python -m pip install "${CUDA_CCCL_WHEEL_PATH}[minimal-${ctk_flavor}${cuda_major_version}]"
+python -m pip install pytest pytest-xdist
+
+cd "${repo_root}/python/cuda_cccl/tests/"
+python -m pytest -n 6 -v compute/test_no_numba.py
+if [[ "${py_version}" == "3.14t" ]]; then
+  # Select only tests that support the minimal extra so pytest does not collect
+  # tests that import numba-cuda and re-enable the GIL. These tests provide their
+  # own worker threads, so keep pytest itself in a single process.
+  # The serialization node-ids are module-skipped on the v2 backend today and
+  # will start running there automatically once v2 gains serialization support.
+  python -m pytest -n 0 -v \
+    compute/test_free_threading_stress.py \
+    compute/test_multi_cc_serialization.py::test_aot_build_result_load_failure_is_shared_and_retryable \
+    compute/test_multi_cc_serialization.py::test_aot_serialization_waits_for_canonical_first_load
+
+  # Broad thread-safety sweep (pytest-run-parallel): re-run the numba-free
+  # functional suite with each test executed concurrently across threads
+  # (barrier-synchronized start), stressing the process-wide build cache,
+  # single-flight coordination, and the Cython bindings from many threads at
+  # once. Complements test_free_threading_stress.py above, which targets specific
+  # shared-object scenarios by hand. -n 0 so the threads share one interpreter.
+  #
+  # --parallel-threads=2 matches CuPy's free-threading CI (the closest GPU
+  # precedent); a small fixed count bounds GPU-memory pressure from concurrent
+  # kernels and stays reproducible across runners, unlike =auto (the runner's
+  # logical-core count).
+  #
+  # pytest-run-parallel is only used by this sweep, so install it on the 3.14t
+  # path rather than for every minimal (e.g. non-free-threaded 3.14) run.
+  python -m pip install pytest-run-parallel
+
+  # Fail fast if the interpreter is not actually GIL-free (wrong build /
+  # PYTHON_GIL=1): pytest-run-parallel does NOT catch a GIL that is enabled from
+  # the start -- it would run threads GIL-serialized and pass vacuously. (A GIL
+  # *re-enabled mid-run* by a non-free-threaded import IS caught by the plugin,
+  # which is why we do not pass --ignore-gil-enabled.)
+  python -c "import sys; assert not sys._is_gil_enabled(), 'GIL is enabled; parallel sweep has no signal'"
+  python -m pytest -n 0 -v --parallel-threads=2 compute/test_no_numba.py
+fi

cccl_upstream/ci/test_cuda_compute_minimal_python_tsan.sh

@@ -0,0 +1,107 @@
+#!/usr/bin/env bash
+
+# ThreadSanitizer variant of the minimal free-threaded cuda.compute lane.
+#
+# Installs the TSan-instrumented cuda_cccl wheel (produced by the `python_tsan`
+# build, which compiles c.parallel v1 host code with -fsanitize=thread) and runs
+# the free-threaded stress tests + the pytest-run-parallel sweep under the TSan
+# runtime. A real data race in c.parallel (e.g. build-owned state mutated at
+# launch and shared across threads) fails the nightly here instead of silently
+# corrupting results under some interleaving.
+#
+# Only meaningful on a free-threaded (3.14t) interpreter; the GIL would serialize
+# the threads and hide the very races we are looking for.
+
+set -euo pipefail
+
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+repo_root="$(cd "$ci_dir/.." && pwd)"
+source "$ci_dir/pyenv_helper.sh"
+
+# Parse common arguments
+source "$ci_dir/util/python/common_arg_parser.sh"
+parse_python_args "$@"
+require_py_version "Usage: $0 -py-version <python_version>"
+
+if [[ "${py_version}" != *t ]]; then
+  echo "ERROR: the TSan lane requires a free-threaded (…t) interpreter; got '${py_version}'." >&2
+  echo "On a GIL interpreter the sweep serializes and TSan has no signal." >&2
+  exit 1
+fi
+
+# Instrument c.parallel when this script builds the wheel itself (local runs). In
+# CI the wheel is the pre-built `python_tsan` artifact, already instrumented; the
+# export is harmless there.
+export CCCL_C_PARALLEL_SANITIZE_THREAD=1
+
+cuda_major_version=$(nvcc --version | grep release | awk '{print $6}' | tr -d ',' | cut -d '.' -f 1 | cut -d 'V' -f 2)
+
+# Setup Python environment
+setup_python_env "${py_version}"
+
+# Fetch or build the TSan-instrumented cuda_cccl wheel. Under project
+# `python_tsan`, get_wheel_artifact_name.sh resolves to the distinct `-tsan`
+# artifact, so this never grabs an uninstrumented wheel.
+if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+  wheel_artifact_name=$("$ci_dir/util/workflow/get_wheel_artifact_name.sh")
+  "$ci_dir/util/artifacts/download.sh" "${wheel_artifact_name}" "${repo_root}/"
+  wheelhouse_dir="${repo_root}/wheelhouse"
+else
+  "$ci_dir/build_cuda_cccl_python.sh" -py-version "${py_version}"
+  wheelhouse_dir="${repo_root}/wheelhouse"
+fi
+
+# minimal-cu* extra intentionally avoids numba/numba-cuda (which re-enable the
+# GIL). pytest-run-parallel drives the concurrent sweep.
+CUDA_CCCL_WHEEL_PATH="$(ls "${wheelhouse_dir}"/cuda_cccl-*.whl)"
+python -m pip install "${CUDA_CCCL_WHEEL_PATH}[minimal-cu${cuda_major_version}]"
+python -m pip install pytest pytest-xdist pytest-run-parallel
+
+# The instrumented .so links libtsan but keeps it external (auditwheel --exclude),
+# so the TSan runtime must be present from process start -- LD_PRELOAD the
+# runner's libtsan (same soname/major as the gcc-13 build). Without preload the
+# .so fails to load ("cannot allocate memory in static TLS block").
+tsan_runtime="$(gcc -print-file-name=libtsan.so.2)"
+if [[ ! -e "${tsan_runtime}" ]]; then
+  echo "ERROR: libtsan.so.2 not found (gcc -print-file-name returned '${tsan_runtime}')." >&2
+  exit 1
+fi
+
+# setarch -R disables ASLR for the process tree. Required: TSan reserves fixed
+# shadow-memory regions and aborts ("unexpected memory mapping") when ASLR drops
+# something into them (google/sanitizers#1686). Uses personality(ADDR_NO_RANDOMIZE)
+# -- if a runner's seccomp profile blocks it, this call fails and must be allowed
+# (e.g. --security-opt seccomp=unconfined on the job's container).
+#
+# ignore_noninstrumented_modules=1: only c.parallel is instrumented, so ignore
+# races inside uninstrumented CPython / CUDA libs (avoids boundary false
+# positives). halt_on_error=1: stop at the first race -- it is usually the root
+# cause, and later reports are typically downstream noise.
+# exitcode=66: exit non-zero when any (unsuppressed) race is found, failing the
+# job even though pytest itself passes.
+run_under_tsan() {
+  setarch -R env \
+    LD_PRELOAD="${tsan_runtime}" \
+    TSAN_OPTIONS="ignore_noninstrumented_modules=1 halt_on_error=1 history_size=7 exitcode=66" \
+    "$@"
+}
+
+# Fail fast if the interpreter is not actually GIL-free (wrong build /
+# PYTHON_GIL=1): the sweep would run GIL-serialized and pass vacuously.
+run_under_tsan python -c "import sys; assert not sys._is_gil_enabled(), 'GIL is enabled; TSan sweep has no signal'"
+
+cd "${repo_root}/python/cuda_cccl/tests/"
+
+# Hand-written free-threaded stress scenarios (they spawn their own worker
+# threads) + the serialization node-ids, all sharing one interpreter (-n 0).
+run_under_tsan python -m pytest -n 0 -v \
+  compute/test_free_threading_stress.py \
+  compute/test_multi_cc_serialization.py::test_aot_build_result_load_failure_is_shared_and_retryable \
+  compute/test_multi_cc_serialization.py::test_aot_serialization_waits_for_canonical_first_load
+
+# Broad sweep: run the numba-free functional suite with each test executed
+# concurrently across threads (barrier-synchronized), exercising many more
+# c.parallel algorithms under contention than the hand-written stress tests.
+# -n 0 so the threads share one interpreter; --parallel-threads=2 bounds
+# GPU-memory pressure and stays reproducible across runners.
+run_under_tsan python -m pytest -n 0 -v --parallel-threads=2 compute/test_no_numba.py

cccl_upstream/ci/test_cuda_compute_minimal_python_v2.sh

@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+# Run the minimal cuda.compute suite against a wheel built with the v2
+# (HostJIT) backend. The shared minimal script owns dependency installation and
+# selects the free-threading stress and serialization tests for Python 3.14t.
+set -euo pipefail
+
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+export CCCL_PYTHON_USE_V2=1
+exec "$ci_dir/test_cuda_compute_minimal_python.sh" "$@"

cccl_upstream/ci/test_cuda_compute_python.sh

@@ -0,0 +1,50 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+ci_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "$ci_dir/pyenv_helper.sh"
+
+# Parse common arguments
+source "$ci_dir/util/python/common_arg_parser.sh"
+parse_python_args "$@"
+# Pin cuda-toolkit to the container's CTK minor and set cuda_version /
+# cuda_major_version (-ctk-mode latest opts out). See pyenv_helper.sh.
+pin_cuda_toolkit "${ctk_mode}"
+
+# Setup Python environment
+setup_python_env "${py_version}"
+
+# Fetch or build the cuda_cccl wheel:
+if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
+  wheel_artifact_name=$("$ci_dir/util/workflow/get_wheel_artifact_name.sh")
+  "$ci_dir/util/artifacts/download.sh" "${wheel_artifact_name}" /home/coder/cccl/
+else
+  "$ci_dir/build_cuda_cccl_python.sh" -py-version "${py_version}"
+fi
+
+# Install cuda_cccl. The extra flavor is "cu" (pip-installed toolkit) or "sysctk"
+# (system-provided toolkit) depending on the -ctk-mode arg.
+CUDA_CCCL_WHEEL_PATH="$(ls /home/coder/cccl/wheelhouse/cuda_cccl-*.whl)"
+ctk_flavor="$(ctk_extra_flavor "${ctk_mode}")"
+python -m pip install "${CUDA_CCCL_WHEEL_PATH}[test-${ctk_flavor}${cuda_major_version}]"
+
+# Run tests for compute module.
+# On the v2 (HostJIT) backend, abort on first failure — the suite is still
+# stabilizing and a single early failure is enough signal to investigate
+# without scrolling through hundreds of subsequent passes.
+pytest_extra=()
+if [[ "${CCCL_PYTHON_USE_V2:-}" =~ ^(1|true|TRUE|on|ON)$ ]]; then
+  pytest_extra+=(-x)
+fi
+
+cd "/home/coder/cccl/python/cuda_cccl/tests/"
+if [[ "${CCCL_PYTHON_USE_V2:-}" =~ ^(1|true|TRUE|on|ON)$ ]]; then
+  # The test isolates itself in a fresh subprocess (LLVM initialization is
+  # process-wide and only cold once), but it carries the free_threading marker,
+  # so it must be selected by node-id here or the sweeps below never run it.
+  python -m pytest "${pytest_extra[@]}" -n 0 -v \
+    compute/test_free_threading_stress.py::test_v2_concurrent_cold_llvm_initialization
+fi
+python -m pytest "${pytest_extra[@]}" -n 6 -v compute/ -m "not large and not free_threading"
+python -m pytest "${pytest_extra[@]}" -n 0 -v compute/ -m "large and not free_threading"