computility-run.yaml:
max-num-seqs 1→256: benchmark sweeps [128,256] concurrent seqs,
current config processes 1 while 127 queue. KV cache budget:
256 seqs × 2048 tokens × 80KB/token = 41.9GB < 45GB available.
max-num-batched-tokens 8192→32768: support 256 concurrent prefills.
gpu-memory-utilization 0.9→0.95: provide KV cache headroom.
Dockerfile:
Deploy paged_attention_v2_triton.py to vllm package path so
try-triton-first logic in _custom_ops.py can find it. Falls back
to PyTorch V2 automatically if Triton V2 fails (SMEM/runtime).
muh/tuning/common.cuh:
scale_mem_bound max_smem now a parameter (default 48KB). Allows
policy_selectors to pass hw.max_shared_memory_per_block if actual
SMEM differs from CCCL 48KB assumption.
muh/tuning/tuning_transform.cuh:
bytes_in_flight 16KB→32KB. Old derivation used 900/50=18 GB/s/SM
(wrong, SM=16 confirmed). Actual per-SM BW = 56 GB/s.
32KB is estimate pending benchmark sweep.
SM count 50→16 corrections across all affected files.
Previous version used base topk policy's bits (11 for key>=2B),
causing SMEM overflow: 512*4*key_size + 2048*4*batches > 49152.
Fix: force bits=8 (same as radix_sort decision for BI-V100).
SMEM: 512*4*key_size + 256*4*batches = manageable.
Also adds while-loop SMEM check on max_batches.
Detected by test_smem_safety.py: 3 overflows at key_size=2,4,8.
The previous version had a `portioned_smem_per_warp` field that doesn't
exist in CCCL. The actual CCCL RadixSortOnesweepPolicy has:
threads, items, store_algorithm, rank_algorithm, scan_algorithm,
rank_private_partitions, radix_bits
Also adds proper SMEM calculation:
total = max(keys_tile, values_tile, rank_smem) + offsets
with 2KB headroom for kernel stack/locals.
rank_private_partitions set to 1 to minimize SMEM pressure.
Defensive guard: if SMEM cap computes max_threads_by_smem < 32
(or rounds to 0), floor at 32 (one warp). Prevents launching
0 threads which is undefined behavior.
scale_mem_bound now returns {items, threads} (items-first) to match
CCCL's scaling_result struct. All 7 call sites in this file updated.
Previously: auto [t, i] bound threads→t, items→i
Now: auto [i, t] binds items→i, threads→t
The ReducePassPolicy{t, i, ...} constructors remain correct because
they take (threads, items, ...) — t is threads, i is items in both cases.
The old code worked by accident (two reversals canceling out).
1. Return order: {threads, items} → {items, threads} matching CCCL scaling_result
2. Upper clamp: nominal*1 → nominal*2 (CCCL allows small types to double items)
3. Add threads SMEM cap: min(nominal, round_up(48KB/(ts*items), 32))
Verified against all 8 test vectors from CCCL catch2_test_util_arch.cu.
The old code was only safe because current bi100_* structs don't hit the
edge cases — but any future CCCL code copy would silently produce wrong
values.
Audit results:
- 20/20 files had IDENTICAL if-branch and fallback (dead code)
- 787 lines total, 5% coverage of 15116 lines in CCCL originals
- No type specializations, no offset_size branches, no benchmark data
- 0 of 20 algorithms appear on vllm's Qwen3.6 inference hot path
The 6 headers that remain (reduce, topk, scan, transform, batch_memcpy, for)
are the only algorithms that execute during vllm decode/prefill/cache operations.
These 6 have real type specializations and CCCL SM100 reference values.
CCCL has 26 algorithms because it's a general-purpose library.
muh targets one workload: Qwen3.6-35B-A3B on 4× BI-V100.
Covering algorithms that don't execute is worse than not covering them —
it creates the illusion of completeness.
Problems fixed:
1. gen_patch.py was reading .muh YAML (all nulls) instead of C++ headers.
Now it parses bi100_* structs directly from tuning_*.cuh via regex,
extracts constexpr values, and maps them to vllm injection points.
Verified: 11 patches generated from 6 algorithms.
2. C++ headers had no build system or tests.
Added CMakeLists.txt (header-only library target) and compile_test.cpp.
Verified: g++ -std=c++17 compiles all headers, 17/17 runtime checks pass.
Also added cuda_compile_test.cu for when nvcc is available.
3. baseline.muh had a tuning section full of nulls duplicating C++ values.
Stripped to vllm launch config only. Tuning values live exclusively
in muh/include/muh/tuning/tuning_*.cuh bi100_* structs.
4. Fixed constexpr goto in tuning_scan.cuh (C++17 doesn't allow goto in
constexpr; replaced with early-return + default: break pattern).
Data flow is now:
tuning_*.cuh (bi100_* constexpr) ──→ gen_patch.py ──→ vllm patches
baseline.muh (launch config) ──→ gen_yaml.py ──→ computility-run.yaml
compile_test.cpp ──→ g++/nvcc ──→ verify values are real
