5f880bb279
The core of muh is now C++ headers that mirror CCCL's tuning architecture:
muh/include/muh/
├── hardware.cuh — hardware_capability descriptor (replaces cuda::compute_capability)
├── muh.cuh — top-level include + scoring formula
└── tuning/
├── common.cuh — shared types, compatible with CCCL's common.cuh
├── tuning_reduce.cuh — P0: attention reduction (5 type specializations)
├── tuning_topk.cuh — P0: sampling top-k/top-p (2B/4B key specializations)
├── tuning_scan.cuh — P0: prefix scan (6 lookback + 6 lookahead specializations)
├── tuning_transform.cuh — P1: activation elementwise (SiLU/GELU/RMSNorm)
├── tuning_batch_memcpy.cuh — P1: KV cache block copy
└── tuning_for.cuh — P2: RoPE position encoding
Architecture:
- Each tuning header has a policy_selector struct with operator()(hardware_capability)
- Dispatches on muh::hardware_capability instead of cuda::compute_capability
- bi100_* structs hold per-type tuning values (initialized from CCCL SM100 reference)
- When CCCL headers are available, re-exports their enum types
- When standalone, provides compatible enum definitions
Python files (extract.py, parse.py, gen_yaml.py, gen_patch.py) remain as tooling.
The C++ headers are what actually gets compiled into the vllm binary.