diff --git a/PIPELINE_STATUS.md b/PIPELINE_STATUS.md new file mode 100644 index 00000000..eabf151d --- /dev/null +++ b/PIPELINE_STATUS.md @@ -0,0 +1,86 @@ +# muh Pipeline Status — Ground Truth + +**Last verified**: 2026-08-07T01:45:31Z by automated analysis + +## Architecture Summary + +``` +CCCL policy_selector(compute_capability) → ReducePolicy{threads, items, vec, algo, load_mod} + ↕ mirrors +muh policy_selector(hardware_capability) → same struct types, BI-V100 values + ↕ gen_patch.py extracts bi100_* values +vllm patch_ops.sh → full-file Python replacements with tuning values baked in +``` + +## Injection Reality + +### What gen_patch.py THINKS (csrc/*.cu — DEAD) +``` +tuning_reduce.cuh → csrc/attention/attention_kernels.cu NUM_THREADS ← NO .cu SOURCE +tuning_scan.cuh → csrc/attention/paged_attention_v1.cu SCAN_BLOCK_SIZE ← NO .cu SOURCE +tuning_topk.cuh → csrc/sampling/sampling_kernels.cu SAMPLING_BLOCK_SIZE ← NO .cu SOURCE +``` + +### What ACTUALLY happens (Python runtime — ALIVE) +``` +_custom_ops.py → SMEM 49152 (was 32768) ← DEPLOYED ✓ +paged_attn.py → _PARTITION_SIZE=512 ← DEPLOYED ✓ (V2 partition, NOT CTA tile) +xformers.py → _Q_CHUNK=256, sdpa_fallback ← DEPLOYED ✓ +sampler.py → torch.topk fast path ← DEPLOYED ✓ +prefix_prefill.py → Triton BLOCK_M/N/warps ← DEPLOYED ✓ (but Triton not available) +computility-run.yaml → vllm server args ← DEPLOYED ✓ +``` + +### The Gap +muh C++ headers define precise per-type-per-op tuning values (14 reduce structs, 22 scan structs). +But the vllm engine on BI-V100 runs ixformer .so (precompiled, not tunable) + Python fallbacks. +The C++ headers' values cannot be injected into the precompiled .so. +They CAN inform: +1. Python fallback implementations (paged_attn.py, xformers.py) — tile sizes, chunk sizes +2. Triton JIT configs — if Triton were available (it's not on BI-V100 base image) +3. Future EngineX releases that expose tuning knobs + +## Asset Inventory + +| Asset | Count | Status | +|-------|-------|--------| +| CCCL tuning headers (upstream) | 27 | Complete | +| muh BI-V100 headers | 27 | Complete (14 reduce + 22 scan + others) | +| muh schema YAMLs | 27 | Complete | +| CUB benchmarks | 91 | Synced to NVIDIA/cccl main | +| CUB tests | 243 | Complete | +| CUB examples | 18 | Complete | +| Thrust examples | 60 | Complete | +| Deployed patches | 15 files | Via patch_ops.sh full replacement | +| bench_bi100.py search spaces | 5 algos | Defined, needs BI-V100 hardware to run | + +## Tool Chain Status + +| Tool | Input | Output | Status | +|------|-------|--------|--------| +| parse.py | baseline.muh | JSON config | ✓ Working | +| gen_patch.py | tuning_*.cuh | Patch report | ⚠ Reports structs but generates 0 patches (injection mapping mismatch) | +| gen_yaml.py | baseline.muh | computility-run.yaml | ✓ Working | +| bench_bi100.py | algo+dtype | CCCL-format speedup data | Needs BI-V100 hardware | +| patch_ops.sh | qwen3_6_scripts/ | Docker vllm patches | ✓ Working | +| muh_dispatch.py | hw+dtype+head_dim | AttentionConfig | ✓ Working (needs torch) | +| scale_mem_bound | (threads, items, type_size) | (items, threads) | ✓ CCCL parity verified | + +## Critical Numbers + +| Metric | Competition Threshold | Current Status | +|--------|----------------------|----------------| +| Functional tests | 50+ pass | 13 items In Progress (all FEA) | +| Effect deviation | ≤ ±4% | Untested (needs hardware) | +| Token throughput weighted | ≥ 8000 | Untested | +| Output TPS weight | 83% (×16.796) | Reduce/scan/topk optimization focus | +| SMEM limit | 49152 bytes | All 36 scan+reduce structs verified ✓ | +| SM count | 16 (confirmed) | All headers updated | + +## Next Actions (Ranked by Competition Impact) + +1. **Run bench_bi100.py on BI-V100** → get real speedup data for reduce/scan/topk +2. **Backfill speedup data to muh headers** → replace TBD/theoretical values +3. **Optimize Python fallback tile sizes** → paged_attn.py, xformers.py Q_CHUNK +4. **Tune computility-run.yaml** → max-num-seqs, max-batched-tokens, gpu-mem-util +5. **Enable prefix caching benchmark** → cached_tokens > 0 for repeat prompts