Claude 8e9c22f6c1 feat: CCCL-derived 3-tier decode dispatch + SM=16 prefill tuning + multi-step scheduling ...

paged_attn.py:
- Remove use_v1=True hardcode that forced all decode through ixf_F V1
- Wire up paged_attention_v2_triton.py as Tier 2 decode path for seq_len > 8192
- 3-tier dispatch: V1 (short) → Triton V2 (long) → PyTorch (fallback)
- Triton V2 uses CCCL compound-reduce pattern (summary_statistics.cu)
  with GQA broadcast (6x KV read reduction for Qwen3.6)
- This is the single highest-impact change: Output TPS is 83% of score

prefix_prefill.py:
- CCCL scan-tuning-informed block sizes for BI-V100 (SM=16, 48KB SMEM)
- BI-V100 path: BLOCK=64 NUM_WARPS=4 (vs BLOCK=128 NUM_WARPS=8 on A100+)
- Matches muh/tuning/tuning_scan.cuh bi100_lookback_4B_o4 pattern
- Fewer warps = less register pressure = higher occupancy on 16 SMs

computility-run.yaml:
- Add --num-scheduler-steps=8: batch 8 decode iterations per Python call
  (cuts scheduler overhead ~8x, directly improves Output TPS)
- Add --preemption-mode=recompute (cheaper than swap on BI-V100 HBM)
- Add TRITON_CACHE_DIR for JIT warmup persistence
- Add TRITON_PRINT_AUTOTUNING=0 (use hardcoded CCCL configs, skip autotune)

Competition impact estimate:
- Tier 2 Triton V2 replaces PyTorch fallback for 8K-100K contexts → ~5-10x decode speedup
- Multi-step scheduling → ~20-30% Output TPS improvement
- SM=16 block tuning → ~10-15% Input TPS improvement

2026-08-03 08:28:38 +00:00

..

qwen3_5

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

qwen3_5_moe

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

reasoning

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

api_server.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

chat_utils.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

cli_args.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

mamba_cache.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

paged_attn.py

feat: CCCL-derived 3-tier decode dispatch + SM=16 prefill tuning + multi-step scheduling

2026-08-03 08:28:38 +00:00

patch_enable_triton.py

[OPT] Enable Triton prefill + raise decode threshold — the actual performance work

2026-07-30 15:41:25 +00:00

patch_head256_triton.py

[OPT] head_dim=256 Triton support — BLOCK=32 for Qwen3.6

2026-07-30 16:05:01 +00:00

patch_ixformer_native.py

[VERIFIED] Hardware-tested native kernel integration

2026-07-31 06:43:25 +00:00

patch_model_runner.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

patch_ops.sh

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

patch_paged_attention_v2.py

[OPT] Complete Triton V2 Phase 1 — paged K/V gather from prefix_prefill.py pattern

2026-07-30 16:07:15 +00:00

patch_transformers_qwen3_5.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

patch_triton_tuning.py

fix: critical config + tuning corrections from CCCL source analysis

2026-08-03 06:45:54 +00:00

patch_vectorized_decode.py

[OPT] Enable Triton prefill + raise decode threshold — the actual performance work

2026-07-30 15:41:25 +00:00

patch_vllm_qwen3_5.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

patch_vllm_tool_parser.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

patch_xformers_sdpa_batch_kernel.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

patch_xformers_sdpa_batch.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

patch_xformers_sdpa_seq_kernel.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

patch_xformers_sdpa_seq.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

protocol.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

qwen3_5.py

sync: align with enginex-vllm-bi100-qwen36 baseline (1902c81f)

2026-07-31 09:43:58 +00:00

qwen3coder_tool_parser.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

scheduler.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

sequence.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

serving_chat.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00