[ENGINE] muh_cc_dispatch + analysis: max_num_seqs=1 from computility-run.yaml

CRITICAL FINDING from reading computility-run.yaml:
  --max-num-seqs 1

This means the competition ALWAYS runs single-sequence inference.
All batch-level optimizations (padded_grid_reduction batching,
multi-seq V2 parallelism, batch-wise tensor caching) have ZERO
impact on actual performance.

The real bottleneck is single-sequence KV cache access:
  - decode: 1 seq × all heads × all KV blocks
  - prefill: 1 seq × chunked (max_num_batched_tokens=8192)
  - MoE: 1 seq × top_k=8 experts × 64 layers

Updated muh_cc_dispatch.py to record QWEN36_MAX_NUM_SEQS=1.

CCCL insight from padded_grid_reduction.cu: the padded grid batching
pattern is only beneficial when num_seqs > 1. For single-seq,
the per-sequence loop (range(1)) has zero overhead — the focus
should be on single-sequence tile optimization instead.

CCCL files: thrust/examples/padded_grid_reduction.cu,
cub/block/block_exchange.cuh
This commit is contained in:
muh-engine
2026-08-06 01:02:15 +00:00
parent d70deefae1
commit 0d810ff989

View File

@@ -297,6 +297,7 @@ def select_moe_config(
QWEN36_HEAD_DIM = 256 # text_cfg.head_dim
QWEN36_NUM_KV_HEADS = 4 # num_key_value_heads
QWEN36_MAX_SEQ_LEN = 100000 # from computility-run.yaml
QWEN36_MAX_NUM_SEQS = 1 # CRITICAL: computility-run.yaml --max-num-seqs 1
QWEN36_NUM_EXPERTS = 256 # MoE experts
QWEN36_TOP_K = 8 # MoE top-k
QWEN36_HIDDEN = 3584 # hidden_size