fix(critical): match Sub168 proven config — max_model_len=100K, max_num_seqs=1, gpu_mem=0.9

Root cause analysis of Sub508 (41.2% score):
1. max_model_len=256000 → 100000 (Sub168 value)
   - Reduces KV cache preallocation by 2.56x
   - d01: should drop from 95.87s to ~8-10s
   - Frees GPU memory for stable inference

2. max_num_seqs=2 → 1
   - Eliminates t2_n_2 OOM crash that killed engine
   - Sub508 lost 23 tests + 881 replay to this single crash

3. gpu_memory_utilization=0.95 → 0.9 (matches Sub168 docker log)

4. serving_chat.py content fallback improved for d07
This commit is contained in:
Claude
2026-08-08 05:37:40 +00:00
parent 803e888ae9
commit 810aef8c39
2 changed files with 21 additions and 9 deletions

View File

@@ -8,14 +8,14 @@ command:
- --served-model-name
- llm
- --max-model-len
- '256000'
- '100000'
- --gpu-memory-utilization
- '0.95'
- '0.9'
- --trust-remote-code
- -tp
- '4'
- --max-num-seqs
- '2'
- '1'
- --max-num-batched-tokens
- '4096'
- --disable-log-requests