record 245k platform-equivalent benchmark

worklogs/decode_analysis_report_2026-07-14.md

@@ -949,3 +949,68 @@ ixsmi
 2. 在干净显存下重新验证 `max_num_batched_tokens=32768` 是否必然 OOM。
 3. 对官方相似负载做参数矩阵：`max_num_batched_tokens`、`gpu_memory_utilization`、`max_model_len`、`max_num_seqs`。
 4. 若参数调优收益不足，继续深入 `PREFIX_FLASH` 和 MoE routed expert 的 native/kernel 路径 profiling。
+
+## 2026-07-15：245K 平台等价配置验证
+
+### 重要启动细节
+
+远端手工测试时，不能从仓库根目录启动：
+
+```bash
+cd /root/data-disk-1/enginex-vllm-bi100-qwen36
+python3 -m vllm.entrypoints.openai.api_server ...
+```
+
+这样会优先导入仓库内未补丁的 `vllm/`，导致：
+
+```text
+KeyError: 'invalid tool call parser: qwen3_coder (chose from { hermes,internlm,llama3_json,mistral })'
+```
+
+平台 Dockerfile 只复制 `qwen3_6_scripts` 并执行 `patch_ops.sh`，不会复制仓库根目录的 `vllm/`，因此平台会使用补丁后的 corex vLLM。远端手工测试需要从 `/root` 或 `/workspace` 等非仓库目录启动，以匹配平台行为。
+
+### 245K 启动结果
+
+从 `/root` 启动平台等价配置成功：
+
+- `--max-model-len 245000`
+- `--max-seq-len-to-capture 245000`
+- `--enforce-eager`
+- `--max-num-batched-tokens 16384`
+- `--max-num-seqs 1`
+- `--tool-call-parser qwen3_coder`
+- `--reasoning-parser qwen3`
+- `--chat-template /workspace/chat_template_multi_system.jinja`
+
+启动后显存约为：
+
+- GPU0: 28.4GB
+- GPU1: 28.3GB
+- GPU2: 28.3GB
+- GPU3: 28.2GB
+
+说明 `--enforce-eager` 后 245K 正确性配置可以启动，但剩余显存空间已经很紧。
+
+### 245K 官方相似压测
+
+本地归档：
+
+- `worklogs/remote_results/2026-07-15-official-like-baseline/eager_245k_c1_r8_t256_cap40k.json`
+
+| 指标 | 100K 基线 | 245K 配置 | 目标 |
+| --- | ---: | ---: | ---: |
+| Success Rate | 100.00% | 100.00% | >= 99% |
+| Cache Hit Rate | 74.17% | 74.17% | >= 50% |
+| TTFT P90 | 9.803s | 24.886s | <= 5s |
+| Output TPS P10 | 5.825 | 5.532 | >= 20 |
+| Aggregate Output TPS | 5.789 | 4.631 | - |
+| Uncached Input TPS | 178.330 | 139.628 | - |
+| Cache TPS | 512.015 | 400.894 | - |
+| Weighted Throughput | 883.109 | 693.102 | >= 8000 |
+
+245K 配置满足上下文窗口正确性，但对当前小样本性能明显更差。后续优化策略：
+
+1. 平台提交配置保留 245K，以覆盖官方 235K+ 输入上限。
+2. 快速性能迭代优先用 100K/cap40K 官方相似数据，减少单轮成本。
+3. 确定有效优化后，再回到 245K 配置做确认。
+4. 当前离目标最大的差距仍是 prefill/cache 吞吐和 decode MoE 吞吐，而不是缓存命中率或请求成功率。

worklogs/remote_results/2026-07-15-official-like-baseline/eager_245k_c1_r8_t256_cap40k.json

@@ -0,0 +1,124 @@
+{
+  "created_at": "2026-07-14T19:47:25",
+  "url": "http://127.0.0.1:1111",
+  "model": "llm",
+  "dataset": "/root/work/logs/synthetic_cumulative_8_s3_cap40k.jsonl",
+  "concurrency": 1,
+  "max_requests": 8,
+  "max_tokens": 256,
+  "wall_sec": 417.18738799728453,
+  "success_rate": 1.0,
+  "ttft_p90_sec": 24.886156552471217,
+  "output_tps_p10_per_request": 5.532290437092943,
+  "aggregate_output_tps": 4.631012479247276,
+  "aggregate_input_tps_uncached": 139.62790265457198,
+  "aggregate_cache_tps": 400.8941900254392,
+  "cache_hit_rate": 0.7416795639891973,
+  "weighted_token_throughput": 693.1017315458303,
+  "totals": {
+    "requests": 8,
+    "success": 8,
+    "prompt_tokens": 225499,
+    "input_tokens_uncached": 58251,
+    "cached_tokens": 167248,
+    "completion_tokens": 1932,
+    "reasoning_tokens": 1825
+  },
+  "targets": {
+    "output_tps_p10_per_request_gte_20": false,
+    "ttft_p90_lte_5": false,
+    "cache_hit_rate_gte_50pct": true,
+    "success_rate_gte_99pct": true,
+    "weighted_token_throughput_gte_8000": false
+  },
+  "results": [
+    {
+      "ok": true,
+      "elapsed_sec": 46.63851813226938,
+      "ttft_sec": 13.72975979372859,
+      "prompt_tokens": 7498,
+      "cached_tokens": 0,
+      "completion_tokens": 256,
+      "reasoning_tokens": 196,
+      "output_tps": 7.77908413822432,
+      "error": null
+    },
+    {
+      "ok": true,
+      "elapsed_sec": 62.5033224709332,
+      "ttft_sec": 37.72389439865947,
+      "prompt_tokens": 34177,
+      "cached_tokens": 7488,
+      "completion_tokens": 140,
+      "reasoning_tokens": 93,
+      "output_tps": 5.649847913828536,
+      "error": null
+    },
+    {
+      "ok": true,
+      "elapsed_sec": 54.076735051348805,
+      "ttft_sec": 7.701814688742161,
+      "prompt_tokens": 35384,
+      "cached_tokens": 34160,
+      "completion_tokens": 256,
+      "reasoning_tokens": 256,
+      "output_tps": 5.520225113020781,
+      "error": null
+    },
+    {
+      "ok": true,
+      "elapsed_sec": 52.38991864770651,
+      "ttft_sec": 19.38426890410483,
+      "prompt_tokens": 19274,
+      "cached_tokens": 7472,
+      "completion_tokens": 256,
+      "reasoning_tokens": 256,
+      "output_tps": 7.756247854191296,
+      "error": null
+    },
+    {
+      "ok": true,
+      "elapsed_sec": 53.290865218266845,
+      "ttft_sec": 8.00927284732461,
+      "prompt_tokens": 37099,
+      "cached_tokens": 35376,
+      "completion_tokens": 256,
+      "reasoning_tokens": 256,
+      "output_tps": 5.65351142916693,
+      "error": null
+    },
+    {
+      "ok": true,
+      "elapsed_sec": 42.13185533322394,
+      "ttft_sec": 10.382675435394049,
+      "prompt_tokens": 14661,
+      "cached_tokens": 7472,
+      "completion_tokens": 256,
+      "reasoning_tokens": 256,
+      "output_tps": 8.063200398366764,
+      "error": null
+    },
+    {
+      "ok": true,
+      "elapsed_sec": 52.711524257436395,
+      "ttft_sec": 7.415150867775083,
+      "prompt_tokens": 38199,
+      "cached_tokens": 37088,
+      "completion_tokens": 256,
+      "reasoning_tokens": 256,
+      "output_tps": 5.651666587030361,
+      "error": null
+    },
+    {
+      "ok": true,
+      "elapsed_sec": 53.436059853062034,
+      "ttft_sec": 7.205488372594118,
+      "prompt_tokens": 39207,
+      "cached_tokens": 38192,
+      "completion_tokens": 256,
+      "reasoning_tokens": 256,
+      "output_tps": 5.537461290266727,
+      "error": null
+    }
+  ]
+}