v3b: A/B对照 — 移除 --hf-overrides 以逐字节对齐成功配置(ouys/sub400)

add --hf-overrides architectures=Qwen3_5MoeForCausalLM (fix model config arch mismatch)
2026-10-02 20:45:45 +08:00 · 2026-10-02 20:00:19 +08:00 · 2026-10-02 19:53:25 +08:00 · 2026-10-01 23:29:16 +08:00 · 2026-09-24 07:57:33 +08:00
1 changed files with 52 additions and 31 deletions
--- a/computility-run.yaml
+++ b/computility-run.yaml
@@ -1,34 +1,55 @@
 concurrency: 1
 command:
-    - python3
-    - -m
-    - vllm.entrypoints.openai.api_server
-    - --model
-    - /model
-    - --served-model-name
-    - llm
-    - --max-model-len
-    - '100000'
-    - --gpu-memory-utilization
-    - '0.9'
-    - --trust-remote-code
-    - -tp
-    - '4'
-    - --max-num-seqs
-    - '1'
-    - --disable-log-requests
-    - --disable-frontend-multiprocessing
-    - --max-num-batched-tokens
-    - '8192'
-    - --enable-chunked-prefill
-    - --max-seq-len-to-capture
-    - '32768'
-    - --enable-auto-tool-choice
-    - --tool-call-parser
-    - qwen3_coder
-    - --reasoning-parser
-    - qwen3
-    - --enable-prefix-caching
+  - python3
+  - -m
+  - vllm.entrypoints.openai.api_server
+  - --model
+  - /model
+  - --served-model-name
+  - llm
+  - --max-model-len
+  - '256000'
+  - --gpu-memory-utilization
+  - '0.95'
+  - --trust-remote-code
+  - -tp
+  - '4'
+  - --max-num-seqs
+  - '4'
+  - --disable-log-requests
+  - --disable-frontend-multiprocessing
+  - --max-num-batched-tokens
+  - '8192'
+  - --enable-chunked-prefill
+  - --max-seq-len-to-capture
+  - '32768'
+  - --enforce-eager
+  - --enable-auto-tool-choice
+  - --tool-call-parser
+  - qwen3_coder
+  - --reasoning-parser
+  - qwen3
+  - --enable-prefix-caching
+  - --dtype
+  - half
 env:
-    - name: VLLM_ENGINE_ITERATION_TIMEOUT_S
-      value: 3600
+  - name: VLLM_ENGINE_ITERATION_TIMEOUT_S
+    value: 3600
+  - name: VLLM_ATTENTION_BACKEND
+    value: XFORMERS
+  - name: ENABLE_CUSTOM_IPC
+    value: 1
+  - name: PYTHONPATH
+    value: /usr/local/corex/lib/python3/dist-packages:/usr/local/corex/lib64/python3/dist-packages
+  - name: LD_LIBRARY_PATH
+    value: /usr/local/corex/lib64:/usr/local/openmpi/lib
+  - name: VLLM_COREX_FA2_LIBRARY
+    value: /usr/local/corex/lib64/libcorex_fa2.so
+  - name: VLLM_COREX_GDN_LIBRARY
+    value: /usr/local/corex/lib64/libcorex_gdn.so
+  - name: VLLM_COREX_MOE_LIBRARY
+    value: /usr/local/corex/lib64/libcorex_moe.so
+  - name: VLLM_REQUEST_METRICS_FILE
+    value: /tmp/vllm-request-metrics.jsonl
+  - name: VLLM_CACHE_BLOCK_SIZE
+    value: 16
Author	SHA1	Message	Date
wanglin	8d3ee4a9e3	v3b: A/B对照 — 移除 --hf-overrides 以逐字节对齐成功配置(ouys/sub400) v3b: A/B对照 — 移除 --hf-overrides 以逐字节对齐成功配置(ouys/sub400)	2026-10-02 20:45:45 +08:00
wanglin	ee1c3c4c9b	add --hf-overrides architectures=Qwen3_5MoeForCausalLM (fix model config arch mismatch)	2026-10-02 20:00:19 +08:00
wanglin	941c42a04f	v3: 对齐成功配置 — 补 --enforce-eager 与 corex 运行库环境变量, max-model-len 256K / gmu 0.95 / seqs 4 / bt 8192 v3: 对齐成功配置 — 补 --enforce-eager 与 corex 运行库环境变量, max-model-len 256K / gmu 0.95 / seqs 4 / bt 8192	2026-10-02 19:53:25 +08:00
wanglin	d6dfff0918	v2: conservative max-num-seqs=8 + batched-tokens=16384, revert dtype/256K (v1 OOM crash)	2026-10-01 23:29:16 +08:00
wanglin	a56835199e	更新 computility-run.yaml 优化启动参数：max-model-len→256K, max-num-seqs→8, batched-tokens→16384, 加prefix-caching和dtype	2026-09-24 07:57:33 +08:00