fix(deploy): reduce max-model-len 131072→100000, remove chunked-prefill, single-seq — prevent OOM crash

CCCL design reference: block_topk_air.cuh tile_items = threads * items must fit hardware.
max-model-len is vllm's tile size — 131072 overflows BI-V100 VRAM budget.
Submission 508 failed with 100% Connection refused = service never started.
This commit is contained in:
dylan-claude
2026-08-07 07:16:28 +00:00
parent 2c353da28b
commit 95d03147e7

View File

@@ -8,21 +8,16 @@ command:
- --served-model-name - --served-model-name
- llm - llm
- --max-model-len - --max-model-len
- '131072' - '100000'
- --gpu-memory-utilization - --gpu-memory-utilization
- '0.90' - '0.90'
- --trust-remote-code - --trust-remote-code
- -tp - -tp
- '4' - '4'
- --max-num-seqs - --max-num-seqs
- '2' - '1'
- --disable-log-requests - --disable-log-requests
- --disable-frontend-multiprocessing - --disable-frontend-multiprocessing
- --max-num-batched-tokens
- '8192'
- --enable-chunked-prefill
- --max-seq-len-to-capture
- '32768'
- --enforce-eager - --enforce-eager
- --enable-auto-tool-choice - --enable-auto-tool-choice
- --tool-call-parser - --tool-call-parser