fix(deploy): reduce max-model-len 131072→100000, remove chunked-prefill, single-seq — prevent OOM crash

CCCL design reference: block_topk_air.cuh tile_items = threads * items must fit hardware.
max-model-len is vllm's tile size — 131072 overflows BI-V100 VRAM budget.
Submission 508 failed with 100% Connection refused = service never started.
This commit is contained in:
dylan-claude
2026-08-07 07:16:28 +00:00
parent 2c353da28b
commit 95d03147e7

View File

@@ -8,21 +8,16 @@ command:
- --served-model-name
- llm
- --max-model-len
- '131072'
- '100000'
- --gpu-memory-utilization
- '0.90'
- --trust-remote-code
- -tp
- '4'
- --max-num-seqs
- '2'
- '1'
- --disable-log-requests
- --disable-frontend-multiprocessing
- --max-num-batched-tokens
- '8192'
- --enable-chunked-prefill
- --max-seq-len-to-capture
- '32768'
- --enforce-eager
- --enable-auto-tool-choice
- --tool-call-parser