fix(CRITICAL): add --limit-mm-per-prompt image=5 — multimodal request kills engine
Engine crash: ValueError: You set image=0 (or defaulted to 1) in --limit-mm-per-prompt, but found 1 items in the same prompt. This kills the entire vLLM engine (AsyncEngineDeadError), making all subsequent requests return 503. Competition sends image requests in functional tests (d08/d09 multimodal). Fix: --limit-mm-per-prompt image=5 allows up to 5 images per prompt.
This commit is contained in:
@@ -33,7 +33,7 @@ command:
|
|||||||
- --dtype
|
- --dtype
|
||||||
- half
|
- half
|
||||||
- --limit-mm-per-prompt
|
- --limit-mm-per-prompt
|
||||||
- image=1
|
- image=5
|
||||||
env:
|
env:
|
||||||
- name: VLLM_ENGINE_ITERATION_TIMEOUT_S
|
- name: VLLM_ENGINE_ITERATION_TIMEOUT_S
|
||||||
value: '3600'
|
value: '3600'
|
||||||
|
|||||||
Reference in New Issue
Block a user