[Doc] Add Qwen2.5-VL eager mode doc (#1394)

### What this PR does / why we need it?
Add Qwen2.5-VL eager mode doc.

---------

Signed-off-by: shen-shanshan <467638484@qq.com>
This commit is contained in:
Shanshan Shen
2025-06-28 09:08:51 +08:00
committed by GitHub
parent d59e7fa095
commit 99e685532d

View File

@@ -29,6 +29,9 @@ docker run --rm \
Setup environment variables: Setup environment variables:
```bash ```bash
# Use vllm v1 engine
export VLLM_USE_V1=1
# Load model from ModelScope to speed up download # Load model from ModelScope to speed up download
export VLLM_USE_MODELSCOPE=True export VLLM_USE_MODELSCOPE=True
@@ -57,6 +60,7 @@ llm = LLM(
model=MODEL_PATH, model=MODEL_PATH,
max_model_len=16384, max_model_len=16384,
limit_mm_per_prompt={"image": 10}, limit_mm_per_prompt={"image": 10},
enforce_eager=True,
) )
sampling_params = SamplingParams( sampling_params = SamplingParams(
@@ -103,13 +107,11 @@ outputs = llm.generate([llm_inputs], sampling_params=sampling_params)
generated_text = outputs[0].outputs[0].text generated_text = outputs[0].outputs[0].text
print(generated_text) print(generated_text)
``` ```
If you run this script successfully, you can see the info shown below: If you run this script successfully, you can see the info shown below:
```bash ```bash
Processed prompts: 100%|███████████████| 1/1 [00:11<00:00, 11.29s/it, est. speed input: 9.48 toks/s, output: 20.55 toks/s]
The image displays a logo consisting of two main elements: a stylized geometric design and a pair of text elements. The image displays a logo consisting of two main elements: a stylized geometric design and a pair of text elements.
1. **Geometric Design**: On the left side of the image, there is a blue geometric design that appears to be made up of interconnected shapes. These shapes resemble a network or a complex polygonal structure, possibly hinting at a technological or interconnected theme. The design is monochromatic and uses only blue as its color, which could be indicative of a specific brand or company. 1. **Geometric Design**: On the left side of the image, there is a blue geometric design that appears to be made up of interconnected shapes. These shapes resemble a network or a complex polygonal structure, possibly hinting at a technological or interconnected theme. The design is monochromatic and uses only blue as its color, which could be indicative of a specific brand or company.
@@ -141,10 +143,15 @@ docker run --rm \
-v /etc/ascend_install.info:/etc/ascend_install.info \ -v /etc/ascend_install.info:/etc/ascend_install.info \
-v /root/.cache:/root/.cache \ -v /root/.cache:/root/.cache \
-p 8000:8000 \ -p 8000:8000 \
-e VLLM_USE_V1=1 \
-e VLLM_USE_MODELSCOPE=True \ -e VLLM_USE_MODELSCOPE=True \
-e PYTORCH_NPU_ALLOC_CONF=max_split_size_mb:256 \ -e PYTORCH_NPU_ALLOC_CONF=max_split_size_mb:256 \
-it $IMAGE \ -it $IMAGE \
vllm serve Qwen/Qwen2.5-VL-7B-Instruct --dtype bfloat16 --max_model_len 16384 --max-num-batched-tokens 16384 vllm serve Qwen/Qwen2.5-VL-7B-Instruct \
--dtype bfloat16 \
--max_model_len 16384 \
--max-num-batched-tokens 16384 \
--enforce-eager
``` ```
:::{note} :::{note}