Honour vllm flags appended after the image, highest priority
Previously any arguments after the image in docker run were silently dropped: entrypoint.sh never forwarded "$@", so docker run ... <image> --served-model-name mymodel started fine but kept serving as llm with no warning. - entrypoint.sh: forward "$@"; skip script-generated flags the user already provided so vllm never sees duplicate keys; strip those keys from EXTRA_ARGS when the command line also sets them - precedence: command line > EXTRA_ARGS > same-name -e > detected default - warn when --port / --served-model-name is overridden, since the host -p side and the client model field must change too - README: document the three ways and their precedence
This commit is contained in:
25
README.md
25
README.md
@@ -355,6 +355,31 @@ Maverick 占用全部 16 张卡,与其它模型互斥;Scout 只占 8 张,
|
||||
**注意**:两个变体都映射到宿主 `24396`,串行部署没问题;并行时必须改 `-p` 左侧端口。
|
||||
`-e SERVED_NAME=llm` 让 `/v1/models` 返回 `llm`,客户端不用为换模型改 `model` 字段。
|
||||
|
||||
### 5.2.1 覆盖参数:三种写法,优先级从高到低
|
||||
|
||||
| 优先级 | 写法 | 例子 |
|
||||
|---|---|---|
|
||||
| 1 | 在镜像后面直接跟 vllm 参数 | `... llama-4-bi150:v0.1.2 --served-model-name mymodel` |
|
||||
| 2 | `-e EXTRA_ARGS="..."` | `-e EXTRA_ARGS="--max-model-len 8192"` |
|
||||
| 3 | 同名 `-e` | `-e MAX_MODEL_LEN=8192 -e SERVED_NAME=mymodel` |
|
||||
| 4 | 默认值 / config.json 检测 | 什么都不传 |
|
||||
|
||||
三种写法都能生效,不会重复传参。覆盖 `--port` / `--served-model-name` 时会打印 WARN,
|
||||
因为宿主机 `-p` 右侧端口和客户端 `model` 字段也要同步改。
|
||||
|
||||
```bash
|
||||
# 镜像后直接跟参数
|
||||
docker run -d --name llama4-scout \
|
||||
--entrypoint /usr/local/bin/llama4-entrypoint.sh --workdir /opt/llama4 \
|
||||
--privileged --pid host --ipc host -v /dev:/dev \
|
||||
-v /mnt/disk0/models/LLM-Research/Llama-4-Scout-17B-16E-Instruct:/models/llama4:ro \
|
||||
--ulimit memlock=-1 --ulimit stack=67108864 \
|
||||
-p 24396:80 \
|
||||
-e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 -e MODEL_PATH=/models/llama4 -e AUTO_DETECT=1 \
|
||||
harbor.4pd.io/modelhubxc/xiejiahao/llama-4-bi150:v0.1.2 \
|
||||
--served-model-name mymodel --max-model-len 8192
|
||||
```
|
||||
|
||||
### 5.3 用 run.sh 启动 CI 镜像
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user