Honour vllm flags appended after the image, highest priority
Previously any arguments after the image in docker run were silently dropped: entrypoint.sh never forwarded "$@", so docker run ... <image> --served-model-name mymodel started fine but kept serving as llm with no warning. - entrypoint.sh: forward "$@"; skip script-generated flags the user already provided so vllm never sees duplicate keys; strip those keys from EXTRA_ARGS when the command line also sets them - precedence: command line > EXTRA_ARGS > same-name -e > detected default - warn when --port / --served-model-name is overridden, since the host -p side and the client model field must change too - README: document the three ways and their precedence
This commit is contained in:
25
README.md
25
README.md
@@ -421,7 +421,30 @@ docker run -d --name gev-26b-decide \
|
||||
两个变体共用挂载点 `/models/gemma4`,靠 `config.json` 区分;TP / eager / 显存 / 并发数都不用传,入口脚本会自动判定(见第 3 节)。
|
||||
|
||||
容器内端口恒为 80、模型名恒为 `llm`,所以两个变体可以复用同一个宿主端口 24396,串行切换时客户端不用改地址也不用改 `model` 字段。
|
||||
需要覆盖时加同名 `-e`,例如 `-e MAX_MODEL_LEN=16384`。
|
||||
### 覆盖参数:三种写法,优先级从高到低
|
||||
|
||||
| 优先级 | 写法 | 例子 |
|
||||
|---|---|---|
|
||||
| 1 | 在镜像后面直接跟 vllm 参数 | `... gemma4-bi150:v0.1.0 --served-model-name mymodel` |
|
||||
| 2 | `-e EXTRA_ARGS="..."` | `-e EXTRA_ARGS="--max-model-len 16384"` |
|
||||
| 3 | 同名 `-e` | `-e MAX_MODEL_LEN=16384 -e SERVED_NAME=mymodel` |
|
||||
| 4 | 默认值 / config.json 检测 | 什么都不传 |
|
||||
|
||||
三种写法都能生效,不会重复传参(脚本会跳过自己已经不需要传的项)。
|
||||
注意覆盖 `--port` 或 `--served-model-name` 时会打印 WARN,因为宿主机 `-p` 的右侧端口和客户端 `model` 字段也要同步改。
|
||||
|
||||
```bash
|
||||
# 写法 1:镜像后直接跟参数(最符合 vllm 习惯)
|
||||
docker run -d --name gemma4-31b \
|
||||
--entrypoint /usr/local/bin/gemma4-entrypoint.sh --workdir /opt/gemma4 \
|
||||
--privileged --pid host --ipc host -v /dev:/dev \
|
||||
-v /mnt/disk0/models/google/gemma-4-31B-it:/models/gemma4:ro \
|
||||
--ulimit memlock=-1 --ulimit stack=67108864 \
|
||||
-p 24396:80 \
|
||||
-e CUDA_VISIBLE_DEVICES=15,14,13,12 -e MODEL_PATH=/models/gemma4 -e AUTO_DETECT=1 \
|
||||
harbor.4pd.io/modelhubxc/xiejiahao/gemma4-bi150:v0.1.0 \
|
||||
--served-model-name mymodel --max-model-len 16384
|
||||
```
|
||||
|
||||
**注意**:并行跑两个变体时必须给不同的 `-p` 左侧端口,例如 `-p 24397:80`。
|
||||
|
||||
|
||||
Reference in New Issue
Block a user