[bugfix] Add modelscope package to avoid docker image without modelscope (#1520)

This commit is contained in:
Kylin
2024-09-29 03:43:22 +08:00
committed by GitHub
parent 840c5dbcb3
commit f42e9bfb52
3 changed files with 16 additions and 1 deletions

View File

@@ -286,6 +286,17 @@ Launch [Qwen2-7B-Instruct](https://www.modelscope.cn/models/qwen/qwen2-7b-instru
```
SGLANG_USE_MODELSCOPE=true python -m sglang.launch_server --model-path qwen/Qwen2-7B-Instruct --port 30000
```
Or start it by docker.
```bash
docker run --gpus all \
-p 30000:30000 \
-v ~/.cache/modelscope:/root/.cache/modelscope \
--env "SGLANG_USE_MODELSCOPE=true" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server --model-path Qwen/Qwen2.5-7B-Instruct --host 0.0.0.0 --port 30000
```
</details>