forked from EngineX-Iluvatar/enginex-vllm-bi100-qwen36
main
10-17 那次推送依据「官方仓库用 git.modelhub.org.cn:9443」把 FROM 改回内网地址, 但按 blob 实测:16 次 A 层「Docker 镜像构建失败」发生在 10-01~10-04,当时 FROM 正是该内网地址; a3a160df(10-10 01:35) 才把它换成 harbor.4pd.io。故那次推送是撤销修复、属回归,现予回退。 边界:harbor 版从未被任何提交实际跑过,两个地址孰对仍未验证。
天数智芯 天垓100 文本生成引擎(基于 vLLM 优化适配Qwen3.6-35B-A3B)
# 本地构建
docker build -t enginex-iluvatar-vllm:bi100-qwen3.6 -f Dockerfile .
启动容器镜像
下载Qwen3.6-35B-A3B模型,并且需要将模型的config.json文件中architectures字段改成
"architectures": [
"Qwen3_5MoeForCausalLM"
]
docker run -dit --network=host --ipc=host \
-v /usr/src:/usr/src -v /lib/modules:/lib/modules -v /dev:/dev --privileged \
-v /mnt/disk1/models/Qwen3.6-35B-A3B:/model:ro --entrypoint=python3 \
-e CUDA_VISIBLE_DEVICES=4,5,6,7 -e VLLM_ENGINE_ITERATION_TIMEOUT_S=3600 \
enginex-iluvatar-vllm:bi100-qwen3.6 \
-m vllm.entrypoints.openai.api_server \
--model /model --port 1111 --served-model-name llm \
--max-model-len 100000 --trust-remote-code -tp 4 --gpu-memory-utilization 0.95 \
--max-num-seqs 1 --disable-log-requests --disable-frontend-multiprocessing \
--max-num-batched-tokens 4096 --enable-chunked-prefill \
--max-seq-len-to-capture 32768 --enable-auto-tool-choice \
--tool-call-parser qwen3_coder --reasoning-parser qwen3
请求
curl http://localhost:1111/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "llm",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Can you tell me the story of Snow White?"}
],
"max_tokens": 200,
"temperature": 0.7
}'
Description
Qwen3.6-35B-A3B on Iluvatar BI-V100 VLLM Engine
96 MiB
1 Stars
1 Watchers
0 Forks
Languages
Python
99.8%
Shell
0.1%
C
0.1%