enginex-mlu370-vl-qwen

运行于寒武纪 mlu370 算力卡的【视觉多模态理解】引擎，支持 Qwen2.5-VL-7B-Instruct、Qwen2.5-VL-32B-Instruct、Qwen2.5-VL-72B-Instruct 模型

QuickStart

1、从 modelscope上下载模型 Qwen2.5-VL-7B-Instruct、Qwen2.5-VL-32B-Instruct、Qwen2.5-VL-72B-Instruct 到目录 /mnt/contest_ceph/zhoushasha/models/Qwen/Qwen2.5-VL-7B-Instruct

2、使用Dockerfile生成镜像 使用 Dockerfile 生成 镜像

docker build -f Dockerfile -t test-cambricon:Qwen2.5-VL-32B-Instruct .

3、启动docker

docker run -it --privileged \
  -p 10091:80 \
  --device=/dev/cambricon_dev0:/dev/cambricon_dev0 \
  --device=/dev/cambricon_dev1:/dev/cambricon_dev1 \
  --device=/dev/cambricon_dev2:/dev/cambricon_dev2 \
  --device=/dev/cambricon_dev3:/dev/cambricon_dev3 \
  --device=/dev/cambricon_ctl \
  --device=/dev/cambricon_ipcm0:/dev/cambricon_ipcm0 \
  --device=/dev/cambricon_ipcm1:/dev/cambricon_ipcm1 \
  --device=/dev/cambricon_ipcm2:/dev/cambricon_ipcm2 \
  --device=/dev/cambricon_ipcm3:/dev/cambricon_ipcm3 \
  -v /mnt/contest_ceph/zhoushasha/models/Qwen/Qwen2.5-VL-32B-Instruct:/models:ro \
  test-cambricon:Qwen2.5-VL-32B-Instruct

4、测试服务

curl -X POST http://localhost:10091/predict \
  -F "image=@demo.jpeg" \
  -F "prompt=What is happening in this image?"