39 lines
1.3 KiB
Markdown
39 lines
1.3 KiB
Markdown
|
# enginex-mlu370-vl-qwen
|
||
|
|
|
||
|
|
运行于寒武纪 mlu370 算力卡的【视觉多模态理解】引擎，支持 Qwen2.5-VL-7B-Instruct、Qwen2.5-VL-32B-Instruct、Qwen2.5-VL-72B-Instruct 模型
|
||
|
|
|
||
|
|
## QuickStart
|
||
|
|
|
||
|
|
1、从 modelscope上下载模型 Qwen2.5-VL-7B-Instruct、Qwen2.5-VL-32B-Instruct、Qwen2.5-VL-72B-Instruct
|
||
|
|
到目录 /mnt/contest_ceph/zhoushasha/models/Qwen/Qwen2.5-VL-7B-Instruct
|
||
|
|
|
||
|
|
2、使用Dockerfile生成镜像
|
||
|
|
使用 Dockerfile 生成 镜像
|
||
|
|
```python
|
||
|
|
docker build -f Dockerfile -t test-cambricon:Qwen2.5-VL-32B-Instruct .
|
||
|
|
```
|
||
|
|
|
||
|
|
3、启动docker
|
||
|
|
```python
|
||
|
|
docker run -it --privileged \
|
||
|
|
-p 10091:80 \
|
||
|
|
--device=/dev/cambricon_dev0:/dev/cambricon_dev0 \
|
||
|
|
--device=/dev/cambricon_dev1:/dev/cambricon_dev1 \
|
||
|
|
--device=/dev/cambricon_dev2:/dev/cambricon_dev2 \
|
||
|
|
--device=/dev/cambricon_dev3:/dev/cambricon_dev3 \
|
||
|
|
--device=/dev/cambricon_ctl \
|
||
|
|
--device=/dev/cambricon_ipcm0:/dev/cambricon_ipcm0 \
|
||
|
|
--device=/dev/cambricon_ipcm1:/dev/cambricon_ipcm1 \
|
||
|
|
--device=/dev/cambricon_ipcm2:/dev/cambricon_ipcm2 \
|
||
|
|
--device=/dev/cambricon_ipcm3:/dev/cambricon_ipcm3 \
|
||
|
|
-v /mnt/contest_ceph/zhoushasha/models/Qwen/Qwen2.5-VL-32B-Instruct:/models:ro \
|
||
|
|
test-cambricon:Qwen2.5-VL-32B-Instruct
|
||
|
|
```
|
||
|
|
|
||
|
|
4、测试服务
|
||
|
|
```python
|
||
|
|
curl -X POST http://localhost:10091/predict \
|
||
|
|
-F "image=@demo.jpeg" \
|
||
|
|
-F "prompt=What is happening in this image?"
|
||
|
|
```
|