enginex-mlu370-any2any/README.md

# enginex-mlu370-any2any

# 寒武纪 mlu370 统一多模态

该模型测试框架在寒武纪mlu370 （X8/X4）加速卡上，基于Transfomer框架，适配了 Qwen/Qwen3-Omni-30B-A3B-Instruct 模型。

* 详见 https://modelscope.cn/models/Qwen/Qwen3-Omni-30B-A3B-Instruct


## Quick Start
1. 首先从modelscope上下载vlm，如`gemma-3-4b-it`  
```bash
modelscope download --model Qwen/Qwen3-Omni-30B-A3B-Instruct --local_dir /models/Qwen3-Omni-30B-A3B-Instruct
```
2. 构建镜像
```bash
docker build -t qwen:omni .
```

3. 启动docker 
```bash
docker run -it --rm \
  -v /models/:/mnt/models \
  --device=/dev/cambricon_dev0:/dev/cambricon_dev0 \
  --device=/dev/cambricon_dev1:/dev/cambricon_dev1 \
  --device=/dev/cambricon_dev2:/dev/cambricon_dev2 \
  --device=/dev/cambricon_dev3:/dev/cambricon_dev3 \
  --device=/dev/cambricon_ctl:/dev/cambricon_ctl \
  -p 8080:80 \
  qwen:omni
```  
注意需要在本地使用寒武纪mlu370 芯片   

4. 测试服务   

4.1 测试视觉理解    
```bash
python request.py
```
4.2 测试统一多模态

启动容器时指定入口点为 /bin/bash

```bash
docker run -it --rm \
  -v /models/:/mnt/models \
  --device=/dev/cambricon_dev0:/dev/cambricon_dev0 \
  --device=/dev/cambricon_dev1:/dev/cambricon_dev1 \
  --device=/dev/cambricon_dev2:/dev/cambricon_dev2 \
  --device=/dev/cambricon_dev3:/dev/cambricon_dev3 \
  --device=/dev/cambricon_ctl:/dev/cambricon_ctl \
  --entrypoint /bin/bash \
  -p 8080:80 \
  qwen:omni
```  

将 test.py 拷贝到容器内
```
docker cp ./test.py <container_id>:/workspace/test.py
```

进入容器执行测试脚本

```bash
python test.py
```