22 lines
695 B
Markdown
22 lines
695 B
Markdown
# enginex-bi_150-llama.cpp
|
||
|
||
运行于【天数智芯-天垓150】算力卡的【文本生成】引擎，基于 llama.cpp (b7516) 引擎进行架构特别适配优化。
|
||
|
||
## Build Docker Image
|
||
|
||
```bash
|
||
docker build -t enginex-iluvatar/iluvatar-llama.cpp:b7516-bi150 .
|
||
```
|
||
|
||
最新镜像：git.modelhub.org.cn:9443/enginex-iluvatar/iluvatar-llama.cpp:b7516-bi150
|
||
|
||
运行容器
|
||
**注意**：必须使用 `--no-mmap` 参数关闭内存映射，否则会报错
|
||
```bash
|
||
docker run -it --rm \
|
||
-v <model_dir>:/app/models \
|
||
--privileged \
|
||
-e CUDA_VISIBLE_DEVICES=0 \
|
||
git.modelhub.org.cn:9443/enginex-iluvatar/iluvatar-llama.cpp:b7516-bi150
|
||
/app/llama-cli -m /app/models/xxx.gguf --no-mmap -p "你好"
|
||
``` |