Qwen/Qwen1.5-32B-Chat-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Cherrytest
2024-04-05 14:05:18 +00:00
parent 58311d1479
commit bc6485e6fd
1 changed files with 8 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
README.md
View File

@@ -48,14 +48,17 @@ We advise you to clone [`llama.cpp`](https://github.com/ggerganov/llama.cpp) and
## How to use
Cloning the repo may be inefficient, and thus you can manually download the GGUF file that you need or use `huggingface-cli` (`pip install huggingface_hub`) as shown below:
```shell
huggingface-cli download Qwen/Qwen1.5-32B-Chat-GGUF qwen1_5-32b-chat-q5_k_m.gguf --local-dir . --local-dir-use-symlinks False
Cloning the repo may be inefficient, and thus you can manually download the GGUF file that you need or use `modelscope` (`pip install modelscope`) as shown below:
```python
from modelscope.hub.file_download import model_file_download
model_dir = model_file_download(model_id='qwen/Qwen1.5-32B-Chat-GGUF',file_path='qwen1_5-32b-chat-q5_k_m.gguf',revision='master',cache_dir='/mnt/workspace/')
```
We demonstrate how to use `llama.cpp` to run Qwen1.5:
We demonstrate how to install and use `llama.cpp` to run Qwen1.5:
```shell
./main -m qwen1_5-32b-chat-q5_k_m.gguf -n 512 --color -i -cml -f prompts/chat-with-qwen.txt
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
make -j && ./main -m /mnt/workspace/qwen/Qwen1.5-32B-Chat-GGUF/qwen1_5-32b-chat-q5_k_m.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 400 -e
```