diff --git a/README.md b/README.md index fcd839e..a7c50fa 100644 --- a/README.md +++ b/README.md @@ -48,14 +48,17 @@ We advise you to clone [`llama.cpp`](https://github.com/ggerganov/llama.cpp) and ## How to use -Cloning the repo may be inefficient, and thus you can manually download the GGUF file that you need or use `huggingface-cli` (`pip install huggingface_hub`) as shown below: -```shell -huggingface-cli download Qwen/Qwen1.5-32B-Chat-GGUF qwen1_5-32b-chat-q5_k_m.gguf --local-dir . --local-dir-use-symlinks False +Cloning the repo may be inefficient, and thus you can manually download the GGUF file that you need or use `modelscope` (`pip install modelscope`) as shown below: +```python +from modelscope.hub.file_download import model_file_download +model_dir = model_file_download(model_id='qwen/Qwen1.5-32B-Chat-GGUF',file_path='qwen1_5-32b-chat-q5_k_m.gguf',revision='master',cache_dir='/mnt/workspace/') ``` -We demonstrate how to use `llama.cpp` to run Qwen1.5: +We demonstrate how to install and use `llama.cpp` to run Qwen1.5: ```shell -./main -m qwen1_5-32b-chat-q5_k_m.gguf -n 512 --color -i -cml -f prompts/chat-with-qwen.txt +git clone https://github.com/ggerganov/llama.cpp.git +cd llama.cpp +make -j && ./main -m /mnt/workspace/qwen/Qwen1.5-32B-Chat-GGUF/qwen1_5-32b-chat-q5_k_m.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 400 -e ```