Upload README.md

README.md

@@ -1,47 +1,73 @@
 ---
-license: Apache License 2.0
-
-#model-type:
-##如 gpt、phi、llama、chatglm、baichuan 等
-#- gpt
-
-#domain:
-##如 nlp、cv、audio、multi-modal
-#- nlp
-
-#language:
-##语言代码列表 https://help.aliyun.com/document_detail/215387.html?spm=a2c4g.11186623.0.0.9f8d7467kni6Aa
-#- cn 
-
-#metrics:
-##如 CIDEr、Blue、ROUGE 等
-#- CIDEr
-
-#tags:
-##各种自定义，包括 pretrained、fine-tuned、instruction-tuned、RL-tuned 等训练方法和其他
-#- pretrained
-
-#tools:
-##如 vllm、fastchat、llamacpp、AdaSeq 等
-#- vllm
+license: other
+license_name: license
+license_link: LICENSE
 ---
-### 当前模型的贡献者未提供更加详细的模型介绍。模型文件和权重，可浏览“模型文件”页面获取。
-#### 您可以通过如下git clone命令，或者ModelScope SDK来下载模型
+<div align="center">
+<h1>
+  Index-1.9B-Chat-GGUF
+</h1>
+</div>
 
-SDK下载
-```bash
-#安装ModelScope
-pip install modelscope
+This repository is the GGUF version of [Index-1.9B-Chat](https://huggingface.co/IndexTeam/Index-1.9B-Chat), which adapts to llama.cpp and also provides ModelFile adaptation for Ollma.
+
+For more details, see our [GitHub](https://github.com/bilibili/Index-1.9B) and [Index-1.9B Technical Report](https://github.com/bilibili/Index-1.9B/blob/main/Index-1.9B%20%E6%8A%80%E6%9C%AF%E6%8A%A5%E5%91%8A.pdf)
+
+### LLAMA.CPP
+```shell 
+# Install llama.cpp(https://github.com/ggerganov/llama.cpp)
+git clone https://github.com/ggerganov/llama.cpp
+cd llama.cpp
+make
+
+# Install llama-cpp-python(https://github.com/abetlen/llama-cpp-python)
+pip install llama-cpp-python
 ```
+llama.cpp terminal
+```shell
+./build/bin/llama-cli -m models/Index-1.9B-Chat/ggml-model-bf16.gguf --color -if
+```
+**Note!!** llama.cpp does not support custom chat_template, so you need to splice prompt yourself. The chat_template of Index-1.9B is
+```shell
+# The three delimiters are <unk>(token_id=0), reserved_0(token_id=3), reserved_1(token_id=4)
+[<unk>]sytem_message[reserved_0]user_message[reserved_1]response
+```
+Use llama-cpp-python to support custom chat_template (already written to GGUF and can be used directly)
 ```python
-#SDK模型下载
-from modelscope import snapshot_download
-model_dir = snapshot_download('IndexTeam/Index-1.9B-Chat-GGUF')
-```
-Git下载
-```
-#Git模型下载
-git clone https://www.modelscope.cn/IndexTeam/Index-1.9B-Chat-GGUF.git
-```
+from llama_cpp import Llama
 
-<p style="color: lightgrey;">如果您是本模型的贡献者，我们邀请您根据<a href="https://modelscope.cn/docs/ModelScope%E6%A8%A1%E5%9E%8B%E6%8E%A5%E5%85%A5%E6%B5%81%E7%A8%8B%E6%A6%82%E8%A7%88" style="color: lightgrey; text-decoration: underline;">模型贡献文档</a>，及时完善模型卡片内容。</p>
+model_path = "Index-1.9B-Chat-GGUF/ggml-model-Q6_K.gguf"
+llm = Llama(model_path =model_path, verbose=True)
+output = llm.create_chat_completion(
+      messages = [
+          {"role": "system", "content": "你是由哔哩哔哩自主研发的大语言模型，名为“Index”。你能够根据用户传入的信息，帮助用户完成指定的任务，并生成恰当的、符合要求的回复。"},
+          #{"role": "system", "content": "你需要扮演B站评论区老哥，用评论区阴阳怪气的话术回复，不要说你是AI"},
+          {"role": "user","content": "篮球和鸡有什么关系"}
+      ]
+)
+print(output)
+```
+### OLLAMA
+- Install [Ollama](https://github.com/ollama/ollama)
+```shell
+curl -fsSL https://ollama.com/install.sh | sh
+```
+```shell
+# Start server
+ollama serve
+
+# Adaptation model, model file and System Message can be modified in OllamaModelFile
+ollama create Index-1.9B-Chat -f Index-1.9B-Chat-GGUF/OllamaModelFile
+
+# Start Terminal
+ollama run Index-1.9B-Chat
+
+# System Message can be specified dynamically
+curl http://localhost:11434/api/chat -d '{
+  "model": "Index-1.9B-Chat",
+  "messages": [
+      { "role": "system", "content": "你是由哔哩哔哩自主研发的大语言模型，名为“Index”。你能够根据用户传入的信息，帮助用户完成指定的任务，并生成恰当的、符合要求的回复。" },
+    { "role": "user", "content": "续写 金坷垃" }
+  ]
+}'
+```