upload model

.gitattributes

@@ -36,3 +36,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 ggml-model-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
 ggml-model-q2_k.gguf filter=lfs diff=lfs merge=lfs -text
 ggml-model-q3_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q4_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q5_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q6_k.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -2,8 +2,13 @@
 frameworks:
 - other
 license: Apache License 2.0
-tasks:
+model-type:
-- text-generation
+- llama
+language:
+- zh
+- en
+tools:
+- llamacpp
 
 #model-type:
 ##如 gpt、phi、llama、chatglm、baichuan 等
@@ -29,23 +34,76 @@ tasks:
 ##如 vllm、fastchat、llamacpp、AdaSeq 等
 #- vllm
 ---
-### 当前模型的贡献者未提供更加详细的模型介绍。模型文件和权重，可浏览“模型文件”页面获取。
-#### 您可以通过如下git clone命令，或者ModelScope SDK来下载模型
 
-SDK下载
+---
-```bash
+license: apache-2.0
-#安装ModelScope
+language:
-pip install modelscope
+- zh
-```
+- en
-```python
+---
-#SDK模型下载
-from modelscope import snapshot_download
-model_dir = snapshot_download('ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3-gguf')
-```
-Git下载
-```
-#Git模型下载
-git clone https://www.modelscope.cn/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3-gguf.git
-```
 
-<p style="color: lightgrey;">如果您是本模型的贡献者，我们邀请您根据<a href="https://modelscope.cn/docs/ModelScope%E6%A8%A1%E5%9E%8B%E6%8E%A5%E5%85%A5%E6%B5%81%E7%A8%8B%E6%A6%82%E8%A7%88" style="color: lightgrey; text-decoration: underline;">模型贡献文档</a>，及时完善模型卡片内容。</p>
+# Llama-3-Chinese-8B-Instruct-v3-GGUF
+
+<p align="center">
+    <a href="https://github.com/ymcui/Chinese-LLaMA-Alpaca-3"><img src="https://ymcui.com/images/chinese-llama-alpaca-3-banner.png" width="600"/></a>
+</p>
+
+这个仓库包含了**Llama-3-Chinese-8B-Instruct-v3-GGUF**（兼容llama.cpp/ollama等），是[Llama-3-Chinese-8B-Instruct-v3](https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3)模型的量化版本。
+
+**注意：这是一个指令模型，可以直接适用于对话、问答等任务。**
+
+更多细节（性能、使用方法等）请参考GitHub项目页面：https://github.com/ymcui/Chinese-LLaMA-Alpaca-3
+
+## 量化性能
+
+评测指标：PPL，**越低越好**
+
+| Quant |     Size |                 PPL |
+| :---: | -------: | ------------------: |
+| Q2_K  |  2.96 GB | 10.0534 +/- 0.13135 |
+| Q3_K  |  3.74 GB |  6.3295 +/- 0.07816 |
+| Q4_0  |  4.34 GB |  6.3200 +/- 0.07893 |
+| Q4_K  |  4.58 GB |  6.0042 +/- 0.07431 |
+| Q5_0  |  5.21 GB |  6.0437 +/- 0.07526 |
+| Q5_K  |  5.34 GB |  5.9484 +/- 0.07399 |
+| Q6_K  |  6.14 GB |  5.9469 +/- 0.07404 |
+| Q8_0  |  7.95 GB |  5.8933 +/- 0.07305 |
+|  F16  | 14.97 GB |  5.8902 +/- 0.07303 |
+
+
+## 其他
+
+- 完整模型：https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3
+
+- 关于本模型的提问，请通过 https://github.com/ymcui/Chinese-LLaMA-Alpaca-3 提交issue
+
+
+----
+
+This repository contains **Llama-3-Chinese-8B-Instruct-v3-GGUF** (llama.cpp/ollama/tgw, etc. compatible), which is the quantized version of [Llama-3-Chinese-8B-Instruct-v3](https://huggingface.co/hfl/llama-3-chinese-8b-instruct-v3).
+
+**Note: this is an instruction (chat) model, which can be used for conversation, QA, etc.** 
+
+Further details (performance, usage, etc.) should refer to GitHub project page: https://github.com/ymcui/Chinese-LLaMA-Alpaca-3
+
+## Performance
+
+Metric: PPL, lower is better
+
+| Quant |     Size |                 PPL |
+| :---: | -------: | ------------------: |
+| Q2_K  |  2.96 GB | 10.0534 +/- 0.13135 |
+| Q3_K  |  3.74 GB |  6.3295 +/- 0.07816 |
+| Q4_0  |  4.34 GB |  6.3200 +/- 0.07893 |
+| Q4_K  |  4.58 GB |  6.0042 +/- 0.07431 |
+| Q5_0  |  5.21 GB |  6.0437 +/- 0.07526 |
+| Q5_K  |  5.34 GB |  5.9484 +/- 0.07399 |
+| Q6_K  |  6.14 GB |  5.9469 +/- 0.07404 |
+| Q8_0  |  7.95 GB |  5.8933 +/- 0.07305 |
+|  F16  | 14.97 GB |  5.8902 +/- 0.07303 |
+
+## Others
+
+- For full model, please see: https://huggingface.co/hfl/llama-3-chinese-8b-instruct-v3
+
+- If you have questions/issues regarding this model, please submit an issue through https://github.com/ymcui/Chinese-LLaMA-Alpaca-3

ggml-model-q2_k.gguf

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1be784d9da6cce6d34aa63617d30c8e8826592ab0904039c7d11fe13b548448e
+oid sha256:91a7e83aeb02591215f8cef4f1f15ff3c6f67249726f7b31fdd7165c0cac4a5c
-size 3179131232
+size 3179131168

ggml-model-q3_k.gguf

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:72f75c4339e8992d5ffc757e17f9d40d1f5f1ea3c7e39a5bcfb06ea403ac327b
+oid sha256:ef57478d2c79bee719038561d6251199ff2f5b7d72bc2c7b0ae5ac7a0414fae2
-size 4018917728
+size 4018917664

ggml-model-q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd28b953e2a49e5b607baf729d1b70972720e8f1617098e75ebf1d30ecfd6237
+size 4661211424

ggml-model-q4_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d8740bc8bb0487f6d84eb3aefdbc2f861ec933ec8e48ef34a5daaea7a78d83a9
+size 4920733984

ggml-model-q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16659a3da16377cb6b98f03638899a8e62e655a9fbfd2c8b3e030d706c4c32ea
+size 5599293728

ggml-model-q5_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7580ccaf9674586c2bbae2fd81773b704863486587943df631801c25cdc87229
+size 5732987168

ggml-model-q6_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ca2cc87d5544a0f3d80b93c38daa44823d4d3b1418571affac8d777cf0fca93
+size 6596006176

ggml-model-q8_0.gguf

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9c8d978a7a197974a1be63ce92bc0df1c4273f6ce74c1e81ceb421a76c651723
+oid sha256:d22bd6413c0c600bdd51d846d6a9024b7601efa57e607807f60b8a10e1329425
-size 8540770656
+size 8540770592