初始化项目，由ModelHub XC社区提供模型

Model: ChineseAlpacaGroup/llama-3-chinese-8b-instruct-gguf
Source: Original Platform

.gitattributes

@@ -0,0 +1,50 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*.tfevents* filter=lfs diff=lfs merge=lfs -text
+*.db* filter=lfs diff=lfs merge=lfs -text
+*.ark* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+ggml-model-q2_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q2_k-im.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q3_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q3_k-im.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q4_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q4_k-im.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q4_0-im.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q5_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q5_k-im.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q5_0-im.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q6_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q6_k-im.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q8_0-im.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,117 @@
+---
+frameworks:
+- other
+license: Apache License 2.0
+model-type:
+- llama
+language:
+- zh
+- en
+tools:
+- llamacpp
+
+#model-type:
+##如 gpt、phi、llama、chatglm、baichuan 等
+#- gpt
+
+#domain:
+##如 nlp、cv、audio、multi-modal
+#- nlp
+
+#language:
+##语言代码列表 https://help.aliyun.com/document_detail/215387.html?spm=a2c4g.11186623.0.0.9f8d7467kni6Aa
+#- cn 
+
+#metrics:
+##如 CIDEr、Blue、ROUGE 等
+#- CIDEr
+
+#tags:
+##各种自定义，包括 pretrained、fine-tuned、instruction-tuned、RL-tuned 等训练方法和其他
+#- pretrained
+
+#tools:
+##如 vllm、fastchat、llamacpp、AdaSeq 等
+#- vllm
+---
+
+---
+license: apache-2.0
+language:
+- zh
+- en
+---
+## **📣📣📣 v3版指令模型已发布，欢迎使用 👉 [[HF版]](https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3) [[GGUF版]](https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3-gguf)**
+
+# Llama-3-Chinese-8B-Instruct-GGUF
+
+## 提醒: GGUF文件已重新生成，由于llama.cpp仍然有可能对其作出修改，因此建议保留HF版模型。
+
+<p align="center">
+    <a href="https://github.com/ymcui/Chinese-LLaMA-Alpaca-3"><img src="https://ymcui.com/images/chinese-llama-alpaca-3-banner.png" width="600"/></a>
+</p>
+
+这个仓库包含了**Llama-3-Chinese-8B-Instruct-GGUF**（兼容llama.cpp/ollama等），是[Llama-3-Chinese-8B-Instruct](https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct)模型的量化版本。
+
+**注意：这是一个指令模型，可以直接适用于对话、问答等任务。**
+
+更多细节（性能、使用方法等）请参考GitHub项目页面：https://github.com/ymcui/Chinese-LLaMA-Alpaca-3
+
+## 量化性能
+
+评测指标：PPL，**越低越好**
+
+| Quant |     Size |   PPL (old model)  | 👍🏻 PPL (new model) |
+| :---: | -------: | -----------------: | ------------------: |
+| Q2_K  |  2.96 GB | 10.3918 +/- 0.13288 | 9.1168 +/- 0.10711 |
+| Q3_K  |  3.74 GB |  6.3018 +/- 0.07849 | 5.4082 +/- 0.05955 |
+| Q4_0  |  4.34 GB |  6.0628 +/- 0.07501 | 5.2048 +/- 0.05725 |
+| Q4_K  |  4.58 GB |  5.9066 +/- 0.07419 | 5.0189 +/- 0.05520 |
+| Q5_0  |  5.21 GB |  5.8562 +/- 0.07355 | 4.9803 +/- 0.05493 |
+| Q5_K  |  5.34 GB |  5.8062 +/- 0.07331 | 4.9195 +/- 0.05436 |
+| Q6_K  |  6.14 GB |  5.7757 +/- 0.07298 | 4.8966 +/- 0.05413 |
+| Q8_0  |  7.95 GB |  5.7626 +/- 0.07272 | 4.8822 +/- 0.05396 |
+|  F16  | 14.97 GB |  5.7628 +/- 0.07275 | 4.8802 +/- 0.05392 |
+
+
+## 其他
+
+- 完整模型：https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct
+
+- LoRA模型：https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-lora
+
+- 关于本模型的提问，请通过 https://github.com/ymcui/Chinese-LLaMA-Alpaca-3 提交issue
+
+
+----
+
+This repository contains **Llama-3-Chinese-8B-Instruct-GGUF** (llama.cpp/ollama/tgw, etc. compatible), which is the quantized version of [Llama-3-Chinese-8B-Instruct](https://huggingface.co/hfl/llama-3-chinese-8b-instruct).
+
+**Note: this is an instruction (chat) model, which can be used for conversation, QA, etc.** 
+
+Further details (performance, usage, etc.) should refer to GitHub project page: https://github.com/ymcui/Chinese-LLaMA-Alpaca-3
+
+## Performance
+
+Metric: PPL, lower is better
+
+| Quant |     Size |   PPL (old model)  | 👍🏻 PPL (new model) |
+| :---: | -------: | -----------------: | ------------------: |
+| Q2_K  |  2.96 GB | 10.3918 +/- 0.13288 | 9.1168 +/- 0.10711 |
+| Q3_K  |  3.74 GB |  6.3018 +/- 0.07849 | 5.4082 +/- 0.05955 |
+| Q4_0  |  4.34 GB |  6.0628 +/- 0.07501 | 5.2048 +/- 0.05725 |
+| Q4_K  |  4.58 GB |  5.9066 +/- 0.07419 | 5.0189 +/- 0.05520 |
+| Q5_0  |  5.21 GB |  5.8562 +/- 0.07355 | 4.9803 +/- 0.05493 |
+| Q5_K  |  5.34 GB |  5.8062 +/- 0.07331 | 4.9195 +/- 0.05436 |
+| Q6_K  |  6.14 GB |  5.7757 +/- 0.07298 | 4.8966 +/- 0.05413 |
+| Q8_0  |  7.95 GB |  5.7626 +/- 0.07272 | 4.8822 +/- 0.05396 |
+|  F16  | 14.97 GB |  5.7628 +/- 0.07275 | 4.8802 +/- 0.05392 |
+
+
+## Others
+
+- For full model, please see: https://huggingface.co/hfl/llama-3-chinese-8b-instruct
+
+- For LoRA-only model, please see: https://huggingface.co/hfl/llama-3-chinese-8b-instruct-lora
+
+- If you have questions/issues regarding this model, please submit an issue through https://github.com/ymcui/Chinese-LLaMA-Alpaca-3

configuration.json

@@ -0,0 +1 @@
+{"framework":"Pytorch","task":"fill-mask"}

ggml-model-q2_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27cc458ed121fac94e516c2b1df1fa841a8b0611c15ab285cf808bc4bf258de5
+size 3179131168

ggml-model-q3_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54c8ab805b5979e9cc181a29bf60d40459c05c7d0b32df84e0cf404d141fe5f5
+size 4018917664

ggml-model-q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9147332466a9d49fa568571b114e84150d1d2db7013ac4e9545b4217a8a4cc5f
+size 4661211424

ggml-model-q4_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5560533f34bacd67a68e78d8b31cbac4871864a6563248f1aca09762bd264dc9
+size 4920733984

ggml-model-q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e474be4c4e38dd85a9f16ba9385848c7fe19e2ce8475c493678d97be312f67be
+size 5599293728

ggml-model-q5_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:203502e82441aed006e3ed68727edb1c26c50d624719a76829c9f6d249ab5c69
+size 5732987168

ggml-model-q6_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e80e225ac6890d828a1ca16dda3a63cc0886dd463a786e84568fdde93608755e
+size 6596006176

ggml-model-q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:296046e494feba1698ecc3e89b86547329ad73184197835c9d8964ee5ff6c0b2
+size 8540770592