初始化项目，由ModelHub XC社区提供模型

Model: ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v2-gguf
Source: Original Platform

.gitattributes

@@ -0,0 +1,42 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*.tfevents* filter=lfs diff=lfs merge=lfs -text
+*.db* filter=lfs diff=lfs merge=lfs -text
+*.ark* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+ggml-model-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q2_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q3_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q6_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q5_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-model-q4_k.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,116 @@
+---
+frameworks:
+- other
+license: Apache License 2.0
+model-type:
+- llama
+language:
+- zh
+- en
+tools:
+- llamacpp
+
+#model-type:
+##如 gpt、phi、llama、chatglm、baichuan 等
+#- gpt
+
+#domain:
+##如 nlp、cv、audio、multi-modal
+#- nlp
+
+#language:
+##语言代码列表 https://help.aliyun.com/document_detail/215387.html?spm=a2c4g.11186623.0.0.9f8d7467kni6Aa
+#- cn 
+
+#metrics:
+##如 CIDEr、Blue、ROUGE 等
+#- CIDEr
+
+#tags:
+##各种自定义，包括 pretrained、fine-tuned、instruction-tuned、RL-tuned 等训练方法和其他
+#- pretrained
+
+#tools:
+##如 vllm、fastchat、llamacpp、AdaSeq 等
+#- vllm
+---
+
+---
+license: apache-2.0
+language:
+- zh
+- en
+---
+
+## **📣📣📣 v3版指令模型已发布，欢迎使用 👉 [[HF版]](https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3) [[GGUF版]](https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3-gguf)**
+
+# Llama-3-Chinese-8B-Instruct-v2-GGUF
+
+<p align="center">
+    <a href="https://github.com/ymcui/Chinese-LLaMA-Alpaca-3"><img src="https://ymcui.com/images/chinese-llama-alpaca-3-banner.png" width="600"/></a>
+</p>
+
+这个仓库包含了**Llama-3-Chinese-8B-Instruct-v2-GGUF**（兼容llama.cpp/ollama等），是[Llama-3-Chinese-8B-Instruct-v2](https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v2)模型的量化版本。
+
+**注意：这是一个指令模型，可以直接适用于对话、问答等任务。**
+
+更多细节（性能、使用方法等）请参考GitHub项目页面：https://github.com/ymcui/Chinese-LLaMA-Alpaca-3
+
+## 量化性能
+
+评测指标：PPL，**越低越好**
+
+| Quant |     Size |                 PPL |
+| :---: | -------: | ------------------: |
+| Q2_K  |  2.96 GB | 13.2488 +/- 0.17217 |
+| Q3_K  |  3.74 GB |  6.9618 +/- 0.08420 |
+| Q4_0  |  4.34 GB |  6.8925 +/- 0.08437 |
+| Q4_K  |  4.58 GB |  6.4851 +/- 0.07892 |
+| Q5_0  |  5.21 GB |  6.4608 +/- 0.07862 |
+| Q5_K  |  5.34 GB |  6.3742 +/- 0.07740 |
+| Q6_K  |  6.14 GB |  6.3494 +/- 0.07703 |
+| Q8_0  |  7.95 GB |  6.3110 +/- 0.07673 |
+|  F16  | 14.97 GB |  6.3005 +/- 0.07658 |
+
+
+## 其他
+
+- 完整模型：https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v2
+
+- LoRA模型：https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v2-lora
+
+- 关于本模型的提问，请通过 https://github.com/ymcui/Chinese-LLaMA-Alpaca-3 提交issue
+
+
+----
+
+This repository contains **Llama-3-Chinese-8B-Instruct-v2-GGUF** (llama.cpp/ollama/tgw, etc. compatible), which is the quantized version of [Llama-3-Chinese-8B-Instruct-v2](https://huggingface.co/hfl/llama-3-chinese-8b-instruct-v2).
+
+**Note: this is an instruction (chat) model, which can be used for conversation, QA, etc.** 
+
+Further details (performance, usage, etc.) should refer to GitHub project page: https://github.com/ymcui/Chinese-LLaMA-Alpaca-3
+
+## Performance
+
+Metric: PPL, lower is better
+
+| Quant |     Size |                 PPL |
+| :---: | -------: | ------------------: |
+| Q2_K  |  2.96 GB | 13.2488 +/- 0.17217 |
+| Q3_K  |  3.74 GB |  6.9618 +/- 0.08420 |
+| Q4_0  |  4.34 GB |  6.8925 +/- 0.08437 |
+| Q4_K  |  4.58 GB |  6.4851 +/- 0.07892 |
+| Q5_0  |  5.21 GB |  6.4608 +/- 0.07862 |
+| Q5_K  |  5.34 GB |  6.3742 +/- 0.07740 |
+| Q6_K  |  6.14 GB |  6.3494 +/- 0.07703 |
+| Q8_0  |  7.95 GB |  6.3110 +/- 0.07673 |
+|  F16  | 14.97 GB |  6.3005 +/- 0.07658 |
+
+
+## Others
+
+- For full model, please see: https://huggingface.co/hfl/llama-3-chinese-8b-instruct-v2
+
+- For LoRA-only model, please see: https://huggingface.co/hfl/llama-3-chinese-8b-instruct-v2-lora
+
+- If you have questions/issues regarding this model, please submit an issue through https://github.com/ymcui/Chinese-LLaMA-Alpaca-3

configuration.json

@@ -0,0 +1 @@
+{"framework":"other","task":"fill-mask"}

ggml-model-q2_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02f9b38ebb1a39f8378b32b3583b862641412c30f319fb2dfe0c198b1f5717b1
+size 3179131232

ggml-model-q3_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c43e69f22c227a758e4d99aed57336e59cba603e8739eeeccc88b704c6f862b9
+size 4018917728

ggml-model-q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55bd47f7fcf749cfc559c13c278e89154d40b81724978a9bdbf7471004d3a561
+size 4661211488

ggml-model-q4_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:823ed11f3c30a79ee49e9374be0403c46907e680af6733e3802a8054aae2da66
+size 4920734048

ggml-model-q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de52c6e71fe41a26a92486ec6b6d3261965c68399597f49ef8090c6b991e954a
+size 5599293792

ggml-model-q5_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:014fa21b97adf5ab025f9a9cb2b4ba2cbc9c59862356bb297ce325c83f55518b
+size 5732987232

ggml-model-q6_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a536ed925d6b1f423cb8b029f36d11991f49edbd51a63e22e636511809ab7ba2
+size 6596006240

ggml-model-q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41ead1861f50bd60c5683a941e2528a47e76012041bcb5d0f3e9008b28e65980
+size 8540770656