commit 4a49ed1163a257235e457673ea7a306233e0a47c Author: ModelHub XC Date: Fri Aug 14 18:11:12 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v2-gguf Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..277ddb0 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,42 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bin.* filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zstandard filter=lfs diff=lfs merge=lfs -text +*.tfevents* filter=lfs diff=lfs merge=lfs -text +*.db* filter=lfs diff=lfs merge=lfs -text +*.ark* filter=lfs diff=lfs merge=lfs -text +**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text +**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text +**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +ggml-model-q8_0.gguf filter=lfs diff=lfs merge=lfs -text +ggml-model-q2_k.gguf filter=lfs diff=lfs merge=lfs -text +ggml-model-q3_k.gguf filter=lfs diff=lfs merge=lfs -text +ggml-model-q6_k.gguf filter=lfs diff=lfs merge=lfs -text +ggml-model-q5_0.gguf filter=lfs diff=lfs merge=lfs -text +ggml-model-q5_k.gguf filter=lfs diff=lfs merge=lfs -text +ggml-model-q4_0.gguf filter=lfs diff=lfs merge=lfs -text +ggml-model-q4_k.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..c53ffc3 --- /dev/null +++ b/README.md @@ -0,0 +1,116 @@ +--- +frameworks: +- other +license: Apache License 2.0 +model-type: +- llama +language: +- zh +- en +tools: +- llamacpp + +#model-type: +##如 gpt、phi、llama、chatglm、baichuan 等 +#- gpt + +#domain: +##如 nlp、cv、audio、multi-modal +#- nlp + +#language: +##语言代码列表 https://help.aliyun.com/document_detail/215387.html?spm=a2c4g.11186623.0.0.9f8d7467kni6Aa +#- cn + +#metrics: +##如 CIDEr、Blue、ROUGE 等 +#- CIDEr + +#tags: +##各种自定义，包括 pretrained、fine-tuned、instruction-tuned、RL-tuned 等训练方法和其他 +#- pretrained + +#tools: +##如 vllm、fastchat、llamacpp、AdaSeq 等 +#- vllm +--- + +--- +license: apache-2.0 +language: +- zh +- en +--- + +## **📣📣📣 v3版指令模型已发布，欢迎使用 👉 [[HF版]](https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3) [[GGUF版]](https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3-gguf)** + +# Llama-3-Chinese-8B-Instruct-v2-GGUF + +

+ +

+ +这个仓库包含了**Llama-3-Chinese-8B-Instruct-v2-GGUF**（兼容llama.cpp/ollama等），是[Llama-3-Chinese-8B-Instruct-v2](https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v2)模型的量化版本。 + +**注意：这是一个指令模型，可以直接适用于对话、问答等任务。** + +更多细节（性能、使用方法等）请参考GitHub项目页面：https://github.com/ymcui/Chinese-LLaMA-Alpaca-3 + +## 量化性能 + +评测指标：PPL，**越低越好** + +| Quant | Size | PPL | +| :---: | -------: | ------------------: | +| Q2_K | 2.96 GB | 13.2488 +/- 0.17217 | +| Q3_K | 3.74 GB | 6.9618 +/- 0.08420 | +| Q4_0 | 4.34 GB | 6.8925 +/- 0.08437 | +| Q4_K | 4.58 GB | 6.4851 +/- 0.07892 | +| Q5_0 | 5.21 GB | 6.4608 +/- 0.07862 | +| Q5_K | 5.34 GB | 6.3742 +/- 0.07740 | +| Q6_K | 6.14 GB | 6.3494 +/- 0.07703 | +| Q8_0 | 7.95 GB | 6.3110 +/- 0.07673 | +| F16 | 14.97 GB | 6.3005 +/- 0.07658 | + + +## 其他 + +- 完整模型：https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v2 + +- LoRA模型：https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v2-lora + +- 关于本模型的提问，请通过 https://github.com/ymcui/Chinese-LLaMA-Alpaca-3 提交issue + + +---- + +This repository contains **Llama-3-Chinese-8B-Instruct-v2-GGUF** (llama.cpp/ollama/tgw, etc. compatible), which is the quantized version of [Llama-3-Chinese-8B-Instruct-v2](https://huggingface.co/hfl/llama-3-chinese-8b-instruct-v2). + +**Note: this is an instruction (chat) model, which can be used for conversation, QA, etc.** + +Further details (performance, usage, etc.) should refer to GitHub project page: https://github.com/ymcui/Chinese-LLaMA-Alpaca-3 + +## Performance + +Metric: PPL, lower is better + +| Quant | Size | PPL | +| :---: | -------: | ------------------: | +| Q2_K | 2.96 GB | 13.2488 +/- 0.17217 | +| Q3_K | 3.74 GB | 6.9618 +/- 0.08420 | +| Q4_0 | 4.34 GB | 6.8925 +/- 0.08437 | +| Q4_K | 4.58 GB | 6.4851 +/- 0.07892 | +| Q5_0 | 5.21 GB | 6.4608 +/- 0.07862 | +| Q5_K | 5.34 GB | 6.3742 +/- 0.07740 | +| Q6_K | 6.14 GB | 6.3494 +/- 0.07703 | +| Q8_0 | 7.95 GB | 6.3110 +/- 0.07673 | +| F16 | 14.97 GB | 6.3005 +/- 0.07658 | + + +## Others + +- For full model, please see: https://huggingface.co/hfl/llama-3-chinese-8b-instruct-v2 + +- For LoRA-only model, please see: https://huggingface.co/hfl/llama-3-chinese-8b-instruct-v2-lora + +- If you have questions/issues regarding this model, please submit an issue through https://github.com/ymcui/Chinese-LLaMA-Alpaca-3 \ No newline at end of file diff --git a/configuration.json b/configuration.json new file mode 100644 index 0000000..b87d082 --- /dev/null +++ b/configuration.json @@ -0,0 +1 @@ +{"framework":"other","task":"fill-mask"} \ No newline at end of file diff --git a/ggml-model-q2_k.gguf b/ggml-model-q2_k.gguf new file mode 100644 index 0000000..56f5eed --- /dev/null +++ b/ggml-model-q2_k.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:02f9b38ebb1a39f8378b32b3583b862641412c30f319fb2dfe0c198b1f5717b1 +size 3179131232 diff --git a/ggml-model-q3_k.gguf b/ggml-model-q3_k.gguf new file mode 100644 index 0000000..4922f83 --- /dev/null +++ b/ggml-model-q3_k.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c43e69f22c227a758e4d99aed57336e59cba603e8739eeeccc88b704c6f862b9 +size 4018917728 diff --git a/ggml-model-q4_0.gguf b/ggml-model-q4_0.gguf new file mode 100644 index 0000000..ed010f1 --- /dev/null +++ b/ggml-model-q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:55bd47f7fcf749cfc559c13c278e89154d40b81724978a9bdbf7471004d3a561 +size 4661211488 diff --git a/ggml-model-q4_k.gguf b/ggml-model-q4_k.gguf new file mode 100644 index 0000000..74f0d3e --- /dev/null +++ b/ggml-model-q4_k.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:823ed11f3c30a79ee49e9374be0403c46907e680af6733e3802a8054aae2da66 +size 4920734048 diff --git a/ggml-model-q5_0.gguf b/ggml-model-q5_0.gguf new file mode 100644 index 0000000..2f1918b --- /dev/null +++ b/ggml-model-q5_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de52c6e71fe41a26a92486ec6b6d3261965c68399597f49ef8090c6b991e954a +size 5599293792 diff --git a/ggml-model-q5_k.gguf b/ggml-model-q5_k.gguf new file mode 100644 index 0000000..a407ecc --- /dev/null +++ b/ggml-model-q5_k.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:014fa21b97adf5ab025f9a9cb2b4ba2cbc9c59862356bb297ce325c83f55518b +size 5732987232 diff --git a/ggml-model-q6_k.gguf b/ggml-model-q6_k.gguf new file mode 100644 index 0000000..524ca67 --- /dev/null +++ b/ggml-model-q6_k.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a536ed925d6b1f423cb8b029f36d11991f49edbd51a63e22e636511809ab7ba2 +size 6596006240 diff --git a/ggml-model-q8_0.gguf b/ggml-model-q8_0.gguf new file mode 100644 index 0000000..5689087 --- /dev/null +++ b/ggml-model-q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:41ead1861f50bd60c5683a941e2528a47e76012041bcb5d0f3e9008b28e65980 +size 8540770656