初始化项目，由ModelHub XC社区提供模型

Model: fedric95/gemma-2-9b-GGUF
Source: Original Platform

.gitattributes

@@ -0,0 +1,46 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+gemma-2-9b.FP32.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-2-9b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-2-9b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-2-9b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-2-9b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-2-9b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-2-9b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-2-9b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-2-9b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-2-9b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-2-9b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,91 @@
+---
+base_model: google/gemma-2-9b
+library_name: transformers
+license: gemma
+pipeline_tag: text-generation
+tags:
+- conversational
+quantized_by: fedric95
+extra_gated_heading: Access Gemma on Hugging Face
+extra_gated_prompt: To access Gemma on Hugging Face, you’re required to review and
+  agree to Google’s usage license. To do this, please ensure you’re logged in to Hugging
+  Face and click below. Requests are processed immediately.
+extra_gated_button_content: Acknowledge license
+---
+
+## Llamacpp Quantizations of Meta-Llama-3.1-8B
+
+Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3583">b3583</a> for quantization.
+
+Original model: https://huggingface.co/google/gemma-2-9b
+
+## Download a file (not the whole branch) from below:
+
+| Filename | Quant type | File Size | Perplexity (wikitext-2-raw-v1.test) |
+| -------- | ---------- | --------- | ----------- |
+| [gemma-2-9b.FP32.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b.FP32.gguf) | FP32 | 37.00GB | 6.9209 +/- 0.04660 |
+| [gemma-2-9b-Q8_0.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q8_0.gguf) | Q8_0 | 9.83GB | 6.9222 +/- 0.04660 |
+| [gemma-2-9b-Q6_K.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q6_K.gguf) | Q6_K | 7.59GB | 6.9353 +/- 0.04675 |
+| [gemma-2-9b-Q5_K_M.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q5_K_M.gguf) | Q5_K_M | 6.65GB | 6.9571 +/- 0.04687 |
+| [gemma-2-9b-Q5_K_S.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q5_K_S.gguf) | Q5_K_S | 6.48GB | 6.9623 +/- 0.04690 |
+| [gemma-2-9b-Q4_K_M.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q4_K_M.gguf) | Q4_K_M | 5.76GB | 7.0220 +/- 0.04737 |
+| [gemma-2-9b-Q4_K_S.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q4_K_S.gguf) | Q4_K_S | 5.48GB | 7.0622 +/- 0.04777 |
+| [gemma-2-9b-Q3_K_L.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q3_K_L.gguf) | Q3_K_L | 5.13GB | 7.2144 +/- 0.04910 |
+| [gemma-2-9b-Q3_K_M.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q3_K_M.gguf) | Q3_K_M | 4.76GB | 7.2849 +/- 0.04970 |
+| [gemma-2-9b-Q3_K_S.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q3_K_S.gguf) | Q3_K_S | 4.34GB | 7.6869 +/- 0.05373 |
+| [gemma-2-9b-Q2_K.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q2_K.gguf) | Q2_K | 3.81GB | 8.7979 +/- 0.06191 |
+
+## Benchmark Results 
+
+Results have been computed using:
+
+[hellaswage_val_full](https://raw.githubusercontent.com/klosax/hellaswag_text_data/main/hellaswag_val_full.txt)
+
+[winogrande-debiased-eval](https://huggingface.co/datasets/ikawrakow/winogrande-eval-for-llama.cpp/raw/main/winogrande-debiased-eval.csv)
+
+[mmlu-validation](https://huggingface.co/datasets/ikawrakow/validation-datasets-for-llama.cpp/resolve/main/mmlu-validation.bin)
+
+| Benchmark | Quant type | Metric |
+| -------- | ---------- | --------- |
+| WinoGrande (0-shot) | Q8_0 | 74.4278 +/- 1.2261 |
+| WinoGrande (0-shot) | Q4_K_M | 74.8224 +/- 1.2198 |
+| WinoGrande (0-shot) | Q3_K_M | 74.1910 +/- 1.2298 |
+| WinoGrande (0-shot) | Q3_K_S | 72.6125 +/- 1.2533 |
+| WinoGrande (0-shot) | Q2_K | 71.4286 +/- 1.2697 |
+| HellaSwag (0-shot) | Q8_0 | 78.39075881 |
+| HellaSwag (0-shot) | Q4_K_M | 77.87293368 |
+| HellaSwag (0-shot) | Q3_K_M | 76.64807807 |
+| HellaSwag (0-shot) | Q3_K_S | 76.08046206 |
+| HellaSwag (0-shot) | Q2_K | 73.07309301 |
+| MMLU (0-shot) | Q8_0 | 42.5065 +/- 1.2569 |
+| MMLU (0-shot) | Q4_K_M | 42.5065 +/- 1.2569 |
+| MMLU (0-shot) | Q3_K_M | 41.3437 +/- 1.2520 |
+| MMLU (0-shot) | Q3_K_S | 40.5685 +/- 1.2484 |
+| MMLU (0-shot) | Q2_K | 38.1137 +/- 1.2348 |
+
+## Downloading using huggingface-cli
+
+First, make sure you have hugginface-cli installed:
+
+```
+pip install -U "huggingface_hub[cli]"
+```
+
+Then, you can target the specific file you want:
+
+```
+huggingface-cli download fedric95/gemma-2-9b-GGUF --include "gemma-2-9b-Q4_K_M.gguf" --local-dir ./
+```
+
+If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
+
+```
+huggingface-cli download fedric95/gemma-2-9b-GGUF --include "gemma-2-9b-Q8_0.gguf/*" --local-dir gemma-2-9b-Q8_0
+```
+
+You can either specify a new local-dir (gemma-2-9b-Q8_0) or download them all in place (./)
+
+
+## Reproducibility
+
+https://github.com/ggerganov/llama.cpp/discussions/9020#discussioncomment-10335638

gemma-2-9b-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e1a13e9d0c38c9cde607fc4521dff99200993388dadb117c47c8f867c4aa5df
+size 3805397248

gemma-2-9b-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbccc68594c41445f698d61f2ea7d9d5d2704a5c5aca2ca5875da7278e6e0e31
+size 5132452096

gemma-2-9b-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f402dc68e096b95a675c5cafbd51bcb8ad38cc3036050577ae968a324b171c9
+size 4761780480

gemma-2-9b-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81a9ded182ad56ae5349df7ee1e3829aac3cad0c0c2203c16abfe80258f52118
+size 4337664256

gemma-2-9b-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5eead6bb1817a6bad0a2634bfb4c4fded5a4b0f7a5952ce6b83134b2fbcd761c
+size 5761057024

gemma-2-9b-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f342fa3bb100e370de6d6dfcfdf7cc306cbfa5c6cdfb76b9ec1dc75e9e6c315
+size 5478924544

gemma-2-9b-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76550eb54adefb2efb6e4886aaa6a6cb881e15640cfc06d0c2af16d5d2ac5fef
+size 6647365888

gemma-2-9b-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:370a793626b4114aba88d19f3cfa4d5a820abd6826a01fd2d47450163ec1a181
+size 6483591424

gemma-2-9b-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b8bfa779c132ca7b5842d226a2e31c591bb51cd8368d7a53a3750ba75c66605
+size 7589069056

gemma-2-9b-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:619ffa1c11b1a3b54299f326c1b0a300a9acf7fdfb698d643172cefb211b41d0
+size 9827148032

gemma-2-9b.FP32.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4e6340b7ec393d777b2484ebf8d8d7054921a00ab9e352c0c1f7c03ff719972
+size 36972880128