fedric95/gemma-2-9b-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: fedric95/gemma-2-9b-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-22 13:21:34 +08:00
commit 2e06fcb8e4
25 changed files with 34395 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

46
.gitattributes vendored Normal file
View File

@@ -0,0 +1,46 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
gemma-2-9b.FP32.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text

91
README.md Normal file
View File

@@ -0,0 +1,91 @@
---
base_model: google/gemma-2-9b
library_name: transformers
license: gemma
pipeline_tag: text-generation
tags:
- conversational
quantized_by: fedric95
extra_gated_heading: Access Gemma on Hugging Face
extra_gated_prompt: To access Gemma on Hugging Face, you’re required to review and
agree to Google’s usage license. To do this, please ensure you’re logged in to Hugging
Face and click below. Requests are processed immediately.
extra_gated_button_content: Acknowledge license
---
## Llamacpp Quantizations of Meta-Llama-3.1-8B
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3583">b3583</a> for quantization.
Original model: https://huggingface.co/google/gemma-2-9b
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Perplexity (wikitext-2-raw-v1.test) |
| -------- | ---------- | --------- | ----------- |
| [gemma-2-9b.FP32.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b.FP32.gguf) | FP32 | 37.00GB | 6.9209 +/- 0.04660 |
| [gemma-2-9b-Q8_0.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q8_0.gguf) | Q8_0 | 9.83GB | 6.9222 +/- 0.04660 |
| [gemma-2-9b-Q6_K.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q6_K.gguf) | Q6_K | 7.59GB | 6.9353 +/- 0.04675 |
| [gemma-2-9b-Q5_K_M.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q5_K_M.gguf) | Q5_K_M | 6.65GB | 6.9571 +/- 0.04687 |
| [gemma-2-9b-Q5_K_S.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q5_K_S.gguf) | Q5_K_S | 6.48GB | 6.9623 +/- 0.04690 |
| [gemma-2-9b-Q4_K_M.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q4_K_M.gguf) | Q4_K_M | 5.76GB | 7.0220 +/- 0.04737 |
| [gemma-2-9b-Q4_K_S.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q4_K_S.gguf) | Q4_K_S | 5.48GB | 7.0622 +/- 0.04777 |
| [gemma-2-9b-Q3_K_L.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q3_K_L.gguf) | Q3_K_L | 5.13GB | 7.2144 +/- 0.04910 |
| [gemma-2-9b-Q3_K_M.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q3_K_M.gguf) | Q3_K_M | 4.76GB | 7.2849 +/- 0.04970 |
| [gemma-2-9b-Q3_K_S.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q3_K_S.gguf) | Q3_K_S | 4.34GB | 7.6869 +/- 0.05373 |
| [gemma-2-9b-Q2_K.gguf](https://huggingface.co/fedric95/gemma-2-9b-GGUF/blob/main/gemma-2-9b-Q2_K.gguf) | Q2_K | 3.81GB | 8.7979 +/- 0.06191 |
## Benchmark Results
Results have been computed using:
[hellaswage_val_full](https://raw.githubusercontent.com/klosax/hellaswag_text_data/main/hellaswag_val_full.txt)
[winogrande-debiased-eval](https://huggingface.co/datasets/ikawrakow/winogrande-eval-for-llama.cpp/raw/main/winogrande-debiased-eval.csv)
[mmlu-validation](https://huggingface.co/datasets/ikawrakow/validation-datasets-for-llama.cpp/resolve/main/mmlu-validation.bin)
| Benchmark | Quant type | Metric |
| -------- | ---------- | --------- |
| WinoGrande (0-shot) | Q8_0 | 74.4278 +/- 1.2261 |
| WinoGrande (0-shot) | Q4_K_M | 74.8224 +/- 1.2198 |
| WinoGrande (0-shot) | Q3_K_M | 74.1910 +/- 1.2298 |
| WinoGrande (0-shot) | Q3_K_S | 72.6125 +/- 1.2533 |
| WinoGrande (0-shot) | Q2_K | 71.4286 +/- 1.2697 |
| HellaSwag (0-shot) | Q8_0 | 78.39075881 |
| HellaSwag (0-shot) | Q4_K_M | 77.87293368 |
| HellaSwag (0-shot) | Q3_K_M | 76.64807807 |
| HellaSwag (0-shot) | Q3_K_S | 76.08046206 |
| HellaSwag (0-shot) | Q2_K | 73.07309301 |
| MMLU (0-shot) | Q8_0 | 42.5065 +/- 1.2569 |
| MMLU (0-shot) | Q4_K_M | 42.5065 +/- 1.2569 |
| MMLU (0-shot) | Q3_K_M | 41.3437 +/- 1.2520 |
| MMLU (0-shot) | Q3_K_S | 40.5685 +/- 1.2484 |
| MMLU (0-shot) | Q2_K | 38.1137 +/- 1.2348 |
## Downloading using huggingface-cli
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download fedric95/gemma-2-9b-GGUF --include "gemma-2-9b-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download fedric95/gemma-2-9b-GGUF --include "gemma-2-9b-Q8_0.gguf/*" --local-dir gemma-2-9b-Q8_0
```
You can either specify a new local-dir (gemma-2-9b-Q8_0) or download them all in place (./)
## Reproducibility
https://github.com/ggerganov/llama.cpp/discussions/9020#discussioncomment-10335638

3
gemma-2-9b-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6e1a13e9d0c38c9cde607fc4521dff99200993388dadb117c47c8f867c4aa5df
size 3805397248

10045
gemma-2-9b-Q2_K.hellaswag.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1558
gemma-2-9b-Q2_K.mmlu.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1269
gemma-2-9b-Q2_K.winogrande.txt Normal file
View File

File diff suppressed because it is too large Load Diff

3
gemma-2-9b-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fbccc68594c41445f698d61f2ea7d9d5d2704a5c5aca2ca5875da7278e6e0e31
size 5132452096

3
gemma-2-9b-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6f402dc68e096b95a675c5cafbd51bcb8ad38cc3036050577ae968a324b171c9
size 4761780480

1558
gemma-2-9b-Q3_K_M.mmlu.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1269
gemma-2-9b-Q3_K_M.winogrande.txt Normal file
View File

File diff suppressed because it is too large Load Diff

3
gemma-2-9b-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:81a9ded182ad56ae5349df7ee1e3829aac3cad0c0c2203c16abfe80258f52118
size 4337664256

10045
gemma-2-9b-Q3_K_S.hellaswag.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1558
gemma-2-9b-Q3_K_S.mmlu.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1269
gemma-2-9b-Q3_K_S.winogrande.txt Normal file
View File

File diff suppressed because it is too large Load Diff

3
gemma-2-9b-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5eead6bb1817a6bad0a2634bfb4c4fded5a4b0f7a5952ce6b83134b2fbcd761c
size 5761057024

1558
gemma-2-9b-Q4_K_M.mmlu.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1269
gemma-2-9b-Q4_K_M.winogrande.txt Normal file
View File

File diff suppressed because it is too large Load Diff

3
gemma-2-9b-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0f342fa3bb100e370de6d6dfcfdf7cc306cbfa5c6cdfb76b9ec1dc75e9e6c315
size 5478924544

3
gemma-2-9b-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:76550eb54adefb2efb6e4886aaa6a6cb881e15640cfc06d0c2af16d5d2ac5fef
size 6647365888

3
gemma-2-9b-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:370a793626b4114aba88d19f3cfa4d5a820abd6826a01fd2d47450163ec1a181
size 6483591424

3
gemma-2-9b-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0b8bfa779c132ca7b5842d226a2e31c591bb51cd8368d7a53a3750ba75c66605
size 7589069056

3
gemma-2-9b-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:619ffa1c11b1a3b54299f326c1b0a300a9acf7fdfb698d643172cefb211b41d0
size 9827148032

1558
gemma-2-9b-Q8_0.mmlu.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1269
gemma-2-9b-Q8_0.winogrande.txt Normal file
View File

File diff suppressed because it is too large Load Diff

3
gemma-2-9b.FP32.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f4e6340b7ec393d777b2484ebf8d8d7054921a00ab9e352c0c1f7c03ff719972
size 36972880128