mradermacher/gemma-2-9b-it-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/gemma-2-9b-it-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-14 16:16:18 +08:00
commit a7e1b7b5ae
24 changed files with 203 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
.gitattributes vendored Normal file
View File

@@ -0,0 +1,57 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text

80
README.md Normal file
View File

@@ -0,0 +1,80 @@
---
base_model: google/gemma-2-9b-it
extra_gated_button_content: Acknowledge license
extra_gated_heading: Access Gemma on Hugging Face
extra_gated_prompt: To access Gemma on Hugging Face, you’re required to review and
agree to Google’s usage license. To do this, please ensure you’re logged in to Hugging
Face and click below. Requests are processed immediately.
language:
- en
library_name: transformers
license: gemma
quantized_by: mradermacher
tags:
- conversational
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
weighted/imatrix quants of https://huggingface.co/google/gemma-2-9b-it
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/gemma-2-9b-it-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-IQ1_S.gguf) | i1-IQ1_S | 2.5 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-IQ1_M.gguf) | i1-IQ1_M | 2.6 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.9 | |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-IQ2_XS.gguf) | i1-IQ2_XS | 3.2 | |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-IQ2_S.gguf) | i1-IQ2_S | 3.3 | |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-IQ2_M.gguf) | i1-IQ2_M | 3.5 | |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.9 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-Q2_K.gguf) | i1-Q2_K | 3.9 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-IQ3_XS.gguf) | i1-IQ3_XS | 4.2 | |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-IQ3_S.gguf) | i1-IQ3_S | 4.4 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-Q3_K_S.gguf) | i1-Q3_K_S | 4.4 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-IQ3_M.gguf) | i1-IQ3_M | 4.6 | |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.9 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-Q3_K_L.gguf) | i1-Q3_K_L | 5.2 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-IQ4_XS.gguf) | i1-IQ4_XS | 5.3 | |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-Q4_0.gguf) | i1-Q4_0 | 5.6 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-Q4_K_S.gguf) | i1-Q4_K_S | 5.6 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.9 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-Q5_K_S.gguf) | i1-Q5_K_S | 6.6 | |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-Q5_K_M.gguf) | i1-Q5_K_M | 6.7 | |
| [GGUF](https://huggingface.co/mradermacher/gemma-2-9b-it-i1-GGUF/resolve/main/gemma-2-9b-it.i1-Q6_K.gguf) | i1-Q6_K | 7.7 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
gemma-2-9b-it.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:472b1285b0367bae88850bca2bed092f6a07549bee2c4b2f63f65323c2edf0cc
size 2545953056

3
gemma-2-9b-it.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e148951a783e1665d88b8add30fa1c53b673c6239f82e98bfbf5c4dc82a7b8df
size 2378565920

3
gemma-2-9b-it.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:155d6ba5404f4b05f298583d4c2dd0acdd2c82726c848dd93d3a88e85ace8743
size 3434670368

3
gemma-2-9b-it.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:466d801d5b71663af7aa54d348bc87ade36dbe0d5e57ea1de50312ee51e0fe49
size 3211487520

3
gemma-2-9b-it.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8ba956cca32a342a6a35e394ad647a71a6f11cf24eb43970d5456987eda7eae6
size 3067382048

3
gemma-2-9b-it.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5ceae8022dff318f6a226cf2f0dd1ffe76e4fa1b5e66210aa55ca8590ff59fea
size 2824931616

3
gemma-2-9b-it.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:015adaf66a7b0aaf649f417b8ac3a85160bb61fcb1122958d9ad35a212ca40e0
size 4494616864

3
gemma-2-9b-it.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:715683fba6bbfe08289cdaf7df8f9df4767a95bbbaf03ae3171b254d1129d16a
size 4337666336

3
gemma-2-9b-it.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:331ff555e8944a8c1cd489f64730cd8faf94cd9cc610a91b80322062cbea350b
size 4144990496

3
gemma-2-9b-it.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7898fd12a3610d0fad59e26e721b2ed38b4ddc9a6871445449e7e5875a6aeb37
size 3796740384

3
gemma-2-9b-it.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:43402f46dff03b9db2f283f52a38d986c4acd30bc9b50dd21a06f5692c0cec11
size 5183031584

3
gemma-2-9b-it.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c3f93897cda3f57a86e4b67825af8d856b77ed0fc6178c69370d6356b51e040a
size 3805399328

3
gemma-2-9b-it.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ac4309b4cbafa6245a12fbba6a0cc925ccdc8472a6c1007e6ad4e77329479616
size 5132454176

3
gemma-2-9b-it.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a02dbe889b7e2122734116074bfc3ebce9290d75f10d9c47c28e1edc30e9ee31
size 4761782560

3
gemma-2-9b-it.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:11b1f957a633ac3c9eb2139fd0f344cba66217c5b67a72d0fd508df38035613c
size 4337666336

3
gemma-2-9b-it.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e2300f461ca3b7344cc2652374da251fca95941a8490f534d7126ed1f572289e
size 5459200288

3
gemma-2-9b-it.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:acdbb7695db264a3fe45066437509c1f6ce86acae348a317aed40a8922d7a604
size 5761059104

3
gemma-2-9b-it.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d5034d149c73a0f9728d57926136b08529bdea7232faedcf69f576c13c73c7ef
size 5478926624

3
gemma-2-9b-it.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:19998b2f45caa26cc429f8394012c03b18c2a7c61ba9741b1c00cc3607839d17
size 6647367968

3
gemma-2-9b-it.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0133a0c6af164ed5dceaad87aa697509bac5eece8529bc072f2efc62e6c1bb91
size 6483593504

3
gemma-2-9b-it.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7d4b1493b27c0d7a683ae030b3a3e9e83599cd96ef866b3d0205b938de44659a
size 7589071136

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7949e076395611b02b76ebe68b3e60b6a69283ceb0d83ffdf0f4e426423d3d72
size 6116891