mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-10 04:55:54 +08:00
commit 3c8ed81f85
27 changed files with 240 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5499f8bb9db331a28e35a494f0dbc6aad5534ff0b2906c008ee823cdf8e8636d
size 2545954112

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c36b326a8527e77b2474c67dae88650d0c32cc8d433fa706bd319b301925d747
size 2378566976

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1cf074ddb6baf6bd661195ec59292a9fe7c7a08b27889749940835ff65919d59
size 3434671424

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:74030e0e4c3731268e0b657a68c8541062f1dbe1eebbe514265030d8bfed89af
size 3211488576

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1fb910e2ffe3a6c33a3e0b0af1afcb5be90f9646013b0287e51e6d6a5000ba2c
size 3067383104

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a0f16c75df3b531058c4401c685a9625d9e4bfb9b6eef6e0f5ff02a21f5165b9
size 2824932672

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5269fbcbf34d446bbb7724a3a8988c49c0b83b27678338874b61ea92a578e91f
size 4494617920

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:476173e22524fae6bfd68b6538c5c3e58fc7923d4a18746fc85e2fb5c78ea97b
size 4337667392

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e812dd834320fd9f9754189ec32a689d471c5ca99b1a4be776603cadec9e92ba
size 4144991552

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:417f799c140e7cc68aed771418d9709c05ce6d2e99e264a42cf826f118c8389d
size 3796741440

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f8ba1439bd1fc619acdfd4a33671dad00635d2840b2157a2d5d5b53e14ded6ec
size 5443145024

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e5a863fbc201e247c02d9d0e5d2d45b10f9e90344e96d4966caaea5510a111c1
size 5183032640

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:acffbbb60b3c5a82edb0cf14e387e9917ada2c6001a683b5a77aa0f39d67eaed
size 3805400384

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3207c2c94b3a520c9700f8177b5d3423c5229a6854589583e971086b628e41c2
size 3552513344

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fb77e4d7aeb14355802d119547cf1653f53ba8b250b2f3583a0a8c23c0e1e91d
size 5132455232

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cf291c93e72abdd07e8d1fd0d3c136f458aeabcd2597f9f25e44a1c126da5fb7
size 4761783616

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:11d1ef34a6afb2307c71cd908634075ca170e0686691f3a38bd0c31b850db1d6
size 4337667392

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5e57c6283e949d4faad7b530a5b6e335100e50864762193d5f558dad3a4ad07a
size 5459201344

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9226aa296775846583624fadcd59359a448c35deaaff2d91d72d6e9f61bfc228
size 5963369792

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3377858e244fbb3c382eb50d31ed142c11da1b5e013c153cfe3432ad3cc41480
size 5761060160

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:19ddd2c89f71689cb4489010fe87a403ebfcf7ace9f898822b07e2c69be00688
size 5478927680

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d829d81eb546506bdcbae4fd9afa93c1360815c1bd7ed7c8f4af2e38dba1e6b9
size 6647369024

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:89e46dd1ac9420d4097cb4104a61d5b27db64e4b01db138eef6231b4e09d840c
size 6483594560

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f145abbf991fff55d440cea52333f2c8253998b30e25fdfc9eef0916bfe4fb64
size 7589072192

3
Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a7f7e16387716ecdcf2e871cf9719ab03f6baebc32e318ec794cf6f96a615447
size 6152352

105
README.md Normal file
View File

@@ -0,0 +1,105 @@
---
base_model: ChiKoi7/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic
datasets:
- tokyotech-llm/lmsys-chat-1m-synth
- tokyotech-llm/swallow-magpie-ultra-v0.1
- tokyotech-llm/swallow-gemma-magpie-v0.1
- lmsys/lmsys-chat-1m
- argilla/magpie-ultra-v0.1
language:
- en
- ja
library_name: transformers
license:
- gemma
- llama3.3
mradermacher:
readme_rev: 1
quantized_by: mradermacher
tags:
- Gemma
- Llama
- Swallow
- 9b
- Instruct
- Heretic
- Japanese
- Uncensored
- Abliterated
- Safetensors
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
<!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
<!-- ### quants_skip: -->
<!-- ### skip_mmproj: -->
weighted/imatrix quants of https://huggingface.co/ChiKoi7/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic
<!-- provided-files -->
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF).***
static quants are available at https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ1_S.gguf) | i1-IQ1_S | 2.5 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ1_M.gguf) | i1-IQ1_M | 2.6 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.9 | |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ2_XS.gguf) | i1-IQ2_XS | 3.2 | |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ2_S.gguf) | i1-IQ2_S | 3.3 | |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ2_M.gguf) | i1-IQ2_M | 3.5 | |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.7 | very low quality |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.9 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q2_K.gguf) | i1-Q2_K | 3.9 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ3_XS.gguf) | i1-IQ3_XS | 4.2 | |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ3_S.gguf) | i1-IQ3_S | 4.4 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q3_K_S.gguf) | i1-Q3_K_S | 4.4 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ3_M.gguf) | i1-IQ3_M | 4.6 | |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.9 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q3_K_L.gguf) | i1-Q3_K_L | 5.2 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ4_XS.gguf) | i1-IQ4_XS | 5.3 | |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-IQ4_NL.gguf) | i1-IQ4_NL | 5.5 | prefer IQ4_XS |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q4_0.gguf) | i1-Q4_0 | 5.6 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q4_K_S.gguf) | i1-Q4_K_S | 5.6 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.9 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q4_1.gguf) | i1-Q4_1 | 6.1 | |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q5_K_S.gguf) | i1-Q5_K_S | 6.6 | |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q5_K_M.gguf) | i1-Q5_K_M | 6.7 | |
| [GGUF](https://huggingface.co/mradermacher/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic-i1-GGUF/resolve/main/Gemma-2-Llama-Swallow-9b-it-v0.1-Heretic.i1-Q6_K.gguf) | i1-Q6_K | 7.7 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->