commit 0264667a517f69f8d91d64441ec49a5419615fdd Author: ModelHub XC Date: Wed May 27 10:22:16 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Qwen3-32B-heretic-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..07c50c3 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,59 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-32B-heretic.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen3-32B-heretic.i1-IQ1_M.gguf b/Qwen3-32B-heretic.i1-IQ1_M.gguf new file mode 100644 index 0000000..e247297 --- /dev/null +++ b/Qwen3-32B-heretic.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b2cb792cce37bff6d30fe179c2b6ffc1adc23f5ab08eb66a01f322c894683eab +size 7959869184 diff --git a/Qwen3-32B-heretic.i1-IQ1_S.gguf b/Qwen3-32B-heretic.i1-IQ1_S.gguf new file mode 100644 index 0000000..cacb7de --- /dev/null +++ b/Qwen3-32B-heretic.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:80155f47fe32782fbb6457d2f70675210146aee22ca5811cb8c2e626b6559b0d +size 7323842304 diff --git a/Qwen3-32B-heretic.i1-IQ2_M.gguf b/Qwen3-32B-heretic.i1-IQ2_M.gguf new file mode 100644 index 0000000..a97b4ba --- /dev/null +++ b/Qwen3-32B-heretic.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:31ea5e8083c8b7204bf8171cb2d62215b0c6f040024c380b5411fee31c418e93 +size 11362861824 diff --git a/Qwen3-32B-heretic.i1-IQ2_S.gguf b/Qwen3-32B-heretic.i1-IQ2_S.gguf new file mode 100644 index 0000000..348f278 --- /dev/null +++ b/Qwen3-32B-heretic.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a786b420a33863958da80a7b8ad034e01be015f9e88de95758ff734d21caa4c +size 10514825984 diff --git a/Qwen3-32B-heretic.i1-IQ2_XS.gguf b/Qwen3-32B-heretic.i1-IQ2_XS.gguf new file mode 100644 index 0000000..a1a6a9e --- /dev/null +++ b/Qwen3-32B-heretic.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:95917e4e6b462e0e7865246b87505c8c8404efb26e497759c2b225324e5acbff +size 9951835904 diff --git a/Qwen3-32B-heretic.i1-IQ2_XXS.gguf b/Qwen3-32B-heretic.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..d84676f --- /dev/null +++ b/Qwen3-32B-heretic.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d10e200689882d3880c7fd3fcc25f8db675998ea8ac7ff58030e5b4228ee10a6 +size 9019913984 diff --git a/Qwen3-32B-heretic.i1-IQ3_M.gguf b/Qwen3-32B-heretic.i1-IQ3_M.gguf new file mode 100644 index 0000000..bd6a045 --- /dev/null +++ b/Qwen3-32B-heretic.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7397ac7cccba28cdebbc9899cd4e3d25e2baa441f791918d5c58bf85ce921c6e +size 14930083584 diff --git a/Qwen3-32B-heretic.i1-IQ3_S.gguf b/Qwen3-32B-heretic.i1-IQ3_S.gguf new file mode 100644 index 0000000..4459d77 --- /dev/null +++ b/Qwen3-32B-heretic.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1742b77283651c9c8d690537b8762baff78fb1b31e92322bf8f2d69f4be5243e +size 14434303744 diff --git a/Qwen3-32B-heretic.i1-IQ3_XS.gguf b/Qwen3-32B-heretic.i1-IQ3_XS.gguf new file mode 100644 index 0000000..12e4458 --- /dev/null +++ b/Qwen3-32B-heretic.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae32ee67e55cd0f5cdc79dca716ec801b3b9fca465781b92430308e13bfa4bd8 +size 13702921984 diff --git a/Qwen3-32B-heretic.i1-IQ3_XXS.gguf b/Qwen3-32B-heretic.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..3fc4c5d --- /dev/null +++ b/Qwen3-32B-heretic.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2422ae07b27f6efa4dd98a9e272a2b149fd276c428a3c1b84fa0f144586eee88 +size 12821037824 diff --git a/Qwen3-32B-heretic.i1-IQ4_XS.gguf b/Qwen3-32B-heretic.i1-IQ4_XS.gguf new file mode 100644 index 0000000..c190cc7 --- /dev/null +++ b/Qwen3-32B-heretic.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1baa70491b323353e4da49b93cfed1f70ebdf5c407da8f470c99ac22c9b8bb64 +size 17690495744 diff --git a/Qwen3-32B-heretic.i1-Q2_K.gguf b/Qwen3-32B-heretic.i1-Q2_K.gguf new file mode 100644 index 0000000..e78e5a0 --- /dev/null +++ b/Qwen3-32B-heretic.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7cf8e31a29eceb9355faf52929f52d6b22ae655561e1df5d8eab8792ca11f777 +size 12344652544 diff --git a/Qwen3-32B-heretic.i1-Q2_K_S.gguf b/Qwen3-32B-heretic.i1-Q2_K_S.gguf new file mode 100644 index 0000000..9b3d264 --- /dev/null +++ b/Qwen3-32B-heretic.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:af352a7d3f101e2f7b46db773048467dc12b12276db57045fff272e262292426 +size 11465814784 diff --git a/Qwen3-32B-heretic.i1-Q3_K_L.gguf b/Qwen3-32B-heretic.i1-Q3_K_L.gguf new file mode 100644 index 0000000..f034bef --- /dev/null +++ b/Qwen3-32B-heretic.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7966f588d6d286dd3278b82fbd21a20071c07d4b814a998853550e9700be5450 +size 17330994944 diff --git a/Qwen3-32B-heretic.i1-Q3_K_M.gguf b/Qwen3-32B-heretic.i1-Q3_K_M.gguf new file mode 100644 index 0000000..499cf1f --- /dev/null +++ b/Qwen3-32B-heretic.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7bde32fd4a625da3bd80264266b5846af2418b72a5346a0ff9f99f2addd36eeb +size 15971778304 diff --git a/Qwen3-32B-heretic.i1-Q3_K_S.gguf b/Qwen3-32B-heretic.i1-Q3_K_S.gguf new file mode 100644 index 0000000..f8f71c7 --- /dev/null +++ b/Qwen3-32B-heretic.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4920cbad20213905f4790fe0a5f8a9f563f9de1e5a0142295d9036aa6d12fc54 +size 14389739264 diff --git a/Qwen3-32B-heretic.i1-Q4_0.gguf b/Qwen3-32B-heretic.i1-Q4_0.gguf new file mode 100644 index 0000000..46b5f96 --- /dev/null +++ b/Qwen3-32B-heretic.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae1af848dce639632617b4a56d3f540473a66fdab0af3b898017be138ad82150 +size 18703088384 diff --git a/Qwen3-32B-heretic.i1-Q4_1.gguf b/Qwen3-32B-heretic.i1-Q4_1.gguf new file mode 100644 index 0000000..d1fc362 --- /dev/null +++ b/Qwen3-32B-heretic.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d174c93ca1383468193f2d46f7f6c1317286a1d6647228d6c32064e89d95e7d9 +size 20636523264 diff --git a/Qwen3-32B-heretic.i1-Q4_K_M.gguf b/Qwen3-32B-heretic.i1-Q4_K_M.gguf new file mode 100644 index 0000000..a46ccdb --- /dev/null +++ b/Qwen3-32B-heretic.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9107562dc5a470679432a4acddabf097ee7626e14607f832672e0591384bbe91 +size 19762150144 diff --git a/Qwen3-32B-heretic.i1-Q4_K_S.gguf b/Qwen3-32B-heretic.i1-Q4_K_S.gguf new file mode 100644 index 0000000..e1e4233 --- /dev/null +++ b/Qwen3-32B-heretic.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1812aa439cf5e7eaa3e751a159ea94e905d66b0171ec062ddb72ba45b8b82425 +size 18771245824 diff --git a/Qwen3-32B-heretic.i1-Q5_K_M.gguf b/Qwen3-32B-heretic.i1-Q5_K_M.gguf new file mode 100644 index 0000000..a98b304 --- /dev/null +++ b/Qwen3-32B-heretic.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:40cfe4a0c5baca7860c083fd76d890955855a7051c9a26c81eec7a6b9ffba2af +size 23214832384 diff --git a/Qwen3-32B-heretic.i1-Q5_K_S.gguf b/Qwen3-32B-heretic.i1-Q5_K_S.gguf new file mode 100644 index 0000000..2faa4a1 --- /dev/null +++ b/Qwen3-32B-heretic.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c95b4780b75061c5ed98977d7e8fb0b8a16fe7e777993901f12409fc3f281c45 +size 22635494144 diff --git a/Qwen3-32B-heretic.i1-Q6_K.gguf b/Qwen3-32B-heretic.i1-Q6_K.gguf new file mode 100644 index 0000000..443cc07 --- /dev/null +++ b/Qwen3-32B-heretic.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:066cfd9fc8de00ec86dc80d5240a98291a4702314e0b41349426ecd23958c697 +size 26883307264 diff --git a/Qwen3-32B-heretic.imatrix.gguf b/Qwen3-32B-heretic.imatrix.gguf new file mode 100644 index 0000000..1783356 --- /dev/null +++ b/Qwen3-32B-heretic.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e1a50c041ca76bb9d5921022370500ead7c37f86f03aba01b0b2ee310b898232 +size 15273216 diff --git a/README.md b/README.md new file mode 100644 index 0000000..efb4d3c --- /dev/null +++ b/README.md @@ -0,0 +1,90 @@ +--- +base_model: igriv/Qwen3-32B-heretic +language: +- en +library_name: transformers +license: apache-2.0 +license_link: https://huggingface.co/Qwen/Qwen3-32B/blob/main/LICENSE +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- heretic +- uncensored +- decensored +- abliterated +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/igriv/Qwen3-32B-heretic + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Qwen3-32B-heretic-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Qwen3-32B-heretic-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-IQ1_S.gguf) | i1-IQ1_S | 7.4 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-IQ1_M.gguf) | i1-IQ1_M | 8.1 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 9.1 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-IQ2_XS.gguf) | i1-IQ2_XS | 10.1 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-IQ2_S.gguf) | i1-IQ2_S | 10.6 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-IQ2_M.gguf) | i1-IQ2_M | 11.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-Q2_K_S.gguf) | i1-Q2_K_S | 11.6 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-Q2_K.gguf) | i1-Q2_K | 12.4 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 12.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-IQ3_XS.gguf) | i1-IQ3_XS | 13.8 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-Q3_K_S.gguf) | i1-Q3_K_S | 14.5 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-IQ3_S.gguf) | i1-IQ3_S | 14.5 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-IQ3_M.gguf) | i1-IQ3_M | 15.0 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-Q3_K_M.gguf) | i1-Q3_K_M | 16.1 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-Q3_K_L.gguf) | i1-Q3_K_L | 17.4 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-IQ4_XS.gguf) | i1-IQ4_XS | 17.8 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-Q4_0.gguf) | i1-Q4_0 | 18.8 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-Q4_K_S.gguf) | i1-Q4_K_S | 18.9 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-Q4_K_M.gguf) | i1-Q4_K_M | 19.9 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-Q4_1.gguf) | i1-Q4_1 | 20.7 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-Q5_K_S.gguf) | i1-Q5_K_S | 22.7 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-Q5_K_M.gguf) | i1-Q5_K_M | 23.3 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-32B-heretic-i1-GGUF/resolve/main/Qwen3-32B-heretic.i1-Q6_K.gguf) | i1-Q6_K | 27.0 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +