commit 45813afcc4c7c607d068b2378ea72f92ad8e0bf1 Author: ModelHub XC Date: Fri Jun 19 01:27:20 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..d9fc300 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ1_M.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ1_M.gguf new file mode 100644 index 0000000..b37c8ec --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dd3723e9b60beee03efcb101e71a2260ffe82f44f8cc36e09785a856578ef6de +size 2042197408 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ1_S.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ1_S.gguf new file mode 100644 index 0000000..01cfc80 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:72cd7903299bc99b8f853e0951da8e213c19f0458fe8f690a1f1b60de0dd491d +size 1903668640 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_M.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_M.gguf new file mode 100644 index 0000000..c0084a4 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f305627a978213a59d40f919e93f415c9c071104d2e776d8c1e79d8a61ef0077 +size 2780343712 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_S.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_S.gguf new file mode 100644 index 0000000..2760f17 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8618ef2fb0e81f263eb4d01cf410eba1ec5ebb147ecef1a085c5b96b5699dc0f +size 2595638688 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_XS.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_XS.gguf new file mode 100644 index 0000000..6bba6c4 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7e01b4304ee91336214a25c6be0e94dcb88392044ff79beecd7861e2ce091ce +size 2469023136 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_XXS.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..d894f38 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:498b0f4c634dec20f95a5a159f32556361fb508ed35e1aeffa8205c187f3e007 +size 2273078688 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_M.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_M.gguf new file mode 100644 index 0000000..daac3a3 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:84b7a6c3ca8fa4c60b6251dd9c2f9db1df0fedb9c8a81de31c252c38d5a46d0d +size 3574013344 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_S.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_S.gguf new file mode 100644 index 0000000..8fed859 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37b4a71deb6c7fb7fef099b22edd75d956f3fa66b3be92ac867560241f447ec6 +size 3499193760 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_XS.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_XS.gguf new file mode 100644 index 0000000..6692638 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e2f4e3a233313b1fb4e80dd5240a146573025adb081793ccf35a0df16c4b3e71 +size 3346257312 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_XXS.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..067ea3f --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:599c10426c6dd1cc029b24e007d4ff30fbdb5625f086a5693b4a37272f65e8d0 +size 3114515872 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ4_NL.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ4_NL.gguf new file mode 100644 index 0000000..04c4e5c --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f0173af1be02a7a3c74338477e67ca0ff5f1f4ee9f768784e156d26154030737 +size 4437814688 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ4_XS.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ4_XS.gguf new file mode 100644 index 0000000..f840cfb --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce6e1190fbb9e58ffe6aabcb24865a6e911300c3f4e4cd004d3d41055cc75174 +size 4218473888 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q2_K.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q2_K.gguf new file mode 100644 index 0000000..ff6d5ae --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d8935cfab1ddc9fa73600ee156aa464f61b773e8ac5bb3f2d81e08731b668389 +size 3015941536 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q2_K_S.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q2_K_S.gguf new file mode 100644 index 0000000..bf9e2c3 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f67a040ac41650be90f2746d8bbbe299753746589f2af9ce85c51478d7a2d097 +size 2834075040 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_L.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_L.gguf new file mode 100644 index 0000000..fe407ca --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:91e8a0bd868b306790f451169af21f0d671846120e4e6daea0db2ccc3e30508d +size 4088460704 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_M.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_M.gguf new file mode 100644 index 0000000..f2e837c --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:76685c244ed86e69578c758135697d1fb619104221268ab09047e9eb2821a70f +size 3808392608 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_S.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_S.gguf new file mode 100644 index 0000000..dc74646 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7f78c2a7d9738a8a3bbc837b4e17b364afe119a6776a1ae98bdbd5ba79aec63e +size 3492369824 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_0.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_0.gguf new file mode 100644 index 0000000..20aa6cf --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d459e96b3c2c3074a585adb5cc33d8be22d8be5bf14e136ca322cfb7f734153f +size 4444122528 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_1.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_1.gguf new file mode 100644 index 0000000..666c9a9 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:442de8b47b098b5bf554f685e7562bf75a6c87224a1c404867642f0aa83b5c85 +size 4873285024 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_K_M.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_K_M.gguf new file mode 100644 index 0000000..56a7c3b --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:810851488341d54cbfe61dc6b969ba950f06099459da695947c6b933c39b2a75 +size 4683074976 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_K_S.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_K_S.gguf new file mode 100644 index 0000000..9c8c057 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:934c15976f47669e60ae245c50345848bbe20d6f3a53ad84ac47b9fcbe216e75 +size 4457770400 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q5_K_M.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q5_K_M.gguf new file mode 100644 index 0000000..14031d2 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c69e8bc91dd78f1b00db6cecd61e638b07789394fd486906e79dc62985af0a1d +size 5444832672 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q5_K_S.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q5_K_S.gguf new file mode 100644 index 0000000..90ffc96 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d163c7a9d7c6cdef75ce68e26ec3a892d05f80895dd16f511eca5ba2d0be9c6a +size 5315177888 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q6_K.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q6_K.gguf new file mode 100644 index 0000000..6e743fa --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:53807f5ae9a4113d90eb1e39b63bb098735cc2054ae440fa5c88a823fc34de9c +size 6254200224 diff --git a/Qwen2.5-Coder-7B-Instruct-heretic.imatrix.gguf b/Qwen2.5-Coder-7B-Instruct-heretic.imatrix.gguf new file mode 100644 index 0000000..7a9c242 --- /dev/null +++ b/Qwen2.5-Coder-7B-Instruct-heretic.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9e050c750c96ad21ea52fcbd99ffac554ac17b49e4479802069469ec9a1c20ae +size 4560352 diff --git a/README.md b/README.md new file mode 100644 index 0000000..ced47ee --- /dev/null +++ b/README.md @@ -0,0 +1,96 @@ +--- +base_model: trohrbaugh/Qwen2.5-Coder-7B-Instruct-heretic +language: +- en +library_name: transformers +license: apache-2.0 +license_link: https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/blob/main/LICENSE +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- code +- codeqwen +- chat +- qwen +- qwen-coder +- heretic +- uncensored +- decensored +- abliterated +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/trohrbaugh/Qwen2.5-Coder-7B-Instruct-heretic + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ1_S.gguf) | i1-IQ1_S | 2.0 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ1_M.gguf) | i1-IQ1_M | 2.1 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.6 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_S.gguf) | i1-IQ2_S | 2.7 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ2_M.gguf) | i1-IQ2_M | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q2_K_S.gguf) | i1-Q2_K_S | 2.9 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q2_K.gguf) | i1-Q2_K | 3.1 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.2 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_S.gguf) | i1-IQ3_S | 3.6 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ3_M.gguf) | i1-IQ3_M | 3.7 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_M.gguf) | i1-Q3_K_M | 3.9 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.2 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.3 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-IQ4_NL.gguf) | i1-IQ4_NL | 4.5 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_0.gguf) | i1-Q4_0 | 4.5 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.6 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_K_M.gguf) | i1-Q4_K_M | 4.8 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q4_1.gguf) | i1-Q4_1 | 5.0 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-heretic.i1-Q6_K.gguf) | i1-Q6_K | 6.4 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +