From e2347728253d2e14709e59e25ec051313840919a Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Mon, 17 Aug 2026 16:11:16 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-Imatrix-GGUF Source: Original Platform --- .gitattributes | 66 +++++++++++ ...inking-2507-Heretic-Q4_0-pure-imatrix.gguf | 3 + Qwen3-4B-Thinking-2507-Heretic-bf16.gguf | 3 + Qwen3-4B-Thinking-2507-Heretic-f16.gguf | 3 + ...B-Thinking-2507-Heretic-imatrix-IQ1_M.gguf | 3 + ...B-Thinking-2507-Heretic-imatrix-IQ1_S.gguf | 3 + ...B-Thinking-2507-Heretic-imatrix-IQ2_M.gguf | 3 + ...B-Thinking-2507-Heretic-imatrix-IQ2_S.gguf | 3 + ...-Thinking-2507-Heretic-imatrix-IQ2_XS.gguf | 3 + ...Thinking-2507-Heretic-imatrix-IQ2_XXS.gguf | 3 + ...B-Thinking-2507-Heretic-imatrix-IQ3_M.gguf | 3 + ...B-Thinking-2507-Heretic-imatrix-IQ3_S.gguf | 3 + ...-Thinking-2507-Heretic-imatrix-IQ3_XS.gguf | 3 + ...Thinking-2507-Heretic-imatrix-IQ3_XXS.gguf | 3 + ...-Thinking-2507-Heretic-imatrix-IQ4_NL.gguf | 3 + ...-Thinking-2507-Heretic-imatrix-IQ4_XS.gguf | 3 + ...4B-Thinking-2507-Heretic-imatrix-Q2_K.gguf | 3 + ...-Thinking-2507-Heretic-imatrix-Q3_K_L.gguf | 3 + ...-Thinking-2507-Heretic-imatrix-Q3_K_M.gguf | 3 + ...-Thinking-2507-Heretic-imatrix-Q3_K_S.gguf | 3 + ...4B-Thinking-2507-Heretic-imatrix-Q4_0.gguf | 3 + ...4B-Thinking-2507-Heretic-imatrix-Q4_1.gguf | 3 + ...4B-Thinking-2507-Heretic-imatrix-Q4_K.gguf | 3 + ...-Thinking-2507-Heretic-imatrix-Q4_K_M.gguf | 3 + ...-Thinking-2507-Heretic-imatrix-Q4_K_S.gguf | 3 + ...4B-Thinking-2507-Heretic-imatrix-Q5_0.gguf | 3 + ...4B-Thinking-2507-Heretic-imatrix-Q5_1.gguf | 3 + ...-Thinking-2507-Heretic-imatrix-Q5_K_M.gguf | 3 + ...-Thinking-2507-Heretic-imatrix-Q5_K_S.gguf | 3 + ...4B-Thinking-2507-Heretic-imatrix-Q6_K.gguf | 3 + ...4B-Thinking-2507-Heretic-imatrix-Q8_0.gguf | 3 + README.md | 111 ++++++++++++++++++ 32 files changed, 267 insertions(+) create mode 100644 .gitattributes create mode 100644 Qwen3-4B-Thinking-2507-Heretic-Q4_0-pure-imatrix.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-bf16.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-f16.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ1_M.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ1_S.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_M.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_S.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_XS.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_XXS.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_M.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_S.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_XS.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_XXS.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ4_NL.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ4_XS.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q2_K.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_L.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_M.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_S.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_0.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_1.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K_M.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K_S.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_0.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_1.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_K_M.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_K_S.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q6_K.gguf create mode 100644 Qwen3-4B-Thinking-2507-Heretic-imatrix-Q8_0.gguf create mode 100644 README.md diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..0752069 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,66 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-f16.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q2_K-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q3_K_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q3_K_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q5_K_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q5_K_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q4_0-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q4_K_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q8_0-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q6_K-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q5_0-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q4_K_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-IQ1_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-IQ1_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-IQ2_XS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-IQ2_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-IQ2_XXS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-IQ3_XS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-IQ3_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-IQ3_XXS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q3_K_L-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-IQ4_NL-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-IQ4_XS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic_q4_0_pure_imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-bf16.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-IQ2_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q5_1-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q4_K-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-IQ3_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q4_0-pure-imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q4_1-imatrix.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen3-4B-Thinking-2507-Heretic-Q4_0-pure-imatrix.gguf b/Qwen3-4B-Thinking-2507-Heretic-Q4_0-pure-imatrix.gguf new file mode 100644 index 0000000..011d905 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-Q4_0-pure-imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5846bbee56d53389235e17e50d554b5312538dc26c56ef6366776ecfa4ed5511 +size 2269269760 diff --git a/Qwen3-4B-Thinking-2507-Heretic-bf16.gguf b/Qwen3-4B-Thinking-2507-Heretic-bf16.gguf new file mode 100644 index 0000000..a103aa5 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-bf16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:505e4e2a24fc10dd06ff4478df67cd300c4196d7801958a711cb824700794525 +size 8051285504 diff --git a/Qwen3-4B-Thinking-2507-Heretic-f16.gguf b/Qwen3-4B-Thinking-2507-Heretic-f16.gguf new file mode 100644 index 0000000..01e4e3b --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:578afc3fbff3cff0d78cb8ac7f168f9942a495e4da7f36224a0fa409af25b1dc +size 8051285504 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ1_M.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ1_M.gguf new file mode 100644 index 0000000..61f3546 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89aa2fb972c8865d9e4f110402ceac45fe7766965b0d5f71866b331cf3d4eb27 +size 1127018240 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ1_S.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ1_S.gguf new file mode 100644 index 0000000..212579e --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4cec06e1416b746c74c9a634ac7fe3b08dbc34a96627ca76be360ace46380de +size 1055256320 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_M.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_M.gguf new file mode 100644 index 0000000..a8f38d3 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:11e3ef30f07511831fdfeb8a604639db72a9b06b0b091006535c0492669469f6 +size 1512984320 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_S.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_S.gguf new file mode 100644 index 0000000..0269d53 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fdf7add0ee88abf2b712d293ba700554e1e3c015ebbbeb974218450592a6986c +size 1417301760 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_XS.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_XS.gguf new file mode 100644 index 0000000..13f9dde --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:966c43c0ed9c8cd2dcb4606cbcd91934df89f801f370f642d25dac4ece58ca9d +size 1354100480 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_XXS.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_XXS.gguf new file mode 100644 index 0000000..1455dd9 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bdb459a478967d4ea9909c6742f3f8e51be9e9f039bbe7ac7313471d229e8da5 +size 1246621440 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_M.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_M.gguf new file mode 100644 index 0000000..df291e2 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1781919d103abd7c7ba752956b6ad0f3c7d9283ab9ae4065540135614cc12d9a +size 1962896640 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_S.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_S.gguf new file mode 100644 index 0000000..6f12ac7 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3e426f8642cf628ef88b26145f9c20c63d2e646124d578b22f01ff9af0b2f566 +size 1899531520 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_XS.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_XS.gguf new file mode 100644 index 0000000..cd9806d --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a5c1cc91c3a8734c0e9e6ccb276fbf1bf5a6056898a2c2acb555d231701297d +size 1814375680 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_XXS.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_XXS.gguf new file mode 100644 index 0000000..863d544 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:48e09fb770a7ddd23be81ea796b09f215a9b0729dc401b04c819ba65c602036e +size 1670188800 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ4_NL.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ4_NL.gguf new file mode 100644 index 0000000..1c824ca --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6fd4ba3b43b84486d6a462a14774e6075dfbba7d633a7554be8ae5a29d3ca298 +size 2381344000 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ4_XS.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ4_XS.gguf new file mode 100644 index 0000000..dfac3a0 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e56825a2be2d9e99197688e750119a57cceba76231626a4d8fa1863406052916 +size 2270752000 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q2_K.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q2_K.gguf new file mode 100644 index 0000000..1118524 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a9ad7f14280f0d90c41866fb7c9311f8e7d17a981ab3fbff39e9bf980ad9c833 +size 1669500160 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_L.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_L.gguf new file mode 100644 index 0000000..b100ff2 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:375aa56a18dbd263e93a70ed01059dcb216a1dcfad5eb2d69052dc1b6ee94319 +size 2239786240 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_M.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_M.gguf new file mode 100644 index 0000000..daf58bb --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:65ddf7a7c493504f68bc2a2253da63a16811310b2ee25138b9901fb52623a269 +size 2075618560 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_S.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_S.gguf new file mode 100644 index 0000000..a867cc4 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1a386d76ff8c0d1743cf1fa84fa2cc9710de316f492a87a300e9b93cbaaf6800 +size 1886997760 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_0.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_0.gguf new file mode 100644 index 0000000..a437e62 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:074b633f48c636feae9f4d0ea47cd74879f922dad67b7a5aa3a1b25b8c514ff4 +size 2375773440 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_1.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_1.gguf new file mode 100644 index 0000000..7b76589 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:46cefc7553ec5c4567adcd42639b4964b511ce4fb8192c72ca2c6b799284989b +size 2596629760 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K.gguf new file mode 100644 index 0000000..c83c331 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:877a87d131672bfadfd46f86fca8a6f0859726f72fb46841939ad2c02aca5769 +size 2497281280 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K_M.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K_M.gguf new file mode 100644 index 0000000..c83c331 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:877a87d131672bfadfd46f86fca8a6f0859726f72fb46841939ad2c02aca5769 +size 2497281280 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K_S.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K_S.gguf new file mode 100644 index 0000000..f81f388 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d9902569406020d89102ed15bcabb9367f1ad6bdd5cc2d44718e489f6a54c3f8 +size 2383310080 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_0.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_0.gguf new file mode 100644 index 0000000..eb722a3 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c36e59de9a99da4d3f74c02e3736b0c5fc85531783863316986b7a0264499756 +size 2829937920 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_1.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_1.gguf new file mode 100644 index 0000000..059ecce --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b2710dadb4d598c7c7f021a9e4924329ca5ea23ab7b3ddb6bc4cb25a481a6fe6 +size 3050794240 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_K_M.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_K_M.gguf new file mode 100644 index 0000000..851c196 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ab1245b0395c1f4b13b4f5479955b935552418c1d1d700c7da9375ca1781b71 +size 2889514240 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_K_S.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_K_S.gguf new file mode 100644 index 0000000..c651b74 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da32fc7438993895527ab03a1c23730af7035ed62c52fdb9e1915cafcbde6969 +size 2823712000 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q6_K.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q6_K.gguf new file mode 100644 index 0000000..377d809 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:28c35ecdce25709d02a54075624b5eb57cf2f8955a180a30031bcee9473a2df3 +size 3306261760 diff --git a/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q8_0.gguf b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q8_0.gguf new file mode 100644 index 0000000..c66e675 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-imatrix-Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fdf0244392d9da5c3a8803f05c5305413c4bb401fb315dff55c3165d8986b2bc +size 4280405760 diff --git a/README.md b/README.md new file mode 100644 index 0000000..5eead4b --- /dev/null +++ b/README.md @@ -0,0 +1,111 @@ +--- +license: apache-2.0 +license_link: https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507/blob/main/LICENSE +pipeline_tag: text-generation +base_model: +- becnic/Qwen3-4B-Thinking-2507-Heretic +language: +- en +- de +- fr +- it +- pt +- hi +- es +- th +--- +# Qwen3-4B-Thinking-2507-Heretic-GGUF + +## Llamacpp imatrix Quantizations of Qwen3-4B-Thinking-2507-Heretic by becnic (from original Qwen3-4B-Thinking-2507) + +Using llama.cpp release b7120 for quantization. + +Original model: https://huggingface.co/becnic/Qwen3-4B-Thinking-2507-Heretic + +Run them in [LM Studio](https://lmstudio.ai/) + +Run them directly with [llama.cpp](https://github.com/ggerganov/llama.cpp), or any other llama.cpp based project + +## Download a file (not the whole branch) from below: + +| Filename | Quant type | File Size | Split | Description | +| -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | --------- | ----- | ---------------------------------------- | +| [Qwen3-4B-Thinking-2507-Heretic-f16.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-f16.gguf) | f16 | 8.05GB | false | Full precision, highest possible quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf) | Q8_0 | 4.28GB | false | Extremely high quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf) | Q6_K | 3.31GB | false | Near-lossless high quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf) | Q5_K_S | 2.82GB | false | Premium high quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf) | Q5_K_M | 2.89GB | false | Very high quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf) | Q5_0 | 2.82GB | false | High quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf) | Q4_K_S | 2.38GB | false | Strong mid-high quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf) | Q4_K_M | 2.50GB | false | Balanced mid-high quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf) | Q4_0 | 2.37GB | false | Good balance of size and quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf) | Q3_K_S | 1.89GB | false | Higher tier Q3 | +| [Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf) | Q3_K_M | 2.08GB | false | Mid-range | +| [Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf) | Q2_K | 1.67GB | false | Smallest size, lowest quality | + + +## Downloading using huggingface-cli + +
+ Click to view download instructions + +First, make sure you have hugginface-cli installed: + +``` +pip install -U "huggingface_hub[cli]" +``` + +Then, you can target the specific file you want: + +``` +huggingface-cli download ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF --include "Qwen3-4B-Thinking-2507-Q8_0.gguf" --local-dir ./ +``` + +If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: + +``` +huggingface-cli download ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF --include "Qwen3-4B-Thinking-2507-Q8_0.gguf/*" --local-dir ./ +``` + +
+ +## Abliteration parameters + +| Parameter | Value | +| :-------- | :---: | +| **direction_index** | 19.42 | +| **attn.o_proj.max_weight** | 1.23 | +| **attn.o_proj.max_weight_position** | 22.34 | +| **attn.o_proj.min_weight** | 0.69 | +| **attn.o_proj.min_weight_distance** | 10.42 | +| **mlp.down_proj.max_weight** | 1.12 | +| **mlp.down_proj.max_weight_position** | 29.64 | +| **mlp.down_proj.min_weight** | 1.08 | +| **mlp.down_proj.min_weight_distance** | 20.24 | + +## Performance + +| Metric | This model | Original model ([Qwen/Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507)) | +| :----- | :--------: | :---------------------------: | +| **KL divergence** | 0.06 | 0 *(by definition)* | +| **Refusals** | 6/100 | 96/100 | + +## Model Overview + +**Qwen3-4B-Thinking-2507** has the following features: +- Type: Causal Language Models +- Training Stage: Pretraining & Post-training +- Number of Parameters: 4.0B +- Number of Paramaters (Non-Embedding): 3.6B +- Number of Layers: 36 +- Number of Attention Heads (GQA): 32 for Q and 8 for KV +- Context Length: **262,144 natively**. + +**NOTE: This model supports only thinking mode. Meanwhile, specifying `enable_thinking=True` is no longer required.** + +Additionally, to enforce model thinking, the default chat template automatically includes ``. Therefore, it is normal for the model's output to contain only `` without an explicit opening `` tag. + +For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our [blog](https://qwenlm.github.io/blog/qwen3/), [GitHub](https://github.com/QwenLM/Qwen3), and [Documentation](https://qwen.readthedocs.io/en/latest/). + +**Supported languages:** English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai. +