初始化项目,由ModelHub XC社区提供模型
Model: ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-Imatrix-GGUF Source: Original Platform
This commit is contained in:
66
.gitattributes
vendored
Normal file
66
.gitattributes
vendored
Normal file
@@ -0,0 +1,66 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-f16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q2_K-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q3_K_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q3_K_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q5_K_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q5_K_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q4_0-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q4_K_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q8_0-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q6_K-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q5_0-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q4_K_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-IQ1_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-IQ1_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-IQ2_XS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-IQ2_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-IQ2_XXS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-IQ3_XS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-IQ3_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-IQ3_XXS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q3_K_L-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-IQ4_NL-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-IQ4_XS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic_q4_0_pure_imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-bf16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-IQ2_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q5_1-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q4_K-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-IQ3_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q4_0-pure-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-4B-Thinking-2507-Heretic-Q4_1-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-Q4_0-pure-imatrix.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-Q4_0-pure-imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5846bbee56d53389235e17e50d554b5312538dc26c56ef6366776ecfa4ed5511
|
||||
size 2269269760
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-bf16.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-bf16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:505e4e2a24fc10dd06ff4478df67cd300c4196d7801958a711cb824700794525
|
||||
size 8051285504
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-f16.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-f16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:578afc3fbff3cff0d78cb8ac7f168f9942a495e4da7f36224a0fa409af25b1dc
|
||||
size 8051285504
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ1_M.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ1_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:89aa2fb972c8865d9e4f110402ceac45fe7766965b0d5f71866b331cf3d4eb27
|
||||
size 1127018240
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ1_S.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ1_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f4cec06e1416b746c74c9a634ac7fe3b08dbc34a96627ca76be360ace46380de
|
||||
size 1055256320
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_M.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:11e3ef30f07511831fdfeb8a604639db72a9b06b0b091006535c0492669469f6
|
||||
size 1512984320
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_S.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:fdf7add0ee88abf2b712d293ba700554e1e3c015ebbbeb974218450592a6986c
|
||||
size 1417301760
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_XS.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:966c43c0ed9c8cd2dcb4606cbcd91934df89f801f370f642d25dac4ece58ca9d
|
||||
size 1354100480
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_XXS.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ2_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:bdb459a478967d4ea9909c6742f3f8e51be9e9f039bbe7ac7313471d229e8da5
|
||||
size 1246621440
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_M.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1781919d103abd7c7ba752956b6ad0f3c7d9283ab9ae4065540135614cc12d9a
|
||||
size 1962896640
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_S.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3e426f8642cf628ef88b26145f9c20c63d2e646124d578b22f01ff9af0b2f566
|
||||
size 1899531520
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_XS.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9a5c1cc91c3a8734c0e9e6ccb276fbf1bf5a6056898a2c2acb555d231701297d
|
||||
size 1814375680
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_XXS.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ3_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:48e09fb770a7ddd23be81ea796b09f215a9b0729dc401b04c819ba65c602036e
|
||||
size 1670188800
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ4_NL.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ4_NL.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6fd4ba3b43b84486d6a462a14774e6075dfbba7d633a7554be8ae5a29d3ca298
|
||||
size 2381344000
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ4_XS.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-IQ4_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e56825a2be2d9e99197688e750119a57cceba76231626a4d8fa1863406052916
|
||||
size 2270752000
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q2_K.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a9ad7f14280f0d90c41866fb7c9311f8e7d17a981ab3fbff39e9bf980ad9c833
|
||||
size 1669500160
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_L.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_L.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:375aa56a18dbd263e93a70ed01059dcb216a1dcfad5eb2d69052dc1b6ee94319
|
||||
size 2239786240
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_M.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:65ddf7a7c493504f68bc2a2253da63a16811310b2ee25138b9901fb52623a269
|
||||
size 2075618560
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_S.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q3_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1a386d76ff8c0d1743cf1fa84fa2cc9710de316f492a87a300e9b93cbaaf6800
|
||||
size 1886997760
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_0.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:074b633f48c636feae9f4d0ea47cd74879f922dad67b7a5aa3a1b25b8c514ff4
|
||||
size 2375773440
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_1.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:46cefc7553ec5c4567adcd42639b4964b511ce4fb8192c72ca2c6b799284989b
|
||||
size 2596629760
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:877a87d131672bfadfd46f86fca8a6f0859726f72fb46841939ad2c02aca5769
|
||||
size 2497281280
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K_M.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:877a87d131672bfadfd46f86fca8a6f0859726f72fb46841939ad2c02aca5769
|
||||
size 2497281280
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K_S.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q4_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d9902569406020d89102ed15bcabb9367f1ad6bdd5cc2d44718e489f6a54c3f8
|
||||
size 2383310080
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_0.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c36e59de9a99da4d3f74c02e3736b0c5fc85531783863316986b7a0264499756
|
||||
size 2829937920
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_1.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b2710dadb4d598c7c7f021a9e4924329ca5ea23ab7b3ddb6bc4cb25a481a6fe6
|
||||
size 3050794240
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_K_M.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:2ab1245b0395c1f4b13b4f5479955b935552418c1d1d700c7da9375ca1781b71
|
||||
size 2889514240
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_K_S.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q5_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:da32fc7438993895527ab03a1c23730af7035ed62c52fdb9e1915cafcbde6969
|
||||
size 2823712000
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q6_K.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:28c35ecdce25709d02a54075624b5eb57cf2f8955a180a30031bcee9473a2df3
|
||||
size 3306261760
|
||||
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q8_0.gguf
Normal file
3
Qwen3-4B-Thinking-2507-Heretic-imatrix-Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:fdf0244392d9da5c3a8803f05c5305413c4bb401fb315dff55c3165d8986b2bc
|
||||
size 4280405760
|
||||
111
README.md
Normal file
111
README.md
Normal file
@@ -0,0 +1,111 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
license_link: https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507/blob/main/LICENSE
|
||||
pipeline_tag: text-generation
|
||||
base_model:
|
||||
- becnic/Qwen3-4B-Thinking-2507-Heretic
|
||||
language:
|
||||
- en
|
||||
- de
|
||||
- fr
|
||||
- it
|
||||
- pt
|
||||
- hi
|
||||
- es
|
||||
- th
|
||||
---
|
||||
# Qwen3-4B-Thinking-2507-Heretic-GGUF
|
||||
|
||||
## Llamacpp imatrix Quantizations of Qwen3-4B-Thinking-2507-Heretic by becnic (from original Qwen3-4B-Thinking-2507)
|
||||
|
||||
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b7120">b7120</a> for quantization.
|
||||
|
||||
Original model: https://huggingface.co/becnic/Qwen3-4B-Thinking-2507-Heretic
|
||||
|
||||
Run them in [LM Studio](https://lmstudio.ai/)
|
||||
|
||||
Run them directly with [llama.cpp](https://github.com/ggerganov/llama.cpp), or any other llama.cpp based project
|
||||
|
||||
## Download a file (not the whole branch) from below:
|
||||
|
||||
| Filename | Quant type | File Size | Split | Description |
|
||||
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | --------- | ----- | ---------------------------------------- |
|
||||
| [Qwen3-4B-Thinking-2507-Heretic-f16.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-f16.gguf) | f16 | 8.05GB | false | Full precision, highest possible quality |
|
||||
| [Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf) | Q8_0 | 4.28GB | false | Extremely high quality |
|
||||
| [Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf) | Q6_K | 3.31GB | false | Near-lossless high quality |
|
||||
| [Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf) | Q5_K_S | 2.82GB | false | Premium high quality |
|
||||
| [Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf) | Q5_K_M | 2.89GB | false | Very high quality |
|
||||
| [Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf) | Q5_0 | 2.82GB | false | High quality |
|
||||
| [Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf) | Q4_K_S | 2.38GB | false | Strong mid-high quality |
|
||||
| [Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf) | Q4_K_M | 2.50GB | false | Balanced mid-high quality |
|
||||
| [Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf) | Q4_0 | 2.37GB | false | Good balance of size and quality |
|
||||
| [Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf) | Q3_K_S | 1.89GB | false | Higher tier Q3 |
|
||||
| [Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf) | Q3_K_M | 2.08GB | false | Mid-range |
|
||||
| [Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf) | Q2_K | 1.67GB | false | Smallest size, lowest quality |
|
||||
|
||||
|
||||
## Downloading using huggingface-cli
|
||||
|
||||
<details>
|
||||
<summary>Click to view download instructions</summary>
|
||||
|
||||
First, make sure you have hugginface-cli installed:
|
||||
|
||||
```
|
||||
pip install -U "huggingface_hub[cli]"
|
||||
```
|
||||
|
||||
Then, you can target the specific file you want:
|
||||
|
||||
```
|
||||
huggingface-cli download ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF --include "Qwen3-4B-Thinking-2507-Q8_0.gguf" --local-dir ./
|
||||
```
|
||||
|
||||
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
|
||||
|
||||
```
|
||||
huggingface-cli download ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF --include "Qwen3-4B-Thinking-2507-Q8_0.gguf/*" --local-dir ./
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
## Abliteration parameters
|
||||
|
||||
| Parameter | Value |
|
||||
| :-------- | :---: |
|
||||
| **direction_index** | 19.42 |
|
||||
| **attn.o_proj.max_weight** | 1.23 |
|
||||
| **attn.o_proj.max_weight_position** | 22.34 |
|
||||
| **attn.o_proj.min_weight** | 0.69 |
|
||||
| **attn.o_proj.min_weight_distance** | 10.42 |
|
||||
| **mlp.down_proj.max_weight** | 1.12 |
|
||||
| **mlp.down_proj.max_weight_position** | 29.64 |
|
||||
| **mlp.down_proj.min_weight** | 1.08 |
|
||||
| **mlp.down_proj.min_weight_distance** | 20.24 |
|
||||
|
||||
## Performance
|
||||
|
||||
| Metric | This model | Original model ([Qwen/Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507)) |
|
||||
| :----- | :--------: | :---------------------------: |
|
||||
| **KL divergence** | 0.06 | 0 *(by definition)* |
|
||||
| **Refusals** | 6/100 | 96/100 |
|
||||
|
||||
## Model Overview
|
||||
|
||||
**Qwen3-4B-Thinking-2507** has the following features:
|
||||
- Type: Causal Language Models
|
||||
- Training Stage: Pretraining & Post-training
|
||||
- Number of Parameters: 4.0B
|
||||
- Number of Paramaters (Non-Embedding): 3.6B
|
||||
- Number of Layers: 36
|
||||
- Number of Attention Heads (GQA): 32 for Q and 8 for KV
|
||||
- Context Length: **262,144 natively**.
|
||||
|
||||
**NOTE: This model supports only thinking mode. Meanwhile, specifying `enable_thinking=True` is no longer required.**
|
||||
|
||||
Additionally, to enforce model thinking, the default chat template automatically includes `<think>`. Therefore, it is normal for the model's output to contain only `</think>` without an explicit opening `<think>` tag.
|
||||
|
||||
For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our [blog](https://qwenlm.github.io/blog/qwen3/), [GitHub](https://github.com/QwenLM/Qwen3), and [Documentation](https://qwen.readthedocs.io/en/latest/).
|
||||
|
||||
**Supported languages:** English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
|
||||
|
||||
Reference in New Issue
Block a user