初始化项目,由ModelHub XC社区提供模型

Model: ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-Imatrix-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-17 16:11:16 +08:00
commit e234772825
32 changed files with 267 additions and 0 deletions

66
.gitattributes vendored Normal file
View File

@@ -0,0 +1,66 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-f16.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q2_K-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q3_K_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q3_K_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q5_K_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q5_K_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q4_0-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q4_K_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q8_0-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q6_K-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q5_0-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q4_K_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-IQ1_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-IQ1_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-IQ2_XS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-IQ2_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-IQ2_XXS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-IQ3_XS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-IQ3_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-IQ3_XXS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q3_K_L-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-IQ4_NL-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-IQ4_XS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic_q4_0_pure_imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-bf16.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-IQ2_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q5_1-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q4_K-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-IQ3_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q4_0-pure-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Qwen3-4B-Thinking-2507-Heretic-Q4_1-imatrix.gguf filter=lfs diff=lfs merge=lfs -text

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5846bbee56d53389235e17e50d554b5312538dc26c56ef6366776ecfa4ed5511
size 2269269760

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:505e4e2a24fc10dd06ff4478df67cd300c4196d7801958a711cb824700794525
size 8051285504

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:578afc3fbff3cff0d78cb8ac7f168f9942a495e4da7f36224a0fa409af25b1dc
size 8051285504

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:89aa2fb972c8865d9e4f110402ceac45fe7766965b0d5f71866b331cf3d4eb27
size 1127018240

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f4cec06e1416b746c74c9a634ac7fe3b08dbc34a96627ca76be360ace46380de
size 1055256320

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:11e3ef30f07511831fdfeb8a604639db72a9b06b0b091006535c0492669469f6
size 1512984320

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fdf7add0ee88abf2b712d293ba700554e1e3c015ebbbeb974218450592a6986c
size 1417301760

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:966c43c0ed9c8cd2dcb4606cbcd91934df89f801f370f642d25dac4ece58ca9d
size 1354100480

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bdb459a478967d4ea9909c6742f3f8e51be9e9f039bbe7ac7313471d229e8da5
size 1246621440

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1781919d103abd7c7ba752956b6ad0f3c7d9283ab9ae4065540135614cc12d9a
size 1962896640

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3e426f8642cf628ef88b26145f9c20c63d2e646124d578b22f01ff9af0b2f566
size 1899531520

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9a5c1cc91c3a8734c0e9e6ccb276fbf1bf5a6056898a2c2acb555d231701297d
size 1814375680

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:48e09fb770a7ddd23be81ea796b09f215a9b0729dc401b04c819ba65c602036e
size 1670188800

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6fd4ba3b43b84486d6a462a14774e6075dfbba7d633a7554be8ae5a29d3ca298
size 2381344000

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e56825a2be2d9e99197688e750119a57cceba76231626a4d8fa1863406052916
size 2270752000

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a9ad7f14280f0d90c41866fb7c9311f8e7d17a981ab3fbff39e9bf980ad9c833
size 1669500160

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:375aa56a18dbd263e93a70ed01059dcb216a1dcfad5eb2d69052dc1b6ee94319
size 2239786240

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:65ddf7a7c493504f68bc2a2253da63a16811310b2ee25138b9901fb52623a269
size 2075618560

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1a386d76ff8c0d1743cf1fa84fa2cc9710de316f492a87a300e9b93cbaaf6800
size 1886997760

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:074b633f48c636feae9f4d0ea47cd74879f922dad67b7a5aa3a1b25b8c514ff4
size 2375773440

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:46cefc7553ec5c4567adcd42639b4964b511ce4fb8192c72ca2c6b799284989b
size 2596629760

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:877a87d131672bfadfd46f86fca8a6f0859726f72fb46841939ad2c02aca5769
size 2497281280

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:877a87d131672bfadfd46f86fca8a6f0859726f72fb46841939ad2c02aca5769
size 2497281280

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d9902569406020d89102ed15bcabb9367f1ad6bdd5cc2d44718e489f6a54c3f8
size 2383310080

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c36e59de9a99da4d3f74c02e3736b0c5fc85531783863316986b7a0264499756
size 2829937920

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b2710dadb4d598c7c7f021a9e4924329ca5ea23ab7b3ddb6bc4cb25a481a6fe6
size 3050794240

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2ab1245b0395c1f4b13b4f5479955b935552418c1d1d700c7da9375ca1781b71
size 2889514240

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:da32fc7438993895527ab03a1c23730af7035ed62c52fdb9e1915cafcbde6969
size 2823712000

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:28c35ecdce25709d02a54075624b5eb57cf2f8955a180a30031bcee9473a2df3
size 3306261760

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fdf0244392d9da5c3a8803f05c5305413c4bb401fb315dff55c3165d8986b2bc
size 4280405760

111
README.md Normal file
View File

@@ -0,0 +1,111 @@
---
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507/blob/main/LICENSE
pipeline_tag: text-generation
base_model:
- becnic/Qwen3-4B-Thinking-2507-Heretic
language:
- en
- de
- fr
- it
- pt
- hi
- es
- th
---
# Qwen3-4B-Thinking-2507-Heretic-GGUF
## Llamacpp imatrix Quantizations of Qwen3-4B-Thinking-2507-Heretic by becnic (from original Qwen3-4B-Thinking-2507)
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b7120">b7120</a> for quantization.
Original model: https://huggingface.co/becnic/Qwen3-4B-Thinking-2507-Heretic
Run them in [LM Studio](https://lmstudio.ai/)
Run them directly with [llama.cpp](https://github.com/ggerganov/llama.cpp), or any other llama.cpp based project
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | --------- | ----- | ---------------------------------------- |
| [Qwen3-4B-Thinking-2507-Heretic-f16.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-f16.gguf) | f16 | 8.05GB | false | Full precision, highest possible quality |
| [Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf) | Q8_0 | 4.28GB | false | Extremely high quality |
| [Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf) | Q6_K | 3.31GB | false | Near-lossless high quality |
| [Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf) | Q5_K_S | 2.82GB | false | Premium high quality |
| [Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf) | Q5_K_M | 2.89GB | false | Very high quality |
| [Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf) | Q5_0 | 2.82GB | false | High quality |
| [Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf) | Q4_K_S | 2.38GB | false | Strong mid-high quality |
| [Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf) | Q4_K_M | 2.50GB | false | Balanced mid-high quality |
| [Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf) | Q4_0 | 2.37GB | false | Good balance of size and quality |
| [Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf) | Q3_K_S | 1.89GB | false | Higher tier Q3 |
| [Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf) | Q3_K_M | 2.08GB | false | Mid-range |
| [Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf) | Q2_K | 1.67GB | false | Smallest size, lowest quality |
## Downloading using huggingface-cli
<details>
<summary>Click to view download instructions</summary>
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF --include "Qwen3-4B-Thinking-2507-Q8_0.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF --include "Qwen3-4B-Thinking-2507-Q8_0.gguf/*" --local-dir ./
```
</details>
## Abliteration parameters
| Parameter | Value |
| :-------- | :---: |
| **direction_index** | 19.42 |
| **attn.o_proj.max_weight** | 1.23 |
| **attn.o_proj.max_weight_position** | 22.34 |
| **attn.o_proj.min_weight** | 0.69 |
| **attn.o_proj.min_weight_distance** | 10.42 |
| **mlp.down_proj.max_weight** | 1.12 |
| **mlp.down_proj.max_weight_position** | 29.64 |
| **mlp.down_proj.min_weight** | 1.08 |
| **mlp.down_proj.min_weight_distance** | 20.24 |
## Performance
| Metric | This model | Original model ([Qwen/Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507)) |
| :----- | :--------: | :---------------------------: |
| **KL divergence** | 0.06 | 0 *(by definition)* |
| **Refusals** | 6/100 | 96/100 |
## Model Overview
**Qwen3-4B-Thinking-2507** has the following features:
- Type: Causal Language Models
- Training Stage: Pretraining & Post-training
- Number of Parameters: 4.0B
- Number of Paramaters (Non-Embedding): 3.6B
- Number of Layers: 36
- Number of Attention Heads (GQA): 32 for Q and 8 for KV
- Context Length: **262,144 natively**.
**NOTE: This model supports only thinking mode. Meanwhile, specifying `enable_thinking=True` is no longer required.**
Additionally, to enforce model thinking, the default chat template automatically includes `<think>`. Therefore, it is normal for the model's output to contain only `</think>` without an explicit opening `<think>` tag.
For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our [blog](https://qwenlm.github.io/blog/qwen3/), [GitHub](https://github.com/QwenLM/Qwen3), and [Documentation](https://qwen.readthedocs.io/en/latest/).
**Supported languages:** English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.