初始化项目，由ModelHub XC社区提供模型

Model: ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF Source: Original Platform
2026-09-03 01:43:17 +08:00
commit 064643fd0d
14 changed files with 194 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,47 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-4B-Thinking-2507-Heretic-f16.gguf filter=lfs diff=lfs merge=lfs -text
--- a/Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf
+++ b/Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44b0ebae0be5435800b904040da2921823cb69cbc85c715cb8a15375bea0b6bc
+size 1669499744
--- a/Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf
+++ b/Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc7974bce5837d92f57ffa3353507f9688f0246407ed50f7c5a929098e74de1d
+size 2075618144
--- a/Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf
+++ b/Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3dbadc1271a61cf36e2e31a6901322222c643cbf33eedcc07337a0892dae73ba
+size 1886997344
--- a/Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf
+++ b/Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f77b96077e75581877d81dfa945b8befff4ba0780a2931d0f7f8902dda05e62
+size 2369547104
--- a/Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf
+++ b/Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff1fd9b00f6a08b9e0ec8a0b6e75feb091bcf73e68d4579f6f2a07cb32f3d7ec
+size 2497280864
--- a/Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf
+++ b/Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f86decd28faafd24beb992e89f480e2a00f61f0c7a6b02e268fca8427baa71bf
+size 2383309664
--- a/Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf
+++ b/Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89f03dab3f0c7dfb2a4cb3744f3b5c4407afee8abfb58f62b19edb92ef5a7fe2
+size 2823711584
--- a/Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf
+++ b/Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ad50ae37d6593a1a9390b59b79ff87055118c41469dca8fe2c93ad753e2a28c
+size 2889513824
--- a/Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf
+++ b/Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d010e8086d8742611469214f5004458eff3b099062b13226d72a67c98a2c6d5d
+size 2823711584
--- a/Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf
+++ b/Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3dee58c60115251ee6abf761d7530d9f7d10942036a6aae1150b3989e85581ab
+size 3306261344
--- a/Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf
+++ b/Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0ddc2ca12b2f5afdb9ea9c423427c16aa67d2489670903afecd89cd8400549d
+size 4280405344
--- a/Qwen3-4B-Thinking-2507-Heretic-f16.gguf
+++ b/Qwen3-4B-Thinking-2507-Heretic-f16.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7387495c06f4d95734854f7b7dfb1106c32b1a6817cdf08da5780422e2663cbb
+size 8051285344
--- a/README.md
+++ b/README.md
@@ -0,0 +1,111 @@
+---
+license: apache-2.0
+license_link: https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507/blob/main/LICENSE
+pipeline_tag: text-generation
+base_model:
+- becnic/Qwen3-4B-Thinking-2507-Heretic
+language:
+- en
+- de
+- fr
+- it
+- pt
+- hi
+- es
+- th
+---
+# Qwen3-4B-Thinking-2507-Heretic-GGUF
+
+## Llamacpp imatrix Quantizations of Qwen3-4B-Thinking-2507-Heretic by becnic (from original Qwen3-4B-Thinking-2507)
+
+Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b7120">b7120</a> for quantization.
+
+Original model: https://huggingface.co/becnic/Qwen3-4B-Thinking-2507-Heretic
+
+Run them in [LM Studio](https://lmstudio.ai/)
+
+Run them directly with [llama.cpp](https://github.com/ggerganov/llama.cpp), or any other llama.cpp based project
+
+## Download a file (not the whole branch) from below:
+
+| Filename                                                                                                                                                             | Quant type | File Size | Split | Description                              |
+| -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | --------- | ----- | ---------------------------------------- |
+| [Qwen3-4B-Thinking-2507-Heretic-f16.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-f16.gguf)       | f16        | 8.05GB    | false | Full precision, highest possible quality |
+| [Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf)     | Q8_0       | 4.28GB   | false | Extremely high quality                   |
+| [Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf)     | Q6_K       | 3.31GB    | false | Near-lossless high quality               |
+| [Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf) | Q5_K_S     | 2.82GB   | false | Premium high quality                     |
+| [Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf) | Q5_K_M     | 2.89GB    | false | Very high quality                        |
+| [Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf)     | Q5_0       | 2.82GB    | false | High quality                             |
+| [Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf) | Q4_K_S     | 2.38GB    | false | Strong mid-high quality                  |
+| [Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf) | Q4_K_M     | 2.50GB    | false | Balanced mid-high quality                |
+| [Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf)     | Q4_0       | 2.37GB    | false | Good balance of size and quality         |
+| [Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf) | Q3_K_S     | 1.89GB    | false | Higher tier Q3                           |
+| [Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf) | Q3_K_M     | 2.08GB    | false | Mid-range                                |
+| [Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf)     | Q2_K       | 1.67GB    | false | Smallest size, lowest quality            |
+
+
+## Downloading using huggingface-cli
+
+<details>
+  <summary>Click to view download instructions</summary>
+
+First, make sure you have hugginface-cli installed:
+
+```
+pip install -U "huggingface_hub[cli]"
+```
+
+Then, you can target the specific file you want:
+
+```
+huggingface-cli download ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF --include "Qwen3-4B-Thinking-2507-Q8_0.gguf" --local-dir ./
+```
+
+If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
+
+```
+huggingface-cli download ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF --include "Qwen3-4B-Thinking-2507-Q8_0.gguf/*" --local-dir ./
+```
+
+</details>
+
+## Abliteration parameters
+
+| Parameter | Value |
+| :-------- | :---: |
+| **direction_index** | 19.42 |
+| **attn.o_proj.max_weight** | 1.23 |
+| **attn.o_proj.max_weight_position** | 22.34 |
+| **attn.o_proj.min_weight** | 0.69 |
+| **attn.o_proj.min_weight_distance** | 10.42 |
+| **mlp.down_proj.max_weight** | 1.12 |
+| **mlp.down_proj.max_weight_position** | 29.64 |
+| **mlp.down_proj.min_weight** | 1.08 |
+| **mlp.down_proj.min_weight_distance** | 20.24 |
+
+## Performance
+
+| Metric | This model | Original model ([Qwen/Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507)) |
+| :----- | :--------: | :---------------------------: |
+| **KL divergence** | 0.06 | 0 *(by definition)* |
+| **Refusals** | 6/100 | 96/100 |
+
+## Model Overview
+
+**Qwen3-4B-Thinking-2507** has the following features:
+- Type: Causal Language Models
+- Training Stage: Pretraining & Post-training
+- Number of Parameters: 4.0B
+- Number of Paramaters (Non-Embedding): 3.6B
+- Number of Layers: 36
+- Number of Attention Heads (GQA): 32 for Q and 8 for KV
+- Context Length: **262,144 natively**. 
+
+**NOTE: This model supports only thinking mode. Meanwhile, specifying `enable_thinking=True` is no longer required.**
+
+Additionally, to enforce model thinking, the default chat template automatically includes `<think>`. Therefore, it is normal for the model's output to contain only `</think>` without an explicit opening `<think>` tag.
+
+For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our [blog](https://qwenlm.github.io/blog/qwen3/), [GitHub](https://github.com/QwenLM/Qwen3), and [Documentation](https://qwen.readthedocs.io/en/latest/).
+
+**Supported languages:** English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
+