commit 064643fd0d48ea84e6ddf878761ade64e9c1a647 Author: ModelHub XC Date: Thu Sep 3 01:43:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..c8f22f1 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,47 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Thinking-2507-Heretic-f16.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf b/Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf new file mode 100644 index 0000000..52356ee --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44b0ebae0be5435800b904040da2921823cb69cbc85c715cb8a15375bea0b6bc +size 1669499744 diff --git a/Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf b/Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf new file mode 100644 index 0000000..4dfd639 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc7974bce5837d92f57ffa3353507f9688f0246407ed50f7c5a929098e74de1d +size 2075618144 diff --git a/Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf b/Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf new file mode 100644 index 0000000..56b91f8 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3dbadc1271a61cf36e2e31a6901322222c643cbf33eedcc07337a0892dae73ba +size 1886997344 diff --git a/Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf b/Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf new file mode 100644 index 0000000..cb7a4df --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f77b96077e75581877d81dfa945b8befff4ba0780a2931d0f7f8902dda05e62 +size 2369547104 diff --git a/Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf b/Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf new file mode 100644 index 0000000..a1e071e --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ff1fd9b00f6a08b9e0ec8a0b6e75feb091bcf73e68d4579f6f2a07cb32f3d7ec +size 2497280864 diff --git a/Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf b/Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf new file mode 100644 index 0000000..d498ce9 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f86decd28faafd24beb992e89f480e2a00f61f0c7a6b02e268fca8427baa71bf +size 2383309664 diff --git a/Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf b/Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf new file mode 100644 index 0000000..f6e5e36 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89f03dab3f0c7dfb2a4cb3744f3b5c4407afee8abfb58f62b19edb92ef5a7fe2 +size 2823711584 diff --git a/Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf b/Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf new file mode 100644 index 0000000..89ac383 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3ad50ae37d6593a1a9390b59b79ff87055118c41469dca8fe2c93ad753e2a28c +size 2889513824 diff --git a/Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf b/Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf new file mode 100644 index 0000000..0331cc8 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d010e8086d8742611469214f5004458eff3b099062b13226d72a67c98a2c6d5d +size 2823711584 diff --git a/Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf b/Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf new file mode 100644 index 0000000..fa4a715 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3dee58c60115251ee6abf761d7530d9f7d10942036a6aae1150b3989e85581ab +size 3306261344 diff --git a/Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf b/Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf new file mode 100644 index 0000000..701448b --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e0ddc2ca12b2f5afdb9ea9c423427c16aa67d2489670903afecd89cd8400549d +size 4280405344 diff --git a/Qwen3-4B-Thinking-2507-Heretic-f16.gguf b/Qwen3-4B-Thinking-2507-Heretic-f16.gguf new file mode 100644 index 0000000..db17644 --- /dev/null +++ b/Qwen3-4B-Thinking-2507-Heretic-f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7387495c06f4d95734854f7b7dfb1106c32b1a6817cdf08da5780422e2663cbb +size 8051285344 diff --git a/README.md b/README.md new file mode 100644 index 0000000..5eead4b --- /dev/null +++ b/README.md @@ -0,0 +1,111 @@ +--- +license: apache-2.0 +license_link: https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507/blob/main/LICENSE +pipeline_tag: text-generation +base_model: +- becnic/Qwen3-4B-Thinking-2507-Heretic +language: +- en +- de +- fr +- it +- pt +- hi +- es +- th +--- +# Qwen3-4B-Thinking-2507-Heretic-GGUF + +## Llamacpp imatrix Quantizations of Qwen3-4B-Thinking-2507-Heretic by becnic (from original Qwen3-4B-Thinking-2507) + +Using llama.cpp release b7120 for quantization. + +Original model: https://huggingface.co/becnic/Qwen3-4B-Thinking-2507-Heretic + +Run them in [LM Studio](https://lmstudio.ai/) + +Run them directly with [llama.cpp](https://github.com/ggerganov/llama.cpp), or any other llama.cpp based project + +## Download a file (not the whole branch) from below: + +| Filename | Quant type | File Size | Split | Description | +| -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | --------- | ----- | ---------------------------------------- | +| [Qwen3-4B-Thinking-2507-Heretic-f16.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-f16.gguf) | f16 | 8.05GB | false | Full precision, highest possible quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q8_0.gguf) | Q8_0 | 4.28GB | false | Extremely high quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q6_K.gguf) | Q6_K | 3.31GB | false | Near-lossless high quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_K_S.gguf) | Q5_K_S | 2.82GB | false | Premium high quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_K_M.gguf) | Q5_K_M | 2.89GB | false | Very high quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q5_0.gguf) | Q5_0 | 2.82GB | false | High quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_K_S.gguf) | Q4_K_S | 2.38GB | false | Strong mid-high quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_K_M.gguf) | Q4_K_M | 2.50GB | false | Balanced mid-high quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q4_0.gguf) | Q4_0 | 2.37GB | false | Good balance of size and quality | +| [Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q3_K_S.gguf) | Q3_K_S | 1.89GB | false | Higher tier Q3 | +| [Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q3_K_M.gguf) | Q3_K_M | 2.08GB | false | Mid-range | +| [Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf](https://huggingface.co/ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF/blob/main/Qwen3-4B-Thinking-2507-Heretic-Q2_K.gguf) | Q2_K | 1.67GB | false | Smallest size, lowest quality | + + +## Downloading using huggingface-cli + +
+ Click to view download instructions + +First, make sure you have hugginface-cli installed: + +``` +pip install -U "huggingface_hub[cli]" +``` + +Then, you can target the specific file you want: + +``` +huggingface-cli download ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF --include "Qwen3-4B-Thinking-2507-Q8_0.gguf" --local-dir ./ +``` + +If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: + +``` +huggingface-cli download ZuzeTt/Qwen3-4B-Thinking-2507-Heretic-GGUF --include "Qwen3-4B-Thinking-2507-Q8_0.gguf/*" --local-dir ./ +``` + +
+ +## Abliteration parameters + +| Parameter | Value | +| :-------- | :---: | +| **direction_index** | 19.42 | +| **attn.o_proj.max_weight** | 1.23 | +| **attn.o_proj.max_weight_position** | 22.34 | +| **attn.o_proj.min_weight** | 0.69 | +| **attn.o_proj.min_weight_distance** | 10.42 | +| **mlp.down_proj.max_weight** | 1.12 | +| **mlp.down_proj.max_weight_position** | 29.64 | +| **mlp.down_proj.min_weight** | 1.08 | +| **mlp.down_proj.min_weight_distance** | 20.24 | + +## Performance + +| Metric | This model | Original model ([Qwen/Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507)) | +| :----- | :--------: | :---------------------------: | +| **KL divergence** | 0.06 | 0 *(by definition)* | +| **Refusals** | 6/100 | 96/100 | + +## Model Overview + +**Qwen3-4B-Thinking-2507** has the following features: +- Type: Causal Language Models +- Training Stage: Pretraining & Post-training +- Number of Parameters: 4.0B +- Number of Paramaters (Non-Embedding): 3.6B +- Number of Layers: 36 +- Number of Attention Heads (GQA): 32 for Q and 8 for KV +- Context Length: **262,144 natively**. + +**NOTE: This model supports only thinking mode. Meanwhile, specifying `enable_thinking=True` is no longer required.** + +Additionally, to enforce model thinking, the default chat template automatically includes ``. Therefore, it is normal for the model's output to contain only `` without an explicit opening `` tag. + +For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our [blog](https://qwenlm.github.io/blog/qwen3/), [GitHub](https://github.com/QwenLM/Qwen3), and [Documentation](https://qwen.readthedocs.io/en/latest/). + +**Supported languages:** English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai. +