commit f40798e86af50ab6d6e5321ada985f1b626005ae Author: ModelHub XC Date: Wed May 27 09:24:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..7f1d6bf --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ1_M.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ1_M.gguf new file mode 100644 index 0000000..0731009 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6885c55584c4c465ab209bb737cd52235e2483943ddc55add034b7a954498042 +size 1127019808 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ1_S.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ1_S.gguf new file mode 100644 index 0000000..e90cb31 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:410636798ad22d69bd364453ee78ee4235e1abf3986966ae7d5025fe45b3a276 +size 1055257888 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_M.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_M.gguf new file mode 100644 index 0000000..da95c73 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d037fc155c41a0a5a7140428b65bc9526514e42d33995598ec8cd9419617602a +size 1512985888 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_S.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_S.gguf new file mode 100644 index 0000000..d8279bb --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cdeb6648633fc4f945d717ab9d959bbf4096717206d72ffdc475655762aa07c5 +size 1417303328 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_XS.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_XS.gguf new file mode 100644 index 0000000..d100b85 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2cdc8ecfd69aa1fd6d751a134e185770de926a9411ac95e65395787b39d13b11 +size 1354102048 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_XXS.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..422d425 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:28383ae3aae47c36e758d837c6109c725d7eb40e0cd4700aa947213de0f2a5f1 +size 1246623008 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_M.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_M.gguf new file mode 100644 index 0000000..4ad9bc8 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce9529b654f5a896c446d96282c64fecbb5a78d17b906b60d858bfdf261c17dc +size 1962898208 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_S.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_S.gguf new file mode 100644 index 0000000..616d98c --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d9859ac285aa04dd223197b7982af441f166e05bc0fe037fdbd124f29e7751d9 +size 1899533088 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_XS.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_XS.gguf new file mode 100644 index 0000000..bedac47 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ba59f687609c15dcee9b0956d5fc9fc2fe7e1cb1cc83047a8ba095f3306ef09 +size 1814377248 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_XXS.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..d754f93 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7e328df111a7e0af5d472bc01d3e33000aca64037bc986b1828d6193ebd3e63 +size 1670190368 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ4_NL.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ4_NL.gguf new file mode 100644 index 0000000..e1a61a0 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:13b136012b3b5e229a6c34b2bb70be4080a57e19c15e4110e9a101a38fd89ef6 +size 2381345568 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ4_XS.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ4_XS.gguf new file mode 100644 index 0000000..468fb1f --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4147b1bf034677f102ed13309b867e03d193dbc4d86b94ae4db3f3256e779b95 +size 2270753568 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q2_K.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q2_K.gguf new file mode 100644 index 0000000..7ed7c8a --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37b83e9a0b6600442c1932cb891ad0376184d6e28d9755e94c673bc7bd220572 +size 1669501728 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q2_K_S.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q2_K_S.gguf new file mode 100644 index 0000000..5893e16 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:871081e40e38e547c8bc9477453d3407b46a7e69f9a50e0fa44b98087bb22a98 +size 1563456288 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_L.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_L.gguf new file mode 100644 index 0000000..771bbf3 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4092e1a5c0388ab752f59cf384bbe786c80442b25bdfeb91de47891de2027240 +size 2239787808 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_M.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_M.gguf new file mode 100644 index 0000000..498c02b --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:17f651fc811b54adc04f164f4d4f2aa92be7c75a8b35c5ac3a40d5229dc0f0be +size 2075620128 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_S.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_S.gguf new file mode 100644 index 0000000..9d1610c --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2bbe7181d17b1a89fc12303f0913b8ebeee657ff494ba0f866d1c5a5c2c126c +size 1886999328 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_0.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_0.gguf new file mode 100644 index 0000000..393e656 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:69775fe3cbca879edd18c05f1bb697d257d2b8947f4bfd17d8cc802019914a6c +size 2375775008 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_1.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_1.gguf new file mode 100644 index 0000000..52885c6 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0068fee6f02cdc49a2418c6d8d494b46ff132a1e10c1a7b1503e56296e06b82d +size 2596631328 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_K_M.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_K_M.gguf new file mode 100644 index 0000000..0373bff --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e187d88598b1474cc48c9d830ccc3eac92fe29729ea6a2140067cfcb6e3db330 +size 2497282848 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_K_S.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_K_S.gguf new file mode 100644 index 0000000..84dfe06 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ed3b61815a5c6ee98ce0e12855779f5c1b646e6ca1262c8dd2f0a32e99378fa1 +size 2383311648 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q5_K_M.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q5_K_M.gguf new file mode 100644 index 0000000..8d6e5a0 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:82fb70d2ecca35024264a817dcc207a13c2a5f6cf9e26dacff9972f4874ba332 +size 2889515808 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q5_K_S.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q5_K_S.gguf new file mode 100644 index 0000000..14e4818 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:db1ad510904578886f982a761bbb3775bf36bd22ad51476591e1a9fa56bb2fd9 +size 2823713568 diff --git a/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q6_K.gguf b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q6_K.gguf new file mode 100644 index 0000000..8e13e71 --- /dev/null +++ b/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:21a576dc414706cf0831a7fb79270836b60c6ae30203bc5e0527288d37da8847 +size 3306263328 diff --git a/README.md b/README.md new file mode 100644 index 0000000..ce91b8c --- /dev/null +++ b/README.md @@ -0,0 +1,111 @@ +--- +base_model: DavidAU/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000 +language: +- en +- fr +- zh +- de +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- creative +- creative writing +- fiction writing +- plot generation +- sub-plot generation +- fiction writing +- story generation +- scene continue +- storytelling +- fiction story +- science fiction +- romance +- all genres +- story +- writing +- vivid prose +- vivid writing +- fiction +- roleplaying +- bfloat16 +- swearing +- rp +- qwen3 +- horror +- finetune +- merge +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/DavidAU/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000 + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ1_S.gguf) | i1-IQ1_S | 1.2 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ1_M.gguf) | i1-IQ1_M | 1.2 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_S.gguf) | i1-IQ2_S | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ2_M.gguf) | i1-IQ2_M | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.7 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q2_K.gguf) | i1-Q2_K | 1.8 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.8 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_S.gguf) | i1-Q3_K_S | 2.0 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_S.gguf) | i1-IQ3_S | 2.0 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ3_M.gguf) | i1-IQ3_M | 2.1 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_M.gguf) | i1-Q3_K_M | 2.2 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q3_K_L.gguf) | i1-Q3_K_L | 2.3 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_0.gguf) | i1-Q4_0 | 2.5 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.5 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.5 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.6 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q4_1.gguf) | i1-Q4_1 | 2.7 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q5_K_M.gguf) | i1-Q5_K_M | 3.0 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000-i1-GGUF/resolve/main/Qwen3-4B-Fiction-On-Fire-Series-6-Model-1000.i1-Q6_K.gguf) | i1-Q6_K | 3.4 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..d3746f4 --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b3889f4750ae1fe2dc673ec51a7700d821debae415331f2d3559d1780a91400 +size 3842209