commit 8dd97a3aa24feded395adb67486f8ae8fd99e092 Author: ModelHub XC Date: Sun Jun 21 15:42:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..ff4a8ce --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-IQ1_M.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ1_M.gguf new file mode 100644 index 0000000..9c00a82 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5242d0fd20bafb1e50752175d95419cd952afaa3b761d507fb230f21d7d42abc +size 2256149888 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-IQ1_S.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ1_S.gguf new file mode 100644 index 0000000..0c1f5b0 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7e175b56198bc8850f2225e418187ee5c44cbac4f70c7c30add6870a30c8b7a2 +size 2115771776 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_M.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_M.gguf new file mode 100644 index 0000000..22c1b72 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4a115156d5acb0490679da5e2a29b9922d48a6d37ef7a9711d4b493be1121033 +size 3051916672 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_S.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_S.gguf new file mode 100644 index 0000000..6b2bc27 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:00f1ebbabfa6c1cde393faeac7bc98ba4d07dd5d061f45c5f23d7b9390079112 +size 2864745856 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_XS.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_XS.gguf new file mode 100644 index 0000000..ace9625 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ba05537e35d1a49277ce25320800e2664686ca9c52e6fd12802dd45cc1dce53 +size 2696158592 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_XXS.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..7ddedf3 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1f9614ac8e9e47bf87d688a3769667f4ce917034799e1f7b7843699c33e38102 +size 2490113408 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_M.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_M.gguf new file mode 100644 index 0000000..20d20a2 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:32e94dadb7291e96138e6aef04e2079715dd274a9b2e5ca59a0279ab8f4f61b0 +size 3896622464 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_S.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_S.gguf new file mode 100644 index 0000000..7d29a39 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:606ef20924644425e32577149c3fab78b7992dbdc474b605fc12a9a02c7806fc +size 3789667712 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_XS.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_XS.gguf new file mode 100644 index 0000000..96b2c6d --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:df1838f42546bb9d547fffa93b574bdda95719e45bf9633e194e3b05fbdae443 +size 3626876288 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_XXS.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..332a195 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3f06cb8c90f3ec302be43a510a99b8b731659833f61b5f6865a06bf24b8f8bfc +size 3369635200 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-IQ4_NL.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ4_NL.gguf new file mode 100644 index 0000000..d081c60 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a886f11953a380fe2d8943d171b4b5542572013e9105ffbc07d41527d97c2526 +size 4793625984 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-IQ4_XS.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ4_XS.gguf new file mode 100644 index 0000000..ff4f186 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:455547e98f85f07414556fe3613d95d420098bcb3c806dd571eb84d7611d231d +size 4561841536 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-Q2_K.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-Q2_K.gguf new file mode 100644 index 0000000..6344ef5 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:552f02be804b8c503d71e623c9e1c936a4b85093f797896b338874894c0e2ff9 +size 3281735040 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-Q2_K_S.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-Q2_K_S.gguf new file mode 100644 index 0000000..03c71e7 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ed1643d9ce5500cecd9a215892be4d380762fac709aa3d314b7afbf5a3d2472 +size 3083554176 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_L.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_L.gguf new file mode 100644 index 0000000..76bcca5 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:51d8433b05b01a8f627e62f24fdc6b8fe6b6fbfc300ae636ec2697fc5b3daa01 +size 4431396224 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_M.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_M.gguf new file mode 100644 index 0000000..e2ad762 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:81766fdd85263c75e296f8e1e2a0a29721e870f3b4c4897b031a3db674f45272 +size 4124163456 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_S.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_S.gguf new file mode 100644 index 0000000..fa638c7 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc5e42b81fb818dda550fa27d4ac64354eeb154499060ab7046958b95cb22b10 +size 3769613696 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_0.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_0.gguf new file mode 100644 index 0000000..37e7911 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1dc992981cc65e8dd66133129c283f44d049fb3ec2dd7cfb700f82ab7aab5d9a +size 4787334528 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_1.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_1.gguf new file mode 100644 index 0000000..c2d331e --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0ff3c5994d3d219b2ca091c2146db3dbadcb0f40d5e85a9923be7ca939b1e7eb +size 5247757696 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_K_M.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_K_M.gguf new file mode 100644 index 0000000..3cf293d --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b4003b14ab78a08b39a8920089f09abadeaf4df3bbabd15fdddc9a259349b97e +size 5027786112 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_K_S.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_K_S.gguf new file mode 100644 index 0000000..087009a --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4d79329d1a036e4272deaa4e9ea649b6cb25458e0af0311c3e52a476431ea74 +size 4802014592 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-Q5_K_M.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-Q5_K_M.gguf new file mode 100644 index 0000000..24386c0 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ee947e3b8426fb348da91e78f585ffe6dda5ae519dd91e6a0e9e5823a02c90a +size 5851114880 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-Q5_K_S.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-Q5_K_S.gguf new file mode 100644 index 0000000..3a3774e --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a78447a6a8bc0b441ec7dfc5298d80d3fc0ee45db97767a9f90927a8471d5e0b +size 5720763776 diff --git a/Qwen-3-VL-8B-Instruct-heretic.i1-Q6_K.gguf b/Qwen-3-VL-8B-Instruct-heretic.i1-Q6_K.gguf new file mode 100644 index 0000000..fcddb5b --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d768ba0729aa3805760e798b39747b216c3d9a128a21fa6421529490efd3a60b +size 6725901696 diff --git a/Qwen-3-VL-8B-Instruct-heretic.imatrix.gguf b/Qwen-3-VL-8B-Instruct-heretic.imatrix.gguf new file mode 100644 index 0000000..69dc452 --- /dev/null +++ b/Qwen-3-VL-8B-Instruct-heretic.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2621462491b9c9b2341bf06b714ba3d8f5e5c498b3404bba2249e5caf89ce618 +size 5347200 diff --git a/README.md b/README.md new file mode 100644 index 0000000..3578c1b --- /dev/null +++ b/README.md @@ -0,0 +1,93 @@ +--- +base_model: heretic-org/Qwen-3-VL-8B-Instruct-heretic +language: +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- heretic +- uncensored +- decensored +- abliterated +- reproducible +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/heretic-org/Qwen-3-VL-8B-Instruct-heretic + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Qwen-3-VL-8B-Instruct-heretic-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-GGUF + +**This is a vision model - mmproj files (if any) will be in the [static repository](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-GGUF).** +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-IQ1_S.gguf) | i1-IQ1_S | 2.2 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-IQ1_M.gguf) | i1-IQ1_M | 2.4 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.6 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.8 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_S.gguf) | i1-IQ2_S | 3.0 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-IQ2_M.gguf) | i1-IQ2_M | 3.2 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.2 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-Q2_K.gguf) | i1-Q2_K | 3.4 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.5 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.7 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.9 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_S.gguf) | i1-IQ3_S | 3.9 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-IQ3_M.gguf) | i1-IQ3_M | 4.0 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.2 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.5 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.7 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_0.gguf) | i1-Q4_0 | 4.9 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-IQ4_NL.gguf) | i1-IQ4_NL | 4.9 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.9 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.1 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-Q4_1.gguf) | i1-Q4_1 | 5.3 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.8 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-Q5_K_M.gguf) | i1-Q5_K_M | 6.0 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-i1-GGUF/resolve/main/Qwen-3-VL-8B-Instruct-heretic.i1-Q6_K.gguf) | i1-Q6_K | 6.8 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +