commit 3a343d7e637a4a68d7a67fceca0042998c20d464 Author: ModelHub XC Date: Sat Jul 25 12:09:09 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..4c1adaf --- /dev/null +++ b/.gitattributes @@ -0,0 +1,59 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ1_M.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ1_M.gguf new file mode 100644 index 0000000..69bbe0f --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3b70bb9fd747a3f4f5b9d802058ecd26dd0f544d42273efa0cec6d72aa4cebbb +size 7932161216 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ1_S.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ1_S.gguf new file mode 100644 index 0000000..9ffa02b --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8936095517b9007ff08e0dafc1cd5dfdd17820d87aaeb7040b4f4baf2546afe0 +size 7274507456 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_M.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_M.gguf new file mode 100644 index 0000000..9486410 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:269617d0f4c8db6908e4864ca3305411f939ea6589a7863812a09670df1637b9 +size 11264441536 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_S.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_S.gguf new file mode 100644 index 0000000..c9c5e66 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b7683717aa68f48b565434889ccc2e8bc6849acdb5f1e1ff1d1a6e337036c9b6 +size 10387569856 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_XS.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_XS.gguf new file mode 100644 index 0000000..07f0ab9 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:23dc386e93e802c8ee61a0d3ed268f86f37dcc4dec820dd78be12dcb6ed071cf +size 9957551296 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_XXS.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..3f0d2ea --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bfaf23f374a3a4916ffb6722bed43116b13d4ba205f108ca1d625d81e058c610 +size 9028250816 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_M.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_M.gguf new file mode 100644 index 0000000..083fc38 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:06dde0cb97912a2a80194f27609e6182e61de3991e9f43e01d99a6ad4d8bafed +size 14810123456 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_S.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_S.gguf new file mode 100644 index 0000000..ae3df26 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fae1fccba67ebb2c164a8cb1e457638929dbcaf386b61b00a54976a6f934e820 +size 14436895936 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_XS.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_XS.gguf new file mode 100644 index 0000000..4bf1c16 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fb268fc8beccfcc380faf8503a62c09d8f71756b2f90ecb0adde4e383fbcfeb6 +size 13705514176 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_XXS.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..37e7b28 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0be9ef452eea13933e992bc1b5443b3f071f636bb22c891814fe0db20745861d +size 12839271616 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ4_XS.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ4_XS.gguf new file mode 100644 index 0000000..021ec0c --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ffb666c644c5c122ebce423790be992acad26babc8c26cc472a1cfd3b6564f45 +size 17693154496 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-Q2_K.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q2_K.gguf new file mode 100644 index 0000000..bd4cbb2 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ced6dab68b37b3573ed7ee354b080e2fe78f58609a1ed7f8f12cab9e382b13c6 +size 12313099456 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-Q2_K_S.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q2_K_S.gguf new file mode 100644 index 0000000..fc02a11 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f5edf22f4686d63d8ddda7cae1c8c0613468b27f9047a85ebd093ec25c5ef923 +size 11488001216 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_L.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_L.gguf new file mode 100644 index 0000000..aad648c --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1d4bfc326140a4135bdaafbefab70f94c700d943b01b689593e7de25357b317d +size 17247079616 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_M.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_M.gguf new file mode 100644 index 0000000..b08373c --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d447e27f307eabe2404e63a3e0b8463081d1f323c6d47f8bf8fd615b070e7534 +size 15935048896 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_S.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_S.gguf new file mode 100644 index 0000000..6e88326 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4d46358656ee281351cd8e0e0aa035c3bc1ff2dc1cfc468579ea2aa66a8dfce1 +size 14392331456 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_0.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_0.gguf new file mode 100644 index 0000000..9347efb --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:39be8840cbed1cbea1ab03cb72bdd6500bbd44418391bb401e6227ca57661dea +size 18711010496 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_1.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_1.gguf new file mode 100644 index 0000000..ea00541 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7107f4e222d1acc2303f5d2543de6c190acf10412c1be4ee6ab926826b785c71 +size 20639243456 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_K_M.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_K_M.gguf new file mode 100644 index 0000000..679c554 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4be525151ad4946babfcc509d553cf7122f0b567a96c4844ad166d7f9383f221 +size 19851336896 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_K_S.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_K_S.gguf new file mode 100644 index 0000000..d6a5f34 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2c9bb2705a69577aeab4cbcd451d054e365f7fdd8a0c8b10465e0314b24ea294 +size 18784410816 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-Q5_K_M.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q5_K_M.gguf new file mode 100644 index 0000000..db9695b --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b7eed3c1d8842cd7515a6005d2b083206ed3e81eb7c77a2d764386dd7b2245ca +size 23262158016 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-Q5_K_S.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q5_K_S.gguf new file mode 100644 index 0000000..d998d0e --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d0502dd112be160a3a26ffe5b77489a7b8bdce87478424efe895c6a390e89b55 +size 22638255296 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.i1-Q6_K.gguf b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q6_K.gguf new file mode 100644 index 0000000..ed7848b --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:edbe79743e31ba1bb370ba261aa9b13991f219346791eefc6dbd4fd953850391 +size 26886155456 diff --git a/Qwen2.5-Coder-32B-Python-Specialist.imatrix.gguf b/Qwen2.5-Coder-32B-Python-Specialist.imatrix.gguf new file mode 100644 index 0000000..0e584a0 --- /dev/null +++ b/Qwen2.5-Coder-32B-Python-Specialist.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c157d8cae2854e3f783a0ac8b216549c311b5387e2ca2cd4d7cff5f1a2d0a0f6 +size 15011072 diff --git a/README.md b/README.md new file mode 100644 index 0000000..3aa07da --- /dev/null +++ b/README.md @@ -0,0 +1,90 @@ +--- +base_model: TobiasLogic/Qwen2.5-Coder-32B-Python-Specialist +language: +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- coding +- qwen2.5 +- python +- unsloth +- 32b +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/TobiasLogic/Qwen2.5-Coder-32B-Python-Specialist + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ1_S.gguf) | i1-IQ1_S | 7.4 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ1_M.gguf) | i1-IQ1_M | 8.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 9.1 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_XS.gguf) | i1-IQ2_XS | 10.1 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_S.gguf) | i1-IQ2_S | 10.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ2_M.gguf) | i1-IQ2_M | 11.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-Q2_K_S.gguf) | i1-Q2_K_S | 11.6 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-Q2_K.gguf) | i1-Q2_K | 12.4 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 12.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_XS.gguf) | i1-IQ3_XS | 13.8 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_S.gguf) | i1-Q3_K_S | 14.5 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_S.gguf) | i1-IQ3_S | 14.5 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ3_M.gguf) | i1-IQ3_M | 14.9 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_M.gguf) | i1-Q3_K_M | 16.0 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-Q3_K_L.gguf) | i1-Q3_K_L | 17.3 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-IQ4_XS.gguf) | i1-IQ4_XS | 17.8 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_0.gguf) | i1-Q4_0 | 18.8 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_K_S.gguf) | i1-Q4_K_S | 18.9 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_K_M.gguf) | i1-Q4_K_M | 20.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-Q4_1.gguf) | i1-Q4_1 | 20.7 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-Q5_K_S.gguf) | i1-Q5_K_S | 22.7 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-Q5_K_M.gguf) | i1-Q5_K_M | 23.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Python-Specialist-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Python-Specialist.i1-Q6_K.gguf) | i1-Q6_K | 27.0 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +