commit ae1161fa5a5997ee00e663341e442fc2243a1412 Author: ModelHub XC Date: Sun Jul 19 02:18:14 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..11f3010 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,57 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ1_M.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ1_M.gguf new file mode 100644 index 0000000..b70e5de --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b78e3716e7832b470f5331f7f537185c5d79b447c9813c0d7163e048012528a +size 7932160896 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ1_S.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ1_S.gguf new file mode 100644 index 0000000..33b1164 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:99edb6cc90ac96b9e31a7ca035e222779b411b5b616ab59d89e732c535ef36cf +size 7274507136 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_M.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_M.gguf new file mode 100644 index 0000000..38088e1 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1f4ce0e8bb2039eeb6c37e1a3bdd5673d0f0944db023052f5f685bf408808cd8 +size 11264441216 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_S.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_S.gguf new file mode 100644 index 0000000..4154bb7 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d4ab39b8c68f5ad168162b4ea1a28033e8a2e7abe74fc75bd5fde6ef3b28b30 +size 10387569536 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_XS.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_XS.gguf new file mode 100644 index 0000000..68e6e08 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a1bb009373089cbf448d3f909ded6a908536d93230561b0e060a3933d45de2f3 +size 9957550976 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_XXS.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..f897a42 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2f9a3c77b1054a997d36a231dcdfee6e6255366ffc8f5731697ade158c9551d4 +size 9028250496 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_M.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_M.gguf new file mode 100644 index 0000000..513cfc5 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ca3e215799e2619d45fb169d3152595308e6065c236eba87f73bdec44b4837e8 +size 14810123136 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_S.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_S.gguf new file mode 100644 index 0000000..6ce9ca9 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc5ac05f05cf81e04e935ca1473f219e50fff8b3b6fb72b2561fccf096b0d3b2 +size 14436895616 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_XS.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_XS.gguf new file mode 100644 index 0000000..5834c7c --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f10833ec9b069d7bed548093040bb7c942f4ff8db563df752e0cd0bbc2224a3a +size 13705513856 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_XXS.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..c70ab4b --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3fc359e5e11c2f2c8071fbd22e3c8b324b5f400a0bfae7dd11308e21598fa77d +size 12839271296 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ4_XS.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ4_XS.gguf new file mode 100644 index 0000000..699cb73 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:df5554d503e9be0b905d259d3907892f2ebf357aacfdb6e61bd1aa9e71e0ff05 +size 17693154176 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q2_K.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q2_K.gguf new file mode 100644 index 0000000..b73070e --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1ccad892a1d6581d53c80b539acc665241dce424d241f327c8fbae6783e1f2b5 +size 12313099136 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_L.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_L.gguf new file mode 100644 index 0000000..53d10db --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b0321783f37d1fffc02d069e6683f66a0c1751acbcad47bfb1fe07a0c4b45f5 +size 17247079296 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_M.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_M.gguf new file mode 100644 index 0000000..10d0967 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:55d2f1035c34b843fa80095d8e0e33b48c1bffc2c85ae7cdb64a47f26146c3cd +size 15935048576 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_S.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_S.gguf new file mode 100644 index 0000000..360c0dc --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d08a54fc0478c4d354694ae34558a9b480853488d57af95b5dfb71e81ed84c65 +size 14392331136 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_0.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_0.gguf new file mode 100644 index 0000000..3d9a1dd --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0bb701973c0b6d9b031e7969574189d53b2eb2525e2a8ce62e61d7a64726a1f4 +size 18711010176 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_K_M.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_K_M.gguf new file mode 100644 index 0000000..9051996 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:86ac8efb86daf241792ac3d5d35b7da92c54901b4208a6f2829bd03d8f273c9c +size 19851336576 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_K_S.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_K_S.gguf new file mode 100644 index 0000000..2178822 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4df6d10a5b8f6ae4caea62f427f0981a56c5f1de321b0c6de2447e84f6432ed4 +size 18784410496 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q5_K_M.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q5_K_M.gguf new file mode 100644 index 0000000..4aca577 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7b0b226a19f3aa922ec4a188990435221bfbdb0585f25073d0ad3fffe6f5dafb +size 23262157696 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q5_K_S.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q5_K_S.gguf new file mode 100644 index 0000000..7d9bb06 --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d311040625deb5d1ff6a58889135c83e33ba68222e7fe567f1a07141fb0c285a +size 22638254976 diff --git a/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q6_K.gguf b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q6_K.gguf new file mode 100644 index 0000000..ea6356c --- /dev/null +++ b/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f9271b091643f1594add0f5cfb403f84561d72d1e5b067c680ff43ab0f672f7 +size 26886155136 diff --git a/README.md b/README.md new file mode 100644 index 0000000..e3d9d2c --- /dev/null +++ b/README.md @@ -0,0 +1,73 @@ +--- +base_model: thirdeyeai/Qwen2.5-Coder-32B-Instruct-Uncensored +language: +- en +library_name: transformers +quantized_by: mradermacher +tags: [] +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/thirdeyeai/Qwen2.5-Coder-32B-Instruct-Uncensored + + +static quants are available at https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ1_S.gguf) | i1-IQ1_S | 7.4 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ1_M.gguf) | i1-IQ1_M | 8.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 9.1 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_XS.gguf) | i1-IQ2_XS | 10.1 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_S.gguf) | i1-IQ2_S | 10.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ2_M.gguf) | i1-IQ2_M | 11.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q2_K.gguf) | i1-Q2_K | 12.4 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 12.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_XS.gguf) | i1-IQ3_XS | 13.8 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_S.gguf) | i1-Q3_K_S | 14.5 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_S.gguf) | i1-IQ3_S | 14.5 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ3_M.gguf) | i1-IQ3_M | 14.9 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_M.gguf) | i1-Q3_K_M | 16.0 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q3_K_L.gguf) | i1-Q3_K_L | 17.3 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-IQ4_XS.gguf) | i1-IQ4_XS | 17.8 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_0.gguf) | i1-Q4_0 | 18.8 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_K_S.gguf) | i1-Q4_K_S | 18.9 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q4_K_M.gguf) | i1-Q4_K_M | 20.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q5_K_S.gguf) | i1-Q5_K_S | 22.7 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q5_K_M.gguf) | i1-Q5_K_M | 23.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-32B-Instruct-Uncensored-i1-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Uncensored.i1-Q6_K.gguf) | i1-Q6_K | 27.0 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..406347d --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:564f36c54b1548bb6ca328655f076d2c4ea698afdce98438f457c5e347acdc30 +size 14957085