commit e3e90cc417fa32ff3004d4780279b1b18be51507 Author: ModelHub XC Date: Wed Oct 14 02:10:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..90d6f1c --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ1_M.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ1_M.gguf new file mode 100644 index 0000000..01d1bec --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4656ff7cd3c0f19aee44821073919a5c6ac37e47f5aa2fde9b79234de3326004 +size 3870231456 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ1_S.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ1_S.gguf new file mode 100644 index 0000000..cccd811 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cf60b36098215deb25ae235a67ba3db0ec6d90c6633f54819e9a440f06b9d656 +size 3605916576 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_M.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_M.gguf new file mode 100644 index 0000000..fd1e8c0 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2173a6b7f9c0625d8f973102f8ccd69f215b8615a3437d1c196423aa1ba0a76 +size 5353861408 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_S.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_S.gguf new file mode 100644 index 0000000..5079fe7 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4f93c43ae7b3fb80016d97163f386067ef10027b4bb99a798827f42668eaf773 +size 5001441568 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_XS.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_XS.gguf new file mode 100644 index 0000000..12cedb8 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2bc43090fa1f24a303576d09dac830e35e37ba6efa5a8b6f0d19f4223d62d0fa +size 4702497696 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_XXS.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..115ffc0 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a4f43498e13e98f7503791074696d6d5f26649acb93e93b3857941f65e940003 +size 4310756256 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_M.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_M.gguf new file mode 100644 index 0000000..e4f4ad3 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de33374fcb540bffbd136bb5bd2a69b5fcdd097a46b47333fd5b72d5d3eb33c2 +size 6913981792 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_S.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_S.gguf new file mode 100644 index 0000000..f710190 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:51082384e4126668f4d89147b09db10433e36d536a3a65b80b6fd822b9b18fd0 +size 6690463072 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_XS.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_XS.gguf new file mode 100644 index 0000000..f3b43f7 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:38fd14aef680625b2f1f4608c71030c879a4f86ce7b4d28a22688351d504e770 +size 6380805472 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_XXS.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..6d815c6 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2d758c252b7ae7731b209ea88077835a0295d2562d671908baa0a963b045166f +size 5944422688 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ4_NL.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ4_NL.gguf new file mode 100644 index 0000000..8cb9366 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:47f8ba369d2e02dc17f1cef199e9559fb3281c6c18cfe7a2ed1e44774b84172a +size 8546355584 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ4_XS.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ4_XS.gguf new file mode 100644 index 0000000..1d9c9a1 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8bd0f04943334e69d3accf0091bfc28fafcd227b1dff260711e7b517befe284 +size 8117076704 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q2_K.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q2_K.gguf new file mode 100644 index 0000000..4ea142f --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0de01bdb706cf2f1f54c450b0a3104cad4cf154be491d5c0323a1ac48cc42fe6 +size 5768148960 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q2_K_S.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q2_K_S.gguf new file mode 100644 index 0000000..80be5b5 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a87c75ea17e1a54ae8adae8d81624dfaecada91f50977065fcdf47fea102e9d0 +size 5394839520 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_L.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_L.gguf new file mode 100644 index 0000000..eb330c1 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a0bb159cb6aab8f1b41b83656d594ff8b485d979ef27c352f2bed47bd6b36cac +size 7922212192 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_M.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_M.gguf new file mode 100644 index 0000000..3c0cfeb --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8642c36e9f9211a2d118a521b967e662efa7216ebb5a440c6915f0f313b2f67 +size 7336648032 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_S.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_S.gguf new file mode 100644 index 0000000..b4c7eab --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6882c2c6b3166561d7cb295b2f213af456b7256c892cefae762aca8708b9dbf3 +size 6657039712 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_0.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_0.gguf new file mode 100644 index 0000000..39b1a85 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7d13713bb2bcb701476e040d1a28714f592a4db71a6b0e9541e6ca5aaf5c78a6 +size 8541440384 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_1.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_1.gguf new file mode 100644 index 0000000..6146102 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8c2f78d19f2b2830bf992416ee8e64c03af835c9f8d7ace1203a029f83fef03f +size 9389184704 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_K_M.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_K_M.gguf new file mode 100644 index 0000000..d9373b8 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e863a2ca3f9b71dd3fc7ac991e347ce129f6224efb10077cae0d9ed787a38998 +size 8985282944 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_K_S.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_K_S.gguf new file mode 100644 index 0000000..4869553 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:42047a69d2e301a48a09a2ddcd7738bed2625f9c6fa51815ee5f0fe7841274b7 +size 8570603904 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q5_K_M.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q5_K_M.gguf new file mode 100644 index 0000000..771d061 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59aa9c12b329937f1e90fe544f7f0926386ee0a99fcc55bc5b79405aec7a99f8 +size 10505790464 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q5_K_S.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q5_K_S.gguf new file mode 100644 index 0000000..346f413 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7d70891a990ddcfd2ede53627214d197ba9b7caca096198b5456a6b57b028afe +size 10263471104 diff --git a/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q6_K.gguf b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q6_K.gguf new file mode 100644 index 0000000..c4072e1 --- /dev/null +++ b/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4fba5aa27c61614da87cf588088bff2a3c0e5e3ad550d085b4549c5a88320874 +size 12121329728 diff --git a/README.md b/README.md new file mode 100644 index 0000000..67a7e51 --- /dev/null +++ b/README.md @@ -0,0 +1,84 @@ +--- +base_model: spacematt/Qwen2.5-Smooth-Coder-14B-Instruct +language: +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- mergekit +- merge +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/spacematt/Qwen2.5-Smooth-Coder-14B-Instruct + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ1_S.gguf) | i1-IQ1_S | 3.7 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ1_M.gguf) | i1-IQ1_M | 4.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 4.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.8 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_S.gguf) | i1-IQ2_S | 5.1 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ2_M.gguf) | i1-IQ2_M | 5.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q2_K_S.gguf) | i1-Q2_K_S | 5.5 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q2_K.gguf) | i1-Q2_K | 5.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 6.0 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_XS.gguf) | i1-IQ3_XS | 6.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_S.gguf) | i1-Q3_K_S | 6.8 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_S.gguf) | i1-IQ3_S | 6.8 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ3_M.gguf) | i1-IQ3_M | 7.0 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_M.gguf) | i1-Q3_K_M | 7.4 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q3_K_L.gguf) | i1-Q3_K_L | 8.0 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ4_XS.gguf) | i1-IQ4_XS | 8.2 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_0.gguf) | i1-Q4_0 | 8.6 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-IQ4_NL.gguf) | i1-IQ4_NL | 8.6 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_K_S.gguf) | i1-Q4_K_S | 8.7 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_K_M.gguf) | i1-Q4_K_M | 9.1 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q4_1.gguf) | i1-Q4_1 | 9.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q5_K_S.gguf) | i1-Q5_K_S | 10.4 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q5_K_M.gguf) | i1-Q5_K_M | 10.6 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Smooth-Coder-14B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Smooth-Coder-14B-Instruct.i1-Q6_K.gguf) | i1-Q6_K | 12.2 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..ba69dee --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:638280459bc885ae282c9a372ba6012c80a4344692484df87fd12a6c86cc9905 +size 8563597