commit 10d5d47ed5742b7ec7fc93c4ed0376e22edd091b Author: ModelHub XC Date: Sun Oct 4 22:49:20 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Qwen2.5-Coder-3B-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..b4513b7 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-Coder-3B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen2.5-Coder-3B.i1-IQ1_M.gguf b/Qwen2.5-Coder-3B.i1-IQ1_M.gguf new file mode 100644 index 0000000..3b224c2 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:98fab3c7d6dd91d91517359d194c8a8c18e95a6a6c3bc5f7007978ff6bb2fcc6 +size 850028032 diff --git a/Qwen2.5-Coder-3B.i1-IQ1_S.gguf b/Qwen2.5-Coder-3B.i1-IQ1_S.gguf new file mode 100644 index 0000000..ab977a6 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:945505a6a0e930fd7354d6022040cbfa9884e2317386d2c1b16496e7137e6e14 +size 791094784 diff --git a/Qwen2.5-Coder-3B.i1-IQ2_M.gguf b/Qwen2.5-Coder-3B.i1-IQ2_M.gguf new file mode 100644 index 0000000..8b9e318 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f5e3e0da46438b04447253dbd2450aa5d7a465bf5b25ee08a1bd2241777e37dd +size 1140516352 diff --git a/Qwen2.5-Coder-3B.i1-IQ2_S.gguf b/Qwen2.5-Coder-3B.i1-IQ2_S.gguf new file mode 100644 index 0000000..dad3a20 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1d158fab1432eadac649d624697608c624b3f9f359c272c02f68a489995668fb +size 1061938688 diff --git a/Qwen2.5-Coder-3B.i1-IQ2_XS.gguf b/Qwen2.5-Coder-3B.i1-IQ2_XS.gguf new file mode 100644 index 0000000..28e959d --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e181132e510be1aa8fbbc0861791ca9234f1176753fba18dcab85bde72d8dfe7 +size 1031546368 diff --git a/Qwen2.5-Coder-3B.i1-IQ2_XXS.gguf b/Qwen2.5-Coder-3B.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..c945299 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fffc19ddaa696b5db91b702434d602e19ec718e82f87921a1a90619fc5521d79 +size 948250112 diff --git a/Qwen2.5-Coder-3B.i1-IQ3_M.gguf b/Qwen2.5-Coder-3B.i1-IQ3_M.gguf new file mode 100644 index 0000000..b69e775 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1bb036fc91bd49dd962b0abb099fb4b0375267845289f58631de560e928469d2 +size 1488895488 diff --git a/Qwen2.5-Coder-3B.i1-IQ3_S.gguf b/Qwen2.5-Coder-3B.i1-IQ3_S.gguf new file mode 100644 index 0000000..d2817b3 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9aea442f486d7773bcb976d289d975f9c051222954baa29f64860bffad75f916 +size 1456864768 diff --git a/Qwen2.5-Coder-3B.i1-IQ3_XS.gguf b/Qwen2.5-Coder-3B.i1-IQ3_XS.gguf new file mode 100644 index 0000000..99c959c --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:286ab48c2e6042f9b09176803d01a5aa45caeda08abf0ffdf108b01592321d83 +size 1391836672 diff --git a/Qwen2.5-Coder-3B.i1-IQ3_XXS.gguf b/Qwen2.5-Coder-3B.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..0723c14 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:289981b29f6d77626570e4056ed3ec34290f303605b46273372086d66bc33a1b +size 1282827776 diff --git a/Qwen2.5-Coder-3B.i1-IQ4_XS.gguf b/Qwen2.5-Coder-3B.i1-IQ4_XS.gguf new file mode 100644 index 0000000..4b8e647 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37488cc2759f4a5c61baffb9a80931380916a2df6f4cfdbe43acff386194133e +size 1739095552 diff --git a/Qwen2.5-Coder-3B.i1-Q2_K.gguf b/Qwen2.5-Coder-3B.i1-Q2_K.gguf new file mode 100644 index 0000000..a54b869 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dd02831bf444f930a3d00e5f73ccc6460c0959c7605cd09daffdbcb9f6770930 +size 1274756608 diff --git a/Qwen2.5-Coder-3B.i1-Q3_K_L.gguf b/Qwen2.5-Coder-3B.i1-Q3_K_L.gguf new file mode 100644 index 0000000..45fe6b0 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ee9c690fdb4e06d3855e4eee4ced673e17b5d2a65d87e57fcddde351e1f7731c +size 1707392512 diff --git a/Qwen2.5-Coder-3B.i1-Q3_K_M.gguf b/Qwen2.5-Coder-3B.i1-Q3_K_M.gguf new file mode 100644 index 0000000..f49e8c9 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d8fb00c775bc5ce0fb87e0a65e61aeed6a531a60b82236304cafecc9dfac7e4c +size 1590476288 diff --git a/Qwen2.5-Coder-3B.i1-Q3_K_S.gguf b/Qwen2.5-Coder-3B.i1-Q3_K_S.gguf new file mode 100644 index 0000000..ee52f59 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1564df74c448b4a1a8e86305b914ea79ec4b6fb7a3c18a510640a2164e93d974 +size 1454358016 diff --git a/Qwen2.5-Coder-3B.i1-Q4_0.gguf b/Qwen2.5-Coder-3B.i1-Q4_0.gguf new file mode 100644 index 0000000..025e7da --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e5c25a2fa8077da21a0ec34b198f830b023c0676f76e98a2828f2288ed248d4b +size 1828486656 diff --git a/Qwen2.5-Coder-3B.i1-Q4_0_4_4.gguf b/Qwen2.5-Coder-3B.i1-Q4_0_4_4.gguf new file mode 100644 index 0000000..7743572 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q4_0_4_4.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d039d4b3cc888d122e512d198e671fa68a4aa3224d7efa0fa8ddc4472ed43eb +size 1822850560 diff --git a/Qwen2.5-Coder-3B.i1-Q4_0_4_8.gguf b/Qwen2.5-Coder-3B.i1-Q4_0_4_8.gguf new file mode 100644 index 0000000..7ab96f7 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q4_0_4_8.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0bd482ffc5ca7b60d67c94324310a01f1b9e2b00a17f7d464551c1d569ee654e +size 1822850560 diff --git a/Qwen2.5-Coder-3B.i1-Q4_0_8_8.gguf b/Qwen2.5-Coder-3B.i1-Q4_0_8_8.gguf new file mode 100644 index 0000000..32a437d --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q4_0_8_8.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:996bd35368e19a99da0124ca845c4c2afe819ee9681f81d34b20962181c5bcf1 +size 1822850560 diff --git a/Qwen2.5-Coder-3B.i1-Q4_K_M.gguf b/Qwen2.5-Coder-3B.i1-Q4_K_M.gguf new file mode 100644 index 0000000..d05d82a --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2e8103b76e1cd085c575254e65b12d2459f5a890c76f5b96f9f6394c45c2cf10 +size 1929903616 diff --git a/Qwen2.5-Coder-3B.i1-Q4_K_S.gguf b/Qwen2.5-Coder-3B.i1-Q4_K_S.gguf new file mode 100644 index 0000000..8650153 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2b6b89a84476869ad1d85bf02a23e4f9c12a312816283d38fc9449b41cdadb6e +size 1834384896 diff --git a/Qwen2.5-Coder-3B.i1-Q5_K_M.gguf b/Qwen2.5-Coder-3B.i1-Q5_K_M.gguf new file mode 100644 index 0000000..268292a --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:987bd5105b4aba4f3509e1ee96d153be087b9637b06fd5fd40368576976aa69c +size 2224815616 diff --git a/Qwen2.5-Coder-3B.i1-Q5_K_S.gguf b/Qwen2.5-Coder-3B.i1-Q5_K_S.gguf new file mode 100644 index 0000000..92ff0fe --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:16714fadce96d8fd15ff399d6439c4fe726da8df48c5099bfbfb4fa96f29ebf2 +size 2169667072 diff --git a/Qwen2.5-Coder-3B.i1-Q6_K.gguf b/Qwen2.5-Coder-3B.i1-Q6_K.gguf new file mode 100644 index 0000000..2de17e1 --- /dev/null +++ b/Qwen2.5-Coder-3B.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2e24c77ee06b3b5e0cd4e5b7e8c92adf977dfe8edb9e15fd1469ce4af877b3d9 +size 2538159616 diff --git a/README.md b/README.md new file mode 100644 index 0000000..1a537d9 --- /dev/null +++ b/README.md @@ -0,0 +1,83 @@ +--- +base_model: Qwen/Qwen2.5-Coder-3B +language: +- en +library_name: transformers +license: other +license_link: https://huggingface.co/Qwen/Qwen2.5-Coder-3B/blob/main/LICENSE +license_name: qwen-research +quantized_by: mradermacher +tags: +- code +- qwen +- qwen-coder +- codeqwen +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/Qwen/Qwen2.5-Coder-3B + + +static quants are available at https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-IQ1_S.gguf) | i1-IQ1_S | 0.9 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-IQ1_M.gguf) | i1-IQ1_M | 1.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.0 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.1 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-IQ2_S.gguf) | i1-IQ2_S | 1.2 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-IQ2_M.gguf) | i1-IQ2_M | 1.2 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q2_K.gguf) | i1-Q2_K | 1.4 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.4 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 1.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-IQ3_S.gguf) | i1-IQ3_S | 1.6 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-IQ3_M.gguf) | i1-IQ3_M | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 1.7 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 1.8 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 1.8 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q4_0_4_4.gguf) | i1-Q4_0_4_4 | 1.9 | fast on arm, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q4_0_4_8.gguf) | i1-Q4_0_4_8 | 1.9 | fast on arm+i8mm, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q4_0_8_8.gguf) | i1-Q4_0_8_8 | 1.9 | fast on arm+sve, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q4_0.gguf) | i1-Q4_0 | 1.9 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 1.9 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.3 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 2.3 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-3B-i1-GGUF/resolve/main/Qwen2.5-Coder-3B.i1-Q6_K.gguf) | i1-Q6_K | 2.6 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..47cb35b --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9d329719f887c2739c83df7e30c8eabbef6bd2ad32a86f4a5974a75d791bbb27 +size 3362977