commit 46517f5f4ff441ce98cac15ed58be417d3e1965c Author: ModelHub XC Date: Mon May 4 12:36:27 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..172992d --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +DeepSeek-R1-14B-Finetuned.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/DeepSeek-R1-14B-Finetuned.i1-IQ1_M.gguf b/DeepSeek-R1-14B-Finetuned.i1-IQ1_M.gguf new file mode 100644 index 0000000..edd2acd --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:610b86e94e7cf5dea12a71f7c77d8477222b7fc65373517e9cf5627b4f5e47ef +size 3872309664 diff --git a/DeepSeek-R1-14B-Finetuned.i1-IQ1_S.gguf b/DeepSeek-R1-14B-Finetuned.i1-IQ1_S.gguf new file mode 100644 index 0000000..6420a6b --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:627ab7d2115f9c56a8d61903d3eae449263dd0e994d203163459a2f696a205bb +size 3607994784 diff --git a/DeepSeek-R1-14B-Finetuned.i1-IQ2_M.gguf b/DeepSeek-R1-14B-Finetuned.i1-IQ2_M.gguf new file mode 100644 index 0000000..9c06de0 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:537f74d0b86663e7449ccb759783be80845c3af23a8e14d5d4588c79dfa1afd8 +size 5356147104 diff --git a/DeepSeek-R1-14B-Finetuned.i1-IQ2_S.gguf b/DeepSeek-R1-14B-Finetuned.i1-IQ2_S.gguf new file mode 100644 index 0000000..11f34d4 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fe95b4a1a85365b99d5b3a992624bed8b7acd5d2e5dbf6e8f48de22f6b524391 +size 5003727264 diff --git a/DeepSeek-R1-14B-Finetuned.i1-IQ2_XS.gguf b/DeepSeek-R1-14B-Finetuned.i1-IQ2_XS.gguf new file mode 100644 index 0000000..4a546e1 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:25200ff775f28d2433c995c8711b05a4aef05e241ca39062d7e2830cc252a64a +size 4704575904 diff --git a/DeepSeek-R1-14B-Finetuned.i1-IQ2_XXS.gguf b/DeepSeek-R1-14B-Finetuned.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..bacbb52 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c71e8c823bcac4a0e7b322b530bfdd93bcdba71df59171aa00c64085d77cff08 +size 4312834464 diff --git a/DeepSeek-R1-14B-Finetuned.i1-IQ3_M.gguf b/DeepSeek-R1-14B-Finetuned.i1-IQ3_M.gguf new file mode 100644 index 0000000..dc1c0e1 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9257447c97933edc7303c42bf96bd79439986cde3361c56f73d5d3951749436b +size 6916538784 diff --git a/DeepSeek-R1-14B-Finetuned.i1-IQ3_S.gguf b/DeepSeek-R1-14B-Finetuned.i1-IQ3_S.gguf new file mode 100644 index 0000000..8fa2d2d --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0eb7d9b3b986146eb14eb537e1a50870ab197e5d16cec3146ec444bab9cf02ad +size 6693020064 diff --git a/DeepSeek-R1-14B-Finetuned.i1-IQ3_XS.gguf b/DeepSeek-R1-14B-Finetuned.i1-IQ3_XS.gguf new file mode 100644 index 0000000..98eb164 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f5cf43bf40d1ededd853db65933cde69febe172caed04748f269a34a0b288106 +size 6383362464 diff --git a/DeepSeek-R1-14B-Finetuned.i1-IQ3_XXS.gguf b/DeepSeek-R1-14B-Finetuned.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..26eb520 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b3c4a21e6ca25baeded4d62b93f4b4d3a3c8349f8635ae44bf97ff62f91c8014 +size 5946708384 diff --git a/DeepSeek-R1-14B-Finetuned.i1-IQ4_NL.gguf b/DeepSeek-R1-14B-Finetuned.i1-IQ4_NL.gguf new file mode 100644 index 0000000..66e262c --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9288e04d90e1142b1186caebf4091df431e7adf6e97dbe808a339cea75ab8074 +size 8549183904 diff --git a/DeepSeek-R1-14B-Finetuned.i1-IQ4_XS.gguf b/DeepSeek-R1-14B-Finetuned.i1-IQ4_XS.gguf new file mode 100644 index 0000000..e4d1464 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4ff3c7dfa5e921c847d835798a078075e5ccec555c9c4c3eb20a7f6fb7b35d13 +size 8119841184 diff --git a/DeepSeek-R1-14B-Finetuned.i1-Q2_K.gguf b/DeepSeek-R1-14B-Finetuned.i1-Q2_K.gguf new file mode 100644 index 0000000..07e2ff0 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f04b9ac9d4cc0d948e4632f9af06b57fe2feb3e568020f8488b0bc9947de3557 +size 5770498464 diff --git a/DeepSeek-R1-14B-Finetuned.i1-Q2_K_S.gguf b/DeepSeek-R1-14B-Finetuned.i1-Q2_K_S.gguf new file mode 100644 index 0000000..d2a06d8 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b0a1934589a041f4ca326504ae50878df3947faaa056f84a369036f76e7ee165 +size 5397189024 diff --git a/DeepSeek-R1-14B-Finetuned.i1-Q3_K_L.gguf b/DeepSeek-R1-14B-Finetuned.i1-Q3_K_L.gguf new file mode 100644 index 0000000..8ffdeec --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e4c18749f85ef3a15230ce7d07513f685b37ba0841b403c49d95327dbba7fe99 +size 7924769184 diff --git a/DeepSeek-R1-14B-Finetuned.i1-Q3_K_M.gguf b/DeepSeek-R1-14B-Finetuned.i1-Q3_K_M.gguf new file mode 100644 index 0000000..a2f1dd6 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bf9cf67064eb9642385482aa126019b0fcc6f5ea17b115e3caf64ce72b6ee0bb +size 7339205024 diff --git a/DeepSeek-R1-14B-Finetuned.i1-Q3_K_S.gguf b/DeepSeek-R1-14B-Finetuned.i1-Q3_K_S.gguf new file mode 100644 index 0000000..dd6cf9f --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eb3311e5547a42438146bcd173fbca7b16de298017224851b1622f44d2a8c2f8 +size 6659596704 diff --git a/DeepSeek-R1-14B-Finetuned.i1-Q4_0.gguf b/DeepSeek-R1-14B-Finetuned.i1-Q4_0.gguf new file mode 100644 index 0000000..c0d57dc --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d869eb75c3d0235c269fe623175f4c60c692e7709f89d6b6c289fee867202f18 +size 8544268704 diff --git a/DeepSeek-R1-14B-Finetuned.i1-Q4_1.gguf b/DeepSeek-R1-14B-Finetuned.i1-Q4_1.gguf new file mode 100644 index 0000000..476de78 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:657348edc517947d7b8cb88207d182f53e84443cd4c802c77c83d1ed9e91a377 +size 9392140704 diff --git a/DeepSeek-R1-14B-Finetuned.i1-Q4_K_M.gguf b/DeepSeek-R1-14B-Finetuned.i1-Q4_K_M.gguf new file mode 100644 index 0000000..1025aea --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:47d96e35d84151e81e9d4d142fe775bab96ce68e695b33464d4c5640b2655088 +size 8988111264 diff --git a/DeepSeek-R1-14B-Finetuned.i1-Q4_K_S.gguf b/DeepSeek-R1-14B-Finetuned.i1-Q4_K_S.gguf new file mode 100644 index 0000000..183f5d2 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7325426aaaae3bbec435ec94e12d76d5399dfbf57ac6078f83ef8163875142b2 +size 8573432224 diff --git a/DeepSeek-R1-14B-Finetuned.i1-Q5_K_M.gguf b/DeepSeek-R1-14B-Finetuned.i1-Q5_K_M.gguf new file mode 100644 index 0000000..5132ac5 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0ecd04efff1d48a933b51b4093ad119b33b9f6b49891772342e1c060367fd994 +size 10508874144 diff --git a/DeepSeek-R1-14B-Finetuned.i1-Q5_K_S.gguf b/DeepSeek-R1-14B-Finetuned.i1-Q5_K_S.gguf new file mode 100644 index 0000000..56af11a --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cfd89c93837b7459516050b9a4a9cf0ae55b90e5380554da4efa2ccc9209f426 +size 10266554784 diff --git a/DeepSeek-R1-14B-Finetuned.i1-Q6_K.gguf b/DeepSeek-R1-14B-Finetuned.i1-Q6_K.gguf new file mode 100644 index 0000000..e5fcee0 --- /dev/null +++ b/DeepSeek-R1-14B-Finetuned.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:54a9c318c24df997740f24ab69c438f40919337a4e4b0a0567da9e599e154160 +size 12124684704 diff --git a/README.md b/README.md new file mode 100644 index 0000000..2a3faee --- /dev/null +++ b/README.md @@ -0,0 +1,83 @@ +--- +base_model: mathdong01/DeepSeek-R1-14B-Finetuned +language: +- en +library_name: transformers +license: apache-2.0 +quantized_by: mradermacher +tags: +- text-generation-inference +- transformers +- unsloth +- qwen2 +- trl +- sft +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/mathdong01/DeepSeek-R1-14B-Finetuned + + +static quants are available at https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-IQ1_S.gguf) | i1-IQ1_S | 3.7 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-IQ1_M.gguf) | i1-IQ1_M | 4.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 4.4 | | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.8 | | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-IQ2_S.gguf) | i1-IQ2_S | 5.1 | | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-IQ2_M.gguf) | i1-IQ2_M | 5.5 | | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-Q2_K_S.gguf) | i1-Q2_K_S | 5.5 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-Q2_K.gguf) | i1-Q2_K | 5.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 6.0 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-IQ3_XS.gguf) | i1-IQ3_XS | 6.5 | | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-Q3_K_S.gguf) | i1-Q3_K_S | 6.8 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-IQ3_S.gguf) | i1-IQ3_S | 6.8 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-IQ3_M.gguf) | i1-IQ3_M | 7.0 | | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-Q3_K_M.gguf) | i1-Q3_K_M | 7.4 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-Q3_K_L.gguf) | i1-Q3_K_L | 8.0 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-IQ4_XS.gguf) | i1-IQ4_XS | 8.2 | | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-Q4_0.gguf) | i1-Q4_0 | 8.6 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-IQ4_NL.gguf) | i1-IQ4_NL | 8.6 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-Q4_K_S.gguf) | i1-Q4_K_S | 8.7 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-Q4_K_M.gguf) | i1-Q4_K_M | 9.1 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-Q4_1.gguf) | i1-Q4_1 | 9.5 | | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-Q5_K_S.gguf) | i1-Q5_K_S | 10.4 | | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-Q5_K_M.gguf) | i1-Q5_K_M | 10.6 | | +| [GGUF](https://huggingface.co/mradermacher/DeepSeek-R1-14B-Finetuned-i1-GGUF/resolve/main/DeepSeek-R1-14B-Finetuned.i1-Q6_K.gguf) | i1-Q6_K | 12.2 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..5079e57 --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8056900aade2c17067a5a952d71c4a8abb1d73ab3d3c7d0d3baf552a4a57ea6f +size 8563597