commit eaa8aa2df73e99a6c53fa15b2c3b9929f097c9fd Author: ModelHub XC Date: Wed May 13 03:49:33 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Hermes-4-14B-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..c1e8b90 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Hermes-4-14B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Hermes-4-14B.i1-IQ1_M.gguf b/Hermes-4-14B.i1-IQ1_M.gguf new file mode 100644 index 0000000..c0ac2e8 --- /dev/null +++ b/Hermes-4-14B.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:16c374baed356ec2ee489237e295a8fb4e0799beb20993ef5a3507e8c6512748 +size 3849656672 diff --git a/Hermes-4-14B.i1-IQ1_S.gguf b/Hermes-4-14B.i1-IQ1_S.gguf new file mode 100644 index 0000000..f5e37d8 --- /dev/null +++ b/Hermes-4-14B.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:77c7e42a5b2008e240868d4b0e0cfedd3d565d0c87ef0458977896ff49d083d9 +size 3579935072 diff --git a/Hermes-4-14B.i1-IQ2_M.gguf b/Hermes-4-14B.i1-IQ2_M.gguf new file mode 100644 index 0000000..e549666 --- /dev/null +++ b/Hermes-4-14B.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eb08dc847b45246591570b0a3672a2777ca3eeda9caa303e64868a4798728963 +size 5322941792 diff --git a/Hermes-4-14B.i1-IQ2_S.gguf b/Hermes-4-14B.i1-IQ2_S.gguf new file mode 100644 index 0000000..7c70874 --- /dev/null +++ b/Hermes-4-14B.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:57e47de5c3781bcdc765bf002d0ae2993aeb3594dc216c97940c2c90bd44a81e +size 4963312992 diff --git a/Hermes-4-14B.i1-IQ2_XS.gguf b/Hermes-4-14B.i1-IQ2_XS.gguf new file mode 100644 index 0000000..f7a5851 --- /dev/null +++ b/Hermes-4-14B.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:60c3bff00d50b44ef3d624af5ed44c7a219eb956d7065141dd91257a41f289da +size 4691589472 diff --git a/Hermes-4-14B.i1-IQ2_XXS.gguf b/Hermes-4-14B.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..079b3fa --- /dev/null +++ b/Hermes-4-14B.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b448944572baf45b1e31b68a55ffa05bc01f404625476dc2988ba963f72f045d +size 4299192672 diff --git a/Hermes-4-14B.i1-IQ3_M.gguf b/Hermes-4-14B.i1-IQ3_M.gguf new file mode 100644 index 0000000..afd7f89 --- /dev/null +++ b/Hermes-4-14B.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:29277d0e5fc9316d9f6d12aba0993522729b3afd133c502d7e97f092d681873f +size 6883410272 diff --git a/Hermes-4-14B.i1-IQ3_S.gguf b/Hermes-4-14B.i1-IQ3_S.gguf new file mode 100644 index 0000000..c88ad1b --- /dev/null +++ b/Hermes-4-14B.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e8e0995f603e9c80f04049758f446e6b47302bb5a45f5c8c18365efe69882e2b +size 6684959072 diff --git a/Hermes-4-14B.i1-IQ3_XS.gguf b/Hermes-4-14B.i1-IQ3_XS.gguf new file mode 100644 index 0000000..d2eee45 --- /dev/null +++ b/Hermes-4-14B.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b9872aa4a8c1148789755d3024eb2848d50423b82424cd9a1e73b6f640df6b0b +size 6375301472 diff --git a/Hermes-4-14B.i1-IQ3_XXS.gguf b/Hermes-4-14B.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..e5f6b8e --- /dev/null +++ b/Hermes-4-14B.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ee3af2bb6c81965b4b7106c8ac7d83863208967f67f0eb83620136de89373c18 +size 5942666592 diff --git a/Hermes-4-14B.i1-IQ4_NL.gguf b/Hermes-4-14B.i1-IQ4_NL.gguf new file mode 100644 index 0000000..34b25cb --- /dev/null +++ b/Hermes-4-14B.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:48268096f3402a6789d8191de17b561c20f24a292b51d25daeef518df28b32b6 +size 8541363552 diff --git a/Hermes-4-14B.i1-IQ4_XS.gguf b/Hermes-4-14B.i1-IQ4_XS.gguf new file mode 100644 index 0000000..f191cf2 --- /dev/null +++ b/Hermes-4-14B.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2f2d22fc0ec839d098d0e68b4d0e6761e5d882c0983e967e0b48e100de365b39 +size 8110730592 diff --git a/Hermes-4-14B.i1-Q2_K.gguf b/Hermes-4-14B.i1-Q2_K.gguf new file mode 100644 index 0000000..8c64741 --- /dev/null +++ b/Hermes-4-14B.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d0b61697ea601a69ad4b5dc0972ab7a8f596a1289eba0c7eafa1087cefaa3359 +size 5753984352 diff --git a/Hermes-4-14B.i1-Q2_K_S.gguf b/Hermes-4-14B.i1-Q2_K_S.gguf new file mode 100644 index 0000000..e01096f --- /dev/null +++ b/Hermes-4-14B.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:deacb67c2d93cacb8cd4b0563967a11c464762d11a859888acd6d372c650c34b +size 5389849952 diff --git a/Hermes-4-14B.i1-Q3_K_L.gguf b/Hermes-4-14B.i1-Q3_K_L.gguf new file mode 100644 index 0000000..5bfaf05 --- /dev/null +++ b/Hermes-4-14B.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:669d941a6a217cfef3c31629782f1762f1fa40fc66d59ae8a3cde294b7e7c0b5 +size 7900651872 diff --git a/Hermes-4-14B.i1-Q3_K_M.gguf b/Hermes-4-14B.i1-Q3_K_M.gguf new file mode 100644 index 0000000..c6fc7f1 --- /dev/null +++ b/Hermes-4-14B.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1fb3f7e3eb0d72f14e08c051ab13f41ff9bb49bae077bcac9bc4a0e4db8e46e +size 7321313632 diff --git a/Hermes-4-14B.i1-Q3_K_S.gguf b/Hermes-4-14B.i1-Q3_K_S.gguf new file mode 100644 index 0000000..fa3c289 --- /dev/null +++ b/Hermes-4-14B.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8eceda9aee63610c1f4b3119685b3fbf12d92b90cd9f4200167b725dadcd79f3 +size 6657106272 diff --git a/Hermes-4-14B.i1-Q4_0.gguf b/Hermes-4-14B.i1-Q4_0.gguf new file mode 100644 index 0000000..06484c2 --- /dev/null +++ b/Hermes-4-14B.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d6bcc23ca150feaad833b18df21bfcdf6b6588b42faf22e371aeaaddfe12f59f +size 8543001952 diff --git a/Hermes-4-14B.i1-Q4_1.gguf b/Hermes-4-14B.i1-Q4_1.gguf new file mode 100644 index 0000000..de87f04 --- /dev/null +++ b/Hermes-4-14B.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e12342ff95962b022ecf341beab1a787185f8bf6e0cecb7bd9f3328e12ba0228 +size 9389522272 diff --git a/Hermes-4-14B.i1-Q4_K_M.gguf b/Hermes-4-14B.i1-Q4_K_M.gguf new file mode 100644 index 0000000..e62e169 --- /dev/null +++ b/Hermes-4-14B.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:09d2e53cd1131b6c1736864cf0d5faeb4b7d703b21abcb775abd0cfddabae272 +size 9001753952 diff --git a/Hermes-4-14B.i1-Q4_K_S.gguf b/Hermes-4-14B.i1-Q4_K_S.gguf new file mode 100644 index 0000000..81a4fca --- /dev/null +++ b/Hermes-4-14B.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e26fe7fc12ac2defc2e835dba1ff419a5d2ade207c0157f49dd7a63119ba30a4 +size 8573476192 diff --git a/Hermes-4-14B.i1-Q5_K_M.gguf b/Hermes-4-14B.i1-Q5_K_M.gguf new file mode 100644 index 0000000..10a3c26 --- /dev/null +++ b/Hermes-4-14B.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7f010aa83e58b6ebad9c7fa5987f5f1a861b51c757a727372c2bdd19f479c31d +size 10514570592 diff --git a/Hermes-4-14B.i1-Q5_K_S.gguf b/Hermes-4-14B.i1-Q5_K_S.gguf new file mode 100644 index 0000000..490b3d2 --- /dev/null +++ b/Hermes-4-14B.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:81cf6ddacbb4b906d1927f005bc276a4d8c044dd0ab915dff18c206596a15a5b +size 10263895392 diff --git a/Hermes-4-14B.i1-Q6_K.gguf b/Hermes-4-14B.i1-Q6_K.gguf new file mode 100644 index 0000000..fd48df8 --- /dev/null +++ b/Hermes-4-14B.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ce336b6d9074baaae0fa0f92b76e5723918457362e4a59fbcc3caf0532f7bf7 +size 12121938272 diff --git a/Hermes-4-14B.imatrix.gguf b/Hermes-4-14B.imatrix.gguf new file mode 100644 index 0000000..598cd2c --- /dev/null +++ b/Hermes-4-14B.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26c7b895432257ff0e547fc3178f7eb7bdb6eac293bde34457e20e39644911c4 +size 7743552 diff --git a/README.md b/README.md new file mode 100644 index 0000000..d22c0f4 --- /dev/null +++ b/README.md @@ -0,0 +1,101 @@ +--- +base_model: NousResearch/Hermes-4-14B +language: +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- Qwen-3-14B +- instruct +- finetune +- reasoning +- hybrid-mode +- chatml +- function calling +- tool use +- json mode +- structured outputs +- atropos +- dataforge +- long context +- roleplaying +- chat +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/NousResearch/Hermes-4-14B + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Hermes-4-14B-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Hermes-4-14B-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-IQ1_S.gguf) | i1-IQ1_S | 3.7 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-IQ1_M.gguf) | i1-IQ1_M | 3.9 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 4.4 | | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.8 | | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-IQ2_S.gguf) | i1-IQ2_S | 5.1 | | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-IQ2_M.gguf) | i1-IQ2_M | 5.4 | | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 5.5 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-Q2_K.gguf) | i1-Q2_K | 5.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 6.0 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 6.5 | | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 6.8 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-IQ3_S.gguf) | i1-IQ3_S | 6.8 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-IQ3_M.gguf) | i1-IQ3_M | 7.0 | | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 7.4 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 8.0 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 8.2 | | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 8.6 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-Q4_0.gguf) | i1-Q4_0 | 8.6 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 8.7 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 9.1 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-Q4_1.gguf) | i1-Q4_1 | 9.5 | | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 10.4 | | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 10.6 | | +| [GGUF](https://huggingface.co/mradermacher/Hermes-4-14B-i1-GGUF/resolve/main/Hermes-4-14B.i1-Q6_K.gguf) | i1-Q6_K | 12.2 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +