commit a8336f7dfd9b114618516ee329daea75f740d1f0 Author: ModelHub XC Date: Wed Oct 7 06:27:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/L3.1-Haggardv1-12B-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..876e008 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +L3.1-Haggardv1-12B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/L3.1-Haggardv1-12B.i1-IQ1_M.gguf b/L3.1-Haggardv1-12B.i1-IQ1_M.gguf new file mode 100644 index 0000000..11d9005 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:783b6b4195870213f9c3c381d5e11eb939b9f0d861dfa0108ccfe7e10a00e941 +size 3071949888 diff --git a/L3.1-Haggardv1-12B.i1-IQ1_S.gguf b/L3.1-Haggardv1-12B.i1-IQ1_S.gguf new file mode 100644 index 0000000..2195bde --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:072ece6f8d4712e93b2091bba85ba7c87c466f19a034747967d9cad9f357a718 +size 2849193024 diff --git a/L3.1-Haggardv1-12B.i1-IQ2_M.gguf b/L3.1-Haggardv1-12B.i1-IQ2_M.gguf new file mode 100644 index 0000000..59287c5 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59ec9ce7cf738700e1d12ddd74166b79b8e32c211f7500fccf5227959973f068 +size 4270545984 diff --git a/L3.1-Haggardv1-12B.i1-IQ2_S.gguf b/L3.1-Haggardv1-12B.i1-IQ2_S.gguf new file mode 100644 index 0000000..9b6b49c --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:750db0ddcc3f18189133f74ab7f0829cf9eb1b8e0346941158d045a482661c99 +size 3973536832 diff --git a/L3.1-Haggardv1-12B.i1-IQ2_XS.gguf b/L3.1-Haggardv1-12B.i1-IQ2_XS.gguf new file mode 100644 index 0000000..660e859 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f97f09170b994411a64ad024b7d5f15867f2be5e3478340c7efba95509d01d48 +size 3766434880 diff --git a/L3.1-Haggardv1-12B.i1-IQ2_XXS.gguf b/L3.1-Haggardv1-12B.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..b1560ee --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b899fd88ca2ed5ff8c5c5ddd1a79f490e9b3a9ec3c21038339a4a954e4562a9 +size 3443211328 diff --git a/L3.1-Haggardv1-12B.i1-IQ3_M.gguf b/L3.1-Haggardv1-12B.i1-IQ3_M.gguf new file mode 100644 index 0000000..ee65146 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:339da0727c78442a34dcac68d1308d4482916f9f9a64c9a5e17fa0534265e8f7 +size 5538053184 diff --git a/L3.1-Haggardv1-12B.i1-IQ3_S.gguf b/L3.1-Haggardv1-12B.i1-IQ3_S.gguf new file mode 100644 index 0000000..71bd1f0 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4e5c996a7fbc2d66f3f129b3bde269c4c502e3e593eb2e3bab1d4a92ae920a38 +size 5379849280 diff --git a/L3.1-Haggardv1-12B.i1-IQ3_XS.gguf b/L3.1-Haggardv1-12B.i1-IQ3_XS.gguf new file mode 100644 index 0000000..56b9474 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0992e66a4c1395b099b2d0521d1dd0551849a238e01065cc122cbf01e609191d +size 5127011392 diff --git a/L3.1-Haggardv1-12B.i1-IQ3_XXS.gguf b/L3.1-Haggardv1-12B.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..1dfe29c --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:76bc3a4e2c1f722c566b3647989b12e6832a6d9898bdc5528a78e3c1095a1433 +size 4782513216 diff --git a/L3.1-Haggardv1-12B.i1-IQ4_NL.gguf b/L3.1-Haggardv1-12B.i1-IQ4_NL.gguf new file mode 100644 index 0000000..edfb854 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4fa7a597140991b029aace823122a945ecb481327ec23df812750f47231d2d83 +size 6896327744 diff --git a/L3.1-Haggardv1-12B.i1-IQ4_XS.gguf b/L3.1-Haggardv1-12B.i1-IQ4_XS.gguf new file mode 100644 index 0000000..8ffb3b8 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7b987b9daf0bb40b13cbb8141b643132f06bd3a462d287bbe132e15c20ff4d17 +size 6545677376 diff --git a/L3.1-Haggardv1-12B.i1-Q2_K.gguf b/L3.1-Haggardv1-12B.i1-Q2_K.gguf new file mode 100644 index 0000000..42ca740 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:15086db7fee6bd85957d4452dc6fd866a10d3c04d3b105eab8e2f98389cf3c5d +size 4623621184 diff --git a/L3.1-Haggardv1-12B.i1-Q2_K_S.gguf b/L3.1-Haggardv1-12B.i1-Q2_K_S.gguf new file mode 100644 index 0000000..3581a77 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26f6054f3056b58ca79131d6f06ace9c623df8fa137b2944214a049cc7b49c92 +size 4322810944 diff --git a/L3.1-Haggardv1-12B.i1-Q3_K_L.gguf b/L3.1-Haggardv1-12B.i1-Q3_K_L.gguf new file mode 100644 index 0000000..047c477 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6ff4c3834b5af96c213298312f207b4d06129dfcb3156de4458ad5607022295b +size 6379273280 diff --git a/L3.1-Haggardv1-12B.i1-Q3_K_M.gguf b/L3.1-Haggardv1-12B.i1-Q3_K_M.gguf new file mode 100644 index 0000000..9dd8dfe --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:08a7eca15bb96f6aa68e16aabc550b46aedebd934e2f50f02c0d73941137995d +size 5904268352 diff --git a/L3.1-Haggardv1-12B.i1-Q3_K_S.gguf b/L3.1-Haggardv1-12B.i1-Q3_K_S.gguf new file mode 100644 index 0000000..600164b --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ac228674c3facddca01e7a6cebc36d7bbaf5b6de4552194c3374e7fcd11b5a0 +size 5351996480 diff --git a/L3.1-Haggardv1-12B.i1-Q4_0.gguf b/L3.1-Haggardv1-12B.i1-Q4_0.gguf new file mode 100644 index 0000000..c7020b4 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d4dec66346b49bcff44cedfe63e8c73d9dddac5cfcb909a847d4ab6ede48168 +size 6892133440 diff --git a/L3.1-Haggardv1-12B.i1-Q4_1.gguf b/L3.1-Haggardv1-12B.i1-Q4_1.gguf new file mode 100644 index 0000000..3ba6d9c --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ac4083e296f0c228770a89879a01de17db5d4969559ee490cd909662b95ba046 +size 7584521280 diff --git a/L3.1-Haggardv1-12B.i1-Q4_K_M.gguf b/L3.1-Haggardv1-12B.i1-Q4_K_M.gguf new file mode 100644 index 0000000..aa1b8d9 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:23e31e9808e5afbc8507727b26690f13a4a51dba981c49e45db1cffb1b5356f4 +size 7275617344 diff --git a/L3.1-Haggardv1-12B.i1-Q4_K_S.gguf b/L3.1-Haggardv1-12B.i1-Q4_K_S.gguf new file mode 100644 index 0000000..be92ef1 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eca453827fa7cb67c498ead75f6755c1548cb9d1c8aad7db89bfe81e5d508f07 +size 6916250688 diff --git a/L3.1-Haggardv1-12B.i1-Q5_K_M.gguf b/L3.1-Haggardv1-12B.i1-Q5_K_M.gguf new file mode 100644 index 0000000..55cec5a --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ac98d2cc8207c1509c62339188611b5325fdad36c4ec0acf37bd8f2e997add6d +size 8507825216 diff --git a/L3.1-Haggardv1-12B.i1-Q5_K_S.gguf b/L3.1-Haggardv1-12B.i1-Q5_K_S.gguf new file mode 100644 index 0000000..eed6fdd --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1ca6d169dea14655127af817a98db7ab1441337e56485164d0877835432fbe8 +size 8298929216 diff --git a/L3.1-Haggardv1-12B.i1-Q6_K.gguf b/L3.1-Haggardv1-12B.i1-Q6_K.gguf new file mode 100644 index 0000000..08aab73 --- /dev/null +++ b/L3.1-Haggardv1-12B.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:594fac6f23ea3ed700dc0cc1430c3d4067214c28a8f4510af5457b4e8db77fed +size 9817046080 diff --git a/L3.1-Haggardv1-12B.imatrix.gguf b/L3.1-Haggardv1-12B.imatrix.gguf new file mode 100644 index 0000000..b8136f5 --- /dev/null +++ b/L3.1-Haggardv1-12B.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9de68413d48670be8e5cdc72d827ac56d7b6fcddb7ef6302d65bbd1e7aaceb21 +size 7836224 diff --git a/README.md b/README.md new file mode 100644 index 0000000..223db21 --- /dev/null +++ b/README.md @@ -0,0 +1,87 @@ +--- +base_model: kromcomp/L3.1-Haggardv1-12B +language: +- en +library_name: transformers +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- mergekit +- merge +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/kromcomp/L3.1-Haggardv1-12B + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#L3.1-Haggardv1-12B-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-IQ1_S.gguf) | i1-IQ1_S | 2.9 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-IQ1_M.gguf) | i1-IQ1_M | 3.2 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 3.5 | | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 3.9 | | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-IQ2_S.gguf) | i1-IQ2_S | 4.1 | | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-IQ2_M.gguf) | i1-IQ2_M | 4.4 | | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 4.4 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-Q2_K.gguf) | i1-Q2_K | 4.7 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 4.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 5.2 | | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 5.5 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-IQ3_S.gguf) | i1-IQ3_S | 5.5 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-IQ3_M.gguf) | i1-IQ3_M | 5.6 | | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 6.0 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 6.5 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 6.6 | | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-Q4_0.gguf) | i1-Q4_0 | 7.0 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 7.0 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 7.0 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 7.4 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-Q4_1.gguf) | i1-Q4_1 | 7.7 | | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 8.4 | | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 8.6 | | +| [GGUF](https://huggingface.co/mradermacher/L3.1-Haggardv1-12B-i1-GGUF/resolve/main/L3.1-Haggardv1-12B.i1-Q6_K.gguf) | i1-Q6_K | 9.9 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +