commit e0919b068def5806a8cb1625f7b59b40339cb563 Author: ModelHub XC Date: Fri Aug 14 03:03:16 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/LFM2.5-230M-heretic-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..a2bd0bb --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +LFM2.5-230M-heretic.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/LFM2.5-230M-heretic.i1-IQ1_M.gguf b/LFM2.5-230M-heretic.i1-IQ1_M.gguf new file mode 100644 index 0000000..d889fca --- /dev/null +++ b/LFM2.5-230M-heretic.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:25c7f9707d7d80ec5e961395d76df40261b6191e89bc8a316050d2cfebc11713 +size 85176096 diff --git a/LFM2.5-230M-heretic.i1-IQ1_S.gguf b/LFM2.5-230M-heretic.i1-IQ1_S.gguf new file mode 100644 index 0000000..658e480 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:155576998b017cda8a1f5f4394bf764e21fe31aeb52e424a498dacb85531a0e6 +size 81649440 diff --git a/LFM2.5-230M-heretic.i1-IQ2_M.gguf b/LFM2.5-230M-heretic.i1-IQ2_M.gguf new file mode 100644 index 0000000..fad96a8 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:27b16de614d9bd6db936cfa37cebedc216a9e86eee32cbe1f79e0032c19a355f +size 102125344 diff --git a/LFM2.5-230M-heretic.i1-IQ2_S.gguf b/LFM2.5-230M-heretic.i1-IQ2_S.gguf new file mode 100644 index 0000000..0885e11 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cec94f0791b4d0fabaa7db386577cb3e2b1c3ab9fec7c2c29585a6f044c50d8d +size 97423136 diff --git a/LFM2.5-230M-heretic.i1-IQ2_XS.gguf b/LFM2.5-230M-heretic.i1-IQ2_XS.gguf new file mode 100644 index 0000000..d605e34 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bafb3617da5e91f9f13a7b67fe6df9f481ec983a1d77e13ed51944ea1fd5f428 +size 95952672 diff --git a/LFM2.5-230M-heretic.i1-IQ2_XXS.gguf b/LFM2.5-230M-heretic.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..d031f28 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:339460651cd977daaddf702b8e6023c49890a28c217fcda633dc89584d671648 +size 91053856 diff --git a/LFM2.5-230M-heretic.i1-IQ3_M.gguf b/LFM2.5-230M-heretic.i1-IQ3_M.gguf new file mode 100644 index 0000000..a4ff4a6 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:028209652b3f38ee0b343c175160ac0bd918aab68ab35cebe695856323206cb3 +size 129097504 diff --git a/LFM2.5-230M-heretic.i1-IQ3_S.gguf b/LFM2.5-230M-heretic.i1-IQ3_S.gguf new file mode 100644 index 0000000..a34fd2a --- /dev/null +++ b/LFM2.5-230M-heretic.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2712aec052637356989f5585b463f4397b4a7ce65160eaae4bedd7477cd0731d +size 127495968 diff --git a/LFM2.5-230M-heretic.i1-IQ3_XS.gguf b/LFM2.5-230M-heretic.i1-IQ3_XS.gguf new file mode 100644 index 0000000..e54ed69 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4704e988532914b89549a687da4a14d995616338793f19e5aea93768cf8e58ce +size 124350240 diff --git a/LFM2.5-230M-heretic.i1-IQ3_XXS.gguf b/LFM2.5-230M-heretic.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..3c8ed8c --- /dev/null +++ b/LFM2.5-230M-heretic.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f82cb98f564d5b44786bdc2671b57b96587975659cbfc2d6240ac2e8016c15b9 +size 110669600 diff --git a/LFM2.5-230M-heretic.i1-IQ4_NL.gguf b/LFM2.5-230M-heretic.i1-IQ4_NL.gguf new file mode 100644 index 0000000..9adcb3c --- /dev/null +++ b/LFM2.5-230M-heretic.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5789c55f7cb1ecddf91229f7cd3bd0ac1518438036930ad313294e8942e19853 +size 149081888 diff --git a/LFM2.5-230M-heretic.i1-IQ4_XS.gguf b/LFM2.5-230M-heretic.i1-IQ4_XS.gguf new file mode 100644 index 0000000..ff7f7f9 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cbdb7f06bcc98c0bde74e51de52e6839863357b8ba3ef75ee26baa6b88f7d42e +size 144002848 diff --git a/LFM2.5-230M-heretic.i1-Q2_K.gguf b/LFM2.5-230M-heretic.i1-Q2_K.gguf new file mode 100644 index 0000000..93010f9 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5a3c03c140dd9ebb5d3fad18a2e93953394535a1165c56f4a910152ea9e502e4 +size 115674912 diff --git a/LFM2.5-230M-heretic.i1-Q2_K_S.gguf b/LFM2.5-230M-heretic.i1-Q2_K_S.gguf new file mode 100644 index 0000000..4afd819 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3e10830abd6dd6ea324eb6ee2d16220d645575337140b990ae8caf8637904a1 +size 111603488 diff --git a/LFM2.5-230M-heretic.i1-Q3_K_L.gguf b/LFM2.5-230M-heretic.i1-Q3_K_L.gguf new file mode 100644 index 0000000..a7d14ec --- /dev/null +++ b/LFM2.5-230M-heretic.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a1717b1a6105f54b1e9de3e271ca66660822c3930ca5dd56321f271e7c36ae8f +size 139390752 diff --git a/LFM2.5-230M-heretic.i1-Q3_K_M.gguf b/LFM2.5-230M-heretic.i1-Q3_K_M.gguf new file mode 100644 index 0000000..ab52c01 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e1261065e043fee038adc53e5b6a9a13c1a4cca96453895363a78102f1ba3849 +size 133754656 diff --git a/LFM2.5-230M-heretic.i1-Q3_K_S.gguf b/LFM2.5-230M-heretic.i1-Q3_K_S.gguf new file mode 100644 index 0000000..2a7051f --- /dev/null +++ b/LFM2.5-230M-heretic.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a8c612c3d64b0377f8a133bc1606d500b80b316bb9bb088677995bda10dd7484 +size 127495968 diff --git a/LFM2.5-230M-heretic.i1-Q4_0.gguf b/LFM2.5-230M-heretic.i1-Q4_0.gguf new file mode 100644 index 0000000..acbb2d9 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0f17fb1183b9cae90fb6985dc846081f53ac1eae56c024567484bba97058105f +size 149245728 diff --git a/LFM2.5-230M-heretic.i1-Q4_1.gguf b/LFM2.5-230M-heretic.i1-Q4_1.gguf new file mode 100644 index 0000000..57f3f85 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:995707d7ee8dfa3ba8b850c769de50570aec0653a33a718e6cc213b74b0b29ef +size 159239968 diff --git a/LFM2.5-230M-heretic.i1-Q4_K_M.gguf b/LFM2.5-230M-heretic.i1-Q4_K_M.gguf new file mode 100644 index 0000000..87d75e9 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:79416020f8474df8a360ba495914fa87c1f38b125446133e3319d4c2982c525f +size 153407264 diff --git a/LFM2.5-230M-heretic.i1-Q4_K_S.gguf b/LFM2.5-230M-heretic.i1-Q4_K_S.gguf new file mode 100644 index 0000000..eaac8f6 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e041bed9ad99e07c000e6f0722d37e1e548cc3641db2f9fde58b85f4e66bb8c2 +size 149671712 diff --git a/LFM2.5-230M-heretic.i1-Q5_K_M.gguf b/LFM2.5-230M-heretic.i1-Q5_K_M.gguf new file mode 100644 index 0000000..0330302 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd28277dffde67796ac55c6488e531f915ac8a15ea45a7dee2471172f98e8ab1 +size 171626272 diff --git a/LFM2.5-230M-heretic.i1-Q5_K_S.gguf b/LFM2.5-230M-heretic.i1-Q5_K_S.gguf new file mode 100644 index 0000000..82b3329 --- /dev/null +++ b/LFM2.5-230M-heretic.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f6e7b1634a5f4d8588989f2d505534e4021ace8ef8b4f23af7361aef8dcdb628 +size 169398048 diff --git a/LFM2.5-230M-heretic.i1-Q6_K.gguf b/LFM2.5-230M-heretic.i1-Q6_K.gguf new file mode 100644 index 0000000..18d947e --- /dev/null +++ b/LFM2.5-230M-heretic.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b2d949e259be3f0b1eba9f64ba4fa85db3e85064e488d7ed12404868795cc2c +size 190983968 diff --git a/LFM2.5-230M-heretic.imatrix.gguf b/LFM2.5-230M-heretic.imatrix.gguf new file mode 100644 index 0000000..70004f0 --- /dev/null +++ b/LFM2.5-230M-heretic.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:992c2e53a06f51b19257c97eb523cd2bb25941c71bae7592830370c5bbe7db87 +size 434912 diff --git a/README.md b/README.md new file mode 100644 index 0000000..9a22941 --- /dev/null +++ b/README.md @@ -0,0 +1,104 @@ +--- +base_model: Dingdust/LFM2.5-230M-heretic +language: +- en +- ar +- zh +- fr +- de +- ja +- ko +- es +- pt +- it +library_name: transformers +license: other +license_link: LICENSE +license_name: lfm1.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- liquid +- lfm2.5 +- edge +- heretic +- uncensored +- decensored +- abliterated +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/Dingdust/LFM2.5-230M-heretic + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#LFM2.5-230M-heretic-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/LFM2.5-230M-heretic-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-IQ1_S.gguf) | i1-IQ1_S | 0.2 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-IQ1_M.gguf) | i1-IQ1_M | 0.2 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 0.2 | | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-IQ2_XS.gguf) | i1-IQ2_XS | 0.2 | | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-IQ2_S.gguf) | i1-IQ2_S | 0.2 | | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-IQ2_M.gguf) | i1-IQ2_M | 0.2 | | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 0.2 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-Q2_K_S.gguf) | i1-Q2_K_S | 0.2 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-Q2_K.gguf) | i1-Q2_K | 0.2 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-IQ3_XS.gguf) | i1-IQ3_XS | 0.2 | | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-IQ3_S.gguf) | i1-IQ3_S | 0.2 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-Q3_K_S.gguf) | i1-Q3_K_S | 0.2 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-IQ3_M.gguf) | i1-IQ3_M | 0.2 | | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-Q3_K_M.gguf) | i1-Q3_K_M | 0.2 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-Q3_K_L.gguf) | i1-Q3_K_L | 0.2 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-IQ4_XS.gguf) | i1-IQ4_XS | 0.2 | | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-IQ4_NL.gguf) | i1-IQ4_NL | 0.2 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-Q4_0.gguf) | i1-Q4_0 | 0.2 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-Q4_K_S.gguf) | i1-Q4_K_S | 0.2 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-Q4_K_M.gguf) | i1-Q4_K_M | 0.3 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-Q4_1.gguf) | i1-Q4_1 | 0.3 | | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-Q5_K_S.gguf) | i1-Q5_K_S | 0.3 | | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-Q5_K_M.gguf) | i1-Q5_K_M | 0.3 | | +| [GGUF](https://huggingface.co/mradermacher/LFM2.5-230M-heretic-i1-GGUF/resolve/main/LFM2.5-230M-heretic.i1-Q6_K.gguf) | i1-Q6_K | 0.3 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +