commit f731f143cdbab1fda7b18db391bca9c850482e31 Author: ModelHub XC Date: Sat May 16 01:58:30 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..c939dc4 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,59 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-IQ1_M.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ1_M.gguf new file mode 100644 index 0000000..3e3abbb --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:09d207dae02c41db38bbf504fe2a0177a350fc8229e1c4c8dd7fd623f811419e +size 5377872640 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-IQ1_S.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ1_S.gguf new file mode 100644 index 0000000..6041466 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2e5239a7612db10b47dcce244b07a9a811d88eda4322defdb96763da2ee940f +size 4838822656 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_M.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_M.gguf new file mode 100644 index 0000000..06378d0 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f0b288958ba20e207c2a0987044ce1ebd8425b96573678df7b7d5b4646c4a83a +size 7787205376 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_S.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_S.gguf new file mode 100644 index 0000000..2d92d3f --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:16fbb41b73a0755bf696ca212c512a4cfc25d2c713dfe11781c9bad890b41cc8 +size 7068472064 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_XS.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_XS.gguf new file mode 100644 index 0000000..216355b --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1de67335d895bd582c19dde86833e9d23a7a86fa0a864bb287703b95be6b67d1 +size 6996333312 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_XXS.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..89675a0 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9cf1c2b05521620e528812ca653e2e51d1d0a4c244596b736425a26d0398ab10 +size 6276289280 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_M.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_M.gguf new file mode 100644 index 0000000..32ff9a5 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c47fd55c57483a0c39817a2b214a999442a1995309f31afb4b665397b3214a5a +size 10412790528 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_S.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_S.gguf new file mode 100644 index 0000000..845bf1e --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bf10aa36f18783b2f87388c744cd1bc4c72c9556278ee84a074640ad00df2730 +size 10319205120 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_XS.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_XS.gguf new file mode 100644 index 0000000..8df3fd9 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c6bb83718d17cde74dc6e42afd668901adeab7428a0fa8bb1c1c79e799978475 +size 9750516480 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_XXS.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..3d052ea --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:76086d591731dfed6f195bee4edff853de048d5b36696ee0d06a74db0efb167f +size 9188938496 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-IQ4_XS.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ4_XS.gguf new file mode 100644 index 0000000..0b0ae6c --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1ad68cd1685d52a641f00806ab4e17f351164f7825b5ac0503cc45c64971c01 +size 12726653696 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-Q2_K.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-Q2_K.gguf new file mode 100644 index 0000000..2af4c61 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:68673912716e449093505f56f1de2e32a506e3e80a7b0cf4d9536a78698e1384 +size 8698974976 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-Q2_K_S.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-Q2_K_S.gguf new file mode 100644 index 0000000..0815db4 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f715b24818e2fbcea74951d7e271cd449c3e4324392e2b7e774806471a1593b6 +size 8064619264 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_L.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_L.gguf new file mode 100644 index 0000000..f2b2354 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ad0b84ee99a18afe5a1002c99902aa49b5dca02b6ea98bc1213b834a0a787f91 +size 12317528832 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_M.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_M.gguf new file mode 100644 index 0000000..d1ff2c0 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:43d9699dc15c6c35301ed57e83e7c97e5701579a63e2e19e541cebe2f37b52e3 +size 11354936064 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_S.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_S.gguf new file mode 100644 index 0000000..69e7706 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ba344cc769b50925910fa15b3515811a6cf034a491df20051c6e08530f0e6a3c +size 10319205120 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_0.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_0.gguf new file mode 100644 index 0000000..8a4943f --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1ccf58e9fc19cb4d2596c54862fdbe92d3851140c56219bc3d0b870356eba2a8 +size 13508170496 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_1.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_1.gguf new file mode 100644 index 0000000..2647739 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:30dde28f7b85a2ba49b8a0141ad35012933c32a73b4c29107406bedc0db11465 +size 14948913920 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_K_M.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_K_M.gguf new file mode 100644 index 0000000..6f30a3f --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:daa7147e15d48892b6013acb6c496bb0fdeb05d080afcba35e781c58300e7f9c +size 14415475456 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_K_S.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_K_S.gguf new file mode 100644 index 0000000..b1e4579 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2b56650a117f91ea86b29db2705eaf2aaea6e470d8d86d9e75a98dd060f418c2 +size 13549458176 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-Q5_K_M.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-Q5_K_M.gguf new file mode 100644 index 0000000..117c871 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bdd507fb01b9b46b731f947e218930ef00f743060959688ba0ca7cd5f4413b85 +size 16918819584 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-Q5_K_S.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-Q5_K_S.gguf new file mode 100644 index 0000000..e02c575 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ffb58b467aa88134547f5ef4215e50a6e9305faccc1087ea38df39e8f631d8ab +size 16430420736 diff --git a/Huihui-LFM2-24B-A2B-abliterated.i1-Q6_K.gguf b/Huihui-LFM2-24B-A2B-abliterated.i1-Q6_K.gguf new file mode 100644 index 0000000..49ee4a3 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8d078ccde6b135ba5d73d9872527be45cddb53bc7c628683793e49240d949819 +size 19578622720 diff --git a/Huihui-LFM2-24B-A2B-abliterated.imatrix.gguf b/Huihui-LFM2-24B-A2B-abliterated.imatrix.gguf new file mode 100644 index 0000000..d1e9195 --- /dev/null +++ b/Huihui-LFM2-24B-A2B-abliterated.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6c4e1c5ac0293b12eab35f48c50323796807f84b2f3ecd5d379bda7a6dfee56e +size 56115392 diff --git a/README.md b/README.md new file mode 100644 index 0000000..12e08f7 --- /dev/null +++ b/README.md @@ -0,0 +1,100 @@ +--- +base_model: huihui-ai/Huihui-LFM2-24B-A2B-abliterated +language: +- en +- ar +- zh +- fr +- de +- ja +- ko +- es +library_name: transformers +license: other +license_link: LICENSE +license_name: lfm1.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- liquid +- lfm2 +- edge +- moe +- abliterated +- uncensored +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/huihui-ai/Huihui-LFM2-24B-A2B-abliterated + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Huihui-LFM2-24B-A2B-abliterated-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.imatrix.gguf) | imatrix | 0.2 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-IQ1_S.gguf) | i1-IQ1_S | 4.9 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-IQ1_M.gguf) | i1-IQ1_M | 5.5 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 6.4 | | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_XS.gguf) | i1-IQ2_XS | 7.1 | | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_S.gguf) | i1-IQ2_S | 7.2 | | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-IQ2_M.gguf) | i1-IQ2_M | 7.9 | | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-Q2_K_S.gguf) | i1-Q2_K_S | 8.2 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-Q2_K.gguf) | i1-Q2_K | 8.8 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 9.3 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_XS.gguf) | i1-IQ3_XS | 9.9 | | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_S.gguf) | i1-IQ3_S | 10.4 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_S.gguf) | i1-Q3_K_S | 10.4 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-IQ3_M.gguf) | i1-IQ3_M | 10.5 | | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_M.gguf) | i1-Q3_K_M | 11.5 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-Q3_K_L.gguf) | i1-Q3_K_L | 12.4 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-IQ4_XS.gguf) | i1-IQ4_XS | 12.8 | | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_0.gguf) | i1-Q4_0 | 13.6 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_K_S.gguf) | i1-Q4_K_S | 13.6 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_K_M.gguf) | i1-Q4_K_M | 14.5 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-Q4_1.gguf) | i1-Q4_1 | 15.0 | | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-Q5_K_S.gguf) | i1-Q5_K_S | 16.5 | | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-Q5_K_M.gguf) | i1-Q5_K_M | 17.0 | | +| [GGUF](https://huggingface.co/mradermacher/Huihui-LFM2-24B-A2B-abliterated-i1-GGUF/resolve/main/Huihui-LFM2-24B-A2B-abliterated.i1-Q6_K.gguf) | i1-Q6_K | 19.7 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +