commit de6beaf037691115fd9532d9e775734b74a944bf Author: ModelHub XC Date: Fri May 1 07:53:14 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..da29968 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Lune-Mamba-3B-v1-GRPO_IF.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ1_M.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ1_M.gguf new file mode 100644 index 0000000..8d48070 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2b69dc516cdce720ff6e95b2647289b1fa51237dd057c134c749dc2f8b75011d +size 879926208 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ1_S.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ1_S.gguf new file mode 100644 index 0000000..96b1a3b --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:66f1b56e14a73d09a9448dd831ad28f9e94e8a5d30313cc18833f9352ba82d2c +size 812428224 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_M.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_M.gguf new file mode 100644 index 0000000..dac4304 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cde24e7654d7ef937d4b1903b4958a33743093d38b848cde1127958bffc4ca76 +size 1205119936 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_S.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_S.gguf new file mode 100644 index 0000000..b263018 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3e03661f1eb07818bdda846709f7b07510f31e43720a8bd1c8192efaeb985aed +size 1115122624 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_XS.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_XS.gguf new file mode 100644 index 0000000..7297bf9 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1d24c0ba701dc652de2e47336ca24f4be8134e1bf1d77c6fed09746756f7f43 +size 1082944448 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_XXS.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..2f78808 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bb70afc67db601329562e4348d0762a1f0e41b4ba69ff860764aaa91727608b7 +size 992422848 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_M.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_M.gguf new file mode 100644 index 0000000..1127ef5 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eccfd358eac733b8c2c787d649cc4a7ae9abadde495c9ce254ace0ea223a38e8 +size 1561332672 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_S.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_S.gguf new file mode 100644 index 0000000..710830f --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7fd979fb635a18e19a4462c74dfbeecf3f7a801ecc8622b28a49dfb8f4ffce09 +size 1547406272 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_XS.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_XS.gguf new file mode 100644 index 0000000..baa2305 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:462388f54a0e86c11dd73ed82556fe52f46e5f4eb2f7829526cd2473de7fce9e +size 1499237312 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_XXS.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..40c6782 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b76b2e590f7d41a841418df4d590e176d8482a4b2eae79836d1842cb88680f04 +size 1379675072 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ4_NL.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ4_NL.gguf new file mode 100644 index 0000000..c6381ec --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b7a7fecf0f80194b98a50bb375142631414e0a2fe3e91e503112333bdc812e18 +size 1971116992 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ4_XS.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ4_XS.gguf new file mode 100644 index 0000000..84db41f --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f6824f718d095204fc5196288e84c4ec71e058750020aae336b40f5ba9e17aab +size 1871420352 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-Q2_K.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q2_K.gguf new file mode 100644 index 0000000..6d6edc5 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:99dbeff130b6d9e94d02af08f51934127c1cbbea8672b52dc5549e10e0eb4c3c +size 1293679552 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-Q2_K_S.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q2_K_S.gguf new file mode 100644 index 0000000..c4e9d84 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a738c4ce7172972ae804154587f8cf767a659b7381f4d2603106c750e04c0c61 +size 1243052992 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_L.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_L.gguf new file mode 100644 index 0000000..563fc83 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f46d455b3f34134b751d6231f4390713c5deb47e4163d0dba3644413b657178f +size 1725828032 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_M.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_M.gguf new file mode 100644 index 0000000..cb21b51 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4bce990671fee788235d0e519fe31027f05546ddab7f17c3862e687450a361dd +size 1643776960 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_S.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_S.gguf new file mode 100644 index 0000000..e2f32c4 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b4e709779cdfd11452ecbee92ae816a3ef6522d1512655f1525fc0beaa5761f +size 1547406272 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_0.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_0.gguf new file mode 100644 index 0000000..9c36dfc --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:97968f25580344ea13c782e0bea9a56867b26af7035caaeaa6e67cd70195111c +size 1976359872 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_1.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_1.gguf new file mode 100644 index 0000000..52e0658 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a83637c144bb861ea9c12593dba2b638577c1d4de67a637eebbac838aa0b8727 +size 2170510272 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_K_M.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_K_M.gguf new file mode 100644 index 0000000..7778412 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:102a3052f9eaba2a26a9fd6c98c830532817f351964e876868c0451de06e3dc4 +size 2058165184 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_K_S.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_K_S.gguf new file mode 100644 index 0000000..4fb2ff7 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:36c215dac2a6b376a1252abff13b0c4d43c2faeecd00a5f4010ddc348c6512ec +size 1982127040 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-Q5_K_M.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q5_K_M.gguf new file mode 100644 index 0000000..0bbbdd3 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:051560f7d1859ed9816a223578f9a778088579e96eee4f5ff07d64b11a4cc5b5 +size 2414746560 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-Q5_K_S.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q5_K_S.gguf new file mode 100644 index 0000000..5f39681 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:511aa9c196606a5b1cc5ff3b01a9af70af59b18dbc04562672871985b1181b86 +size 2369903552 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.i1-Q6_K.gguf b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q6_K.gguf new file mode 100644 index 0000000..a4d55b6 --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:66586ff9531774e810a1a4073dd74c873edcdfdfec5de1b050ebcb83e61f28c0 +size 2793614272 diff --git a/Lune-Mamba-3B-v1-GRPO_IF.imatrix.gguf b/Lune-Mamba-3B-v1-GRPO_IF.imatrix.gguf new file mode 100644 index 0000000..8d5b1fd --- /dev/null +++ b/Lune-Mamba-3B-v1-GRPO_IF.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26d1c09604e79856c1b6b5f5c02d8727f276193064b53e1ed64558e0b1a8b0eb +size 3013856 diff --git a/README.md b/README.md new file mode 100644 index 0000000..328de9e --- /dev/null +++ b/README.md @@ -0,0 +1,90 @@ +--- +base_model: allura-org/Lune-Mamba-3B-v1-GRPO_IF +language: +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- conversational +- instruct +- mamba +- hybrid +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/allura-org/Lune-Mamba-3B-v1-GRPO_IF + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ1_S.gguf) | i1-IQ1_S | 0.9 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ1_M.gguf) | i1-IQ1_M | 1.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.1 | | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.2 | | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_S.gguf) | i1-IQ2_S | 1.2 | | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ2_M.gguf) | i1-IQ2_M | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.3 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-Q2_K.gguf) | i1-Q2_K | 1.4 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.5 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_S.gguf) | i1-IQ3_S | 1.6 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_S.gguf) | i1-Q3_K_S | 1.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ3_M.gguf) | i1-IQ3_M | 1.7 | | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_M.gguf) | i1-Q3_K_M | 1.7 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-Q3_K_L.gguf) | i1-Q3_K_L | 1.8 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.0 | | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.1 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_0.gguf) | i1-Q4_0 | 2.1 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.1 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.2 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-Q4_1.gguf) | i1-Q4_1 | 2.3 | | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.5 | | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-Q5_K_M.gguf) | i1-Q5_K_M | 2.5 | | +| [GGUF](https://huggingface.co/mradermacher/Lune-Mamba-3B-v1-GRPO_IF-i1-GGUF/resolve/main/Lune-Mamba-3B-v1-GRPO_IF.i1-Q6_K.gguf) | i1-Q6_K | 2.9 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +