From a233a609aeb4f9366fc12cb670c52f6bdd26d455 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Thu, 30 Apr 2026 23:30:18 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF Source: Original Platform --- .gitattributes | 58 +++++++++++++++ OpenMia-Indo-Mistral-7b-v3.i1-IQ1_M.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-IQ1_S.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-IQ2_M.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-IQ2_S.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XS.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XXS.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-IQ3_M.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-IQ3_S.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XS.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XXS.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-IQ4_XS.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-Q2_K.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-Q2_K_S.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_L.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_M.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_S.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-Q4_0.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_M.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_S.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_M.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_S.gguf | 3 + OpenMia-Indo-Mistral-7b-v3.i1-Q6_K.gguf | 3 + README.md | 82 ++++++++++++++++++++++ imatrix.dat | 3 + 25 files changed, 209 insertions(+) create mode 100644 .gitattributes create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-IQ1_M.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-IQ1_S.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-IQ2_M.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-IQ2_S.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XS.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XXS.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-IQ3_M.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-IQ3_S.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XS.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XXS.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-IQ4_XS.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-Q2_K.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-Q2_K_S.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_L.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_M.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_S.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-Q4_0.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_M.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_S.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_M.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_S.gguf create mode 100644 OpenMia-Indo-Mistral-7b-v3.i1-Q6_K.gguf create mode 100644 README.md create mode 100644 imatrix.dat diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..920b4de --- /dev/null +++ b/.gitattributes @@ -0,0 +1,58 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenMia-Indo-Mistral-7b-v3.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-IQ1_M.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-IQ1_M.gguf new file mode 100644 index 0000000..696c637 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7d9fe9bfe7eda3c1916e22d6e98a7f0d049f3449690f6996f5e1fd81f011a842 +size 1754447648 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-IQ1_S.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-IQ1_S.gguf new file mode 100644 index 0000000..6023756 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ccc6bd7cf153e2f2017ab08d56b8c7b766063659eb8179a1c1b48f4f3dcc4f3 +size 1612103456 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_M.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_M.gguf new file mode 100644 index 0000000..8122483 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6efe1aa4c2fa63aa82cd803acc253e973a2a1713baa4a13dc6edd9eba15df44b +size 2500714272 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_S.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_S.gguf new file mode 100644 index 0000000..42a889d --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7d070b28206cf7de537bb8395546c1dbda565f01a43f2d9d3649f49c954d1461 +size 2310922016 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XS.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XS.gguf new file mode 100644 index 0000000..bd19901 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5dd6146aecbcb49b9a15abd6ecbd6f872eebab60f60e340dc029a63cddca5d21 +size 2198257440 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XXS.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..fe29a8f --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:df7d11d27255360ebab386d5b21e4cdb8b8788a1113d187a77045768ae7ec94c +size 1991687968 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_M.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_M.gguf new file mode 100644 index 0000000..8a56579 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0aea357149a54f25ad27f1054dfe40e61a64b3f14ceac537fee815a21febb848 +size 3284893472 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_S.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_S.gguf new file mode 100644 index 0000000..dbbd3ce --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5dcb659c7b181652197a397b6aea7042fe30403acbbfcd4f443acce658ba4527 +size 3182395168 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XS.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XS.gguf new file mode 100644 index 0000000..9d62536 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:826b59d887a828a2c02a6a56404b16250ab65f36b92eecf6d43139e9784521dd +size 3018817312 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XXS.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..73c5b01 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:387f10f92d8e9f29e7e47bf2b4bb28a69614f325c060f650e324e6614ef4b24c +size 2827345696 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-IQ4_XS.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-IQ4_XS.gguf new file mode 100644 index 0000000..dd7a301 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c33b77bd0e32c82a1ea1d9a5f4653a2213d1cf61df707123f60e30dbe34ae55f +size 3907690272 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-Q2_K.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-Q2_K.gguf new file mode 100644 index 0000000..60174ce --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f53563db36a797b7da26f2b00cbcf64296a25a98db7cb76c8b458e247d683f37 +size 2719244064 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-Q2_K_S.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-Q2_K_S.gguf new file mode 100644 index 0000000..f9e6db0 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ad329e166472e137f87626e18569e0cc329976ca53bc745a10a3c2a3f3782c3 +size 2528927520 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_L.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_L.gguf new file mode 100644 index 0000000..a00c4c9 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c5149563fe8e1d86e9a1f01bbb568ace3a49067785f0a7fe5a06d4d131e0a1a7 +size 3822026528 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_M.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_M.gguf new file mode 100644 index 0000000..f0365f8 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:96d96f41c15ce721c130feace59ceeac40c45f96480c8cc7d5c7da09557b36a7 +size 3518988064 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_S.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_S.gguf new file mode 100644 index 0000000..91a2744 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1885d6a12e81a8bd5602df4c22efe997ed573064e25979a709a8cfaae98469f1 +size 3164569376 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-Q4_0.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-Q4_0.gguf new file mode 100644 index 0000000..32ef824 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a32ef64ac0b5e1514c3f07e4b0336b12722a8f1a5c38a7b8ce19a5127f6a828b +size 4123598624 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_M.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_M.gguf new file mode 100644 index 0000000..119c4e3 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b166cff0b30b7d5c4c6a9be318e27ea5e950f7c7b7c3d793354ffd8fed03248e +size 4368441120 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_S.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_S.gguf new file mode 100644 index 0000000..b50b99c --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:be01780f5ea76e8320ea0a891a6c76d8a57958af13bacf7c421126bc63563a48 +size 4140375840 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_M.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_M.gguf new file mode 100644 index 0000000..c3cd529 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da0b2fcc52ac3930e1e425d124789c7b827553c5e297274e2880a09bffc37132 +size 5131411232 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_S.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_S.gguf new file mode 100644 index 0000000..6615a50 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3fdb2184e44669a5f5f9b2c22e54575f72bc9a94b9ff2bf1ca6b4cfcf8b8a9bc +size 4997717792 diff --git a/OpenMia-Indo-Mistral-7b-v3.i1-Q6_K.gguf b/OpenMia-Indo-Mistral-7b-v3.i1-Q6_K.gguf new file mode 100644 index 0000000..70cfac9 --- /dev/null +++ b/OpenMia-Indo-Mistral-7b-v3.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ee976a0d005ed0c1834ee105d6b9f35459f81c94470dce41e38c7cfa9a4c0eba +size 5942066976 diff --git a/README.md b/README.md new file mode 100644 index 0000000..387e8c5 --- /dev/null +++ b/README.md @@ -0,0 +1,82 @@ +--- +base_model: indischepartij/OpenMia-Indo-Mistral-7b-v3 +datasets: +- cahya/alpaca-id-cleaned +language: +- en +- id +library_name: transformers +license: cc-by-nc-4.0 +quantized_by: mradermacher +tags: +- text-generation-inference +- transformers +- mistral +- trl +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/indischepartij/OpenMia-Indo-Mistral-7b-v3 + + +static quants are available at https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-IQ1_S.gguf) | i1-IQ1_S | 1.7 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-IQ1_M.gguf) | i1-IQ1_M | 1.9 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.1 | | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.3 | | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_S.gguf) | i1-IQ2_S | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-IQ2_M.gguf) | i1-IQ2_M | 2.6 | | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-Q2_K_S.gguf) | i1-Q2_K_S | 2.6 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-Q2_K.gguf) | i1-Q2_K | 2.8 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 2.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.1 | | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.3 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_S.gguf) | i1-IQ3_S | 3.3 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-IQ3_M.gguf) | i1-IQ3_M | 3.4 | | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_M.gguf) | i1-Q3_K_M | 3.6 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-Q3_K_L.gguf) | i1-Q3_K_L | 3.9 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.0 | | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-Q4_0.gguf) | i1-Q4_0 | 4.2 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.2 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-Q4_K_M.gguf) | i1-Q4_K_M | 4.5 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.1 | | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.2 | | +| [GGUF](https://huggingface.co/mradermacher/OpenMia-Indo-Mistral-7b-v3-i1-GGUF/resolve/main/OpenMia-Indo-Mistral-7b-v3.i1-Q6_K.gguf) | i1-Q6_K | 6.0 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..0ad1922 --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7875613de29972f0ee508ef421a90f56be3689d9cb6138fd6313ace2b40f1c96 +size 4988157