commit b9bf4842de826a1126dd68188c6e84640d781b4f Author: ModelHub XC Date: Fri Apr 10 22:50:02 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..34fe81e --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Eunoia-Gemma-9B-o1-Indo.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-IQ1_M.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-IQ1_M.gguf new file mode 100644 index 0000000..7de687b --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:57a5a82b9625039180227524bc6a3349751f988ebf369cb35ef185efe165d22a +size 2545952832 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-IQ1_S.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-IQ1_S.gguf new file mode 100644 index 0000000..27c5a4b --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:58ad3f8f6aed20c766e602692b1cc9b6694ba3f8d3d83856c88556cd116d3f51 +size 2378565696 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_M.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_M.gguf new file mode 100644 index 0000000..6c59c8e --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:64664a7e4558df85f42c288556d8133c910057b04792581edc1057c56181d2fc +size 3434670144 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_S.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_S.gguf new file mode 100644 index 0000000..528b4e7 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1be6d4286d00b0bf224988b4887c42f85a520ef0fb061430f63f808ccd7a19f9 +size 3211487296 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_XS.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_XS.gguf new file mode 100644 index 0000000..d4c2e55 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a0f050a52cbdcfff5191bd7a43ad03eb2303bb35851ba6afd7a612a0d394f3d4 +size 3067381824 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_XXS.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..1391747 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9bf9d04df6e09385102f9f2f4be0326cd8872806a93e5dc8dc9c4a56b36e9c9b +size 2824931392 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_M.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_M.gguf new file mode 100644 index 0000000..af8c5cd --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e4926ec4fadee8fe0d9f257d4e5fa1eefa42d9e5c0d0b6acbc9658428a821a04 +size 4494616640 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_S.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_S.gguf new file mode 100644 index 0000000..fe48364 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a53d7854a03b448166cea28b6d3fa5659f639df164badf8f9136f33a218733c6 +size 4337666112 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_XS.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_XS.gguf new file mode 100644 index 0000000..a950e02 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:95408eb2ed0aaae38c0383d9b8362c89dbdd19375e7f65feceea3f32dacbce91 +size 4144990272 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_XXS.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..b971653 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:df18e51f248ffe277ee42d09e16339a569579f02357c566c1b3fec2162b3cbad +size 3796740160 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-IQ4_NL.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-IQ4_NL.gguf new file mode 100644 index 0000000..9f0a1df --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e45db4b4ce3ee32bb3bf757a65f5bb330080a1165cc8eae333b805aa7955109a +size 5443143744 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-IQ4_XS.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-IQ4_XS.gguf new file mode 100644 index 0000000..f71145e --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fdda43bdbd193f9090091e764d4560860ca7a62c89bfbe43f1ef455d46754b2d +size 5183031360 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-Q2_K.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-Q2_K.gguf new file mode 100644 index 0000000..daa7d27 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:698078f64af775b0edfa84ade21ad7ae5303b5f4a769d8e97900823549393d14 +size 3805399104 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-Q2_K_S.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-Q2_K_S.gguf new file mode 100644 index 0000000..1c8c7c0 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:776e8d7437e884f36512b2ec182a8fcf1b155e332c411998a6a74fbb07980388 +size 3552512064 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_L.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_L.gguf new file mode 100644 index 0000000..b6ffa46 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:70266a95b871c557405debc59a01e317adcd65dde9ba76a8e5236883eb377466 +size 5132453952 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_M.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_M.gguf new file mode 100644 index 0000000..b87ef17 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:adde1f7ec7475dbb176d6020da52b9425830ecd73b8a2e6382c427914bc6df4b +size 4761782336 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_S.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_S.gguf new file mode 100644 index 0000000..cdaee8c --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:adc5eb59676e41ed5c222c0ed52b3ed995cfd5010dbf79c5675641d447d848e0 +size 4337666112 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-Q4_0.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-Q4_0.gguf new file mode 100644 index 0000000..0436d96 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59efb5616e6f1ecf4afd77e5027843b35c22de67880476e8d6a11a2691677c99 +size 5459200064 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-Q4_1.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-Q4_1.gguf new file mode 100644 index 0000000..e3881ba --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5aa93b8437dfcf3fcadab6189fa959a77c4f29ff439d97b70ef39c90a7885fd7 +size 5963368512 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-Q4_K_M.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-Q4_K_M.gguf new file mode 100644 index 0000000..ceacf1d --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f07c496a0a7ad3522491d94a9dad8147c2ff722b8911e58b60b133f63823cc42 +size 5761058880 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-Q4_K_S.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-Q4_K_S.gguf new file mode 100644 index 0000000..5595684 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9788c0557bb0674facbd5faf45cfc19b88e54e645c54ca34836f3b7a8f1dcd26 +size 5478926400 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-Q5_K_M.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-Q5_K_M.gguf new file mode 100644 index 0000000..d88dddc --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4ec3d27a9accdf811cdd80526717d0b7471abdc1d7c23f5e18eb01ebd6d2683e +size 6647367744 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-Q5_K_S.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-Q5_K_S.gguf new file mode 100644 index 0000000..f7ec4aa --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0123f4b140b3cd5bae4bd6594d5dcc2f085af6d46c30fb21ad2966961a93a11a +size 6483593280 diff --git a/Eunoia-Gemma-9B-o1-Indo.i1-Q6_K.gguf b/Eunoia-Gemma-9B-o1-Indo.i1-Q6_K.gguf new file mode 100644 index 0000000..474f716 --- /dev/null +++ b/Eunoia-Gemma-9B-o1-Indo.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:58d93b94e95477cf7fa0e0af67be180af80c44dc26bd9d1f0e2ba22b1e0dee01 +size 7589070912 diff --git a/README.md b/README.md new file mode 100644 index 0000000..3983ad3 --- /dev/null +++ b/README.md @@ -0,0 +1,86 @@ +--- +base_model: gmonsoon/Eunoia-Gemma-9B-o1-Indo +datasets: +- gmonsoon/CoT-id +language: +- en +- id +library_name: transformers +license: gemma +quantized_by: mradermacher +tags: +- text-generation-inference +- transformers +- unsloth +- gemma2 +- trl +- sft +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/gmonsoon/Eunoia-Gemma-9B-o1-Indo + + +static quants are available at https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-IQ1_S.gguf) | i1-IQ1_S | 2.5 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-IQ1_M.gguf) | i1-IQ1_M | 2.6 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_XS.gguf) | i1-IQ2_XS | 3.2 | | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_S.gguf) | i1-IQ2_S | 3.3 | | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-IQ2_M.gguf) | i1-IQ2_M | 3.5 | | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.7 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-Q2_K.gguf) | i1-Q2_K | 3.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_XS.gguf) | i1-IQ3_XS | 4.2 | | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_S.gguf) | i1-IQ3_S | 4.4 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_S.gguf) | i1-Q3_K_S | 4.4 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-IQ3_M.gguf) | i1-IQ3_M | 4.6 | | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.9 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-Q3_K_L.gguf) | i1-Q3_K_L | 5.2 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-IQ4_XS.gguf) | i1-IQ4_XS | 5.3 | | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-IQ4_NL.gguf) | i1-IQ4_NL | 5.5 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-Q4_0.gguf) | i1-Q4_0 | 5.6 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-Q4_K_S.gguf) | i1-Q4_K_S | 5.6 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.9 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-Q4_1.gguf) | i1-Q4_1 | 6.1 | | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-Q5_K_S.gguf) | i1-Q5_K_S | 6.6 | | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-Q5_K_M.gguf) | i1-Q5_K_M | 6.7 | | +| [GGUF](https://huggingface.co/mradermacher/Eunoia-Gemma-9B-o1-Indo-i1-GGUF/resolve/main/Eunoia-Gemma-9B-o1-Indo.i1-Q6_K.gguf) | i1-Q6_K | 7.7 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..d1e302e --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1448bab48ff5988088d32a3c6aed5e26934f98d4ed48125dbfb9c6308ef3bf50 +size 6116887