commit 445f476cf84ec6c13349a4a58d1361d062305e2c Author: ModelHub XC Date: Tue Jul 7 19:59:16 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/BaobabAI-v0.3-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..03d0971 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +BaobabAI-v0.3.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/BaobabAI-v0.3.i1-IQ1_M.gguf b/BaobabAI-v0.3.i1-IQ1_M.gguf new file mode 100644 index 0000000..4398522 --- /dev/null +++ b/BaobabAI-v0.3.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7e96b194d1bcae3cc70c15730f56a9a6984e880aa152689062ac93aa2788e048 +size 924191584 diff --git a/BaobabAI-v0.3.i1-IQ1_S.gguf b/BaobabAI-v0.3.i1-IQ1_S.gguf new file mode 100644 index 0000000..7051d84 --- /dev/null +++ b/BaobabAI-v0.3.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:efbef02a0110c787953a9bdc7cdd41f339ea4d743a88af1c36e0dc4de4e326bd +size 868158304 diff --git a/BaobabAI-v0.3.i1-IQ2_M.gguf b/BaobabAI-v0.3.i1-IQ2_M.gguf new file mode 100644 index 0000000..58bffa5 --- /dev/null +++ b/BaobabAI-v0.3.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ba70442f8dc0b33a5ec63bc066fbb51a3474e13d1c1454bce545916b182c2bb4 +size 1229032288 diff --git a/BaobabAI-v0.3.i1-IQ2_S.gguf b/BaobabAI-v0.3.i1-IQ2_S.gguf new file mode 100644 index 0000000..5d309f4 --- /dev/null +++ b/BaobabAI-v0.3.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:828c0c013eac494b47ec8f21e8182d00522ee0469ef4211a5364c69dc8268d58 +size 1154321248 diff --git a/BaobabAI-v0.3.i1-IQ2_XS.gguf b/BaobabAI-v0.3.i1-IQ2_XS.gguf new file mode 100644 index 0000000..bb9499c --- /dev/null +++ b/BaobabAI-v0.3.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2e8699adf8d9a1fe1a6a89f2818f5b729d3318133351e19b5d86547d4c478d43 +size 1100548960 diff --git a/BaobabAI-v0.3.i1-IQ2_XXS.gguf b/BaobabAI-v0.3.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..7c26c65 --- /dev/null +++ b/BaobabAI-v0.3.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:289086fb173ad73a9b32073d374bf751464e072a3c006e586c53550893ace497 +size 1017580384 diff --git a/BaobabAI-v0.3.i1-IQ3_M.gguf b/BaobabAI-v0.3.i1-IQ3_M.gguf new file mode 100644 index 0000000..2984f59 --- /dev/null +++ b/BaobabAI-v0.3.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:02c528544ab574b5cf451688d4616cb82b325015177bb305bb74040037807048 +size 1599669088 diff --git a/BaobabAI-v0.3.i1-IQ3_S.gguf b/BaobabAI-v0.3.i1-IQ3_S.gguf new file mode 100644 index 0000000..0e40642 --- /dev/null +++ b/BaobabAI-v0.3.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:13f8f289ec0021c78d04e268d99b61d463f626eacf3aad7acd58bbbdc952f7a8 +size 1542849376 diff --git a/BaobabAI-v0.3.i1-IQ3_XS.gguf b/BaobabAI-v0.3.i1-IQ3_XS.gguf new file mode 100644 index 0000000..a16ca08 --- /dev/null +++ b/BaobabAI-v0.3.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8e8da41278241c663fdcae0a405098a26744b418939165aa414962988ee18095 +size 1476789088 diff --git a/BaobabAI-v0.3.i1-IQ3_XXS.gguf b/BaobabAI-v0.3.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..a382436 --- /dev/null +++ b/BaobabAI-v0.3.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fda12a2249ae57d753bb520e91d574e9c1728a92e49ecc0e0a6c19a747f0f867 +size 1348766560 diff --git a/BaobabAI-v0.3.i1-IQ4_NL.gguf b/BaobabAI-v0.3.i1-IQ4_NL.gguf new file mode 100644 index 0000000..a67ae17 --- /dev/null +++ b/BaobabAI-v0.3.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f5303c403cbc3c96b36ffaf5275cf9c024d8a55d33279421c6af802142790bf6 +size 1917191008 diff --git a/BaobabAI-v0.3.i1-IQ4_XS.gguf b/BaobabAI-v0.3.i1-IQ4_XS.gguf new file mode 100644 index 0000000..7d3b5a5 --- /dev/null +++ b/BaobabAI-v0.3.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8eb7d4f55ce051f16483af0facd270ca1d8cce610ea6e181df2d0d10864558db +size 1829110624 diff --git a/BaobabAI-v0.3.i1-Q2_K.gguf b/BaobabAI-v0.3.i1-Q2_K.gguf new file mode 100644 index 0000000..94c8d84 --- /dev/null +++ b/BaobabAI-v0.3.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:30bca870b3ec192cbee7190eee5f44012bd083db3040e1fd0a688d91c6ebbe3f +size 1363936096 diff --git a/BaobabAI-v0.3.i1-Q2_K_S.gguf b/BaobabAI-v0.3.i1-Q2_K_S.gguf new file mode 100644 index 0000000..766b75f --- /dev/null +++ b/BaobabAI-v0.3.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e29d9ba7258103b7c5d3c6f55ca380e9e00f85b3be36b885f571e72c3e9ab02 +size 1274282848 diff --git a/BaobabAI-v0.3.i1-Q3_K_L.gguf b/BaobabAI-v0.3.i1-Q3_K_L.gguf new file mode 100644 index 0000000..42559de --- /dev/null +++ b/BaobabAI-v0.3.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:70a40dbbbae69697ca9a6be5f42a91ce4b4720fa2b6e0758eaf12869787e98b8 +size 1815348064 diff --git a/BaobabAI-v0.3.i1-Q3_K_M.gguf b/BaobabAI-v0.3.i1-Q3_K_M.gguf new file mode 100644 index 0000000..2942194 --- /dev/null +++ b/BaobabAI-v0.3.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:928f230f8923eff9d8746fde81ccbbb3ea720a47cb8da99d4a960059b4a4780d +size 1687159648 diff --git a/BaobabAI-v0.3.i1-Q3_K_S.gguf b/BaobabAI-v0.3.i1-Q3_K_S.gguf new file mode 100644 index 0000000..e1fce2f --- /dev/null +++ b/BaobabAI-v0.3.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:623c9bb3f36c51bf211de7439b838340fbd2b3a7204186959f65dbd02e6958fb +size 1542849376 diff --git a/BaobabAI-v0.3.i1-Q4_0.gguf b/BaobabAI-v0.3.i1-Q4_0.gguf new file mode 100644 index 0000000..f5a67e0 --- /dev/null +++ b/BaobabAI-v0.3.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:84d88c39b9f3acf68a09a635c241cd3b94f79e388d93262a3e76044af37c9299 +size 1921909600 diff --git a/BaobabAI-v0.3.i1-Q4_1.gguf b/BaobabAI-v0.3.i1-Q4_1.gguf new file mode 100644 index 0000000..d86ba57 --- /dev/null +++ b/BaobabAI-v0.3.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a4d4b2434d7347f7a8e38d7f94efb420dc1118e7da5c302ef0d5551323ba1a4d +size 2093351776 diff --git a/BaobabAI-v0.3.i1-Q4_K_M.gguf b/BaobabAI-v0.3.i1-Q4_K_M.gguf new file mode 100644 index 0000000..83e7044 --- /dev/null +++ b/BaobabAI-v0.3.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d87bfa7ea865f8bfbb944002930612aa906cd6d0b009b3de6885dc947ef4cd9d +size 2019378016 diff --git a/BaobabAI-v0.3.i1-Q4_K_S.gguf b/BaobabAI-v0.3.i1-Q4_K_S.gguf new file mode 100644 index 0000000..24e8d6b --- /dev/null +++ b/BaobabAI-v0.3.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6ee241af9c76a4edad390ed0060242261835a446749843469fb4cad0172bd683 +size 1928201056 diff --git a/BaobabAI-v0.3.i1-Q5_K_M.gguf b/BaobabAI-v0.3.i1-Q5_K_M.gguf new file mode 100644 index 0000000..32f6d17 --- /dev/null +++ b/BaobabAI-v0.3.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c24b550c40278f93d8ccbe91f1bd83bc9d279129ef0d2d713204b934c924efb3 +size 2322154336 diff --git a/BaobabAI-v0.3.i1-Q5_K_S.gguf b/BaobabAI-v0.3.i1-Q5_K_S.gguf new file mode 100644 index 0000000..e191bb9 --- /dev/null +++ b/BaobabAI-v0.3.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0238bb8686a0fa72dcebab16794dbd7c9e3bb89c0ca6f705909cf8fdbfa4aa8f +size 2269512544 diff --git a/BaobabAI-v0.3.i1-Q6_K.gguf b/BaobabAI-v0.3.i1-Q6_K.gguf new file mode 100644 index 0000000..49bbe13 --- /dev/null +++ b/BaobabAI-v0.3.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:194bcabe77c85fa56a7c185bb584d4ccff96db2b7f08f68a619ea183cb43f5e5 +size 2643854176 diff --git a/BaobabAI-v0.3.imatrix.gguf b/BaobabAI-v0.3.imatrix.gguf new file mode 100644 index 0000000..b112c79 --- /dev/null +++ b/BaobabAI-v0.3.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:547d4ef62ad9de8c3aa81534c5f3997734f718d2d870ec2f9304893b75d9cb68 +size 3012064 diff --git a/README.md b/README.md new file mode 100644 index 0000000..a6c953c --- /dev/null +++ b/README.md @@ -0,0 +1,109 @@ +--- +base_model: okaforpascal40/BaobabAI-v0.3 +language: +- ha +- ig +- pcm +- yo +- sw +- am +- so +- xh +- sn +- lg +- ln +- zu +- wo +- om +- ti +- rn +- tw +- ff +- mg +- rw +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- africa +- llama +- fine-tuned +- multilingual +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/okaforpascal40/BaobabAI-v0.3 + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#BaobabAI-v0.3-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/BaobabAI-v0.3-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-IQ1_S.gguf) | i1-IQ1_S | 1.0 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-IQ1_M.gguf) | i1-IQ1_M | 1.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.1 | | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.2 | | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-IQ2_S.gguf) | i1-IQ2_S | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-IQ2_M.gguf) | i1-IQ2_M | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.4 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.4 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-Q2_K.gguf) | i1-Q2_K | 1.5 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-IQ3_S.gguf) | i1-IQ3_S | 1.6 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-Q3_K_S.gguf) | i1-Q3_K_S | 1.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-IQ3_M.gguf) | i1-IQ3_M | 1.7 | | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-Q3_K_M.gguf) | i1-Q3_K_M | 1.8 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-Q3_K_L.gguf) | i1-Q3_K_L | 1.9 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-IQ4_XS.gguf) | i1-IQ4_XS | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.0 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-Q4_0.gguf) | i1-Q4_0 | 2.0 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.0 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.1 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-Q4_1.gguf) | i1-Q4_1 | 2.2 | | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-Q5_K_M.gguf) | i1-Q5_K_M | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/BaobabAI-v0.3-i1-GGUF/resolve/main/BaobabAI-v0.3.i1-Q6_K.gguf) | i1-Q6_K | 2.7 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +