commit 838e339d65a955a1583a2ba4912f09a78a4d3f66 Author: ModelHub XC Date: Tue Apr 14 13:17:07 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Mistral-Trismegistus-7B-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..f82b3d1 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Mistral-Trismegistus-7B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Mistral-Trismegistus-7B.i1-IQ1_M.gguf b/Mistral-Trismegistus-7B.i1-IQ1_M.gguf new file mode 100644 index 0000000..9252548 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:50276780e9b9df4eaaaf2fef37702adb652f509208344a04001451933c03e5bc +size 1754447040 diff --git a/Mistral-Trismegistus-7B.i1-IQ1_S.gguf b/Mistral-Trismegistus-7B.i1-IQ1_S.gguf new file mode 100644 index 0000000..d9b136d --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cb89f889d590280161aea2b894a85c12adb76bf76e983e214ff6dd59ef012063 +size 1612102848 diff --git a/Mistral-Trismegistus-7B.i1-IQ2_M.gguf b/Mistral-Trismegistus-7B.i1-IQ2_M.gguf new file mode 100644 index 0000000..606a196 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de01b5a152a71c6a27e14f271cd39bf51496f7ce18df04cc8a5b26e0ab31d9b7 +size 2500713664 diff --git a/Mistral-Trismegistus-7B.i1-IQ2_S.gguf b/Mistral-Trismegistus-7B.i1-IQ2_S.gguf new file mode 100644 index 0000000..b8dda1b --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f60f80997068533cb7e0e0445e7256e7773bdf7721f570eeb9a2501a3e1ddfb6 +size 2310921408 diff --git a/Mistral-Trismegistus-7B.i1-IQ2_XS.gguf b/Mistral-Trismegistus-7B.i1-IQ2_XS.gguf new file mode 100644 index 0000000..6193bff --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4a7f5a34e7dce40d977831a05d301df56088c6eded14a5da51f6a87d3c48cb7 +size 2198256832 diff --git a/Mistral-Trismegistus-7B.i1-IQ2_XXS.gguf b/Mistral-Trismegistus-7B.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..382900f --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a9dc1d4a4b157a6a131f6b0769eec961a802f7fb2adc48ee450114efd3f33009 +size 1991687360 diff --git a/Mistral-Trismegistus-7B.i1-IQ3_M.gguf b/Mistral-Trismegistus-7B.i1-IQ3_M.gguf new file mode 100644 index 0000000..2a78d3a --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2b5c43ba3a24a406c34404caf3bba03e500768026a85c0537b5a217428913b89 +size 3284892864 diff --git a/Mistral-Trismegistus-7B.i1-IQ3_S.gguf b/Mistral-Trismegistus-7B.i1-IQ3_S.gguf new file mode 100644 index 0000000..482eab2 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d0d4a19df134f787820793fe7bb4d216c0351790606b681998e0d105314602fc +size 3182394560 diff --git a/Mistral-Trismegistus-7B.i1-IQ3_XS.gguf b/Mistral-Trismegistus-7B.i1-IQ3_XS.gguf new file mode 100644 index 0000000..dc3ca00 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:00fb7ae2969ba251e36eaabe9265078434ae4d576276cb1edff3cf5d20d26486 +size 3018816704 diff --git a/Mistral-Trismegistus-7B.i1-IQ3_XXS.gguf b/Mistral-Trismegistus-7B.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..7191538 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:40d56ec6dbb9c592398088097e2cd43176f4b860b33fddc906640a482451aba8 +size 2827345088 diff --git a/Mistral-Trismegistus-7B.i1-IQ4_NL.gguf b/Mistral-Trismegistus-7B.i1-IQ4_NL.gguf new file mode 100644 index 0000000..a110362 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:85728179a6a7b212d45840404829197474652a6e816bd4f951ba10b1979cacab +size 4125695168 diff --git a/Mistral-Trismegistus-7B.i1-IQ4_XS.gguf b/Mistral-Trismegistus-7B.i1-IQ4_XS.gguf new file mode 100644 index 0000000..cdca5a7 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:09fae685eb37403b95462c0cf3244dff2332dcf12c353e96b1e85455ef4325f6 +size 3907689664 diff --git a/Mistral-Trismegistus-7B.i1-Q2_K.gguf b/Mistral-Trismegistus-7B.i1-Q2_K.gguf new file mode 100644 index 0000000..dc108d0 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b7edb29112a643d3be18dfaa83786883d545cc6eb772c0e7ba552e51c5ed25ab +size 2719243456 diff --git a/Mistral-Trismegistus-7B.i1-Q2_K_S.gguf b/Mistral-Trismegistus-7B.i1-Q2_K_S.gguf new file mode 100644 index 0000000..cba380b --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fb92cff8831cdd23344f7c503afaace70d5835c4efc7fa24b11a8393d169347b +size 2528926912 diff --git a/Mistral-Trismegistus-7B.i1-Q3_K_L.gguf b/Mistral-Trismegistus-7B.i1-Q3_K_L.gguf new file mode 100644 index 0000000..c908321 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ee1548f75d3c1bd21238d42ae211f24c9b8bbb168f275ee77027ec6ff18aa4e4 +size 3822025920 diff --git a/Mistral-Trismegistus-7B.i1-Q3_K_M.gguf b/Mistral-Trismegistus-7B.i1-Q3_K_M.gguf new file mode 100644 index 0000000..cecdfe5 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3d2a8c4dae4f2bc59a0f254983b868b7435ad6c45bc4ba280ddf04f29091b2e +size 3518987456 diff --git a/Mistral-Trismegistus-7B.i1-Q3_K_S.gguf b/Mistral-Trismegistus-7B.i1-Q3_K_S.gguf new file mode 100644 index 0000000..cb9425a --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9335bf8b81df5ecd87d71a7e766a1f8de63029638a259eb6d719f6319e678722 +size 3164568768 diff --git a/Mistral-Trismegistus-7B.i1-Q4_0.gguf b/Mistral-Trismegistus-7B.i1-Q4_0.gguf new file mode 100644 index 0000000..6ce5693 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:15f51c69efe2627f1f8033d040afd93e72c02d00d1f1986d75767fa5c8246d48 +size 4123598016 diff --git a/Mistral-Trismegistus-7B.i1-Q4_1.gguf b/Mistral-Trismegistus-7B.i1-Q4_1.gguf new file mode 100644 index 0000000..4f31907 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1a9f7712036189eb7978cc2f8a6fea130ba9988ab9c314e82b96716683a2149f +size 4553317568 diff --git a/Mistral-Trismegistus-7B.i1-Q4_K_M.gguf b/Mistral-Trismegistus-7B.i1-Q4_K_M.gguf new file mode 100644 index 0000000..dc847ee --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6969c9c6c1ff1fe9e2e4a1bb200de6b4b340a2b1d4be31a3ce9a6853043d9535 +size 4368440512 diff --git a/Mistral-Trismegistus-7B.i1-Q4_K_S.gguf b/Mistral-Trismegistus-7B.i1-Q4_K_S.gguf new file mode 100644 index 0000000..d104689 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f8748ebe80286d2dfa9eccc7bd91052f8333d141777234fcc57d1d30a44f8910 +size 4140375232 diff --git a/Mistral-Trismegistus-7B.i1-Q5_K_M.gguf b/Mistral-Trismegistus-7B.i1-Q5_K_M.gguf new file mode 100644 index 0000000..81f8d68 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:32bbf5ae1be7125fdbda41cdcab95d9c689fef0cd636e19da080a7296d6b538c +size 5131410624 diff --git a/Mistral-Trismegistus-7B.i1-Q5_K_S.gguf b/Mistral-Trismegistus-7B.i1-Q5_K_S.gguf new file mode 100644 index 0000000..4d1589a --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:325d672c5563bbbe385f374a17f27c490c751480b3abc3cced0ef30b3ca81583 +size 4997717184 diff --git a/Mistral-Trismegistus-7B.i1-Q6_K.gguf b/Mistral-Trismegistus-7B.i1-Q6_K.gguf new file mode 100644 index 0000000..872f7d7 --- /dev/null +++ b/Mistral-Trismegistus-7B.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44fa3622ca7e756a572983ec8c9a4fbd4071395daa93886425fdc40292b7e0cf +size 5942066368 diff --git a/Mistral-Trismegistus-7B.imatrix.gguf b/Mistral-Trismegistus-7B.imatrix.gguf new file mode 100644 index 0000000..1cd786f --- /dev/null +++ b/Mistral-Trismegistus-7B.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9666012c89da35ed8a89609a8d082ca9ac951c331a9b5011e66f8b211b6c30ce +size 5015200 diff --git a/README.md b/README.md new file mode 100644 index 0000000..23de235 --- /dev/null +++ b/README.md @@ -0,0 +1,92 @@ +--- +base_model: EducatingAI/Mistral-Trismegistus-7B +language: +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- mistral-7b +- instruct +- finetune +- gpt4 +- synthetic data +- distillation +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/EducatingAI/Mistral-Trismegistus-7B + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Mistral-Trismegistus-7B-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-IQ1_S.gguf) | i1-IQ1_S | 1.7 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-IQ1_M.gguf) | i1-IQ1_M | 1.9 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.1 | | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.3 | | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-IQ2_S.gguf) | i1-IQ2_S | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-IQ2_M.gguf) | i1-IQ2_M | 2.6 | | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 2.6 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-Q2_K.gguf) | i1-Q2_K | 2.8 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 2.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.1 | | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.3 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-IQ3_S.gguf) | i1-IQ3_S | 3.3 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-IQ3_M.gguf) | i1-IQ3_M | 3.4 | | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 3.6 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 3.9 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.0 | | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-Q4_0.gguf) | i1-Q4_0 | 4.2 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 4.2 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.2 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 4.5 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-Q4_1.gguf) | i1-Q4_1 | 4.7 | | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.1 | | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.2 | | +| [GGUF](https://huggingface.co/mradermacher/Mistral-Trismegistus-7B-i1-GGUF/resolve/main/Mistral-Trismegistus-7B.i1-Q6_K.gguf) | i1-Q6_K | 6.0 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +