commit 9365feabb647abeb4059fdd927dd131f77082142 Author: ModelHub XC Date: Sun May 10 00:27:12 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/GaMS-9B-SFT-Translator-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..882f51f --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +GaMS-9B-SFT-Translator.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/GaMS-9B-SFT-Translator.i1-IQ1_M.gguf b/GaMS-9B-SFT-Translator.i1-IQ1_M.gguf new file mode 100644 index 0000000..028490a --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ed212fbe3d4c52258449d46a7d8a0ecf3498ae79df01e20a499b7c0de19b4583 +size 2545952768 diff --git a/GaMS-9B-SFT-Translator.i1-IQ1_S.gguf b/GaMS-9B-SFT-Translator.i1-IQ1_S.gguf new file mode 100644 index 0000000..3af744e --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:344a23949d7f0ca9b7d09e8d773d5e72824c4fd4b8aaa6aceb8ae0099bfb2630 +size 2378565632 diff --git a/GaMS-9B-SFT-Translator.i1-IQ2_M.gguf b/GaMS-9B-SFT-Translator.i1-IQ2_M.gguf new file mode 100644 index 0000000..a8820b2 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:98e182099f3c261376174acf6e1f231e7887b7e68bff1365ae225ce10ae8c55b +size 3434670080 diff --git a/GaMS-9B-SFT-Translator.i1-IQ2_S.gguf b/GaMS-9B-SFT-Translator.i1-IQ2_S.gguf new file mode 100644 index 0000000..86113e0 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a9831d9df553acd3617dadd9902f1a43cd3b62a86982da1f99d50a6ab62f64c +size 3211487232 diff --git a/GaMS-9B-SFT-Translator.i1-IQ2_XS.gguf b/GaMS-9B-SFT-Translator.i1-IQ2_XS.gguf new file mode 100644 index 0000000..83d00ea --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7ecae9ef00ec63348d9c4b2028026159653a3597485fe123e6407ab60094e97a +size 3067381760 diff --git a/GaMS-9B-SFT-Translator.i1-IQ2_XXS.gguf b/GaMS-9B-SFT-Translator.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..5ac752c --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a517c50ccbaa7e4cdaff2852896096c0318671afac38f2a3bdc19bbf54c068fa +size 2824931328 diff --git a/GaMS-9B-SFT-Translator.i1-IQ3_M.gguf b/GaMS-9B-SFT-Translator.i1-IQ3_M.gguf new file mode 100644 index 0000000..dbd36f7 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0e1be1e68b2fdc1defad2ab140ac99b46226ab5a6454e0d3072e6d6e3ca84688 +size 4494616576 diff --git a/GaMS-9B-SFT-Translator.i1-IQ3_S.gguf b/GaMS-9B-SFT-Translator.i1-IQ3_S.gguf new file mode 100644 index 0000000..32f61d1 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:66e131cfcfa5856f0d0154586d519ba0e1e9c54eb61511f3d91dc972e3d40083 +size 4337666048 diff --git a/GaMS-9B-SFT-Translator.i1-IQ3_XS.gguf b/GaMS-9B-SFT-Translator.i1-IQ3_XS.gguf new file mode 100644 index 0000000..fe70603 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6370b73c37002caf6dd9377ab511d36f0b4e5851b970c6c993271a3b940b60fe +size 4144990208 diff --git a/GaMS-9B-SFT-Translator.i1-IQ3_XXS.gguf b/GaMS-9B-SFT-Translator.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..65a9288 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4ee091f27dde6426799bc329735c43b253350a96733d53a933d63ef34d22a5a1 +size 3796740096 diff --git a/GaMS-9B-SFT-Translator.i1-IQ4_NL.gguf b/GaMS-9B-SFT-Translator.i1-IQ4_NL.gguf new file mode 100644 index 0000000..3005e9c --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:84ac0f402143164c005a38f7f4f4e8719401c47cabc162d4a7d65b742eb05ab3 +size 5443143680 diff --git a/GaMS-9B-SFT-Translator.i1-IQ4_XS.gguf b/GaMS-9B-SFT-Translator.i1-IQ4_XS.gguf new file mode 100644 index 0000000..1cff8c7 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ed9d314734a6eeb23fe590d4eb78c3f1d1747a3eb466c70f02e38811a012d600 +size 5183031296 diff --git a/GaMS-9B-SFT-Translator.i1-Q2_K.gguf b/GaMS-9B-SFT-Translator.i1-Q2_K.gguf new file mode 100644 index 0000000..984ec06 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c05701bfce1fe868fdc4aeeaafe62af4e12082cfaa22d724f6c4d7713a9c2aca +size 3805399040 diff --git a/GaMS-9B-SFT-Translator.i1-Q2_K_S.gguf b/GaMS-9B-SFT-Translator.i1-Q2_K_S.gguf new file mode 100644 index 0000000..2cd183e --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0352fb7eaf8aa9b316a93db84b4b76f9b3f803412cdb2db9f454dac66270e6cb +size 3552512000 diff --git a/GaMS-9B-SFT-Translator.i1-Q3_K_L.gguf b/GaMS-9B-SFT-Translator.i1-Q3_K_L.gguf new file mode 100644 index 0000000..343cad2 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f745cb2b540af8135a36cfb5c9c6a54d3f11ccbdb8d19561598e45851f1dc1b8 +size 5132453888 diff --git a/GaMS-9B-SFT-Translator.i1-Q3_K_M.gguf b/GaMS-9B-SFT-Translator.i1-Q3_K_M.gguf new file mode 100644 index 0000000..4450e34 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5536de3d18ef59d72639b95d5c00961a95b8fe2017913836163041f22124e0f6 +size 4761782272 diff --git a/GaMS-9B-SFT-Translator.i1-Q3_K_S.gguf b/GaMS-9B-SFT-Translator.i1-Q3_K_S.gguf new file mode 100644 index 0000000..62d4017 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d68dff9524ead95fa6fe77b9dedc0b109e0118bfb85ff238041cd545f3164843 +size 4337666048 diff --git a/GaMS-9B-SFT-Translator.i1-Q4_0.gguf b/GaMS-9B-SFT-Translator.i1-Q4_0.gguf new file mode 100644 index 0000000..399ad5f --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f214acd468af122eb028f61558184cdb274ab5c9d36cf67d6d42ba0448c8d168 +size 5459200000 diff --git a/GaMS-9B-SFT-Translator.i1-Q4_1.gguf b/GaMS-9B-SFT-Translator.i1-Q4_1.gguf new file mode 100644 index 0000000..9294970 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0a5247e806dcce05cf78c6b84ea660130399453f444a35ce62dc5ab5620524fa +size 5963368448 diff --git a/GaMS-9B-SFT-Translator.i1-Q4_K_M.gguf b/GaMS-9B-SFT-Translator.i1-Q4_K_M.gguf new file mode 100644 index 0000000..3fde62a --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b1d554eb01045200703fcec25c8444455eedd7403201af54bf42ebee5fc2531e +size 5761058816 diff --git a/GaMS-9B-SFT-Translator.i1-Q4_K_S.gguf b/GaMS-9B-SFT-Translator.i1-Q4_K_S.gguf new file mode 100644 index 0000000..9f3ec59 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4eba6a64599f50eb0997682d81403e2add15e10829dec2a9b0cba9cb3ecd73a3 +size 5478926336 diff --git a/GaMS-9B-SFT-Translator.i1-Q5_K_M.gguf b/GaMS-9B-SFT-Translator.i1-Q5_K_M.gguf new file mode 100644 index 0000000..154469f --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f02a0aa679a7dd43ce6f5cb6e656d42f113e50890f99dfaf632df543f5ddbbca +size 6647367680 diff --git a/GaMS-9B-SFT-Translator.i1-Q5_K_S.gguf b/GaMS-9B-SFT-Translator.i1-Q5_K_S.gguf new file mode 100644 index 0000000..c91e211 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:90b869001c4f015a883709f097504684ee2ab3ac5699d0945cf66055cc59f0f9 +size 6483593216 diff --git a/GaMS-9B-SFT-Translator.i1-Q6_K.gguf b/GaMS-9B-SFT-Translator.i1-Q6_K.gguf new file mode 100644 index 0000000..f2b3c14 --- /dev/null +++ b/GaMS-9B-SFT-Translator.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bbde63d857b50b10e9e8fd69af13508911862d6ff1639f36237763787ab9ca21 +size 7589070848 diff --git a/GaMS-9B-SFT-Translator.imatrix.gguf b/GaMS-9B-SFT-Translator.imatrix.gguf new file mode 100644 index 0000000..65017e2 --- /dev/null +++ b/GaMS-9B-SFT-Translator.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:754c178c9bbd802b0d50a7a27c36763655e2d287ab8ae5d0d35dac1299f7ea5d +size 6152352 diff --git a/README.md b/README.md new file mode 100644 index 0000000..711ca37 --- /dev/null +++ b/README.md @@ -0,0 +1,86 @@ +--- +base_model: GaMS-Beta/GaMS-9B-SFT-Translator +language: +- sl +- en +library_name: transformers +license: gemma +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/GaMS-Beta/GaMS-9B-SFT-Translator + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#GaMS-9B-SFT-Translator-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-IQ1_S.gguf) | i1-IQ1_S | 2.5 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-IQ1_M.gguf) | i1-IQ1_M | 2.6 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-IQ2_XS.gguf) | i1-IQ2_XS | 3.2 | | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-IQ2_S.gguf) | i1-IQ2_S | 3.3 | | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-IQ2_M.gguf) | i1-IQ2_M | 3.5 | | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.7 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-Q2_K.gguf) | i1-Q2_K | 3.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-IQ3_XS.gguf) | i1-IQ3_XS | 4.2 | | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-IQ3_S.gguf) | i1-IQ3_S | 4.4 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-Q3_K_S.gguf) | i1-Q3_K_S | 4.4 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-IQ3_M.gguf) | i1-IQ3_M | 4.6 | | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.9 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-Q3_K_L.gguf) | i1-Q3_K_L | 5.2 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-IQ4_XS.gguf) | i1-IQ4_XS | 5.3 | | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-IQ4_NL.gguf) | i1-IQ4_NL | 5.5 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-Q4_0.gguf) | i1-Q4_0 | 5.6 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-Q4_K_S.gguf) | i1-Q4_K_S | 5.6 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.9 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-Q4_1.gguf) | i1-Q4_1 | 6.1 | | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-Q5_K_S.gguf) | i1-Q5_K_S | 6.6 | | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-Q5_K_M.gguf) | i1-Q5_K_M | 6.7 | | +| [GGUF](https://huggingface.co/mradermacher/GaMS-9B-SFT-Translator-i1-GGUF/resolve/main/GaMS-9B-SFT-Translator.i1-Q6_K.gguf) | i1-Q6_K | 7.7 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +