commit e115d74b1da108c6ee9cc463c9970dc24b421f54 Author: ModelHub XC Date: Thu Jun 25 17:43:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Amethyst-12B-v3-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..8366ced --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Amethyst-12B-v3.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Amethyst-12B-v3.i1-IQ1_M.gguf b/Amethyst-12B-v3.i1-IQ1_M.gguf new file mode 100644 index 0000000..09c7a41 --- /dev/null +++ b/Amethyst-12B-v3.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:399736e302ecc3d7c69932d01c5a9f71871ea562804e2023fd1af7b038ce27bc +size 3221629792 diff --git a/Amethyst-12B-v3.i1-IQ1_S.gguf b/Amethyst-12B-v3.i1-IQ1_S.gguf new file mode 100644 index 0000000..9a81af7 --- /dev/null +++ b/Amethyst-12B-v3.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:224a1a7f07340474d39ff15cd039c312f880c026547f610d93e2fb29a58a3fb3 +size 2999216992 diff --git a/Amethyst-12B-v3.i1-IQ2_M.gguf b/Amethyst-12B-v3.i1-IQ2_M.gguf new file mode 100644 index 0000000..be11245 --- /dev/null +++ b/Amethyst-12B-v3.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a73883a48933e942b05e2f8185b2fbcdf45ed9cc91beb3d873fae549c300297c +size 4435028832 diff --git a/Amethyst-12B-v3.i1-IQ2_S.gguf b/Amethyst-12B-v3.i1-IQ2_S.gguf new file mode 100644 index 0000000..c73b38f --- /dev/null +++ b/Amethyst-12B-v3.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:717b6d43aa9cc7409c445f4b46ef73ec75ad7e6793d7c4a67db0e588ae2c1251 +size 4138478432 diff --git a/Amethyst-12B-v3.i1-IQ2_XS.gguf b/Amethyst-12B-v3.i1-IQ2_XS.gguf new file mode 100644 index 0000000..43a8e7c --- /dev/null +++ b/Amethyst-12B-v3.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c0cebed4dba294740c09d96fbdc559db85a6b1ea5889b26038dfc53a0b11e7ad +size 3915082592 diff --git a/Amethyst-12B-v3.i1-IQ2_XXS.gguf b/Amethyst-12B-v3.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..e1580c4 --- /dev/null +++ b/Amethyst-12B-v3.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:56404424ffc86a9a2519df5e6fcc7df8ce190447e047d7f1d653cd17eee1b570 +size 3592317792 diff --git a/Amethyst-12B-v3.i1-IQ3_M.gguf b/Amethyst-12B-v3.i1-IQ3_M.gguf new file mode 100644 index 0000000..6c9e99d --- /dev/null +++ b/Amethyst-12B-v3.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3b2ff340f5e20c7331734a7a07025aa3a6fd327f3ef5007801e27701171c9136 +size 5722237792 diff --git a/Amethyst-12B-v3.i1-IQ3_S.gguf b/Amethyst-12B-v3.i1-IQ3_S.gguf new file mode 100644 index 0000000..ed7c568 --- /dev/null +++ b/Amethyst-12B-v3.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:24dd096cc4b1b3911924b7ee480b190471fe7a9e26be1b5cce7b65334f4ae285 +size 5562084192 diff --git a/Amethyst-12B-v3.i1-IQ3_XS.gguf b/Amethyst-12B-v3.i1-IQ3_XS.gguf new file mode 100644 index 0000000..e691686 --- /dev/null +++ b/Amethyst-12B-v3.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f946f85e7eb6d0cf3fccef262371a0c13c1af60701c1513ee216585a14faf3d1 +size 5306493792 diff --git a/Amethyst-12B-v3.i1-IQ3_XXS.gguf b/Amethyst-12B-v3.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..6186952 --- /dev/null +++ b/Amethyst-12B-v3.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3cae2e283bdc9fea17dcedf9d20f5e05b98115315c05a69ef78a017c2bb96b8d +size 4945390432 diff --git a/Amethyst-12B-v3.i1-IQ4_NL.gguf b/Amethyst-12B-v3.i1-IQ4_NL.gguf new file mode 100644 index 0000000..8969ef8 --- /dev/null +++ b/Amethyst-12B-v3.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44ce561edc5cbe79cde640d643ef891297179f23f0cc68b42848f42002680577 +size 7097920352 diff --git a/Amethyst-12B-v3.i1-IQ4_XS.gguf b/Amethyst-12B-v3.i1-IQ4_XS.gguf new file mode 100644 index 0000000..48a4b97 --- /dev/null +++ b/Amethyst-12B-v3.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37bda6daf620107b6b44e0f4770a1a33b004653d94813e4807e3eeb2c8cacd04 +size 6742715232 diff --git a/Amethyst-12B-v3.i1-Q2_K.gguf b/Amethyst-12B-v3.i1-Q2_K.gguf new file mode 100644 index 0000000..78af080 --- /dev/null +++ b/Amethyst-12B-v3.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b0a34fa4e7154d42ea7509114f77d698926496484d95e71a145cdc426d30e056 +size 4791053152 diff --git a/Amethyst-12B-v3.i1-Q2_K_S.gguf b/Amethyst-12B-v3.i1-Q2_K_S.gguf new file mode 100644 index 0000000..e30babe --- /dev/null +++ b/Amethyst-12B-v3.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f60654fe98d74447755b711a44fe4441eec94cf7dd24f5406d9010f0615092ef +size 4493683552 diff --git a/Amethyst-12B-v3.i1-Q3_K_L.gguf b/Amethyst-12B-v3.i1-Q3_K_L.gguf new file mode 100644 index 0000000..a87c8b1 --- /dev/null +++ b/Amethyst-12B-v3.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5b91757b1d3075438dedc6dac4edbabb1a48acbad53b3f8bda0ef8e2dddde81f +size 6561508192 diff --git a/Amethyst-12B-v3.i1-Q3_K_M.gguf b/Amethyst-12B-v3.i1-Q3_K_M.gguf new file mode 100644 index 0000000..7bbd6cd --- /dev/null +++ b/Amethyst-12B-v3.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7522cde9b5b99c2573361c28707cd1ff24b3b10b17efd583b84a9c59101c7d01 +size 6083095392 diff --git a/Amethyst-12B-v3.i1-Q3_K_S.gguf b/Amethyst-12B-v3.i1-Q3_K_S.gguf new file mode 100644 index 0000000..188345f --- /dev/null +++ b/Amethyst-12B-v3.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4c2fa23552ee21723387a53411e11e1e3840a0515bdab497b3d6fcc0aed6c813 +size 5534231392 diff --git a/Amethyst-12B-v3.i1-Q4_0.gguf b/Amethyst-12B-v3.i1-Q4_0.gguf new file mode 100644 index 0000000..362a236 --- /dev/null +++ b/Amethyst-12B-v3.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:05288cc7734af458cf5a10ad6fee301304011d05739953bebbc598ac4d904df5 +size 7094643552 diff --git a/Amethyst-12B-v3.i1-Q4_1.gguf b/Amethyst-12B-v3.i1-Q4_1.gguf new file mode 100644 index 0000000..6719b68 --- /dev/null +++ b/Amethyst-12B-v3.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f502547285d421398d6ba7d85f63c99bab8b8a4572f5f1229e0ff0b0b995e142 +size 7795223392 diff --git a/Amethyst-12B-v3.i1-Q4_K_M.gguf b/Amethyst-12B-v3.i1-Q4_K_M.gguf new file mode 100644 index 0000000..d097191 --- /dev/null +++ b/Amethyst-12B-v3.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ddca0af760274702898d441ae28cb9da8fb05c9ac4b27ef0f7ba781090425e4a +size 7477209952 diff --git a/Amethyst-12B-v3.i1-Q4_K_S.gguf b/Amethyst-12B-v3.i1-Q4_K_S.gguf new file mode 100644 index 0000000..8c252d3 --- /dev/null +++ b/Amethyst-12B-v3.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f97a243f215e8edc663f0754dde22ea8c8e2ed45d9190a9dabec95d50475b190 +size 7120202592 diff --git a/Amethyst-12B-v3.i1-Q5_K_M.gguf b/Amethyst-12B-v3.i1-Q5_K_M.gguf new file mode 100644 index 0000000..10cdb1b --- /dev/null +++ b/Amethyst-12B-v3.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3fe7b36eb10865e6e641f48b21c2e2888af5bea101023d4350f90778131d5a0a +size 8727636832 diff --git a/Amethyst-12B-v3.i1-Q5_K_S.gguf b/Amethyst-12B-v3.i1-Q5_K_S.gguf new file mode 100644 index 0000000..4866fc5 --- /dev/null +++ b/Amethyst-12B-v3.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6c46535b8c83ad51aa67c0dea66502d2bdb493482f0fef0d67435d6951cbfa9d +size 8518740832 diff --git a/Amethyst-12B-v3.i1-Q6_K.gguf b/Amethyst-12B-v3.i1-Q6_K.gguf new file mode 100644 index 0000000..4c825c0 --- /dev/null +++ b/Amethyst-12B-v3.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:20b844025810ce36b737e0d36e6d43d8e71744ef836b12de2fcc312395264202 +size 10056215392 diff --git a/Amethyst-12B-v3.imatrix.gguf b/Amethyst-12B-v3.imatrix.gguf new file mode 100644 index 0000000..78b426d --- /dev/null +++ b/Amethyst-12B-v3.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3faefc15a5fe94c292e2c12b8eddd231d3cc208f76d4685154972171a3f2fba3 +size 7088192 diff --git a/README.md b/README.md new file mode 100644 index 0000000..73dd8c2 --- /dev/null +++ b/README.md @@ -0,0 +1,87 @@ +--- +base_model: Lambent/Amethyst-12B-v3 +language: +- en +library_name: transformers +license: cc-by-nc-4.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- not-for-all-audiences +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/Lambent/Amethyst-12B-v3 + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Amethyst-12B-v3-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Amethyst-12B-v3-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-IQ1_S.gguf) | i1-IQ1_S | 3.1 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-IQ1_M.gguf) | i1-IQ1_M | 3.3 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 3.7 | | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.0 | | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-IQ2_S.gguf) | i1-IQ2_S | 4.2 | | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-IQ2_M.gguf) | i1-IQ2_M | 4.5 | | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-Q2_K_S.gguf) | i1-Q2_K_S | 4.6 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-Q2_K.gguf) | i1-Q2_K | 4.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 5.0 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-IQ3_XS.gguf) | i1-IQ3_XS | 5.4 | | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-Q3_K_S.gguf) | i1-Q3_K_S | 5.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-IQ3_S.gguf) | i1-IQ3_S | 5.7 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-IQ3_M.gguf) | i1-IQ3_M | 5.8 | | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-Q3_K_M.gguf) | i1-Q3_K_M | 6.2 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-Q3_K_L.gguf) | i1-Q3_K_L | 6.7 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-IQ4_XS.gguf) | i1-IQ4_XS | 6.8 | | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-Q4_0.gguf) | i1-Q4_0 | 7.2 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-IQ4_NL.gguf) | i1-IQ4_NL | 7.2 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-Q4_K_S.gguf) | i1-Q4_K_S | 7.2 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-Q4_K_M.gguf) | i1-Q4_K_M | 7.6 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-Q4_1.gguf) | i1-Q4_1 | 7.9 | | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-Q5_K_S.gguf) | i1-Q5_K_S | 8.6 | | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-Q5_K_M.gguf) | i1-Q5_K_M | 8.8 | | +| [GGUF](https://huggingface.co/mradermacher/Amethyst-12B-v3-i1-GGUF/resolve/main/Amethyst-12B-v3.i1-Q6_K.gguf) | i1-Q6_K | 10.2 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +