commit 1ac2765bc3c447b769fcbd73af1d00a8ba084c9a Author: ModelHub XC Date: Sat May 9 21:21:07 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..8e90f61 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v2.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Arsenic-Shahrazad-12B-v2.i1-IQ1_M.gguf b/Arsenic-Shahrazad-12B-v2.i1-IQ1_M.gguf new file mode 100644 index 0000000..a00d659 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1de595507e670d243259f5d2d77834761ec6dde1e22cf6b225bd20915bcb35bb +size 3221628704 diff --git a/Arsenic-Shahrazad-12B-v2.i1-IQ1_S.gguf b/Arsenic-Shahrazad-12B-v2.i1-IQ1_S.gguf new file mode 100644 index 0000000..aec7947 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9d6a1c0bf8a137a255822d03153ecca2101be3228cfa8b3abfcf1c2396c764ab +size 2999215904 diff --git a/Arsenic-Shahrazad-12B-v2.i1-IQ2_M.gguf b/Arsenic-Shahrazad-12B-v2.i1-IQ2_M.gguf new file mode 100644 index 0000000..bbec200 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d54556ebc260b25340532f721e8a47078bc8b6f95542f90aa0485442816131a1 +size 4435027744 diff --git a/Arsenic-Shahrazad-12B-v2.i1-IQ2_S.gguf b/Arsenic-Shahrazad-12B-v2.i1-IQ2_S.gguf new file mode 100644 index 0000000..38a8d4b --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9fd721559e7b9fdf5f8ed8271e8f9377399b47760e9bde9d8ad1d070907a6014 +size 4138477344 diff --git a/Arsenic-Shahrazad-12B-v2.i1-IQ2_XS.gguf b/Arsenic-Shahrazad-12B-v2.i1-IQ2_XS.gguf new file mode 100644 index 0000000..f22c8bb --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b850955e918ef4f0606184c3418a934b7c86f315d1fb76a6db0aa14f061846a +size 3915081504 diff --git a/Arsenic-Shahrazad-12B-v2.i1-IQ2_XXS.gguf b/Arsenic-Shahrazad-12B-v2.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..da9a449 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5a1d9e4043b6b8a1c83ee8e8a3a743ebec551e47add25bfcb16477245e515997 +size 3592316704 diff --git a/Arsenic-Shahrazad-12B-v2.i1-IQ3_M.gguf b/Arsenic-Shahrazad-12B-v2.i1-IQ3_M.gguf new file mode 100644 index 0000000..9f5da22 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cce02b4d8b4cec750849a20b9de6d44ab04676a343bc41229e9dd928b4d0cf71 +size 5722236704 diff --git a/Arsenic-Shahrazad-12B-v2.i1-IQ3_S.gguf b/Arsenic-Shahrazad-12B-v2.i1-IQ3_S.gguf new file mode 100644 index 0000000..79c93f5 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:824bdc6a3a29342824a25e67fd171d398d00ae0fe7c3818f01d88904624724d8 +size 5562083104 diff --git a/Arsenic-Shahrazad-12B-v2.i1-IQ3_XS.gguf b/Arsenic-Shahrazad-12B-v2.i1-IQ3_XS.gguf new file mode 100644 index 0000000..34457e0 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:64c217055ece3bc6138bdd07f17d7e5ccef5e01221865a246c0c395819b9d815 +size 5306492704 diff --git a/Arsenic-Shahrazad-12B-v2.i1-IQ3_XXS.gguf b/Arsenic-Shahrazad-12B-v2.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..0bde7e3 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e26bd9b9b9b5685e1d0170e7e481853ded48a39e9e7412c9227e7c443758ce55 +size 4945389344 diff --git a/Arsenic-Shahrazad-12B-v2.i1-IQ4_NL.gguf b/Arsenic-Shahrazad-12B-v2.i1-IQ4_NL.gguf new file mode 100644 index 0000000..28a0f66 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:21d1ddc72666fcb26f1db4d054d22d986c5279af15b56059fbcd251f16fe53ea +size 7097919264 diff --git a/Arsenic-Shahrazad-12B-v2.i1-IQ4_XS.gguf b/Arsenic-Shahrazad-12B-v2.i1-IQ4_XS.gguf new file mode 100644 index 0000000..7effce0 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9dba7ac689058164629dfb49cd6eff5cb9fe92cdf3afa7b2b8162f27410f0417 +size 6742714144 diff --git a/Arsenic-Shahrazad-12B-v2.i1-Q2_K.gguf b/Arsenic-Shahrazad-12B-v2.i1-Q2_K.gguf new file mode 100644 index 0000000..2b2d364 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:81a3c2c9f232baefe746e1f84bdb7c03a2e10296f70e76d9acf1659a5cb5dc40 +size 4791052064 diff --git a/Arsenic-Shahrazad-12B-v2.i1-Q2_K_S.gguf b/Arsenic-Shahrazad-12B-v2.i1-Q2_K_S.gguf new file mode 100644 index 0000000..59bbbf3 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e9221e67b344f94d7439bc94775ebc88c341f353fc7427215d6558773b247624 +size 4493682464 diff --git a/Arsenic-Shahrazad-12B-v2.i1-Q3_K_L.gguf b/Arsenic-Shahrazad-12B-v2.i1-Q3_K_L.gguf new file mode 100644 index 0000000..c3815ef --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:257371ce8a65f039bda9a3d16df9af0f156cfd207ddc376dac490df1761e62d6 +size 6561507104 diff --git a/Arsenic-Shahrazad-12B-v2.i1-Q3_K_M.gguf b/Arsenic-Shahrazad-12B-v2.i1-Q3_K_M.gguf new file mode 100644 index 0000000..196f50e --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2a172a1b06688a3a5d222878cfa8ec36e23be77610281349682fc1185bd6db9e +size 6083094304 diff --git a/Arsenic-Shahrazad-12B-v2.i1-Q3_K_S.gguf b/Arsenic-Shahrazad-12B-v2.i1-Q3_K_S.gguf new file mode 100644 index 0000000..fe8665a --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44498600e75504c9ca07623cb467f4f179ce6b67a2496b49c439b30b8ee55473 +size 5534230304 diff --git a/Arsenic-Shahrazad-12B-v2.i1-Q4_0.gguf b/Arsenic-Shahrazad-12B-v2.i1-Q4_0.gguf new file mode 100644 index 0000000..a3eadb1 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:be176e0caf58422c81f922ea4c47383fb867aec676e04428ce6e01d97c099e04 +size 7094642464 diff --git a/Arsenic-Shahrazad-12B-v2.i1-Q4_1.gguf b/Arsenic-Shahrazad-12B-v2.i1-Q4_1.gguf new file mode 100644 index 0000000..775ed49 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:11c6d0eb4d398d96a1ec2ddaf39aa85f225bf8fcdfc5d29d69c977f19690d8fd +size 7795222304 diff --git a/Arsenic-Shahrazad-12B-v2.i1-Q4_K_M.gguf b/Arsenic-Shahrazad-12B-v2.i1-Q4_K_M.gguf new file mode 100644 index 0000000..fdb0d89 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3876925c1c2d992e6df342c17bf6afd447964da9fd2cc8fcb81dab1bcc9005c1 +size 7477208864 diff --git a/Arsenic-Shahrazad-12B-v2.i1-Q4_K_S.gguf b/Arsenic-Shahrazad-12B-v2.i1-Q4_K_S.gguf new file mode 100644 index 0000000..4fb3ea9 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4f5a1fb212bc8341da3d83bb460f4dfa08a511c59c3e869ee034c125018a2425 +size 7120201504 diff --git a/Arsenic-Shahrazad-12B-v2.i1-Q5_K_M.gguf b/Arsenic-Shahrazad-12B-v2.i1-Q5_K_M.gguf new file mode 100644 index 0000000..78fd6c7 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c7de38a7ae51c64b4ed80365d22620e76958c3c64437e33346f8517896b474b0 +size 8727635744 diff --git a/Arsenic-Shahrazad-12B-v2.i1-Q5_K_S.gguf b/Arsenic-Shahrazad-12B-v2.i1-Q5_K_S.gguf new file mode 100644 index 0000000..7946a5f --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:10527f990eb42f68a0f8d1a988a259b6dea3495942fc9f40edac28592980b922 +size 8518739744 diff --git a/Arsenic-Shahrazad-12B-v2.i1-Q6_K.gguf b/Arsenic-Shahrazad-12B-v2.i1-Q6_K.gguf new file mode 100644 index 0000000..4a9d020 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5a7cc69e6737b66d5ff2a72cb58ca2d0b33c25a7eac828f02d8a3999ccb0cb85 +size 10056214304 diff --git a/Arsenic-Shahrazad-12B-v2.imatrix.gguf b/Arsenic-Shahrazad-12B-v2.imatrix.gguf new file mode 100644 index 0000000..4aaa64a --- /dev/null +++ b/Arsenic-Shahrazad-12B-v2.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:15c270be878becaa403e402abffbe49b029beb2ba45f428e4c9af7ed22e531bd +size 7088192 diff --git a/README.md b/README.md new file mode 100644 index 0000000..317e699 --- /dev/null +++ b/README.md @@ -0,0 +1,87 @@ +--- +base_model: Lambent/Arsenic-Shahrazad-12B-v2 +language: +- en +library_name: transformers +license: cc-by-nc-4.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- not-for-all-audiences +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/Lambent/Arsenic-Shahrazad-12B-v2 + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Arsenic-Shahrazad-12B-v2-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-IQ1_S.gguf) | i1-IQ1_S | 3.1 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-IQ1_M.gguf) | i1-IQ1_M | 3.3 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 3.7 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.0 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-IQ2_S.gguf) | i1-IQ2_S | 4.2 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-IQ2_M.gguf) | i1-IQ2_M | 4.5 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-Q2_K_S.gguf) | i1-Q2_K_S | 4.6 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-Q2_K.gguf) | i1-Q2_K | 4.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 5.0 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-IQ3_XS.gguf) | i1-IQ3_XS | 5.4 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-Q3_K_S.gguf) | i1-Q3_K_S | 5.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-IQ3_S.gguf) | i1-IQ3_S | 5.7 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-IQ3_M.gguf) | i1-IQ3_M | 5.8 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-Q3_K_M.gguf) | i1-Q3_K_M | 6.2 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-Q3_K_L.gguf) | i1-Q3_K_L | 6.7 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-IQ4_XS.gguf) | i1-IQ4_XS | 6.8 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-Q4_0.gguf) | i1-Q4_0 | 7.2 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-IQ4_NL.gguf) | i1-IQ4_NL | 7.2 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-Q4_K_S.gguf) | i1-Q4_K_S | 7.2 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-Q4_K_M.gguf) | i1-Q4_K_M | 7.6 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-Q4_1.gguf) | i1-Q4_1 | 7.9 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-Q5_K_S.gguf) | i1-Q5_K_S | 8.6 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-Q5_K_M.gguf) | i1-Q5_K_M | 8.8 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v2-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v2.i1-Q6_K.gguf) | i1-Q6_K | 10.2 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +