commit 1f3da3e2173d452599e633b85dd42fb5516b444c Author: ModelHub XC Date: Sat May 30 03:00:20 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..106054a --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Arsenic-Shahrazad-12B-v4.1.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-IQ1_M.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-IQ1_M.gguf new file mode 100644 index 0000000..b2f1ed3 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3df6b1fcaeb7d6c393ea85d2d0b33ebb462ece91409a8ddc8f3f8a6549db3de8 +size 3221629792 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-IQ1_S.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-IQ1_S.gguf new file mode 100644 index 0000000..76f2203 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e562bfe01bf45445a4518db2aed3fd19a1559e6a0e95e727aaeecf625b28c80f +size 2999216992 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_M.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_M.gguf new file mode 100644 index 0000000..c38c48a --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:199752b2c9992ce0dd04d3b97f54946939b57914fe62daf714b21954566b88e0 +size 4435028832 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_S.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_S.gguf new file mode 100644 index 0000000..eb4e451 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3b65b37c134fd087066d27736bad1512667fe76a8fb54eb20a354698942880f +size 4138478432 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_XS.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_XS.gguf new file mode 100644 index 0000000..6435c7f --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ddf67a7f39df3312bf6de0133343f5984e93045f8b2c4fe24c17d5945fd10def +size 3915082592 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_XXS.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..89b1115 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc547bdad53a179835738edfe6651c6c475abf798887aee88fce66731f4fd6e0 +size 3592317792 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_M.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_M.gguf new file mode 100644 index 0000000..784a177 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1588602c7197ea74cf558843eb86a42921512215f621127cc54b19260d8aa39d +size 5722237792 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_S.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_S.gguf new file mode 100644 index 0000000..20da1c1 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:24ab3bccca2821e8efd9b7031cae5bfb74fd0e401ddbd6900f140323f028254a +size 5562084192 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_XS.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_XS.gguf new file mode 100644 index 0000000..281faf4 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5dbc80d40749bd433a642ce27e7158e5a8ef0e9224b20c29b2ff7c8b3e97eb98 +size 5306493792 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_XXS.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..929ab48 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e59a5c71ce26af9e9b6a400651ba0a72beabf395835520e9efc8ac9ef7cd3ef6 +size 4945390432 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-IQ4_NL.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-IQ4_NL.gguf new file mode 100644 index 0000000..9903923 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eeb17abc8dda0319a56b2da765ab6420c4bb3377e3c205c9f1f76a041cb90897 +size 7097920352 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-IQ4_XS.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-IQ4_XS.gguf new file mode 100644 index 0000000..00480e7 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0e314e188483c49674f511d6a6c88bf91df1b068a5a9b2e474514b76f0f35e69 +size 6742715232 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-Q2_K.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-Q2_K.gguf new file mode 100644 index 0000000..793be1c --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8b6c266302a42345ed4124b93c2e11082895da7043d3bd5af91665f950df4866 +size 4791053152 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-Q2_K_S.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-Q2_K_S.gguf new file mode 100644 index 0000000..29643b4 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:48056e0f1a5eb771c24853fca07fc49c551f50b73b48981d2b5f53886fde1518 +size 4493683552 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_L.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_L.gguf new file mode 100644 index 0000000..703280d --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:39992cabb2888cba13f309715b188b3b7e62e471c81c49fadc6add5adaab2969 +size 6561508192 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_M.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_M.gguf new file mode 100644 index 0000000..6338099 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cbc4bf368e4db144d930e5a17aee4e3331759ca603a862d5d173bafe0c2eabdd +size 6083095392 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_S.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_S.gguf new file mode 100644 index 0000000..3db4143 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:36a2554a042c7ece96b5b35054aaa928ee7628af6b1c3351dd9a45bea4434f24 +size 5534231392 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-Q4_0.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-Q4_0.gguf new file mode 100644 index 0000000..6ffade0 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:282284a5aebf41048978273a7269766372fe3a2d19b0f1c4bcf2dba3e9792a52 +size 7094643552 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-Q4_1.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-Q4_1.gguf new file mode 100644 index 0000000..737080a --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d9d377db1647f11b4dd2cdd08835a6c82a85a1bd0c51743a5c8c42b0fc9370bb +size 7795223392 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-Q4_K_M.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-Q4_K_M.gguf new file mode 100644 index 0000000..afff34c --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:62ce845e6f29d5bf20a8e108794cdf91f9c5e8cf653727c314e832abede6e4e4 +size 7477209952 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-Q4_K_S.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-Q4_K_S.gguf new file mode 100644 index 0000000..d17700e --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1b9513220a7bd7ab9c0879ef759d7902a7d743a84e95c9e34ec37c87789e702 +size 7120202592 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-Q5_K_M.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-Q5_K_M.gguf new file mode 100644 index 0000000..e331597 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b237601c4164c3b3c46bc1f06b318cdc8f6e0cae791f0ad46ff4bc4a93921cea +size 8727636832 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-Q5_K_S.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-Q5_K_S.gguf new file mode 100644 index 0000000..92fa920 --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e88dcd1be12b4100bb4433369f8468491a824085fd91b54b0cf5c679e191d009 +size 8518740832 diff --git a/Arsenic-Shahrazad-12B-v4.1.i1-Q6_K.gguf b/Arsenic-Shahrazad-12B-v4.1.i1-Q6_K.gguf new file mode 100644 index 0000000..562ae8d --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f742e9e0bd1c7ab1be544fb5be118365185424728ce734b5baf916a457b36141 +size 10056215392 diff --git a/Arsenic-Shahrazad-12B-v4.1.imatrix.gguf b/Arsenic-Shahrazad-12B-v4.1.imatrix.gguf new file mode 100644 index 0000000..e52ca4c --- /dev/null +++ b/Arsenic-Shahrazad-12B-v4.1.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:574dac42361f2e3d751a44f758dce46d8a0f4c46b8d930c77c41f471d7ae758a +size 7088192 diff --git a/README.md b/README.md new file mode 100644 index 0000000..fe56c3d --- /dev/null +++ b/README.md @@ -0,0 +1,87 @@ +--- +base_model: Lambent/Arsenic-Shahrazad-12B-v4.1 +language: +- en +library_name: transformers +license: cc-by-nc-4.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- not-for-all-audiences +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/Lambent/Arsenic-Shahrazad-12B-v4.1 + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Arsenic-Shahrazad-12B-v4.1-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-IQ1_S.gguf) | i1-IQ1_S | 3.1 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-IQ1_M.gguf) | i1-IQ1_M | 3.3 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 3.7 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.0 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_S.gguf) | i1-IQ2_S | 4.2 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-IQ2_M.gguf) | i1-IQ2_M | 4.5 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-Q2_K_S.gguf) | i1-Q2_K_S | 4.6 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-Q2_K.gguf) | i1-Q2_K | 4.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 5.0 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_XS.gguf) | i1-IQ3_XS | 5.4 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_S.gguf) | i1-Q3_K_S | 5.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_S.gguf) | i1-IQ3_S | 5.7 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-IQ3_M.gguf) | i1-IQ3_M | 5.8 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_M.gguf) | i1-Q3_K_M | 6.2 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-Q3_K_L.gguf) | i1-Q3_K_L | 6.7 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-IQ4_XS.gguf) | i1-IQ4_XS | 6.8 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-Q4_0.gguf) | i1-Q4_0 | 7.2 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-IQ4_NL.gguf) | i1-IQ4_NL | 7.2 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-Q4_K_S.gguf) | i1-Q4_K_S | 7.2 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-Q4_K_M.gguf) | i1-Q4_K_M | 7.6 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-Q4_1.gguf) | i1-Q4_1 | 7.9 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-Q5_K_S.gguf) | i1-Q5_K_S | 8.6 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-Q5_K_M.gguf) | i1-Q5_K_M | 8.8 | | +| [GGUF](https://huggingface.co/mradermacher/Arsenic-Shahrazad-12B-v4.1-i1-GGUF/resolve/main/Arsenic-Shahrazad-12B-v4.1.i1-Q6_K.gguf) | i1-Q6_K | 10.2 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +