commit 57090a00a6069b050922684c3bd379d9e1c6983b Author: ModelHub XC Date: Wed May 20 05:53:16 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/IRIS-18B-SFT-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..c17aa47 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +IRIS-18B-SFT.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/IRIS-18B-SFT.i1-IQ1_M.gguf b/IRIS-18B-SFT.i1-IQ1_M.gguf new file mode 100644 index 0000000..bb75dc4 --- /dev/null +++ b/IRIS-18B-SFT.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8aa4f55ce237a215c74764548e49f6ebd0f3629e98e44ab3e9054e367d920c77 +size 4125162016 diff --git a/IRIS-18B-SFT.i1-IQ1_S.gguf b/IRIS-18B-SFT.i1-IQ1_S.gguf new file mode 100644 index 0000000..6cb10ca --- /dev/null +++ b/IRIS-18B-SFT.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:84d6cba3798c1f9b24e950da7b756b84b213f839b159f2f7c21d17905dd31efb +size 3720395296 diff --git a/IRIS-18B-SFT.i1-IQ2_M.gguf b/IRIS-18B-SFT.i1-IQ2_M.gguf new file mode 100644 index 0000000..c8246ea --- /dev/null +++ b/IRIS-18B-SFT.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26c3818462ae3ceb487a6da9c572993a32c181f317c122f0e04b9821c8e12849 +size 5935450656 diff --git a/IRIS-18B-SFT.i1-IQ2_S.gguf b/IRIS-18B-SFT.i1-IQ2_S.gguf new file mode 100644 index 0000000..562e741 --- /dev/null +++ b/IRIS-18B-SFT.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3f36b19e02a74dafbe81686903a3cda7ae0fb9fd84f1626d01cd67a15514db02 +size 5395761696 diff --git a/IRIS-18B-SFT.i1-IQ2_XS.gguf b/IRIS-18B-SFT.i1-IQ2_XS.gguf new file mode 100644 index 0000000..8a9ead4 --- /dev/null +++ b/IRIS-18B-SFT.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1f0aa4989a4f09ce8c74ab4c6ec09e8ed778a1ee87baa1493a144a59ab65ea14 +size 5345196576 diff --git a/IRIS-18B-SFT.i1-IQ2_XXS.gguf b/IRIS-18B-SFT.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..6fb91e8 --- /dev/null +++ b/IRIS-18B-SFT.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2c33b57659ffb4ee8afeeaa092ab6773deae80b0794c41fc70d9792e21a153f0 +size 4799773216 diff --git a/IRIS-18B-SFT.i1-IQ3_M.gguf b/IRIS-18B-SFT.i1-IQ3_M.gguf new file mode 100644 index 0000000..bfe8c19 --- /dev/null +++ b/IRIS-18B-SFT.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1ba553bc5da999a22c29c084da28c331c90556b73b139e411ea66b18e80dc2ec +size 7944477216 diff --git a/IRIS-18B-SFT.i1-IQ3_S.gguf b/IRIS-18B-SFT.i1-IQ3_S.gguf new file mode 100644 index 0000000..5b68bc5 --- /dev/null +++ b/IRIS-18B-SFT.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1e798b7e0f23ba0c98320daf532336a22fcaff72d4d065dcdeac41ffb9ee6f44 +size 7859526176 diff --git a/IRIS-18B-SFT.i1-IQ3_XS.gguf b/IRIS-18B-SFT.i1-IQ3_XS.gguf new file mode 100644 index 0000000..ea54fa7 --- /dev/null +++ b/IRIS-18B-SFT.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:35f4ac312731cc8d5a1b3cb7a86e2b5f3d949859a0127e964c9f7a8ec9d58c3c +size 7431166496 diff --git a/IRIS-18B-SFT.i1-IQ3_XXS.gguf b/IRIS-18B-SFT.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..e53a991 --- /dev/null +++ b/IRIS-18B-SFT.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:16fd76c2456dc0b20723f35045c7b04e9d848292ad1a55e4ea601d5e40a23aa9 +size 6989638176 diff --git a/IRIS-18B-SFT.i1-IQ4_NL.gguf b/IRIS-18B-SFT.i1-IQ4_NL.gguf new file mode 100644 index 0000000..9d62b14 --- /dev/null +++ b/IRIS-18B-SFT.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:681bb0079105c95e3933285b7ad9a4fc787cb1af9bac1efa762ea2f4820eed7a +size 10214541856 diff --git a/IRIS-18B-SFT.i1-IQ4_XS.gguf b/IRIS-18B-SFT.i1-IQ4_XS.gguf new file mode 100644 index 0000000..43c17c7 --- /dev/null +++ b/IRIS-18B-SFT.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f614c6f9fee586cf26d85497eec2e8ae929730154d039fa645335370dd37db58 +size 9661499936 diff --git a/IRIS-18B-SFT.i1-Q2_K.gguf b/IRIS-18B-SFT.i1-Q2_K.gguf new file mode 100644 index 0000000..353f86d --- /dev/null +++ b/IRIS-18B-SFT.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dab76d873195a46267780d359535813a48becd584c6cd2d23e1c441c17fae89c +size 6666238496 diff --git a/IRIS-18B-SFT.i1-Q2_K_S.gguf b/IRIS-18B-SFT.i1-Q2_K_S.gguf new file mode 100644 index 0000000..bd80257 --- /dev/null +++ b/IRIS-18B-SFT.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:18929c8b46bbf9268ba5fdb8693dde690bb49b6c09d5a552e9eb93ef9d9f5794 +size 6169045536 diff --git a/IRIS-18B-SFT.i1-Q3_K_L.gguf b/IRIS-18B-SFT.i1-Q3_K_L.gguf new file mode 100644 index 0000000..ffe7583 --- /dev/null +++ b/IRIS-18B-SFT.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ecdd4b92ecd416fe0371f7791ff683eb89e4b65012e89e935ffbd99327859f59 +size 9397594656 diff --git a/IRIS-18B-SFT.i1-Q3_K_M.gguf b/IRIS-18B-SFT.i1-Q3_K_M.gguf new file mode 100644 index 0000000..d736b25 --- /dev/null +++ b/IRIS-18B-SFT.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:95813e18bead69e5c8bad8ab60ddcd649e5efb78747421e7d334497a21886f7a +size 8653761056 diff --git a/IRIS-18B-SFT.i1-Q3_K_S.gguf b/IRIS-18B-SFT.i1-Q3_K_S.gguf new file mode 100644 index 0000000..e7f28a0 --- /dev/null +++ b/IRIS-18B-SFT.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59e56c34b7d34e064a74054928f43c6820a71eef9a1b0407ead016ed7acdc733 +size 7854651936 diff --git a/IRIS-18B-SFT.i1-Q4_0.gguf b/IRIS-18B-SFT.i1-Q4_0.gguf new file mode 100644 index 0000000..e025d72 --- /dev/null +++ b/IRIS-18B-SFT.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f65fff54ade728630406343722250fb036f354a425a39f62c1497fbe0158cf1a +size 10238462496 diff --git a/IRIS-18B-SFT.i1-Q4_1.gguf b/IRIS-18B-SFT.i1-Q4_1.gguf new file mode 100644 index 0000000..ff89766 --- /dev/null +++ b/IRIS-18B-SFT.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:65a8236168b2137194c751d24105dc92cde469aeb6e77788a0f3e7e56a8b75d9 +size 11318331936 diff --git a/IRIS-18B-SFT.i1-Q4_K_M.gguf b/IRIS-18B-SFT.i1-Q4_K_M.gguf new file mode 100644 index 0000000..20b319a --- /dev/null +++ b/IRIS-18B-SFT.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8f2a4368b1db6b5240db114be6fa6f48bba323763b23a197cb27f4982a415215 +size 10934454816 diff --git a/IRIS-18B-SFT.i1-Q4_K_S.gguf b/IRIS-18B-SFT.i1-Q4_K_S.gguf new file mode 100644 index 0000000..b158ea3 --- /dev/null +++ b/IRIS-18B-SFT.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ea5754575f991b502a8246edd7e0ec5bc989f6e0cd3d5cef57cb14d6e6027d8d +size 10267626016 diff --git a/IRIS-18B-SFT.i1-Q5_K_M.gguf b/IRIS-18B-SFT.i1-Q5_K_M.gguf new file mode 100644 index 0000000..8335320 --- /dev/null +++ b/IRIS-18B-SFT.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0ad23e146aeca39ed2087e92a5467071d96eb4368a225308d885ee5accef6d59 +size 12799937056 diff --git a/IRIS-18B-SFT.i1-Q5_K_S.gguf b/IRIS-18B-SFT.i1-Q5_K_S.gguf new file mode 100644 index 0000000..9c3d741 --- /dev/null +++ b/IRIS-18B-SFT.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9691ce97b0fa2a46931e309b15595ddb17a5632477a506f878ff3d6e5a62d9e4 +size 12426709536 diff --git a/IRIS-18B-SFT.i1-Q6_K.gguf b/IRIS-18B-SFT.i1-Q6_K.gguf new file mode 100644 index 0000000..9845023 --- /dev/null +++ b/IRIS-18B-SFT.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:54c921a0fe7177962e0252604ae888b449461b02c88a7b873ccbee335ea4b8e2 +size 14782011936 diff --git a/IRIS-18B-SFT.imatrix.gguf b/IRIS-18B-SFT.imatrix.gguf new file mode 100644 index 0000000..58b6340 --- /dev/null +++ b/IRIS-18B-SFT.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:930fa2c6d8f8ae81a5e5a120470c748fc1e4ee2c66ffd8562ec3502bc06f8f9b +size 39823424 diff --git a/README.md b/README.md new file mode 100644 index 0000000..11bfcff --- /dev/null +++ b/README.md @@ -0,0 +1,85 @@ +--- +base_model: jerrimu/IRIS-18B-SFT +language: +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/jerrimu/IRIS-18B-SFT + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#IRIS-18B-SFT-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/IRIS-18B-SFT-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-IQ1_S.gguf) | i1-IQ1_S | 3.8 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-IQ1_M.gguf) | i1-IQ1_M | 4.2 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 4.9 | | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-IQ2_XS.gguf) | i1-IQ2_XS | 5.4 | | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-IQ2_S.gguf) | i1-IQ2_S | 5.5 | | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-IQ2_M.gguf) | i1-IQ2_M | 6.0 | | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-Q2_K_S.gguf) | i1-Q2_K_S | 6.3 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-Q2_K.gguf) | i1-Q2_K | 6.8 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 7.1 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-IQ3_XS.gguf) | i1-IQ3_XS | 7.5 | | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-Q3_K_S.gguf) | i1-Q3_K_S | 8.0 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-IQ3_S.gguf) | i1-IQ3_S | 8.0 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-IQ3_M.gguf) | i1-IQ3_M | 8.0 | | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-Q3_K_M.gguf) | i1-Q3_K_M | 8.8 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-Q3_K_L.gguf) | i1-Q3_K_L | 9.5 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-IQ4_XS.gguf) | i1-IQ4_XS | 9.8 | | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-IQ4_NL.gguf) | i1-IQ4_NL | 10.3 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-Q4_0.gguf) | i1-Q4_0 | 10.3 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-Q4_K_S.gguf) | i1-Q4_K_S | 10.4 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-Q4_K_M.gguf) | i1-Q4_K_M | 11.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-Q4_1.gguf) | i1-Q4_1 | 11.4 | | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-Q5_K_S.gguf) | i1-Q5_K_S | 12.5 | | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-Q5_K_M.gguf) | i1-Q5_K_M | 12.9 | | +| [GGUF](https://huggingface.co/mradermacher/IRIS-18B-SFT-i1-GGUF/resolve/main/IRIS-18B-SFT.i1-Q6_K.gguf) | i1-Q6_K | 14.9 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +