commit 41c0743734efe3aece9cf30f0ea9a3a19eb1aa01 Author: ModelHub XC Date: Sun Aug 9 10:47:16 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..30d378a --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +G2-Darkest-Writer-9B-v1.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/G2-Darkest-Writer-9B-v1.i1-IQ1_M.gguf b/G2-Darkest-Writer-9B-v1.i1-IQ1_M.gguf new file mode 100644 index 0000000..5950ac4 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:46efe677a24f3e32f9f1fdbc682a4b23b6e8882c4eb6642235ae6c83607c97f8 +size 2545952352 diff --git a/G2-Darkest-Writer-9B-v1.i1-IQ1_S.gguf b/G2-Darkest-Writer-9B-v1.i1-IQ1_S.gguf new file mode 100644 index 0000000..73edccd --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f2846a8254fe67c8006345b456152322ebc429f19e32839cab0c0a2557de3a5c +size 2378565216 diff --git a/G2-Darkest-Writer-9B-v1.i1-IQ2_M.gguf b/G2-Darkest-Writer-9B-v1.i1-IQ2_M.gguf new file mode 100644 index 0000000..45330a0 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:10f7cd8d2ebee9bc9215b700913c803e3cab34670a6c823db590a58e17f816b4 +size 3434669664 diff --git a/G2-Darkest-Writer-9B-v1.i1-IQ2_S.gguf b/G2-Darkest-Writer-9B-v1.i1-IQ2_S.gguf new file mode 100644 index 0000000..92c7b7f --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e88a2f3fe934d502d5ad45e31c349a427ac0832a1acabf2d60a21cf5a4d5e19e +size 3211486816 diff --git a/G2-Darkest-Writer-9B-v1.i1-IQ2_XS.gguf b/G2-Darkest-Writer-9B-v1.i1-IQ2_XS.gguf new file mode 100644 index 0000000..7c781e2 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:78a1cc466a1566533c250a8d47e4d880ba0e3a21cb7ebc7645894576970fe45a +size 3067381344 diff --git a/G2-Darkest-Writer-9B-v1.i1-IQ2_XXS.gguf b/G2-Darkest-Writer-9B-v1.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..8c98e32 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8e0c7d5123b7acccfb312431063fca3917a38b0e1501f4858fd6d0416c4523f +size 2824930912 diff --git a/G2-Darkest-Writer-9B-v1.i1-IQ3_M.gguf b/G2-Darkest-Writer-9B-v1.i1-IQ3_M.gguf new file mode 100644 index 0000000..ef2a0f1 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:03a7a1cb87b4466eaf0689324ec4f5adad912899a42b4ca589c93cb2c7ead94c +size 4494616160 diff --git a/G2-Darkest-Writer-9B-v1.i1-IQ3_S.gguf b/G2-Darkest-Writer-9B-v1.i1-IQ3_S.gguf new file mode 100644 index 0000000..2d949f0 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:21b2d44b9eb26133b7dec8ff4fe56b45b2cda16cc34ef00cba79c7e7836a7660 +size 4337665632 diff --git a/G2-Darkest-Writer-9B-v1.i1-IQ3_XS.gguf b/G2-Darkest-Writer-9B-v1.i1-IQ3_XS.gguf new file mode 100644 index 0000000..81843a7 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c64882629375d6ea6760c162174d5bc436dd8f72d603345ac0f96499a73801d +size 4144989792 diff --git a/G2-Darkest-Writer-9B-v1.i1-IQ3_XXS.gguf b/G2-Darkest-Writer-9B-v1.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..f4f3904 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4f3d13cdaa05586cb2cb4b9807b898458dc169d5d5618d6629c2ac50472c572d +size 3796739680 diff --git a/G2-Darkest-Writer-9B-v1.i1-IQ4_NL.gguf b/G2-Darkest-Writer-9B-v1.i1-IQ4_NL.gguf new file mode 100644 index 0000000..c27d887 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1600170f2f99a133420089d829e4bbb4b9773b8dcbfb56b0ce9c5d238e4796a +size 5443143264 diff --git a/G2-Darkest-Writer-9B-v1.i1-IQ4_XS.gguf b/G2-Darkest-Writer-9B-v1.i1-IQ4_XS.gguf new file mode 100644 index 0000000..19f2e36 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3d14af858454b24f0d7da26ef171a961a9903fb1b85de51110a87cdc5155722a +size 5183030880 diff --git a/G2-Darkest-Writer-9B-v1.i1-Q2_K.gguf b/G2-Darkest-Writer-9B-v1.i1-Q2_K.gguf new file mode 100644 index 0000000..5e69cb2 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:499070664f15bb3f927616039798dd5a9af53724c4fd296d834b64dc4ca5ff03 +size 3805398624 diff --git a/G2-Darkest-Writer-9B-v1.i1-Q2_K_S.gguf b/G2-Darkest-Writer-9B-v1.i1-Q2_K_S.gguf new file mode 100644 index 0000000..2beeb56 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2748408a4ed03c7b301b7b25d509ad94e7a0e1ef1c9289bc884981b8b2392793 +size 3552511584 diff --git a/G2-Darkest-Writer-9B-v1.i1-Q3_K_L.gguf b/G2-Darkest-Writer-9B-v1.i1-Q3_K_L.gguf new file mode 100644 index 0000000..241555b --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae26b313166ea7680b2f0afcf0a73e4360fbe18b896b895a04f1bc0198929710 +size 5132453472 diff --git a/G2-Darkest-Writer-9B-v1.i1-Q3_K_M.gguf b/G2-Darkest-Writer-9B-v1.i1-Q3_K_M.gguf new file mode 100644 index 0000000..b624efe --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9cf54c7d636ef796b6249f25b4987714636ff36ecf3da8b5cfc53e3c3bf52872 +size 4761781856 diff --git a/G2-Darkest-Writer-9B-v1.i1-Q3_K_S.gguf b/G2-Darkest-Writer-9B-v1.i1-Q3_K_S.gguf new file mode 100644 index 0000000..63d1908 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19762ef129f17a374d4e15de1a55d4aa33404bd0fe51c6fc4ed5bedbf9bbd5bc +size 4337665632 diff --git a/G2-Darkest-Writer-9B-v1.i1-Q4_0.gguf b/G2-Darkest-Writer-9B-v1.i1-Q4_0.gguf new file mode 100644 index 0000000..6373cd1 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1ed5b55316b8f548a5fd9a97a4b867385d4cea71a4067660909ae0d939f0d3ac +size 5459199584 diff --git a/G2-Darkest-Writer-9B-v1.i1-Q4_1.gguf b/G2-Darkest-Writer-9B-v1.i1-Q4_1.gguf new file mode 100644 index 0000000..ef76082 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5aed12ace1b95440f9667e6f88af5f578e980cfa219a71c46ba644b9f21a690b +size 5963368032 diff --git a/G2-Darkest-Writer-9B-v1.i1-Q4_K_M.gguf b/G2-Darkest-Writer-9B-v1.i1-Q4_K_M.gguf new file mode 100644 index 0000000..b63fa38 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:594c50bd7ed48849e66a04fc1f1220871b7da5ae60b6ad9ea8daa536e06ae6f6 +size 5761058400 diff --git a/G2-Darkest-Writer-9B-v1.i1-Q4_K_S.gguf b/G2-Darkest-Writer-9B-v1.i1-Q4_K_S.gguf new file mode 100644 index 0000000..613766c --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:555e05645ef220fbb0926b0cf4f36eafd12144640eed537cd1c218347e7ad258 +size 5478925920 diff --git a/G2-Darkest-Writer-9B-v1.i1-Q5_K_M.gguf b/G2-Darkest-Writer-9B-v1.i1-Q5_K_M.gguf new file mode 100644 index 0000000..a8637af --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7b7a667117f1220a319c0e5f1867cc27dd3d283b8004000c5e5aa425c595712e +size 6647367264 diff --git a/G2-Darkest-Writer-9B-v1.i1-Q5_K_S.gguf b/G2-Darkest-Writer-9B-v1.i1-Q5_K_S.gguf new file mode 100644 index 0000000..5ba590d --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:64b9a12eb68360e2d00c85d285ff25c8c8a8372b2322d3caf056dc286b041f53 +size 6483592800 diff --git a/G2-Darkest-Writer-9B-v1.i1-Q6_K.gguf b/G2-Darkest-Writer-9B-v1.i1-Q6_K.gguf new file mode 100644 index 0000000..6de1b77 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:08bec28bf1f40573e2c89210f916b214dca58ccf32563c40272fe14f0be411c6 +size 7589070432 diff --git a/G2-Darkest-Writer-9B-v1.imatrix.gguf b/G2-Darkest-Writer-9B-v1.imatrix.gguf new file mode 100644 index 0000000..d22cd31 --- /dev/null +++ b/G2-Darkest-Writer-9B-v1.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ca4d87be4f197bc7391294dbc0aaeac023721054fd12810009c2587b5d35be81 +size 6152352 diff --git a/README.md b/README.md new file mode 100644 index 0000000..977edac --- /dev/null +++ b/README.md @@ -0,0 +1,84 @@ +--- +base_model: OccultAI/G2-Darkest-Writer-9B-v1 +language: +- en +library_name: transformers +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/OccultAI/G2-Darkest-Writer-9B-v1 + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#G2-Darkest-Writer-9B-v1-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-IQ1_S.gguf) | i1-IQ1_S | 2.5 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-IQ1_M.gguf) | i1-IQ1_M | 2.6 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-IQ2_XS.gguf) | i1-IQ2_XS | 3.2 | | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-IQ2_S.gguf) | i1-IQ2_S | 3.3 | | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-IQ2_M.gguf) | i1-IQ2_M | 3.5 | | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.7 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-Q2_K.gguf) | i1-Q2_K | 3.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-IQ3_XS.gguf) | i1-IQ3_XS | 4.2 | | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-IQ3_S.gguf) | i1-IQ3_S | 4.4 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-Q3_K_S.gguf) | i1-Q3_K_S | 4.4 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-IQ3_M.gguf) | i1-IQ3_M | 4.6 | | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.9 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-Q3_K_L.gguf) | i1-Q3_K_L | 5.2 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-IQ4_XS.gguf) | i1-IQ4_XS | 5.3 | | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-IQ4_NL.gguf) | i1-IQ4_NL | 5.5 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-Q4_0.gguf) | i1-Q4_0 | 5.6 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-Q4_K_S.gguf) | i1-Q4_K_S | 5.6 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.9 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-Q4_1.gguf) | i1-Q4_1 | 6.1 | | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-Q5_K_S.gguf) | i1-Q5_K_S | 6.6 | | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-Q5_K_M.gguf) | i1-Q5_K_M | 6.7 | | +| [GGUF](https://huggingface.co/mradermacher/G2-Darkest-Writer-9B-v1-i1-GGUF/resolve/main/G2-Darkest-Writer-9B-v1.i1-Q6_K.gguf) | i1-Q6_K | 7.7 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +