commit 8b0d2dd58cb0571dd83a4912bb5ed2348e018911 Author: ModelHub XC Date: Fri May 15 02:51:07 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Kraken-12B-v0-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..b2c5255 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Kraken-12B-v0.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Kraken-12B-v0.i1-IQ1_M.gguf b/Kraken-12B-v0.i1-IQ1_M.gguf new file mode 100644 index 0000000..cdf05e4 --- /dev/null +++ b/Kraken-12B-v0.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:431421a8af6f4387197b8349b0ceaaace02985d672af075aeb99cb7453c68239 +size 3221733856 diff --git a/Kraken-12B-v0.i1-IQ1_S.gguf b/Kraken-12B-v0.i1-IQ1_S.gguf new file mode 100644 index 0000000..8adcbda --- /dev/null +++ b/Kraken-12B-v0.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8e7b30d31b8a425f17d83304718ea0721d4329982e5ff96bb0a4117252e5d963 +size 2999321056 diff --git a/Kraken-12B-v0.i1-IQ2_M.gguf b/Kraken-12B-v0.i1-IQ2_M.gguf new file mode 100644 index 0000000..f1b8dde --- /dev/null +++ b/Kraken-12B-v0.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5db3e77852a9fec0f343a57d801bb8560794f90148155ff15daffbcc925f7d9c +size 4435141216 diff --git a/Kraken-12B-v0.i1-IQ2_S.gguf b/Kraken-12B-v0.i1-IQ2_S.gguf new file mode 100644 index 0000000..95a8c4e --- /dev/null +++ b/Kraken-12B-v0.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6379b5ebab4efa593e33a020647725740451fbeccc8948ce06c2c646d31d3426 +size 4138590816 diff --git a/Kraken-12B-v0.i1-IQ2_XS.gguf b/Kraken-12B-v0.i1-IQ2_XS.gguf new file mode 100644 index 0000000..b484fcc --- /dev/null +++ b/Kraken-12B-v0.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f098a8c4fd38a05cc804decef51356691a72c2d4bce1e91cc8d3d0d7e1ef7d78 +size 3915186656 diff --git a/Kraken-12B-v0.i1-IQ2_XXS.gguf b/Kraken-12B-v0.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..51897a4 --- /dev/null +++ b/Kraken-12B-v0.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b670c62241230a52f80b24b6352ce189c987506daeb819a02666b7099a52b50 +size 3592421856 diff --git a/Kraken-12B-v0.i1-IQ3_M.gguf b/Kraken-12B-v0.i1-IQ3_M.gguf new file mode 100644 index 0000000..2d3e2a0 --- /dev/null +++ b/Kraken-12B-v0.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:267d9b53a188d9840c1b7b87da25330c2c4a44411de6545e10bc3060983d8d59 +size 5722361056 diff --git a/Kraken-12B-v0.i1-IQ3_S.gguf b/Kraken-12B-v0.i1-IQ3_S.gguf new file mode 100644 index 0000000..2293ba7 --- /dev/null +++ b/Kraken-12B-v0.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd1bcc56fe6921769693ac916e1fa45d6cf518e8d21c2f8a3714b511a2da0a86 +size 5562207456 diff --git a/Kraken-12B-v0.i1-IQ3_XS.gguf b/Kraken-12B-v0.i1-IQ3_XS.gguf new file mode 100644 index 0000000..e011688 --- /dev/null +++ b/Kraken-12B-v0.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4b9e2cf599f07111fe1d1d4fc4f1d669a9f054a019a52be3ffe92e9d717f0a1a +size 5306617056 diff --git a/Kraken-12B-v0.i1-IQ3_XXS.gguf b/Kraken-12B-v0.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..716103c --- /dev/null +++ b/Kraken-12B-v0.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7886d99b4ddf072465e7d651a7a3b3dad303de2ea453646da38e410991b57381 +size 4945502816 diff --git a/Kraken-12B-v0.i1-IQ4_NL.gguf b/Kraken-12B-v0.i1-IQ4_NL.gguf new file mode 100644 index 0000000..207cbc4 --- /dev/null +++ b/Kraken-12B-v0.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e92720e293fb0273c9d5568111089b959cd0f40876538715ee031630bb05d30e +size 7098054496 diff --git a/Kraken-12B-v0.i1-IQ4_XS.gguf b/Kraken-12B-v0.i1-IQ4_XS.gguf new file mode 100644 index 0000000..de3fa0f --- /dev/null +++ b/Kraken-12B-v0.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ddf20cf6cfd7c6a4d0f9559784e6a5e9ed04084e9142c8fdbb83f17a94b5601a +size 6742846816 diff --git a/Kraken-12B-v0.i1-Q2_K.gguf b/Kraken-12B-v0.i1-Q2_K.gguf new file mode 100644 index 0000000..cb77a6d --- /dev/null +++ b/Kraken-12B-v0.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ebcb543875163d328efe45003977729bc8aca36c37a02dc7184990b218de50ea +size 4791168096 diff --git a/Kraken-12B-v0.i1-Q2_K_S.gguf b/Kraken-12B-v0.i1-Q2_K_S.gguf new file mode 100644 index 0000000..3ccc85e --- /dev/null +++ b/Kraken-12B-v0.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b58889fec53578e71310f1fa37a3aaef7a3b7045517c75e2c2dfe57b16adbe22 +size 4493798496 diff --git a/Kraken-12B-v0.i1-Q3_K_L.gguf b/Kraken-12B-v0.i1-Q3_K_L.gguf new file mode 100644 index 0000000..41ecd7b --- /dev/null +++ b/Kraken-12B-v0.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44af37de3d5702fbf46781a7379f270e24198a905bdcf6643df260568ee01b2d +size 6561631456 diff --git a/Kraken-12B-v0.i1-Q3_K_M.gguf b/Kraken-12B-v0.i1-Q3_K_M.gguf new file mode 100644 index 0000000..b2c0c34 --- /dev/null +++ b/Kraken-12B-v0.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d0fafd55f6a5412c5e55722e0e26d92545b5ab9c09c39206354e8fd82e31c73e +size 6083218656 diff --git a/Kraken-12B-v0.i1-Q3_K_S.gguf b/Kraken-12B-v0.i1-Q3_K_S.gguf new file mode 100644 index 0000000..38225f6 --- /dev/null +++ b/Kraken-12B-v0.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:30033c183fd34cf506a82fe1e6d82a6183f5a163cc41c9a24388f02b6e516994 +size 5534354656 diff --git a/Kraken-12B-v0.i1-Q4_0.gguf b/Kraken-12B-v0.i1-Q4_0.gguf new file mode 100644 index 0000000..9c02ae6 --- /dev/null +++ b/Kraken-12B-v0.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1aa8c9ceb880b7962641ec81a822466274a077d92cc2e511fdf87b9d9feba7c +size 7094777696 diff --git a/Kraken-12B-v0.i1-Q4_1.gguf b/Kraken-12B-v0.i1-Q4_1.gguf new file mode 100644 index 0000000..8cbb5ad --- /dev/null +++ b/Kraken-12B-v0.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:15bf310fd4806eaf2fe3ec2a417e2d45eddad2e69c067320c6f1b339ef20a2b6 +size 7795362656 diff --git a/Kraken-12B-v0.i1-Q4_K_M.gguf b/Kraken-12B-v0.i1-Q4_K_M.gguf new file mode 100644 index 0000000..1552f31 --- /dev/null +++ b/Kraken-12B-v0.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f08bcae011222bb065d0377ef2d62218e3bdb4c2a40c1db98d0575dbe39ed5eb +size 7477344096 diff --git a/Kraken-12B-v0.i1-Q4_K_S.gguf b/Kraken-12B-v0.i1-Q4_K_S.gguf new file mode 100644 index 0000000..002d7b2 --- /dev/null +++ b/Kraken-12B-v0.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d4311ba467270c319cf09a08769d3076fda6434ca9e1c72faf2a20e4c8a0fd89 +size 7120336736 diff --git a/Kraken-12B-v0.i1-Q5_K_M.gguf b/Kraken-12B-v0.i1-Q5_K_M.gguf new file mode 100644 index 0000000..ebb6f6d --- /dev/null +++ b/Kraken-12B-v0.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d5e3229554e23bcf3ac743d79c8a2ab14aeaf3cb8e399291a7a49c46fcbbc05e +size 8727781216 diff --git a/Kraken-12B-v0.i1-Q5_K_S.gguf b/Kraken-12B-v0.i1-Q5_K_S.gguf new file mode 100644 index 0000000..a0ceb94 --- /dev/null +++ b/Kraken-12B-v0.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b572a1c98eadfa53852b80cd5f97895226a981c80619d249543a317a7291119e +size 8518885216 diff --git a/Kraken-12B-v0.i1-Q6_K.gguf b/Kraken-12B-v0.i1-Q6_K.gguf new file mode 100644 index 0000000..67297e2 --- /dev/null +++ b/Kraken-12B-v0.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd02ee4b51a5c36542b6fbecd2592ddf70405ae0ee92cc7e5beef17f6a2119e5 +size 10056370656 diff --git a/Kraken-12B-v0.imatrix.gguf b/Kraken-12B-v0.imatrix.gguf new file mode 100644 index 0000000..43719ab --- /dev/null +++ b/Kraken-12B-v0.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3c81286585135518540c9d888f973ef7cf6724039a7e883d1bd10625b5f0d70e +size 7088192 diff --git a/README.md b/README.md new file mode 100644 index 0000000..09a486d --- /dev/null +++ b/README.md @@ -0,0 +1,114 @@ +--- +base_model: EldritchLabs/Kraken-12B-v0 +language: +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- creative +- creative writing +- fiction writing +- plot generation +- sub-plot generation +- fiction writing +- story generation +- scene continue +- storytelling +- fiction story +- science fiction +- romance +- all genres +- story +- writing +- vivid prosing +- vivid writing +- fiction +- roleplaying +- float32 +- swearing +- rp +- horror +- della +- mistral +- nemo +- merge +- mergekit +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/EldritchLabs/Kraken-12B-v0 + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Kraken-12B-v0-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Kraken-12B-v0-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-IQ1_S.gguf) | i1-IQ1_S | 3.1 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-IQ1_M.gguf) | i1-IQ1_M | 3.3 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 3.7 | | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.0 | | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-IQ2_S.gguf) | i1-IQ2_S | 4.2 | | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-IQ2_M.gguf) | i1-IQ2_M | 4.5 | | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-Q2_K_S.gguf) | i1-Q2_K_S | 4.6 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-Q2_K.gguf) | i1-Q2_K | 4.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 5.0 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-IQ3_XS.gguf) | i1-IQ3_XS | 5.4 | | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-Q3_K_S.gguf) | i1-Q3_K_S | 5.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-IQ3_S.gguf) | i1-IQ3_S | 5.7 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-IQ3_M.gguf) | i1-IQ3_M | 5.8 | | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-Q3_K_M.gguf) | i1-Q3_K_M | 6.2 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-Q3_K_L.gguf) | i1-Q3_K_L | 6.7 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-IQ4_XS.gguf) | i1-IQ4_XS | 6.8 | | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-Q4_0.gguf) | i1-Q4_0 | 7.2 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-IQ4_NL.gguf) | i1-IQ4_NL | 7.2 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-Q4_K_S.gguf) | i1-Q4_K_S | 7.2 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-Q4_K_M.gguf) | i1-Q4_K_M | 7.6 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-Q4_1.gguf) | i1-Q4_1 | 7.9 | | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-Q5_K_S.gguf) | i1-Q5_K_S | 8.6 | | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-Q5_K_M.gguf) | i1-Q5_K_M | 8.8 | | +| [GGUF](https://huggingface.co/mradermacher/Kraken-12B-v0-i1-GGUF/resolve/main/Kraken-12B-v0.i1-Q6_K.gguf) | i1-Q6_K | 10.2 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +