commit 500ce982286cbca9a45126bec1770362e3f7b2fd Author: ModelHub XC Date: Thu May 7 17:32:16 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/OpenSonnet-Lite-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..026b2f6 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +OpenSonnet-Lite.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/OpenSonnet-Lite.i1-IQ1_M.gguf b/OpenSonnet-Lite.i1-IQ1_M.gguf new file mode 100644 index 0000000..788cfc3 --- /dev/null +++ b/OpenSonnet-Lite.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:85e656f2e326f47df2d3df8d2c4628c6251c9c6f9ded80a81264e0bf53323bdc +size 1127021600 diff --git a/OpenSonnet-Lite.i1-IQ1_S.gguf b/OpenSonnet-Lite.i1-IQ1_S.gguf new file mode 100644 index 0000000..c0a920e --- /dev/null +++ b/OpenSonnet-Lite.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:52a2f1511c5d4f401fe531bc2dfb992968063d6648264fe572be6a85d0edd86d +size 1055259680 diff --git a/OpenSonnet-Lite.i1-IQ2_M.gguf b/OpenSonnet-Lite.i1-IQ2_M.gguf new file mode 100644 index 0000000..76adad9 --- /dev/null +++ b/OpenSonnet-Lite.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0d28a076c8768ca750d5b197ce79c0fce965ddcb513a5e16687a0e17112f978d +size 1512987680 diff --git a/OpenSonnet-Lite.i1-IQ2_S.gguf b/OpenSonnet-Lite.i1-IQ2_S.gguf new file mode 100644 index 0000000..e221366 --- /dev/null +++ b/OpenSonnet-Lite.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2f7bd144eb0dc8d88260758cb0078d7becd956212ed71b438e6554da780c0f24 +size 1417305120 diff --git a/OpenSonnet-Lite.i1-IQ2_XS.gguf b/OpenSonnet-Lite.i1-IQ2_XS.gguf new file mode 100644 index 0000000..1626b55 --- /dev/null +++ b/OpenSonnet-Lite.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ac2d0cec27331a9c02c3c0778779a216f5771c85b911a4be77698ada2059d88a +size 1354103840 diff --git a/OpenSonnet-Lite.i1-IQ2_XXS.gguf b/OpenSonnet-Lite.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..fcb4acf --- /dev/null +++ b/OpenSonnet-Lite.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:875f5ed1f152257d4a7f5aafae7d9c9c51cc85e434d91e248ce63de62959660a +size 1246624800 diff --git a/OpenSonnet-Lite.i1-IQ3_M.gguf b/OpenSonnet-Lite.i1-IQ3_M.gguf new file mode 100644 index 0000000..80358a6 --- /dev/null +++ b/OpenSonnet-Lite.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:295238d255e2da8fd4cc58019eeb71e9d2ec6ac4acedff35d8ea9194e80a8fe4 +size 1962900000 diff --git a/OpenSonnet-Lite.i1-IQ3_S.gguf b/OpenSonnet-Lite.i1-IQ3_S.gguf new file mode 100644 index 0000000..c8091be --- /dev/null +++ b/OpenSonnet-Lite.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2b9eaf39712f9515a60c0a3ad2826fd0d594376fdd68f1826f040b1fb1ca561a +size 1899534880 diff --git a/OpenSonnet-Lite.i1-IQ3_XS.gguf b/OpenSonnet-Lite.i1-IQ3_XS.gguf new file mode 100644 index 0000000..a95c7c7 --- /dev/null +++ b/OpenSonnet-Lite.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ee4c94870acc711f5e31f597c2d0d7b982e92c76de6dbd1870f3fec7648f4df6 +size 1814379040 diff --git a/OpenSonnet-Lite.i1-IQ3_XXS.gguf b/OpenSonnet-Lite.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..615ce47 --- /dev/null +++ b/OpenSonnet-Lite.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1fd2fe18f1d4d397c2fc4bfdeef2e68f4e943b2be86a7e54369a5aed6318132 +size 1670192160 diff --git a/OpenSonnet-Lite.i1-IQ4_NL.gguf b/OpenSonnet-Lite.i1-IQ4_NL.gguf new file mode 100644 index 0000000..db44a0a --- /dev/null +++ b/OpenSonnet-Lite.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:649ef8c46747ab99ddf693772bd770462f6e9b3db71a8f4ec72dd395a19d9755 +size 2381347360 diff --git a/OpenSonnet-Lite.i1-IQ4_XS.gguf b/OpenSonnet-Lite.i1-IQ4_XS.gguf new file mode 100644 index 0000000..b8f0a64 --- /dev/null +++ b/OpenSonnet-Lite.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6d516f3bc21e5e414372177d2d96b13ce44e5af836ac0090e3b9edf9971418b1 +size 2270755360 diff --git a/OpenSonnet-Lite.i1-Q2_K.gguf b/OpenSonnet-Lite.i1-Q2_K.gguf new file mode 100644 index 0000000..012d044 --- /dev/null +++ b/OpenSonnet-Lite.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:99ca22b3817112466f20b1cf90ad8798710172676cee0f48f4d895a2654ad425 +size 1669503520 diff --git a/OpenSonnet-Lite.i1-Q2_K_S.gguf b/OpenSonnet-Lite.i1-Q2_K_S.gguf new file mode 100644 index 0000000..87651ed --- /dev/null +++ b/OpenSonnet-Lite.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aa90874112d396214c8b48ea298e1a85831dbbe5ab7fdab5ab23fc221f724ea4 +size 1563458080 diff --git a/OpenSonnet-Lite.i1-Q3_K_L.gguf b/OpenSonnet-Lite.i1-Q3_K_L.gguf new file mode 100644 index 0000000..b2eb9dc --- /dev/null +++ b/OpenSonnet-Lite.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:27ff0b88807f5e2c283d22f601881e36a269f967fd80f8d0d1f1909b5bd43cd2 +size 2239789600 diff --git a/OpenSonnet-Lite.i1-Q3_K_M.gguf b/OpenSonnet-Lite.i1-Q3_K_M.gguf new file mode 100644 index 0000000..2c1f889 --- /dev/null +++ b/OpenSonnet-Lite.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c5362a48b771cd3c8e17f14598f8611dc7de5797bbd3b55b14f4496c917de22a +size 2075621920 diff --git a/OpenSonnet-Lite.i1-Q3_K_S.gguf b/OpenSonnet-Lite.i1-Q3_K_S.gguf new file mode 100644 index 0000000..ab9bb70 --- /dev/null +++ b/OpenSonnet-Lite.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d04377e1368785c0314618168007db03ed547d8c30b26260085a9e76e9b22433 +size 1887001120 diff --git a/OpenSonnet-Lite.i1-Q4_0.gguf b/OpenSonnet-Lite.i1-Q4_0.gguf new file mode 100644 index 0000000..97c6a2f --- /dev/null +++ b/OpenSonnet-Lite.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c12340ffeb75c94aa5eba6ba8ed537dba3b22138c1a4075153a008349aaf1336 +size 2375776800 diff --git a/OpenSonnet-Lite.i1-Q4_1.gguf b/OpenSonnet-Lite.i1-Q4_1.gguf new file mode 100644 index 0000000..ce06067 --- /dev/null +++ b/OpenSonnet-Lite.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a8dc31013324d38df4f8f5ec5afeb2b2247ccb19f343256f33297c7f34579b07 +size 2596633120 diff --git a/OpenSonnet-Lite.i1-Q4_K_M.gguf b/OpenSonnet-Lite.i1-Q4_K_M.gguf new file mode 100644 index 0000000..6f31fd2 --- /dev/null +++ b/OpenSonnet-Lite.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2b531be9e5de0a5ab6b782a27dd4cce26e8b7d1822f0d8aa1cec34b612f1e06 +size 2497284640 diff --git a/OpenSonnet-Lite.i1-Q4_K_S.gguf b/OpenSonnet-Lite.i1-Q4_K_S.gguf new file mode 100644 index 0000000..542a4c4 --- /dev/null +++ b/OpenSonnet-Lite.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8927eccd6a06747d34e50df08d554319e6c8442e80a74284394ed203922b49bd +size 2383313440 diff --git a/OpenSonnet-Lite.i1-Q5_K_M.gguf b/OpenSonnet-Lite.i1-Q5_K_M.gguf new file mode 100644 index 0000000..1815d66 --- /dev/null +++ b/OpenSonnet-Lite.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0e00ca2ed0d2345ba5c976201524609783adff254730ea6dee3df236235ad9b8 +size 2889517600 diff --git a/OpenSonnet-Lite.i1-Q5_K_S.gguf b/OpenSonnet-Lite.i1-Q5_K_S.gguf new file mode 100644 index 0000000..f9f11eb --- /dev/null +++ b/OpenSonnet-Lite.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:337141b2385b7247ce4ce22ffaa37122704fecf4b6d21ab21bc1ef0ac292d0e3 +size 2823715360 diff --git a/OpenSonnet-Lite.i1-Q6_K.gguf b/OpenSonnet-Lite.i1-Q6_K.gguf new file mode 100644 index 0000000..0c77457 --- /dev/null +++ b/OpenSonnet-Lite.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c555a17f682cc7c2cf1434f7cdaae98ae46202120614e6ddf2fc3c5381c54809 +size 3306265120 diff --git a/OpenSonnet-Lite.imatrix.gguf b/OpenSonnet-Lite.imatrix.gguf new file mode 100644 index 0000000..47aaa69 --- /dev/null +++ b/OpenSonnet-Lite.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6439b56db17c6710a62460e8e830460de3b81458cab483d47237a31404d3acad +size 3872640 diff --git a/README.md b/README.md new file mode 100644 index 0000000..8f4fbae --- /dev/null +++ b/README.md @@ -0,0 +1,103 @@ +--- +base_model: hadadxyz/OpenSonnet-Lite +datasets: +- Roman1111111/claude-sonnet-4.6-120000x +- Roman1111111/claude-sonnet-4.6-100000X-filtered +- TeichAI/lordx64-claude-opus-4.7-max-cleaned +- Crownelius/Opus-4.6-Reasoning-3300x +- TeichAI/claude-4.5-opus-high-reasoning-250x +- TeichAI/claude-haiku-4.5-high-reasoning-1700x +- TeichAI/claude-sonnet-4.5-high-reasoning-250x +- TeichAI/deepseek-v3.2-speciale-openr1-math-3k +- TeichAI/deepseek-v3.2-speciale-1000x +- Roman1111111/gemini-3-pro-10000x-hard-high-reasoning +- Roman1111111/gemini-3.1-pro-hard-high-reasoning +- Jackrong/DeepSeek-V4-Distill-8000x +language: +- en +library_name: transformers +license: apache-2.0 +license_link: https://huggingface.co/hadadxyz/OpenSonnet-Lite/blob/main/LICENSE +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- opensonnet +- claude-sonnet +- sonnet +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/hadadxyz/OpenSonnet-Lite + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#OpenSonnet-Lite-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/OpenSonnet-Lite-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-IQ1_S.gguf) | i1-IQ1_S | 1.2 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-IQ1_M.gguf) | i1-IQ1_M | 1.2 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-IQ2_S.gguf) | i1-IQ2_S | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-IQ2_M.gguf) | i1-IQ2_M | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.7 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-Q2_K.gguf) | i1-Q2_K | 1.8 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.8 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-Q3_K_S.gguf) | i1-Q3_K_S | 2.0 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-IQ3_S.gguf) | i1-IQ3_S | 2.0 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-IQ3_M.gguf) | i1-IQ3_M | 2.1 | | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-Q3_K_M.gguf) | i1-Q3_K_M | 2.2 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-Q3_K_L.gguf) | i1-Q3_K_L | 2.3 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-Q4_0.gguf) | i1-Q4_0 | 2.5 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.5 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.5 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.6 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-Q4_1.gguf) | i1-Q4_1 | 2.7 | | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-Q5_K_M.gguf) | i1-Q5_K_M | 3.0 | | +| [GGUF](https://huggingface.co/mradermacher/OpenSonnet-Lite-i1-GGUF/resolve/main/OpenSonnet-Lite.i1-Q6_K.gguf) | i1-Q6_K | 3.4 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +