commit e7cd7e70bc3af3834de1cbbaa57153adc9a69921 Author: ModelHub XC Date: Wed May 13 17:58:20 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Plano-Orchestrator-4B-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..c354694 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Plano-Orchestrator-4B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Plano-Orchestrator-4B.i1-IQ1_M.gguf b/Plano-Orchestrator-4B.i1-IQ1_M.gguf new file mode 100644 index 0000000..4d88d5c --- /dev/null +++ b/Plano-Orchestrator-4B.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4f4e19bdf366019237be9d9c4fd5b327a0f7ec804efb6a2f93790f01c3c46929 +size 1127017184 diff --git a/Plano-Orchestrator-4B.i1-IQ1_S.gguf b/Plano-Orchestrator-4B.i1-IQ1_S.gguf new file mode 100644 index 0000000..ca9fd60 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0540bd1ad64d0f14511d8439fc59d0aa8bc270e421a8e67d08e0a6fd297fd764 +size 1055255264 diff --git a/Plano-Orchestrator-4B.i1-IQ2_M.gguf b/Plano-Orchestrator-4B.i1-IQ2_M.gguf new file mode 100644 index 0000000..3fdf0dd --- /dev/null +++ b/Plano-Orchestrator-4B.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:797cc568e18f45e2596211fce37521b3a96b5e0f4c58042429338c33b2ee5a35 +size 1512983264 diff --git a/Plano-Orchestrator-4B.i1-IQ2_S.gguf b/Plano-Orchestrator-4B.i1-IQ2_S.gguf new file mode 100644 index 0000000..372e242 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9c3a91a06c2f9b8fd8275837799dea6d651cc1b765c26a2cc01f6ca7f052fd2b +size 1417300704 diff --git a/Plano-Orchestrator-4B.i1-IQ2_XS.gguf b/Plano-Orchestrator-4B.i1-IQ2_XS.gguf new file mode 100644 index 0000000..3863eb5 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0ae37cff0866ba305ea564971d2211a89c8322350b1ad22b8c3372fd55461748 +size 1354099424 diff --git a/Plano-Orchestrator-4B.i1-IQ2_XXS.gguf b/Plano-Orchestrator-4B.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..013f6dc --- /dev/null +++ b/Plano-Orchestrator-4B.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ab09d0267ae31adc60c88e27eef56a8922ef646ad1bc4cc1e2dbfb9bb7a99b4a +size 1246620384 diff --git a/Plano-Orchestrator-4B.i1-IQ3_M.gguf b/Plano-Orchestrator-4B.i1-IQ3_M.gguf new file mode 100644 index 0000000..0c4b9f5 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d013c6f0df30c73b4f4cab000ed8cd43f8660394d784b3091b7921570dfb0ab3 +size 1962895584 diff --git a/Plano-Orchestrator-4B.i1-IQ3_S.gguf b/Plano-Orchestrator-4B.i1-IQ3_S.gguf new file mode 100644 index 0000000..f231420 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ddfef066f8079e5621990b98817968026eb8d27c90b059ddefbbb499e4b951e +size 1899530464 diff --git a/Plano-Orchestrator-4B.i1-IQ3_XS.gguf b/Plano-Orchestrator-4B.i1-IQ3_XS.gguf new file mode 100644 index 0000000..ba94e47 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3b426bfc60c40a2777bb4d4bde3f729e03d508fe26ece2e5295cc670a3dabaad +size 1814374624 diff --git a/Plano-Orchestrator-4B.i1-IQ3_XXS.gguf b/Plano-Orchestrator-4B.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..28d26e4 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2e8e8a65d1ed2a6b720bf3440e6f78b51866ccefe1099fcc3a7bc8bb32939c12 +size 1670187744 diff --git a/Plano-Orchestrator-4B.i1-IQ4_NL.gguf b/Plano-Orchestrator-4B.i1-IQ4_NL.gguf new file mode 100644 index 0000000..49ccc6b --- /dev/null +++ b/Plano-Orchestrator-4B.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:42c5a6d7b52868a8aefc12c5fded945bc1ef02a5a177bf3d0c95df65f076e038 +size 2381342944 diff --git a/Plano-Orchestrator-4B.i1-IQ4_XS.gguf b/Plano-Orchestrator-4B.i1-IQ4_XS.gguf new file mode 100644 index 0000000..6cd5b77 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:87a292814bcd9b5624fab9adc707b8c66d43e4468692ec75a3a58bde947861e0 +size 2270750944 diff --git a/Plano-Orchestrator-4B.i1-Q2_K.gguf b/Plano-Orchestrator-4B.i1-Q2_K.gguf new file mode 100644 index 0000000..9d79aa1 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:53563b752901e196ea9f808d55fc804811fba4d30153d846f6a66aa70c71a709 +size 1669499104 diff --git a/Plano-Orchestrator-4B.i1-Q2_K_S.gguf b/Plano-Orchestrator-4B.i1-Q2_K_S.gguf new file mode 100644 index 0000000..2ca18cc --- /dev/null +++ b/Plano-Orchestrator-4B.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3add3b4f572c9fe64ca1f47407c55803efdf5231df58408957ec3252fc4e4d95 +size 1563453664 diff --git a/Plano-Orchestrator-4B.i1-Q3_K_L.gguf b/Plano-Orchestrator-4B.i1-Q3_K_L.gguf new file mode 100644 index 0000000..82ac633 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0fd1e496dceb392f00384bd515b10f1022cdffb3368a3cc189960f5d85ebf4ee +size 2239785184 diff --git a/Plano-Orchestrator-4B.i1-Q3_K_M.gguf b/Plano-Orchestrator-4B.i1-Q3_K_M.gguf new file mode 100644 index 0000000..b1266ea --- /dev/null +++ b/Plano-Orchestrator-4B.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7542341e63b167c1519343bb7539b3afd367c030c4eed39ae0ad791b22c736d7 +size 2075617504 diff --git a/Plano-Orchestrator-4B.i1-Q3_K_S.gguf b/Plano-Orchestrator-4B.i1-Q3_K_S.gguf new file mode 100644 index 0000000..5336903 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1d57f1a9663e63ae84ce56d29aa83ee09a82986f191af5f4eda88d909e8843c7 +size 1886996704 diff --git a/Plano-Orchestrator-4B.i1-Q4_0.gguf b/Plano-Orchestrator-4B.i1-Q4_0.gguf new file mode 100644 index 0000000..0bcf4a2 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:73a20e25e291e72849d836b2df9aa322a111ffbb922ef87608267cae2da72e53 +size 2375772384 diff --git a/Plano-Orchestrator-4B.i1-Q4_1.gguf b/Plano-Orchestrator-4B.i1-Q4_1.gguf new file mode 100644 index 0000000..1ac41b1 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e42dbc1d652ee3a920fb2c75687a09d752f678014f81fecd9ea5a7384d82fb84 +size 2596628704 diff --git a/Plano-Orchestrator-4B.i1-Q4_K_M.gguf b/Plano-Orchestrator-4B.i1-Q4_K_M.gguf new file mode 100644 index 0000000..37b76fa --- /dev/null +++ b/Plano-Orchestrator-4B.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:66a023af0c72f089083f744f97305c8e5a6422138eeb17865ac8c5758432829e +size 2497280224 diff --git a/Plano-Orchestrator-4B.i1-Q4_K_S.gguf b/Plano-Orchestrator-4B.i1-Q4_K_S.gguf new file mode 100644 index 0000000..c073d61 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a8425f796b0d2fca1bb3092381a4ad19e7ac30594f5291a0e61697f1eab85223 +size 2383309024 diff --git a/Plano-Orchestrator-4B.i1-Q5_K_M.gguf b/Plano-Orchestrator-4B.i1-Q5_K_M.gguf new file mode 100644 index 0000000..336324d --- /dev/null +++ b/Plano-Orchestrator-4B.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9baa87e47f43f87a0ff33b118ac2399c9d5134cf5e0db5c6f491c4d552f90768 +size 2889513184 diff --git a/Plano-Orchestrator-4B.i1-Q5_K_S.gguf b/Plano-Orchestrator-4B.i1-Q5_K_S.gguf new file mode 100644 index 0000000..30ba3c2 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e9ba52afb09b89f7e8afb535d9d3390cd63b4fcb10c69232438e099b116ebc7a +size 2823710944 diff --git a/Plano-Orchestrator-4B.i1-Q6_K.gguf b/Plano-Orchestrator-4B.i1-Q6_K.gguf new file mode 100644 index 0000000..e8269d2 --- /dev/null +++ b/Plano-Orchestrator-4B.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d40aeff3d0477e2111bf6a25c13e74f81b594de057de608e0092e066b02ea25 +size 3306260704 diff --git a/Plano-Orchestrator-4B.imatrix.gguf b/Plano-Orchestrator-4B.imatrix.gguf new file mode 100644 index 0000000..eb05fbd --- /dev/null +++ b/Plano-Orchestrator-4B.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:71d9f9ee27f735ecaa922ab04605c7ceb121d938a6f52b3f235971d6c4e1cfde +size 3872640 diff --git a/README.md b/README.md new file mode 100644 index 0000000..2b6bac2 --- /dev/null +++ b/README.md @@ -0,0 +1,87 @@ +--- +base_model: katanemo/Plano-Orchestrator-4B +language: +- en +library_name: transformers +license: other +license_link: https://huggingface.co/katanemo/Plano-Orchestrator-4B/blob/main/LICENSE +license_name: katanemo-research +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/katanemo/Plano-Orchestrator-4B + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Plano-Orchestrator-4B-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Plano-Orchestrator-4B-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-IQ1_S.gguf) | i1-IQ1_S | 1.2 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-IQ1_M.gguf) | i1-IQ1_M | 1.2 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-IQ2_S.gguf) | i1-IQ2_S | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-IQ2_M.gguf) | i1-IQ2_M | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.7 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-Q2_K.gguf) | i1-Q2_K | 1.8 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.8 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 2.0 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-IQ3_S.gguf) | i1-IQ3_S | 2.0 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-IQ3_M.gguf) | i1-IQ3_M | 2.1 | | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 2.2 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 2.3 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-Q4_0.gguf) | i1-Q4_0 | 2.5 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.5 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.5 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.6 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-Q4_1.gguf) | i1-Q4_1 | 2.7 | | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 3.0 | | +| [GGUF](https://huggingface.co/mradermacher/Plano-Orchestrator-4B-i1-GGUF/resolve/main/Plano-Orchestrator-4B.i1-Q6_K.gguf) | i1-Q6_K | 3.4 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +