commit c0fffd948197b23c0d48c49777cd7a0d5f5966b7 Author: ModelHub XC Date: Sun May 31 02:10:15 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Orpheus-3B-Mini-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..b18d385 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Orpheus-3B-Mini.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Orpheus-3B-Mini.i1-IQ1_M.gguf b/Orpheus-3B-Mini.i1-IQ1_M.gguf new file mode 100644 index 0000000..c2ce8ef --- /dev/null +++ b/Orpheus-3B-Mini.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:af83c910645c1a754dd3aaa84e1073d5efe2d6d1850acb439158d521bf176921 +size 924191296 diff --git a/Orpheus-3B-Mini.i1-IQ1_S.gguf b/Orpheus-3B-Mini.i1-IQ1_S.gguf new file mode 100644 index 0000000..a2337b7 --- /dev/null +++ b/Orpheus-3B-Mini.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:76915389e6a4189043767cd773a7ad64bae37b195fb68c9cf6fbe68b5e5953f0 +size 868158016 diff --git a/Orpheus-3B-Mini.i1-IQ2_M.gguf b/Orpheus-3B-Mini.i1-IQ2_M.gguf new file mode 100644 index 0000000..6debf09 --- /dev/null +++ b/Orpheus-3B-Mini.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:935d5e32022e58a7542e0136dbb851c78e47d51ab0e7f4b5737dfb1a04e218e6 +size 1229032000 diff --git a/Orpheus-3B-Mini.i1-IQ2_S.gguf b/Orpheus-3B-Mini.i1-IQ2_S.gguf new file mode 100644 index 0000000..c300505 --- /dev/null +++ b/Orpheus-3B-Mini.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a96d9f7b46c34e8b549d140152046ccb5ba226112beec7e102670ce929d79ce1 +size 1154320960 diff --git a/Orpheus-3B-Mini.i1-IQ2_XS.gguf b/Orpheus-3B-Mini.i1-IQ2_XS.gguf new file mode 100644 index 0000000..c23e49a --- /dev/null +++ b/Orpheus-3B-Mini.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:97e60695600b276c253608b7d76bb009ca7565b5e4ee9b1e78c43524efd83b77 +size 1100548672 diff --git a/Orpheus-3B-Mini.i1-IQ2_XXS.gguf b/Orpheus-3B-Mini.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..7a46f79 --- /dev/null +++ b/Orpheus-3B-Mini.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a6eb5f7a0a075a2de8424f945985af5781e049c625efdb52098de6c56b0adde2 +size 1017580096 diff --git a/Orpheus-3B-Mini.i1-IQ3_M.gguf b/Orpheus-3B-Mini.i1-IQ3_M.gguf new file mode 100644 index 0000000..b26906e --- /dev/null +++ b/Orpheus-3B-Mini.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6563be8b544cb49941902d7df3a8f3ddb6ca52474ca2d1205183f3c5394bcab8 +size 1599668800 diff --git a/Orpheus-3B-Mini.i1-IQ3_S.gguf b/Orpheus-3B-Mini.i1-IQ3_S.gguf new file mode 100644 index 0000000..948e55f --- /dev/null +++ b/Orpheus-3B-Mini.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:36d81716d74cf11fce32528f147bb02b8b9b303a6dcd6ccec80b13a26cf61084 +size 1542849088 diff --git a/Orpheus-3B-Mini.i1-IQ3_XS.gguf b/Orpheus-3B-Mini.i1-IQ3_XS.gguf new file mode 100644 index 0000000..06a534f --- /dev/null +++ b/Orpheus-3B-Mini.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eb285beb5761beeaa88dad3059c8465982c64530f7864e9f60194cb10bd3fa58 +size 1476788800 diff --git a/Orpheus-3B-Mini.i1-IQ3_XXS.gguf b/Orpheus-3B-Mini.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..f5cfe4e --- /dev/null +++ b/Orpheus-3B-Mini.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1f51de342cf205d6a5a48af400ebaf14b05b83fc449c6b6fb771dbc31fb35220 +size 1348766272 diff --git a/Orpheus-3B-Mini.i1-IQ4_NL.gguf b/Orpheus-3B-Mini.i1-IQ4_NL.gguf new file mode 100644 index 0000000..8520fb4 --- /dev/null +++ b/Orpheus-3B-Mini.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:10d33a463d3bec670e2f7811b2f7538b3ea90d6b60802e3de4ffb15e48b30604 +size 1917190720 diff --git a/Orpheus-3B-Mini.i1-IQ4_XS.gguf b/Orpheus-3B-Mini.i1-IQ4_XS.gguf new file mode 100644 index 0000000..b979914 --- /dev/null +++ b/Orpheus-3B-Mini.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c296fc53f68ab8eab3c03c2c848028d9fcb74e743daa159dae68834d39321095 +size 1829110336 diff --git a/Orpheus-3B-Mini.i1-Q2_K.gguf b/Orpheus-3B-Mini.i1-Q2_K.gguf new file mode 100644 index 0000000..e32728a --- /dev/null +++ b/Orpheus-3B-Mini.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2373ba2bb80c22e88251ba52f154062679430664384ac650a6533274b79b301a +size 1363935808 diff --git a/Orpheus-3B-Mini.i1-Q2_K_S.gguf b/Orpheus-3B-Mini.i1-Q2_K_S.gguf new file mode 100644 index 0000000..db0d710 --- /dev/null +++ b/Orpheus-3B-Mini.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:83780d1a8d0ca922aa74c414fc4c8725ee3b6846d0cb15b2183aa92d0ffdae41 +size 1274282560 diff --git a/Orpheus-3B-Mini.i1-Q3_K_L.gguf b/Orpheus-3B-Mini.i1-Q3_K_L.gguf new file mode 100644 index 0000000..daddadf --- /dev/null +++ b/Orpheus-3B-Mini.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fd601ecd809c263f11dc8cd049e637aa3c6c9a84e97dfe13e149352396f24c02 +size 1815347776 diff --git a/Orpheus-3B-Mini.i1-Q3_K_M.gguf b/Orpheus-3B-Mini.i1-Q3_K_M.gguf new file mode 100644 index 0000000..ac45dc3 --- /dev/null +++ b/Orpheus-3B-Mini.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a48ac9d2199e38ec9d48bf930637cd7d07d2511d6c84daceec0710babb9d3586 +size 1687159360 diff --git a/Orpheus-3B-Mini.i1-Q3_K_S.gguf b/Orpheus-3B-Mini.i1-Q3_K_S.gguf new file mode 100644 index 0000000..c9d51af --- /dev/null +++ b/Orpheus-3B-Mini.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f9dd4f308d4d16e46a9f562fa463bce6bc859ee2eb24db7f1da2b37275580535 +size 1542849088 diff --git a/Orpheus-3B-Mini.i1-Q4_0.gguf b/Orpheus-3B-Mini.i1-Q4_0.gguf new file mode 100644 index 0000000..94d6c34 --- /dev/null +++ b/Orpheus-3B-Mini.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d54707dceab66a1b0ad31b2c88659689e500be6bf700f4b68b0e6c2d2323f3e3 +size 1921909312 diff --git a/Orpheus-3B-Mini.i1-Q4_1.gguf b/Orpheus-3B-Mini.i1-Q4_1.gguf new file mode 100644 index 0000000..2ed2b0a --- /dev/null +++ b/Orpheus-3B-Mini.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:77a73b22b3b34f51ebb2e1cfd0fdbaa7bd8e39be731aa3aa2b3d59ebea4ffa8c +size 2093351488 diff --git a/Orpheus-3B-Mini.i1-Q4_K_M.gguf b/Orpheus-3B-Mini.i1-Q4_K_M.gguf new file mode 100644 index 0000000..80750fe --- /dev/null +++ b/Orpheus-3B-Mini.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2f2e808f1360fa8c49d7bd077b0ddbc65cfe9b1d76c4e715f89b11c28c7871ee +size 2019377728 diff --git a/Orpheus-3B-Mini.i1-Q4_K_S.gguf b/Orpheus-3B-Mini.i1-Q4_K_S.gguf new file mode 100644 index 0000000..4fe758e --- /dev/null +++ b/Orpheus-3B-Mini.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2e3b775dd8788b08358f64215056b58ea9994adef34e5123ab0d7c60da833d5e +size 1928200768 diff --git a/Orpheus-3B-Mini.i1-Q5_K_M.gguf b/Orpheus-3B-Mini.i1-Q5_K_M.gguf new file mode 100644 index 0000000..a435633 --- /dev/null +++ b/Orpheus-3B-Mini.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:04e6c4b512d07f71324fe3c25a1c58e89e13858cd954fbeb521dfe3a8dd443c3 +size 2322154048 diff --git a/Orpheus-3B-Mini.i1-Q5_K_S.gguf b/Orpheus-3B-Mini.i1-Q5_K_S.gguf new file mode 100644 index 0000000..8f2acb2 --- /dev/null +++ b/Orpheus-3B-Mini.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6232b35c4d963701df6990edd6a614383ade48df7137e767ba46ef0f3a764e47 +size 2269512256 diff --git a/Orpheus-3B-Mini.i1-Q6_K.gguf b/Orpheus-3B-Mini.i1-Q6_K.gguf new file mode 100644 index 0000000..dc5e44c --- /dev/null +++ b/Orpheus-3B-Mini.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:313800de9bd47c9b8cfef74e60405f8a97213dafa89d922eb4aa0f1e582c7347 +size 2643853888 diff --git a/Orpheus-3B-Mini.imatrix.gguf b/Orpheus-3B-Mini.imatrix.gguf new file mode 100644 index 0000000..77e5879 --- /dev/null +++ b/Orpheus-3B-Mini.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f872aba56c19a74be7a52356b93805281bf85efb378dad5bd91d86bc7f013e9c +size 3012064 diff --git a/README.md b/README.md new file mode 100644 index 0000000..9470b8a --- /dev/null +++ b/README.md @@ -0,0 +1,84 @@ +--- +base_model: blascotobasco/Orpheus-3B-Mini +language: +- en +library_name: transformers +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/blascotobasco/Orpheus-3B-Mini + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Orpheus-3B-Mini-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Orpheus-3B-Mini-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-IQ1_S.gguf) | i1-IQ1_S | 1.0 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-IQ1_M.gguf) | i1-IQ1_M | 1.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.1 | | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.2 | | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-IQ2_S.gguf) | i1-IQ2_S | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-IQ2_M.gguf) | i1-IQ2_M | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.4 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.4 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-Q2_K.gguf) | i1-Q2_K | 1.5 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-IQ3_S.gguf) | i1-IQ3_S | 1.6 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-Q3_K_S.gguf) | i1-Q3_K_S | 1.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-IQ3_M.gguf) | i1-IQ3_M | 1.7 | | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-Q3_K_M.gguf) | i1-Q3_K_M | 1.8 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-Q3_K_L.gguf) | i1-Q3_K_L | 1.9 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-IQ4_XS.gguf) | i1-IQ4_XS | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.0 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-Q4_0.gguf) | i1-Q4_0 | 2.0 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.0 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.1 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-Q4_1.gguf) | i1-Q4_1 | 2.2 | | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-Q5_K_M.gguf) | i1-Q5_K_M | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/Orpheus-3B-Mini-i1-GGUF/resolve/main/Orpheus-3B-Mini.i1-Q6_K.gguf) | i1-Q6_K | 2.7 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +