commit 894f7c0430f4fb2d4094756f1215a1d0135614ac Author: ModelHub XC Date: Wed Sep 23 21:11:18 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Franken-MoE-Large-Base-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..fc70c7d --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Franken-MoE-Large-Base.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Franken-MoE-Large-Base.i1-IQ1_M.gguf b/Franken-MoE-Large-Base.i1-IQ1_M.gguf new file mode 100644 index 0000000..5be347d --- /dev/null +++ b/Franken-MoE-Large-Base.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:53421894549f732d9d158cd572ed1a86d26fedf90d2afaa70cd7a206842b7c62 +size 1107296096 diff --git a/Franken-MoE-Large-Base.i1-IQ1_S.gguf b/Franken-MoE-Large-Base.i1-IQ1_S.gguf new file mode 100644 index 0000000..14b606c --- /dev/null +++ b/Franken-MoE-Large-Base.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f5d6fca06812d463aee3fe06816cdcced1eb9b853c20db8908c91b9802982480 +size 1009680224 diff --git a/Franken-MoE-Large-Base.i1-IQ2_M.gguf b/Franken-MoE-Large-Base.i1-IQ2_M.gguf new file mode 100644 index 0000000..f64c091 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1f7b2f82b776a1efd558e38e3cb776dc37992aa0bee62d45cb5dd73ca5c2cae7 +size 1555726176 diff --git a/Franken-MoE-Large-Base.i1-IQ2_S.gguf b/Franken-MoE-Large-Base.i1-IQ2_S.gguf new file mode 100644 index 0000000..59a3bc4 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e7d19f807a0e8970a76583f6fa315c3c9017531cd8c7cf26006a149c0f226e7e +size 1425571680 diff --git a/Franken-MoE-Large-Base.i1-IQ2_XS.gguf b/Franken-MoE-Large-Base.i1-IQ2_XS.gguf new file mode 100644 index 0000000..82d4ad8 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b872be10aa38e6857ad652f800e714772ff03386f55a8f5d22c75650164bc155 +size 1401978720 diff --git a/Franken-MoE-Large-Base.i1-IQ2_XXS.gguf b/Franken-MoE-Large-Base.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..e52ee31 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc6bf9d52975c437bc200623b6764ac756aabc8f49d18ce3626d8b68f3ee5d57 +size 1269989216 diff --git a/Franken-MoE-Large-Base.i1-IQ3_M.gguf b/Franken-MoE-Large-Base.i1-IQ3_M.gguf new file mode 100644 index 0000000..5635adb --- /dev/null +++ b/Franken-MoE-Large-Base.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:871a89a6accb96d357003e61713a17e9014cf18777e3daf6f6c96c54fccf7df9 +size 2059783008 diff --git a/Franken-MoE-Large-Base.i1-IQ3_S.gguf b/Franken-MoE-Large-Base.i1-IQ3_S.gguf new file mode 100644 index 0000000..cbb1134 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8c4c3ecbb2920c022c63bc3f9070009e8ecd09e0ba852a544f6f255bcefe06ff +size 2028030816 diff --git a/Franken-MoE-Large-Base.i1-IQ3_XS.gguf b/Franken-MoE-Large-Base.i1-IQ3_XS.gguf new file mode 100644 index 0000000..336308d --- /dev/null +++ b/Franken-MoE-Large-Base.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b332da09fb1e69302e2240ed5287ce0291bdaf134edb8e62839d164567aaf80a +size 1924811616 diff --git a/Franken-MoE-Large-Base.i1-IQ3_XXS.gguf b/Franken-MoE-Large-Base.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..0bf0336 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8c67159ce117f3ca5e9fdb8db6a494931c6326cc195f994211ac0bfee25959d7 +size 1798176608 diff --git a/Franken-MoE-Large-Base.i1-IQ4_NL.gguf b/Franken-MoE-Large-Base.i1-IQ4_NL.gguf new file mode 100644 index 0000000..4ccd472 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fbd0a28dccda7282afbb6a312d9691191d468c5a7b55b974d257116c09c9ca29 +size 2612939616 diff --git a/Franken-MoE-Large-Base.i1-IQ4_XS.gguf b/Franken-MoE-Large-Base.i1-IQ4_XS.gguf new file mode 100644 index 0000000..7d7c23d --- /dev/null +++ b/Franken-MoE-Large-Base.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b7ce0226684b265f8e168771ad77735a73af7e41d35b8cd3b8c8323775b34d5 +size 2475314016 diff --git a/Franken-MoE-Large-Base.i1-Q2_K.gguf b/Franken-MoE-Large-Base.i1-Q2_K.gguf new file mode 100644 index 0000000..d419126 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b268ba67fc550fdf6a6c65b3dab665df8365ff72392d520c54a52bb2469367d9 +size 1732823904 diff --git a/Franken-MoE-Large-Base.i1-Q2_K_S.gguf b/Franken-MoE-Large-Base.i1-Q2_K_S.gguf new file mode 100644 index 0000000..4d6facb --- /dev/null +++ b/Franken-MoE-Large-Base.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:25bf9ce06d31d825f1a53145cd0144e2832af7995e9d1d2938b11960bcf65e84 +size 1616137056 diff --git a/Franken-MoE-Large-Base.i1-Q3_K_L.gguf b/Franken-MoE-Large-Base.i1-Q3_K_L.gguf new file mode 100644 index 0000000..c8e03bc --- /dev/null +++ b/Franken-MoE-Large-Base.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:91885be533f92f53cf544551ef22704e1aced60cf979aea2a36f737343797ae1 +size 2414070624 diff --git a/Franken-MoE-Large-Base.i1-Q3_K_M.gguf b/Franken-MoE-Large-Base.i1-Q3_K_M.gguf new file mode 100644 index 0000000..61fb29b --- /dev/null +++ b/Franken-MoE-Large-Base.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:faccfbf0560e75fdcb52bd60bb6455a75c57ae485fcb2a8c42d162703a171bd0 +size 2233453408 diff --git a/Franken-MoE-Large-Base.i1-Q3_K_S.gguf b/Franken-MoE-Large-Base.i1-Q3_K_S.gguf new file mode 100644 index 0000000..ce3a09b --- /dev/null +++ b/Franken-MoE-Large-Base.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2b470e564545ebf773a89a21bf5532830e3762e964b3e19f44a91f84085a7c7e +size 2028030816 diff --git a/Franken-MoE-Large-Base.i1-Q4_0.gguf b/Franken-MoE-Large-Base.i1-Q4_0.gguf new file mode 100644 index 0000000..25d066a --- /dev/null +++ b/Franken-MoE-Large-Base.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eaeef41075ddbd9b24abcde684d67faab63759da04dc8e24d8792cbb5eaab8d7 +size 2622376800 diff --git a/Franken-MoE-Large-Base.i1-Q4_1.gguf b/Franken-MoE-Large-Base.i1-Q4_1.gguf new file mode 100644 index 0000000..4f45177 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:544af7a1360eab630a59a8fc2a86df7d22074e0768da96f56742b03f121ec256 +size 2888190816 diff --git a/Franken-MoE-Large-Base.i1-Q4_K_M.gguf b/Franken-MoE-Large-Base.i1-Q4_K_M.gguf new file mode 100644 index 0000000..87f0f9e --- /dev/null +++ b/Franken-MoE-Large-Base.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c077cad00de8b9baa65a793d6b50df5cd054b78b0c15f253c600cef677af0b92 +size 2798390112 diff --git a/Franken-MoE-Large-Base.i1-Q4_K_S.gguf b/Franken-MoE-Large-Base.i1-Q4_K_S.gguf new file mode 100644 index 0000000..5e8d6b9 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1f80d95af549ef05335e3c07ebab67ddb4d9968724e9b6711def4846d5c93ab9 +size 2632338272 diff --git a/Franken-MoE-Large-Base.i1-Q5_K_M.gguf b/Franken-MoE-Large-Base.i1-Q5_K_M.gguf new file mode 100644 index 0000000..3f35a88 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:40711a7585294ff120ab314d3bfe5e5c457b2de8acabcc4ed55d35a4fac1de4f +size 3258977120 diff --git a/Franken-MoE-Large-Base.i1-Q5_K_S.gguf b/Franken-MoE-Large-Base.i1-Q5_K_S.gguf new file mode 100644 index 0000000..e8bbde4 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3f27438d9d2b49967091582e712d81f721d1deaec972a447d9c17bf7b5de21b9 +size 3163442016 diff --git a/Franken-MoE-Large-Base.i1-Q6_K.gguf b/Franken-MoE-Large-Base.i1-Q6_K.gguf new file mode 100644 index 0000000..150d9a8 --- /dev/null +++ b/Franken-MoE-Large-Base.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5dce9e548cf936dd39c65d1c957a01bd600b2d9cc2ec7824373a5b35655b0950 +size 3748350816 diff --git a/Franken-MoE-Large-Base.imatrix.gguf b/Franken-MoE-Large-Base.imatrix.gguf new file mode 100644 index 0000000..4c333b4 --- /dev/null +++ b/Franken-MoE-Large-Base.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6fe7b415eb69f4fdd80af4cd9d6ffe29e1698b42328faea1a4f3028c6aef7320 +size 9902752 diff --git a/README.md b/README.md new file mode 100644 index 0000000..ae09ab3 --- /dev/null +++ b/README.md @@ -0,0 +1,93 @@ +--- +base_model: DrRiceIO7/Franken-MoE-Large-Base +language: +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- moe +- merge +- qwen3 +- qwen3_moe +- triton +- unsloth +- pytorch +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/DrRiceIO7/Franken-MoE-Large-Base + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Franken-MoE-Large-Base-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Franken-MoE-Large-Base-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-IQ1_S.gguf) | i1-IQ1_S | 1.1 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-IQ1_M.gguf) | i1-IQ1_M | 1.2 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.4 | | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-IQ2_S.gguf) | i1-IQ2_S | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-IQ2_M.gguf) | i1-IQ2_M | 1.7 | | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.7 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-Q2_K.gguf) | i1-Q2_K | 1.8 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-IQ3_XS.gguf) | i1-IQ3_XS | 2.0 | | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-IQ3_S.gguf) | i1-IQ3_S | 2.1 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-Q3_K_S.gguf) | i1-Q3_K_S | 2.1 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-IQ3_M.gguf) | i1-IQ3_M | 2.2 | | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-Q3_K_M.gguf) | i1-Q3_K_M | 2.3 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-Q3_K_L.gguf) | i1-Q3_K_L | 2.5 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-IQ4_XS.gguf) | i1-IQ4_XS | 2.6 | | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.7 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-Q4_0.gguf) | i1-Q4_0 | 2.7 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.7 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.9 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-Q4_1.gguf) | i1-Q4_1 | 3.0 | | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-Q5_K_S.gguf) | i1-Q5_K_S | 3.3 | | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-Q5_K_M.gguf) | i1-Q5_K_M | 3.4 | | +| [GGUF](https://huggingface.co/mradermacher/Franken-MoE-Large-Base-i1-GGUF/resolve/main/Franken-MoE-Large-Base.i1-Q6_K.gguf) | i1-Q6_K | 3.8 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +