commit b318354de21636fb4dabd66792b0c9a486137320 Author: ModelHub XC Date: Sat Apr 11 09:17:02 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Megatron-Opus-14B-Exp-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..a42a758 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Megatron-Opus-14B-Exp.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Megatron-Opus-14B-Exp.i1-IQ1_M.gguf b/Megatron-Opus-14B-Exp.i1-IQ1_M.gguf new file mode 100644 index 0000000..40154ba --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b8e622b2fe1474cb96fa7659896c4bd396f7e79cadb9685bfc64f4a04882bb47 +size 3870225920 diff --git a/Megatron-Opus-14B-Exp.i1-IQ1_S.gguf b/Megatron-Opus-14B-Exp.i1-IQ1_S.gguf new file mode 100644 index 0000000..0df7071 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5eae51ef4fdc33a1025eb8e1113422cbdb2ac28ff7d8cf3d46bc6ccb00cc41c2 +size 3605911040 diff --git a/Megatron-Opus-14B-Exp.i1-IQ2_M.gguf b/Megatron-Opus-14B-Exp.i1-IQ2_M.gguf new file mode 100644 index 0000000..7081c09 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:29409e39a235c6fdd853068b4146556db6c9b9d8ea103cb7ce36b270018732cf +size 5353855872 diff --git a/Megatron-Opus-14B-Exp.i1-IQ2_S.gguf b/Megatron-Opus-14B-Exp.i1-IQ2_S.gguf new file mode 100644 index 0000000..5f4315f --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f42ab3450c0bee9be3f5158132aa767392bc4d37a781ff4f5d6aed0f0b7ae004 +size 5001436032 diff --git a/Megatron-Opus-14B-Exp.i1-IQ2_XS.gguf b/Megatron-Opus-14B-Exp.i1-IQ2_XS.gguf new file mode 100644 index 0000000..17e5408 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:03c46722de7a0f671dfdb5496554d439e3f344a536af1afdb2d5fb548b3b504f +size 4702492160 diff --git a/Megatron-Opus-14B-Exp.i1-IQ2_XXS.gguf b/Megatron-Opus-14B-Exp.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..bafa474 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da9b5a03d44d1ff21d5ec8621bb3389ca6f4cf6bfb230190e98e26ceda4bc402 +size 4310750720 diff --git a/Megatron-Opus-14B-Exp.i1-IQ3_M.gguf b/Megatron-Opus-14B-Exp.i1-IQ3_M.gguf new file mode 100644 index 0000000..7c57bb7 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:11d75fa0970fc8d9508a34f734dea7e8ea5a12e38d2cd0708ec9b210e83d8442 +size 6913976256 diff --git a/Megatron-Opus-14B-Exp.i1-IQ3_S.gguf b/Megatron-Opus-14B-Exp.i1-IQ3_S.gguf new file mode 100644 index 0000000..46b4f9e --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:be6f483f8fceae8d964d9ff65e3d87062096310409f47fffd7992afff88cb38f +size 6690457536 diff --git a/Megatron-Opus-14B-Exp.i1-IQ3_XS.gguf b/Megatron-Opus-14B-Exp.i1-IQ3_XS.gguf new file mode 100644 index 0000000..71d70eb --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4019cda39ff8ac0e017ae9f190e19bea34c0b1c3b0828391bea72d785a5ff54f +size 6380799936 diff --git a/Megatron-Opus-14B-Exp.i1-IQ3_XXS.gguf b/Megatron-Opus-14B-Exp.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..842f403 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0fdcac5f065d32e0e3dc0e8b0348b5f6ec9207cc9111ef2012828799eb00b47a +size 5944417152 diff --git a/Megatron-Opus-14B-Exp.i1-IQ4_NL.gguf b/Megatron-Opus-14B-Exp.i1-IQ4_NL.gguf new file mode 100644 index 0000000..c280db8 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26a39610c04727d42c9864fe4e02ca48dd0799c85b52738f5e541650e82d7225 +size 8546350048 diff --git a/Megatron-Opus-14B-Exp.i1-IQ4_XS.gguf b/Megatron-Opus-14B-Exp.i1-IQ4_XS.gguf new file mode 100644 index 0000000..a045d04 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b064f108154742f646d2a8afb00f81a44d96c5764e52294839a6f135e104d834 +size 8117071168 diff --git a/Megatron-Opus-14B-Exp.i1-Q2_K.gguf b/Megatron-Opus-14B-Exp.i1-Q2_K.gguf new file mode 100644 index 0000000..52e0b07 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5589e693617400975a0ae9390356de70c5dfb490e16b848b637e1f97c146ddab +size 5768143424 diff --git a/Megatron-Opus-14B-Exp.i1-Q2_K_S.gguf b/Megatron-Opus-14B-Exp.i1-Q2_K_S.gguf new file mode 100644 index 0000000..4fc006d --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:06471d047813e24b0a5354068415b3bdee596c7fbbcc9289a2194ce3c40d8632 +size 5394833984 diff --git a/Megatron-Opus-14B-Exp.i1-Q3_K_L.gguf b/Megatron-Opus-14B-Exp.i1-Q3_K_L.gguf new file mode 100644 index 0000000..247619f --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:82f9fad83ac8d463be0684f07e7fed16ea71b4abc0f737075b02dc2c6124212a +size 7922206656 diff --git a/Megatron-Opus-14B-Exp.i1-Q3_K_M.gguf b/Megatron-Opus-14B-Exp.i1-Q3_K_M.gguf new file mode 100644 index 0000000..bff4af7 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:85ea46107f4df4a5c9a6a4f6172183bc2f0e7d3f12342a10611275c18516e2be +size 7336642496 diff --git a/Megatron-Opus-14B-Exp.i1-Q3_K_S.gguf b/Megatron-Opus-14B-Exp.i1-Q3_K_S.gguf new file mode 100644 index 0000000..b5063fa --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1a998ac06d82dc820ba8cb76e761d99ae529e2ffa0f5e8c02ee1f3c8be888fa9 +size 6657034176 diff --git a/Megatron-Opus-14B-Exp.i1-Q4_0.gguf b/Megatron-Opus-14B-Exp.i1-Q4_0.gguf new file mode 100644 index 0000000..ea98ce3 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:20b1b43e09f28ced065743252cb466db3e6517d94b94449499e82673f80fcf8d +size 8541434848 diff --git a/Megatron-Opus-14B-Exp.i1-Q4_1.gguf b/Megatron-Opus-14B-Exp.i1-Q4_1.gguf new file mode 100644 index 0000000..451fd58 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0a59e5e3a128670a6f9f1ae2114887c7a6bb94e318f89c5b9cbee9c7d7f4b12e +size 9389179168 diff --git a/Megatron-Opus-14B-Exp.i1-Q4_K_M.gguf b/Megatron-Opus-14B-Exp.i1-Q4_K_M.gguf new file mode 100644 index 0000000..4e1fbd4 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e49d566d494e1ea2c5f238ea17b514d56a3d71422e51a47da8ea644b18116bad +size 8985277408 diff --git a/Megatron-Opus-14B-Exp.i1-Q4_K_S.gguf b/Megatron-Opus-14B-Exp.i1-Q4_K_S.gguf new file mode 100644 index 0000000..7f73e9e --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ef4265c356bdd0882bdcb4741d8c0ae2e8634f891607ad09069cd39f9464efc +size 8570598368 diff --git a/Megatron-Opus-14B-Exp.i1-Q5_K_M.gguf b/Megatron-Opus-14B-Exp.i1-Q5_K_M.gguf new file mode 100644 index 0000000..44d3f80 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da54263aa912ff84c684551f80ee7cd683a3325306d330a4b389de6afc2cc8ae +size 10505784928 diff --git a/Megatron-Opus-14B-Exp.i1-Q5_K_S.gguf b/Megatron-Opus-14B-Exp.i1-Q5_K_S.gguf new file mode 100644 index 0000000..0e0f913 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c851113d1e4531c98811711996122bbc66d87ab0501320b1e9b78fb0820e2d35 +size 10263465568 diff --git a/Megatron-Opus-14B-Exp.i1-Q6_K.gguf b/Megatron-Opus-14B-Exp.i1-Q6_K.gguf new file mode 100644 index 0000000..cdbd132 --- /dev/null +++ b/Megatron-Opus-14B-Exp.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:52c757bbdb3b3e6a0359c0188a7348ab2991ce4955352da4a26e8b86bf837ceb +size 12121324192 diff --git a/README.md b/README.md new file mode 100644 index 0000000..3248d40 --- /dev/null +++ b/README.md @@ -0,0 +1,81 @@ +--- +base_model: prithivMLmods/Megatron-Opus-14B-Exp +language: +- en +library_name: transformers +license: apache-2.0 +quantized_by: mradermacher +tags: +- Reasoning +- Math +- text-generation-inference +- eXP +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/prithivMLmods/Megatron-Opus-14B-Exp + + +static quants are available at https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-IQ1_S.gguf) | i1-IQ1_S | 3.7 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-IQ1_M.gguf) | i1-IQ1_M | 4.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 4.4 | | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.8 | | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-IQ2_S.gguf) | i1-IQ2_S | 5.1 | | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-IQ2_M.gguf) | i1-IQ2_M | 5.5 | | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-Q2_K_S.gguf) | i1-Q2_K_S | 5.5 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-Q2_K.gguf) | i1-Q2_K | 5.9 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 6.0 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-IQ3_XS.gguf) | i1-IQ3_XS | 6.5 | | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-Q3_K_S.gguf) | i1-Q3_K_S | 6.8 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-IQ3_S.gguf) | i1-IQ3_S | 6.8 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-IQ3_M.gguf) | i1-IQ3_M | 7.0 | | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-Q3_K_M.gguf) | i1-Q3_K_M | 7.4 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-Q3_K_L.gguf) | i1-Q3_K_L | 8.0 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-IQ4_XS.gguf) | i1-IQ4_XS | 8.2 | | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-Q4_0.gguf) | i1-Q4_0 | 8.6 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-IQ4_NL.gguf) | i1-IQ4_NL | 8.6 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-Q4_K_S.gguf) | i1-Q4_K_S | 8.7 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-Q4_K_M.gguf) | i1-Q4_K_M | 9.1 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-Q4_1.gguf) | i1-Q4_1 | 9.5 | | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-Q5_K_S.gguf) | i1-Q5_K_S | 10.4 | | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-Q5_K_M.gguf) | i1-Q5_K_M | 10.6 | | +| [GGUF](https://huggingface.co/mradermacher/Megatron-Opus-14B-Exp-i1-GGUF/resolve/main/Megatron-Opus-14B-Exp.i1-Q6_K.gguf) | i1-Q6_K | 12.2 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..5a47fbd --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a8c200c234efc12ade35df260150fd2f7675347db1a89cc834f69c76b6e566be +size 8563597