commit b2c17bae6bc5944aa40279503dd523aa5aa21538 Author: ModelHub XC Date: Wed Jun 17 09:20:19 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/hermeo-7b-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..46b2bc3 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text +hermeo-7b.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..ce8ad48 --- /dev/null +++ b/README.md @@ -0,0 +1,85 @@ +--- +base_model: malteos/hermeo-7b +language: +- en +- de +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- merge +- mergekit +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/malteos/hermeo-7b + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#hermeo-7b-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/hermeo-7b-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-IQ1_S.gguf) | i1-IQ1_S | 1.7 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-IQ1_M.gguf) | i1-IQ1_M | 1.9 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.1 | | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.3 | | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-IQ2_S.gguf) | i1-IQ2_S | 2.4 | | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-IQ2_M.gguf) | i1-IQ2_M | 2.6 | | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q2_K.gguf) | i1-Q2_K | 2.8 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 2.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.1 | | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.3 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-IQ3_S.gguf) | i1-IQ3_S | 3.3 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-IQ3_M.gguf) | i1-IQ3_M | 3.4 | | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q3_K_M.gguf) | i1-Q3_K_M | 3.6 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q3_K_L.gguf) | i1-Q3_K_L | 3.9 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.0 | | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q4_0_4_4.gguf) | i1-Q4_0_4_4 | 4.2 | fast on arm, low quality | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q4_0_4_8.gguf) | i1-Q4_0_4_8 | 4.2 | fast on arm+i8mm, low quality | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q4_0_8_8.gguf) | i1-Q4_0_8_8 | 4.2 | fast on arm+sve, low quality | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q4_0.gguf) | i1-Q4_0 | 4.2 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.2 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q4_K_M.gguf) | i1-Q4_K_M | 4.5 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.1 | | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.2 | | +| [GGUF](https://huggingface.co/mradermacher/hermeo-7b-i1-GGUF/resolve/main/hermeo-7b.i1-Q6_K.gguf) | i1-Q6_K | 6.0 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/hermeo-7b.i1-IQ1_M.gguf b/hermeo-7b.i1-IQ1_M.gguf new file mode 100644 index 0000000..704b4ac --- /dev/null +++ b/hermeo-7b.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2227ec5ab0296f1fac815e4b59694efa54847a5723744575a3971feeb41e0838 +size 1754454848 diff --git a/hermeo-7b.i1-IQ1_S.gguf b/hermeo-7b.i1-IQ1_S.gguf new file mode 100644 index 0000000..ae1ccbf --- /dev/null +++ b/hermeo-7b.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f5fac98b64eeb6370abf02f354ef48d3e4101168aefc59c92b0df01156f1fd9c +size 1612110656 diff --git a/hermeo-7b.i1-IQ2_M.gguf b/hermeo-7b.i1-IQ2_M.gguf new file mode 100644 index 0000000..4a65dc2 --- /dev/null +++ b/hermeo-7b.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd81f7c57ff489ef28f56357ad5cd41c06426fa7689ab688747736b8c850506b +size 2500722304 diff --git a/hermeo-7b.i1-IQ2_S.gguf b/hermeo-7b.i1-IQ2_S.gguf new file mode 100644 index 0000000..2aab94f --- /dev/null +++ b/hermeo-7b.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9cc6e034ccbf74a685ae7128a0ad6d12b245fb6134ad5829470006a629a9b5a4 +size 2310930048 diff --git a/hermeo-7b.i1-IQ2_XS.gguf b/hermeo-7b.i1-IQ2_XS.gguf new file mode 100644 index 0000000..1858687 --- /dev/null +++ b/hermeo-7b.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8b3b4c0864c4f0bfeef7285013363055a6d50847dbb8c8972e153d7524e56fb9 +size 2198264640 diff --git a/hermeo-7b.i1-IQ2_XXS.gguf b/hermeo-7b.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..ba698fc --- /dev/null +++ b/hermeo-7b.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3d3e5952f9b6daa92f116c15a7ef28848f8668b659cbb02dc3b18bfdd9eebfad +size 1991695168 diff --git a/hermeo-7b.i1-IQ3_M.gguf b/hermeo-7b.i1-IQ3_M.gguf new file mode 100644 index 0000000..bf38039 --- /dev/null +++ b/hermeo-7b.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:509628a634c1c9c9cae9dbe2f49a1796002187356fddbd7b47af50a7e6fca50a +size 3284902592 diff --git a/hermeo-7b.i1-IQ3_S.gguf b/hermeo-7b.i1-IQ3_S.gguf new file mode 100644 index 0000000..5fe72cf --- /dev/null +++ b/hermeo-7b.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4079614d8f1458a1cd2e8f7ca9d50888e3b3e177c78071bacfb846d13e4ab879 +size 3182404288 diff --git a/hermeo-7b.i1-IQ3_XS.gguf b/hermeo-7b.i1-IQ3_XS.gguf new file mode 100644 index 0000000..c699d81 --- /dev/null +++ b/hermeo-7b.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:995ac70ffb222962f15d4a14f74339384d0843f03864c18368bc606681178fff +size 3018826432 diff --git a/hermeo-7b.i1-IQ3_XXS.gguf b/hermeo-7b.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..7c31dd6 --- /dev/null +++ b/hermeo-7b.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ccb12e82c2a7ed1a017dc19f3d63d991eef70e0daa1e15d190f48b7113907870 +size 2827353728 diff --git a/hermeo-7b.i1-IQ4_XS.gguf b/hermeo-7b.i1-IQ4_XS.gguf new file mode 100644 index 0000000..0f2a2ab --- /dev/null +++ b/hermeo-7b.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d40f3c0bfd240cc571a5e7f7b45f9eefa4f4d800f54d60ec854269f637c8e021 +size 3907700224 diff --git a/hermeo-7b.i1-Q2_K.gguf b/hermeo-7b.i1-Q2_K.gguf new file mode 100644 index 0000000..ddb2962 --- /dev/null +++ b/hermeo-7b.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:936cd169ada4c842551ab7c2be7989de9f7d5e9300f2a3afb8f810aefeb05b4a +size 2719252352 diff --git a/hermeo-7b.i1-Q3_K_L.gguf b/hermeo-7b.i1-Q3_K_L.gguf new file mode 100644 index 0000000..35f6dab --- /dev/null +++ b/hermeo-7b.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:73c644087f66bcab8f85aac950ce5adfaa07b544fb2272be735e8389d5a4dab9 +size 3822035648 diff --git a/hermeo-7b.i1-Q3_K_M.gguf b/hermeo-7b.i1-Q3_K_M.gguf new file mode 100644 index 0000000..3b7987e --- /dev/null +++ b/hermeo-7b.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:06a4238b6f02d0369d078b319e1db983f4bc002e8f3e4e1e448bc1dce020d4da +size 3518997184 diff --git a/hermeo-7b.i1-Q3_K_S.gguf b/hermeo-7b.i1-Q3_K_S.gguf new file mode 100644 index 0000000..2f7b2ef --- /dev/null +++ b/hermeo-7b.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3cfeeb7373a2e281fc1baf1b5bb86ec2fc769d701a6360133028b7716d1b3efb +size 3164578496 diff --git a/hermeo-7b.i1-Q4_0.gguf b/hermeo-7b.i1-Q4_0.gguf new file mode 100644 index 0000000..129b7cf --- /dev/null +++ b/hermeo-7b.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b24c4b94e32576d0b660f1c0f64852c5c5de3246211af2cf9339419d7aaf472 +size 4123608832 diff --git a/hermeo-7b.i1-Q4_0_4_4.gguf b/hermeo-7b.i1-Q4_0_4_4.gguf new file mode 100644 index 0000000..f438a5f --- /dev/null +++ b/hermeo-7b.i1-Q4_0_4_4.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7a1055d8caca534e860f250d0404de59adc00d6fd865a037460d0586e30de62f +size 4108928768 diff --git a/hermeo-7b.i1-Q4_0_4_8.gguf b/hermeo-7b.i1-Q4_0_4_8.gguf new file mode 100644 index 0000000..b59f2aa --- /dev/null +++ b/hermeo-7b.i1-Q4_0_4_8.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e37c941f9af1de2efe8aa48eff78b3dc770670f7b5982b0357ff7538171b57c2 +size 4108928768 diff --git a/hermeo-7b.i1-Q4_0_8_8.gguf b/hermeo-7b.i1-Q4_0_8_8.gguf new file mode 100644 index 0000000..2c7b230 --- /dev/null +++ b/hermeo-7b.i1-Q4_0_8_8.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19e360fce5bc1dc9458e6c63a918b40007f1aecae96a25f54aa8d2295460094c +size 4108928768 diff --git a/hermeo-7b.i1-Q4_K_M.gguf b/hermeo-7b.i1-Q4_K_M.gguf new file mode 100644 index 0000000..8acebf5 --- /dev/null +++ b/hermeo-7b.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:88564725422ee8df03285d1d8fba8581b29b304678c00dc6593b50fb75df6265 +size 4368451328 diff --git a/hermeo-7b.i1-Q4_K_S.gguf b/hermeo-7b.i1-Q4_K_S.gguf new file mode 100644 index 0000000..f925a99 --- /dev/null +++ b/hermeo-7b.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:453a6a6cb091a5ea8648554b423157ff02def7c5f52b992657322801361a0aa6 +size 4140386048 diff --git a/hermeo-7b.i1-Q5_K_M.gguf b/hermeo-7b.i1-Q5_K_M.gguf new file mode 100644 index 0000000..1a75eb5 --- /dev/null +++ b/hermeo-7b.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3033131a9ebda11761c7c4a41609d25f2bd16e6758474b53bd3f7715c9e1a3dc +size 5131422464 diff --git a/hermeo-7b.i1-Q5_K_S.gguf b/hermeo-7b.i1-Q5_K_S.gguf new file mode 100644 index 0000000..712c84b --- /dev/null +++ b/hermeo-7b.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0e61b4ff3c066aff32493b44f6aa17ca99e2e3389d9f27b019673820654a315b +size 4997729024 diff --git a/hermeo-7b.i1-Q6_K.gguf b/hermeo-7b.i1-Q6_K.gguf new file mode 100644 index 0000000..901d379 --- /dev/null +++ b/hermeo-7b.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fdb225268aa4aa2027553b7a656c1dbd67a7bb129aaf8bd3b21e37ad21211d24 +size 5942079296 diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..3b52e6d --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2f93f85b7e71b1bb70bd169804426ed1e268c0b126604a287e850288c77a27c2 +size 4988157