mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-09 07:25:16 +08:00
commit 066bdf9b1b
24 changed files with 211 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
.gitattributes vendored Normal file
View File

@@ -0,0 +1,57 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Llama-3-8B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text

3
Hermes-2-Pro-Llama-3-8B.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:66c6308bed70cd09c2ee89f69734532e79b544d8a1a24604f188b9225695a51f
size 2162111200

3
Hermes-2-Pro-Llama-3-8B.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:594b04970fb3b60aa4b87257bc21b66ec2b3afa155e10a386df49ad66e0e6066
size 2019767008

3
Hermes-2-Pro-Llama-3-8B.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4f564fb4d59467d8aa2d2e51c8c632f8ab6e886395e37b54b85a81c5a36e2e22
size 2948433632

3
Hermes-2-Pro-Llama-3-8B.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:95bc04b0d672a3c0861956142d59c842b180f6bb9887dd15bb144c5e1807d592
size 2758641376

3
Hermes-2-Pro-Llama-3-8B.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:801b165455e5b08367435a8822fc67967f6d678f5282fc23cdf6d5a5bfb44bc1
size 2605920992

3
Hermes-2-Pro-Llama-3-8B.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a9eacf6ee26ffa4d0d0829376fd471375ebf054ee7061dd85af86fadd3ff7523
size 2399351520

3
Hermes-2-Pro-Llama-3-8B.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f7d8afe9861eeee9ac2996f8a823a9db82582e4bade007af4b5c909f0886a601
size 3784993504

3
Hermes-2-Pro-Llama-3-8B.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:38b9c16bbf9e8495d9a8eddac292a14fa6ec9195635b58bb91778ca5b0f9c934
size 3682495200

3
Hermes-2-Pro-Llama-3-8B.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a144af979b2788e0b680da141fb4044e38388faefb9ef873a5c143498f61e3b7
size 3518917344

3
Hermes-2-Pro-Llama-3-8B.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:eabbea6b9da53e10dc5d9c56c74f9b022b7f2bd0e2a3de9804c4e85bb49bfd94
size 3275065056

3
Hermes-2-Pro-Llama-3-8B.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:48bf49581763e4c3fecfe1ff51a47f9f5fb9027c3e962f23845a122c9e3af16d
size 4447846112

3
Hermes-2-Pro-Llama-3-8B.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1ae8f8b6a23496e4eb2049318d3f31160e7b2b30ade46c9f12078bb73994e637
size 3179288288

3
Hermes-2-Pro-Llama-3-8B.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:71fb3a2e1dc6a75794ef5a727838fbc0398f150de79045eebb31e338b5c1362d
size 4322126560

3
Hermes-2-Pro-Llama-3-8B.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a4f1ab299f1c139672a8933131d222d82f0e8f9a9606cbca4c2934ab00d5877a
size 4019088096

3
Hermes-2-Pro-Llama-3-8B.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ea6c02ca8c6d1ad8a59589f7f587583e9c1c3f2f69c87acfa386782b76bc8a45
size 3664669408

3
Hermes-2-Pro-Llama-3-8B.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6999dc0e450f331b4c80ea9949be27be534a8c94415c3d1545b64a04a7707a38
size 4676079328

3
Hermes-2-Pro-Llama-3-8B.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:652caebe51149e2cb9e7e0439ac81adb5d9c87f22a8a25e4ec4523d0a69fec67
size 4920921824

3
Hermes-2-Pro-Llama-3-8B.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b24d4b86c8af76a5b6270452156861d4ad642155365ea51eb108141e1a8f64f7
size 4692856544

3
Hermes-2-Pro-Llama-3-8B.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d60deeff293be310f2f053be4f44e2830ebb2ef953fc0900223db39bd87ed311
size 5733191392

3
Hermes-2-Pro-Llama-3-8B.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2004196dde8106fd73e914ff43b0faaded4834b6bfaa997d8318460b835a5c35
size 5599497952

3
Hermes-2-Pro-Llama-3-8B.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:79421a1bba3055df7a52cbcc7a06996f5594d5797c64ac2f02f2c6ea5f106266
size 6596227808

88
README.md Normal file
View File

@@ -0,0 +1,88 @@
---
base_model: emplitude/Hermes-2-Pro-Llama-3-8B
datasets:
- teknium/OpenHermes-2.5
language:
- en
library_name: transformers
license: llama3
quantized_by: mradermacher
tags:
- Llama-3
- instruct
- finetune
- chatml
- DPO
- RLHF
- gpt4
- synthetic data
- distillation
- function calling
- json mode
- axolotl
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
weighted/imatrix quants of https://huggingface.co/emplitude/Hermes-2-Pro-Llama-3-8B
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.5 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.7 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-IQ2_S.gguf) | i1-IQ2_S | 2.9 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-IQ2_M.gguf) | i1-IQ2_M | 3.0 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-Q2_K.gguf) | i1-Q2_K | 3.3 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.4 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.6 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.8 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-IQ3_S.gguf) | i1-IQ3_S | 3.8 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-IQ3_M.gguf) | i1-IQ3_M | 3.9 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.1 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.4 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.5 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-Q4_0.gguf) | i1-Q4_0 | 4.8 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.0 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.7 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Llama-3-8B-i1-GGUF/resolve/main/Hermes-2-Pro-Llama-3-8B.i1-Q6_K.gguf) | i1-Q6_K | 6.7 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4100cc8a27c0c31f280f2334ea3f11836217c3afb20bb423c68c9259cf8a271f
size 4988157