mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-09 18:27:16 +08:00
commit 339fb1aad2
24 changed files with 210 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
.gitattributes vendored Normal file
View File

@@ -0,0 +1,57 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Hermes-2-Pro-Mistral-7B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text

3
Hermes-2-Pro-Mistral-7B.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ac7fc1928c0b3e55ed9f849f1e3e078f20c37bca12f35188091c62189b343acb
size 1754585344

3
Hermes-2-Pro-Mistral-7B.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3280b549cc8f4dc9232265108eee7298aec1eb886747aea6cfe7c8056580a0ab
size 1612241152

3
Hermes-2-Pro-Mistral-7B.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e2484904f26e74100f82595affdb09cd1e44d26b80aafaee9a6d4de393bb233f
size 2500865280

3
Hermes-2-Pro-Mistral-7B.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6d2c72a4b33b9b5adbd5ed8a29d853b333a5fbb07a9c8c7303be46009a7812b9
size 2311073024

3
Hermes-2-Pro-Mistral-7B.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6aa3d2b1fc88c6f033ac7fec097d2bc894e00c9fe63a033c7fbb42f2332b442d
size 2198395136

3
Hermes-2-Pro-Mistral-7B.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b8e27de8406e6da64f790ece63d24ebb16058784dd296d529f308a3d2d82fa46
size 1991825664

3
Hermes-2-Pro-Mistral-7B.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2a4921ea2813298641ee86ff7232cd0258446428963b0381045a64e2ce23fe3f
size 3285061888

3
Hermes-2-Pro-Mistral-7B.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:298653be203dfb94743329199e77ea72ebf3cd204eb807f7e7b0ab653daacd1d
size 3182563584

3
Hermes-2-Pro-Mistral-7B.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7a95b6fd40e79ea53b8e1145c7b1c4c31b893b42931bb1ddc2aeda3d8c5665d9
size 3018985728

3
Hermes-2-Pro-Mistral-7B.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b7f72ba3b74928d3fe58394abaff0aacb57702b2548b2263d5266aa31fb0aec9
size 2827496704

3
Hermes-2-Pro-Mistral-7B.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dcb28194119e1e3dc8256165a86adb430e845b062d8e6a5814ce5c2c16814e85
size 3907872000

3
Hermes-2-Pro-Mistral-7B.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6fb9104bbf2a747d545bc5e70172cfcb0abc1170a3d3c5432f0e7938655877b5
size 2719399168

3
Hermes-2-Pro-Mistral-7B.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:afc583585e884dcd6890eda103aeb7360dda55d6f0dc0e658e9796e5b85c6aa1
size 3822194944

3
Hermes-2-Pro-Mistral-7B.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1b630a8729eba6523494bf5ae755e05593c3c2563f325c5b5fdb4244f529f88d
size 3519156480

3
Hermes-2-Pro-Mistral-7B.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:439bfdf51be07c799641ca206829aff52ed12701986ea1e99d0145eba4560fb7
size 3164737792

3
Hermes-2-Pro-Mistral-7B.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:195efba96a4d8444b2851cf782b395bc8e6a25184db3ffa4dd1604e71e391728
size 4123784448

3
Hermes-2-Pro-Mistral-7B.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:20b253c25f7d17ae34080804396b107eeb2c0ad893a22109431ee79156c68098
size 4368626944

3
Hermes-2-Pro-Mistral-7B.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:27694783a4ab98a1a14a69b4050a5cda4670f0d26abdd29d62abacaf59dbfc2d
size 4140561664

3
Hermes-2-Pro-Mistral-7B.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f5cb7bc4fdcea507490a5a557cb4f8267c72c4a5b34a40388032a9f55d41b7f8
size 5131613440

3
Hermes-2-Pro-Mistral-7B.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9d62688e7133e588762eb305aa25823edf39c788431de74c69c4b678f20ed1cf
size 4997920000

3
Hermes-2-Pro-Mistral-7B.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:82640ff856b3c73aae2ec02732f6313d428aadfe2215d0660b6867a225823837
size 5942286592

87
README.md Normal file
View File

@@ -0,0 +1,87 @@
---
base_model: NousResearch/Hermes-2-Pro-Mistral-7B
datasets:
- teknium/OpenHermes-2.5
language:
- en
library_name: transformers
license: apache-2.0
quantized_by: mradermacher
tags:
- Mistral
- instruct
- finetune
- chatml
- DPO
- RLHF
- gpt4
- synthetic data
- distillation
- function calling
- json mode
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
weighted/imatrix quants of https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-IQ1_S.gguf) | i1-IQ1_S | 1.7 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-IQ1_M.gguf) | i1-IQ1_M | 1.9 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.1 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.3 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-IQ2_S.gguf) | i1-IQ2_S | 2.4 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-IQ2_M.gguf) | i1-IQ2_M | 2.6 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-Q2_K.gguf) | i1-Q2_K | 2.8 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 2.9 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.1 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.3 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-IQ3_S.gguf) | i1-IQ3_S | 3.3 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-IQ3_M.gguf) | i1-IQ3_M | 3.4 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 3.6 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 3.9 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.0 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-Q4_0.gguf) | i1-Q4_0 | 4.2 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.2 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 4.5 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.1 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.2 | |
| [GGUF](https://huggingface.co/mradermacher/Hermes-2-Pro-Mistral-7B-i1-GGUF/resolve/main/Hermes-2-Pro-Mistral-7B.i1-Q6_K.gguf) | i1-Q6_K | 6.0 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:09b3c0fc65cc97631e9b738c4869f9be72111e6543d2debdc15a4d79ebbcb49a
size 4988161