mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-10-12 04:41:17 +08:00
commit 566d85d96c
26 changed files with 227 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

59
.gitattributes vendored Normal file
View File

@@ -0,0 +1,59 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
DeepHermes-3-Mistral-24B-Preview.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

3
DeepHermes-3-Mistral-24B-Preview.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2fbf1896aba8a90cb58beed108c4a0f4ff66a3bc5067d91bbd09d4fc28e2833b
size 5750527008

3
DeepHermes-3-Mistral-24B-Preview.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9385ea0f4467d6c1b55077b94f9d282c1efc125b9858af8dba122a295d059669
size 5273752608

3
DeepHermes-3-Mistral-24B-Preview.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f8c4578362436ec5411c057e25f028ff9c04989ff17dd80676220164c370dc27
size 8114085984

3
DeepHermes-3-Mistral-24B-Preview.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7573e1bb29c12bb27b21c3132cb18091ff97b21c47b35b0dee67af960a5c7e81
size 7478386784

3
DeepHermes-3-Mistral-24B-Preview.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:91125060289bbffe2867b39de4853071972a5976cc7083d2daf23f0087cc79a3
size 7207064608

3
DeepHermes-3-Mistral-24B-Preview.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f4cf47c5849fd8be9de5b80b56f01b14e1821d4e69e48403e4ff73bf0029cc1d
size 6545151008

3
DeepHermes-3-Mistral-24B-Preview.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8af3ee9cf358f0e874fc7ce7461019ba2e3bd481fc797de3016aa85b091cd697
size 10650988640

3
DeepHermes-3-Mistral-24B-Preview.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:262b501e3f159f20130d94c2184d9119b46422ebe0e63aeac5aff426f4e96140
size 10428166240

3
DeepHermes-3-Mistral-24B-Preview.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d35b4b9b42741a50f6bd1e42c3b879e8fcd56d2b6add7be56dc2f23cbf55a470
size 9907155040

3
DeepHermes-3-Mistral-24B-Preview.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3a05c92b1c5fc04cfcab4c03b45d87a65a4aaed8ad2a752148c190a943c2bf3f
size 9280626784

3
DeepHermes-3-Mistral-24B-Preview.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:851a94350b42af866f09c15fd149e87dbee913b48f3b9de2162031eeafd0c4dc
size 12758957184

3
DeepHermes-3-Mistral-24B-Preview.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7dc0617d702c54be01ae14b88db87aa41bbd19457733a359d768a92c129f36d6
size 8890360864

3
DeepHermes-3-Mistral-24B-Preview.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a05b778ff0608eee5f57ffb5440f345dd117600477835cb6fc0527f64bc0718a
size 8320197664

3
DeepHermes-3-Mistral-24B-Preview.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:785bde79e41bcb8f097dce5c5d2f9c0e13403c683aa1c23e735fec1e4844e45b
size 12400799840

3
DeepHermes-3-Mistral-24B-Preview.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f0b714d0960601a6266e8082c8dc37f5df278ec5dfc96988eccf7d55b70d5939
size 11474120800

3
DeepHermes-3-Mistral-24B-Preview.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:74c82db41779d2fb5dc2a9fc5e2595f8055959ccb613f7a43e2fb2a413d9a348
size 10400313440

3
DeepHermes-3-Mistral-24B-Preview.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:659e32193d0856f58fee18807cecca3a727a3be7284c1c2d20d347d0ea141de9
size 13494272064

3
DeepHermes-3-Mistral-24B-Preview.i1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:48ee5aa35b8ce30ea65d052f301c28ee4e56cd83f0d5f3db642dfaccc15bc780
size 14873151424

3
DeepHermes-3-Mistral-24B-Preview.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:880690ea4c7a1ddffaecf05a25355524c4ef4e3f087edaa98707cf3f0e1e3a63
size 14333952064

3
DeepHermes-3-Mistral-24B-Preview.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:57d1f1b3924277326cac3c0bf58641ea687a9ad66c3709be562802f343f51988
size 13549322304

3
DeepHermes-3-Mistral-24B-Preview.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:eda1acd9600e9bbc222aa69c12a7f81c301e63d8b063ae620168cfe8e6128cb2
size 16764030784

3
DeepHermes-3-Mistral-24B-Preview.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4f2bdc52c3b9e1787c5361db7f1c1a3444ba31dc40a1c50aefc663bf29285f85
size 16304459584

3
DeepHermes-3-Mistral-24B-Preview.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4053570b47af0e928ce26575882ac1688994abd7565d794baec6f17bee01fd86
size 19345989440

96
README.md Normal file
View File

@@ -0,0 +1,96 @@
---
base_model: NousResearch/DeepHermes-3-Mistral-24B-Preview
language:
- en
library_name: transformers
license: apache-2.0
mradermacher:
readme_rev: 1
quantized_by: mradermacher
tags:
- Mistral-Small
- instruct
- finetune
- chatml
- gpt4
- synthetic data
- distillation
- function calling
- json mode
- axolotl
- roleplaying
- chat
- reasoning
- r1
- vllm
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
weighted/imatrix quants of https://huggingface.co/NousResearch/DeepHermes-3-Mistral-24B-Preview
<!-- provided-files -->
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#DeepHermes-3-Mistral-24B-Preview-i1-GGUF).***
static quants are available at https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-IQ1_S.gguf) | i1-IQ1_S | 5.4 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-IQ1_M.gguf) | i1-IQ1_M | 5.9 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 6.6 | |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-IQ2_XS.gguf) | i1-IQ2_XS | 7.3 | |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-IQ2_S.gguf) | i1-IQ2_S | 7.6 | |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-IQ2_M.gguf) | i1-IQ2_M | 8.2 | |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-Q2_K_S.gguf) | i1-Q2_K_S | 8.4 | very low quality |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-Q2_K.gguf) | i1-Q2_K | 9.0 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 9.4 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-IQ3_XS.gguf) | i1-IQ3_XS | 10.0 | |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-Q3_K_S.gguf) | i1-Q3_K_S | 10.5 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-IQ3_S.gguf) | i1-IQ3_S | 10.5 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-IQ3_M.gguf) | i1-IQ3_M | 10.8 | |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-Q3_K_M.gguf) | i1-Q3_K_M | 11.6 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-Q3_K_L.gguf) | i1-Q3_K_L | 12.5 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-IQ4_XS.gguf) | i1-IQ4_XS | 12.9 | |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-Q4_0.gguf) | i1-Q4_0 | 13.6 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-Q4_K_S.gguf) | i1-Q4_K_S | 13.6 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-Q4_K_M.gguf) | i1-Q4_K_M | 14.4 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-Q4_1.gguf) | i1-Q4_1 | 15.0 | |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-Q5_K_S.gguf) | i1-Q5_K_S | 16.4 | |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-Q5_K_M.gguf) | i1-Q5_K_M | 16.9 | |
| [GGUF](https://huggingface.co/mradermacher/DeepHermes-3-Mistral-24B-Preview-i1-GGUF/resolve/main/DeepHermes-3-Mistral-24B-Preview.i1-Q6_K.gguf) | i1-Q6_K | 19.4 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:361eb08642d99d49b1f7016cc6515f081731f8170d668c4993729d1c7f28f358
size 10003525