2.1 KiB
license, base_model, language, pipeline_tag, library_name, tags
| license | base_model | language | pipeline_tag | library_name | tags | ||||
|---|---|---|---|---|---|---|---|---|---|
| apache-2.0 |
|
|
text-generation | transformers |
|
Hermes-3-Llama-3.2-3B-f32-GGUF
Hermes 3 - Llama-3.2 3B by Nous Research is a compact yet powerful generalist language model fine-tuned from the Llama-3.2 3B foundation, designed to deliver advanced agentic capabilities, improved roleplaying, complex reasoning, and coherent multi-turn conversations with long context understanding. It offers powerful user steering and control, supports structured prompting with ChatML format, and enables function calling and JSON output for dynamic interaction and automation.
Trained on high-performance GPU infrastructure, Hermes 3 excels in problem-solving, dialogue, and code generation tasks, demonstrating competitive benchmark scores across GPT4All, AGIEval, and BigBench. This model is optimized for efficiency, supports 4-bit quantization for deployment, and is ideal for building sophisticated AI assistants with controllable and natural user interactions.
Model Files
| Model File name | Size | QuantType |
|---|---|---|
| Hermes-3-Llama-3.2-3B.BF16.gguf | 6.43 GB | BF16 |
| Hermes-3-Llama-3.2-3B.F16.gguf | 6.43 GB | F16 |
| Hermes-3-Llama-3.2-3B.F32.gguf | 12.9 GB | F32 |
| Hermes-3-Llama-3.2-3B.Q2_K.gguf | 1.36 GB | Q2_K |
| Hermes-3-Llama-3.2-3B.Q3_K_L.gguf | 1.82 GB | Q3_K_L |
| Hermes-3-Llama-3.2-3B.Q3_K_M.gguf | 1.69 GB | Q3_K_M |
| Hermes-3-Llama-3.2-3B.Q3_K_S.gguf | 1.54 GB | Q3_K_S |
| Hermes-3-Llama-3.2-3B.Q4_K_M.gguf | 2.02 GB | Q4_K_M |
| Hermes-3-Llama-3.2-3B.Q4_K_S.gguf | 1.93 GB | Q4_K_S |
| Hermes-3-Llama-3.2-3B.Q5_K_M.gguf | 2.32 GB | Q5_K_M |
| Hermes-3-Llama-3.2-3B.Q5_K_S.gguf | 2.27 GB | Q5_K_S |
| Hermes-3-Llama-3.2-3B.Q6_K.gguf | 2.64 GB | Q6_K |
| Hermes-3-Llama-3.2-3B.Q8_0.gguf | 3.42 GB | Q8_0 |
Quants Usage
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):
