---
license: apache-2.0
language:
- en
base_model:
- Nodmix
pipeline_tag: text-generation
library_name: transformers
tags:
- Nodmix
- agent
- code
---

# **Nodmix-Q4**

>  Nodmix is the latest generation of large language models in Nodmix IQ series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Nodmix delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support

## Model Files

| File Name              | Size    | Quantization | Format | Description                      |
| ---------------------- | ------- | ------------ | ------ | -------------------------------- |
| `Nodmix_Q4.F32.gguf`    | 16.1 GB | FP32         | GGUF   | Full precision (float32) version |
| `Nodmix_4B.BF16.gguf`   | 8.05 GB | BF16         | GGUF   | BFloat16 precision version       |
| `Nodmix_4B.F16.gguf`    | 8.05 GB | FP16         | GGUF   | Float16 precision version        |
| `Nodmix_4B.Q3_K_M.gguf` | 2.08 GB | Q3\_K\_M     | GGUF   | 3-bit quantized (K M variant)    |
| `Nodmix_4B.Q3_K_S.gguf` | 1.89 GB | Q3\_K\_S     | GGUF   | 3-bit quantized (K S variant)    |
| `Nodmix_4B.Q4_K_M.gguf` | 2.5 GB  | Q4\_K\_M     | GGUF   | 4-bit quantized (K M variant)    |
| `Nodmix_4B.Q4_K_S.gguf` | 2.38 GB | Q4\_K\_S     | GGUF   | 4-bit quantized (K S variant)    |
| `Nodmix_4B.Q5_K_M.gguf` | 2.89 GB | Q5\_K\_M     | GGUF   | 5-bit quantized (K M variant)    |
| `Nodmix_4B.Q8_0.gguf`   | 4.28 GB | Q8\_0        | GGUF   | 8-bit quantized                  |
| `.gitattributes`       | 2.02 kB | —            | —      | Git LFS tracking file            |
| `config.json`          | 31 B    | —            | —      | Configuration placeholder        |
| `README.md`            | 3.6 kB  | —            | —      | Model documentation              |

## Quants Usage 

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

|
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):