---
license: apache-2.0
base_model: Kolyadual/MIXdevAI-yandexGPT5-8B
tags:
- gguf
- llama.cpp
- text-generation
---

# MIXdevAI-yandexGPT5-8B-GGUF

Full set of GGUF quantizations for [Kolyadual/MIXdevAI-yandexGPT5-8B](https://huggingface.co/Kolyadual/MIXdevAI-yandexGPT5-8B).

## Files

| File | Quantization | Type |
|---|---|---|
| `mixdev-ya-f16.gguf` | F16 | Base Model |
| `mixdev-ya-q8_0.gguf` | Q8_0 | Text Model |
| `mixdev-ya-q6_k.gguf` | Q6_K | Text Model |
| `mixdev-ya-q5_k_m.gguf` | Q5_K_M | Text Model |
| `mixdev-ya-q5_k_s.gguf` | Q5_K_S | Text Model |
| `mixdev-ya-q5_1.gguf` | Q5_1 | Text Model |
| `mixdev-ya-q5_0.gguf` | Q5_0 | Text Model |
| `mixdev-ya-q4_k_m.gguf` | Q4_K_M | Text Model |
| `mixdev-ya-q4_k_s.gguf` | Q4_K_S | Text Model |
| `mixdev-ya-q4_1.gguf` | Q4_1 | Text Model |
| `mixdev-ya-q4_0.gguf` | Q4_0 | Text Model |
| `mixdev-ya-q3_k_l.gguf` | Q3_K_L | Text Model |
| `mixdev-ya-q3_k_m.gguf` | Q3_K_M | Text Model |
| `mixdev-ya-q3_k_s.gguf` | Q3_K_S | Text Model |
| `mixdev-ya-q2_k.gguf` | Q2_K | Text Model |

### Quantization notes
- **Q8_0 / F16**: Almost lossless. Best quality, largest size.
- **Q6_K / Q5_K_M**: Excellent balance between quality and size.
- **Q4_K_M**: Golden standard for local inference.
- **Q3_K_M / Q2_K**: Noticeable quality degradation. Use only if you have severe RAM/VRAM constraints.

## How to run locally

Example using `llama.cpp` server:

```bash
llama-server \
  -m mixdev-ya-q4_k_m.gguf \
  -ngl 999 \
  --host 0.0.0.0 --port 8080 \
  -c 32768