license, base_model, tags

license	base_model	tags
apache-2.0	Kolyadual/MIXdevAI-yandexGPT5-8B	gguf llama.cpp text-generation

MIXdevAI-yandexGPT5-8B-GGUF

Full set of GGUF quantizations for Kolyadual/MIXdevAI-yandexGPT5-8B.

Files

File	Quantization	Type
mixdev-ya-f16.gguf	F16	Base Model
mixdev-ya-q8_0.gguf	Q8_0	Text Model
mixdev-ya-q6_k.gguf	Q6_K	Text Model
mixdev-ya-q5_k_m.gguf	Q5_K_M	Text Model
mixdev-ya-q5_k_s.gguf	Q5_K_S	Text Model
mixdev-ya-q5_1.gguf	Q5_1	Text Model
mixdev-ya-q5_0.gguf	Q5_0	Text Model
mixdev-ya-q4_k_m.gguf	Q4_K_M	Text Model
mixdev-ya-q4_k_s.gguf	Q4_K_S	Text Model
mixdev-ya-q4_1.gguf	Q4_1	Text Model
mixdev-ya-q4_0.gguf	Q4_0	Text Model
mixdev-ya-q3_k_l.gguf	Q3_K_L	Text Model
mixdev-ya-q3_k_m.gguf	Q3_K_M	Text Model
mixdev-ya-q3_k_s.gguf	Q3_K_S	Text Model
mixdev-ya-q2_k.gguf	Q2_K	Text Model

Quantization notes

• Q8_0 / F16: Almost lossless. Best quality, largest size.
• Q6_K / Q5_K_M: Excellent balance between quality and size.
• Q4_K_M: Golden standard for local inference.
• Q3_K_M / Q2_K: Noticeable quality degradation. Use only if you have severe RAM/VRAM constraints.

How to run locally

Example using llama.cpp server:

llama-server \
  -m mixdev-ya-q4_k_m.gguf \
  -ngl 999 \
  --host 0.0.0.0 --port 8080 \
  -c 32768