---
base_model: Qwen/Qwen3-0.6B
library_name: gguf
license: apache-2.0
language:
- en
tags:
- qwen3
- gguf
- quantized
- distillation
pipeline_tag: text-generation
---

# Qwen3-0.6B-DISTILL-glm-4.7-think-GGUF

GGUF quantized versions of Qwen3-0.6B fine-tuned model.

## Available Formats

| Filename | Size | Quant Type | Description |
|----------|------|------------|-------------|
| Qwen3-0.6B-DISTILL-glm-4.7-think-f16.gguf | 1.41 GB | F16 | Largest, original FP16 |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q2_k.gguf | 0.32 GB | Q2_K | Smallest, significant quality loss |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q3_k_l.gguf | 0.41 GB | Q3_K_L | Small, better quality |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q3_k_m.gguf | 0.39 GB | Q3_K_M | Small, balanced |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q3_k_s.gguf | 0.36 GB | Q3_K_S | Small, moderate quality loss |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q4_0.gguf | 0.44 GB | Q4_0 | Medium, legacy format |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q4_1.gguf | 0.47 GB | Q4_1 |  |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q4_k_m.gguf | 0.45 GB | Q4_K_M | **Recommended** - Best balance |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q4_k_s.gguf | 0.44 GB | Q4_K_S | Medium, good balance |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q5_0.gguf | 0.51 GB | Q5_0 | Medium-large, good quality |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q5_1.gguf | 0.54 GB | Q5_1 |  |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q5_k_m.gguf | 0.51 GB | Q5_K_M | Medium-large, high quality |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q5_k_s.gguf | 0.51 GB | Q5_K_S | Medium-large, high quality |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q6_k.gguf | 0.58 GB | Q6_K | Large, very high quality |
| Qwen3-0.6B-DISTILL-glm-4.7-think-q8_0.gguf | 0.75 GB | Q8_0 | Large, near lossless |


## Quick Start

### Ollama
```bash
ollama run hf.co/glogwa68/Qwen3-0.6B-DISTILL-glm-4.7-think-GGUF:Q4_K_M
```

### llama.cpp
```bash
llama-cli --hf-repo glogwa68/Qwen3-0.6B-DISTILL-glm-4.7-think-GGUF --hf-file qwen3-0.6b-distill-glm-4.7-think-q4_k_m.gguf -p "Hello"
```