51 lines
2.0 KiB
Markdown
51 lines
2.0 KiB
Markdown
---
|
|
base_model: Qwen/Qwen3-0.6B
|
|
library_name: gguf
|
|
license: apache-2.0
|
|
language:
|
|
- en
|
|
tags:
|
|
- qwen3
|
|
- gguf
|
|
- quantized
|
|
- distillation
|
|
pipeline_tag: text-generation
|
|
---
|
|
|
|
# Qwen3-0.6B-DISTILL-glm-4.7-think-GGUF
|
|
|
|
GGUF quantized versions of Qwen3-0.6B fine-tuned model.
|
|
|
|
## Available Formats
|
|
|
|
| Filename | Size | Quant Type | Description |
|
|
|----------|------|------------|-------------|
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-f16.gguf | 1.41 GB | F16 | Largest, original FP16 |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q2_k.gguf | 0.32 GB | Q2_K | Smallest, significant quality loss |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q3_k_l.gguf | 0.41 GB | Q3_K_L | Small, better quality |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q3_k_m.gguf | 0.39 GB | Q3_K_M | Small, balanced |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q3_k_s.gguf | 0.36 GB | Q3_K_S | Small, moderate quality loss |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q4_0.gguf | 0.44 GB | Q4_0 | Medium, legacy format |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q4_1.gguf | 0.47 GB | Q4_1 | |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q4_k_m.gguf | 0.45 GB | Q4_K_M | **Recommended** - Best balance |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q4_k_s.gguf | 0.44 GB | Q4_K_S | Medium, good balance |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q5_0.gguf | 0.51 GB | Q5_0 | Medium-large, good quality |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q5_1.gguf | 0.54 GB | Q5_1 | |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q5_k_m.gguf | 0.51 GB | Q5_K_M | Medium-large, high quality |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q5_k_s.gguf | 0.51 GB | Q5_K_S | Medium-large, high quality |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q6_k.gguf | 0.58 GB | Q6_K | Large, very high quality |
|
|
| Qwen3-0.6B-DISTILL-glm-4.7-think-q8_0.gguf | 0.75 GB | Q8_0 | Large, near lossless |
|
|
|
|
|
|
## Quick Start
|
|
|
|
### Ollama
|
|
```bash
|
|
ollama run hf.co/glogwa68/Qwen3-0.6B-DISTILL-glm-4.7-think-GGUF:Q4_K_M
|
|
```
|
|
|
|
### llama.cpp
|
|
```bash
|
|
llama-cli --hf-repo glogwa68/Qwen3-0.6B-DISTILL-glm-4.7-think-GGUF --hf-file qwen3-0.6b-distill-glm-4.7-think-q4_k_m.gguf -p "Hello"
|
|
```
|