Qwen3-0.6B-DISTILL-glm-4.7-think-GGUF/README.md

base_model, library_name, license, language, tags, pipeline_tag

base_model	library_name	license	language	tags	pipeline_tag
Qwen/Qwen3-0.6B	gguf	apache-2.0	en	qwen3 gguf quantized distillation	text-generation

Qwen3-0.6B-DISTILL-glm-4.7-think-GGUF

GGUF quantized versions of Qwen3-0.6B fine-tuned model.

Available Formats

Filename	Size	Quant Type	Description
Qwen3-0.6B-DISTILL-glm-4.7-think-f16.gguf	1.41 GB	F16	Largest, original FP16
Qwen3-0.6B-DISTILL-glm-4.7-think-q2_k.gguf	0.32 GB	Q2_K	Smallest, significant quality loss
Qwen3-0.6B-DISTILL-glm-4.7-think-q3_k_l.gguf	0.41 GB	Q3_K_L	Small, better quality
Qwen3-0.6B-DISTILL-glm-4.7-think-q3_k_m.gguf	0.39 GB	Q3_K_M	Small, balanced
Qwen3-0.6B-DISTILL-glm-4.7-think-q3_k_s.gguf	0.36 GB	Q3_K_S	Small, moderate quality loss
Qwen3-0.6B-DISTILL-glm-4.7-think-q4_0.gguf	0.44 GB	Q4_0	Medium, legacy format
Qwen3-0.6B-DISTILL-glm-4.7-think-q4_1.gguf	0.47 GB	Q4_1
Qwen3-0.6B-DISTILL-glm-4.7-think-q4_k_m.gguf	0.45 GB	Q4_K_M	Recommended - Best balance
Qwen3-0.6B-DISTILL-glm-4.7-think-q4_k_s.gguf	0.44 GB	Q4_K_S	Medium, good balance
Qwen3-0.6B-DISTILL-glm-4.7-think-q5_0.gguf	0.51 GB	Q5_0	Medium-large, good quality
Qwen3-0.6B-DISTILL-glm-4.7-think-q5_1.gguf	0.54 GB	Q5_1
Qwen3-0.6B-DISTILL-glm-4.7-think-q5_k_m.gguf	0.51 GB	Q5_K_M	Medium-large, high quality
Qwen3-0.6B-DISTILL-glm-4.7-think-q5_k_s.gguf	0.51 GB	Q5_K_S	Medium-large, high quality
Qwen3-0.6B-DISTILL-glm-4.7-think-q6_k.gguf	0.58 GB	Q6_K	Large, very high quality
Qwen3-0.6B-DISTILL-glm-4.7-think-q8_0.gguf	0.75 GB	Q8_0	Large, near lossless

Quick Start

Ollama

ollama run hf.co/glogwa68/Qwen3-0.6B-DISTILL-glm-4.7-think-GGUF:Q4_K_M

llama.cpp

llama-cli --hf-repo glogwa68/Qwen3-0.6B-DISTILL-glm-4.7-think-GGUF --hf-file qwen3-0.6b-distill-glm-4.7-think-q4_k_m.gguf -p "Hello"