FastThink-0.5B-Tiny-GGUF/README.md

license, language, base_model, pipeline_tag, library_name, tags

license	language	base_model	pipeline_tag	library_name	tags
apache-2.0	en	prithivMLmods/FastThink-0.5B-Tiny	text-generation	transformers	text-generation-inference

FastThink-0.5B-Tiny-GGUF

FastThink-0.5B-Tiny is a reasoning-focused model based on Qwen2.5. They have released a range of base language models and instruction-tuned language models, spanning from 0.5 billion to 72 billion parameters.

Model Files

File	Size	Description
README.md	31 Bytes	Project documentation
config.json	31 Bytes	Model configuration
.gitattributes	2.39 kB	Git attributes configuration
FastThink-0.5B-Tiny.BF16.gguf	994 MB	BFloat16 quantized model
FastThink-0.5B-Tiny.F16.gguf	994 MB	Float16 quantized model
FastThink-0.5B-Tiny.F32.gguf	1.98 GB	Float32 full precision model
FastThink-0.5B-Tiny.Q2_K.gguf	339 MB	2-bit K-quantized model
FastThink-0.5B-Tiny.Q3_K_L.gguf	369 MB	3-bit K-quantized model (Large)
FastThink-0.5B-Tiny.Q3_K_M.gguf	355 MB	3-bit K-quantized model (Medium)
FastThink-0.5B-Tiny.Q3_K_S.gguf	338 MB	3-bit K-quantized model (Small)
FastThink-0.5B-Tiny.Q4_K_M.gguf	398 MB	4-bit K-quantized model (Medium)
FastThink-0.5B-Tiny.Q4_K_S.gguf	385 MB	4-bit K-quantized model (Small)
FastThink-0.5B-Tiny.Q5_K_M.gguf	420 MB	5-bit K-quantized model (Medium)
FastThink-0.5B-Tiny.Q5_K_S.gguf	413 MB	5-bit K-quantized model (Small)
FastThink-0.5B-Tiny.Q6_K.gguf	506 MB	6-bit K-quantized model
FastThink-0.5B-Tiny.Q8_0.gguf	531 MB	8-bit quantized model

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):