Ferret-3B-GGUF/README.md

base_model, datasets, inference, model_creator, model_name, pipeline_tag, quantized_by, tags

base_model	datasets	inference	model_creator	model_name	pipeline_tag	quantized_by	tags
euclaise/Ferret-3B	euclaise/MiniCoT euclaise/SciCoT euclaise/symtune_mini euclaise/mathoverflow-accepted euirim/goodwiki	false	euclaise	Ferret-3B	text-generation	afrideva	gguf ggml quantized q2_k q3_k_m q4_k_m q5_k_m q6_k q8_0

euclaise/Ferret-3B-GGUF

Quantized GGUF model files for Ferret-3B from euclaise

Name	Quant method	Size
ferret-3b.fp16.gguf	fp16	5.59 GB
ferret-3b.q2_k.gguf	q2_k	1.20 GB
ferret-3b.q3_k_m.gguf	q3_k_m	1.39 GB
ferret-3b.q4_k_m.gguf	q4_k_m	1.71 GB
ferret-3b.q5_k_m.gguf	q5_k_m	1.99 GB
ferret-3b.q6_k.gguf	q6_k	2.30 GB
ferret-3b.q8_0.gguf	q8_0	2.97 GB

Original Model Card: