Zero-Mistral-24B-gguf/README.md

license, datasets, language, tags, inference, pipeline_tag, base_model, library_name, base_model_relation, quantized_by

license	datasets	language	tags	inference	pipeline_tag	base_model	library_name	base_model_relation	quantized_by
mit	ZeroAgency/ru-big-russian-dataset	ru en	mistral chat conversational transformers	parameters temperature 0	text-generation	ZeroAgency/Zero-Mistral-24B	llama.cpp	quantized	bethrezen

Model Card for Zero-Mistral

This is a GGUF version of ZeroAgency/Zero-Mistral-24B.

All quants made with llama.cpp version b5083.

Quants available:

• BF16
• F16
• IQ4_NL
• IQ4_NL_L - same as above but with --leave-output-tensors
• IQ4_XS
• IQ4_XS_L - same as above but with --leave-output-tensors
• Q4_K_M
• Q4_K_M_L - same as above but with --leave-output-tensors
• Q6_K
• Q6_K_L - same as above but with --leave-output-tensors
• Q8_0 - quantized from bf16 gguf
• Q8_0-direct - direct convertation from hf
• Q8_0_L - quantized from bf16 but with --leave-output-tensors