base_model, language, library_name, tags, pipeline_tag

base_model	language	library_name	tags	pipeline_tag
dungnvt/Tini-8B-A1B	en	transformers	gguf lfm2_moe reasoning agent function-calling	text-generation

Tini-8B-A1B-GGUF

This repository contains GGUF format versions of the fine-tuned model Tini-8B-A1B, which is based on the LiquidAI/LFM2.5-8B-A1B architecture and optimized for agent reasoning and function-calling.

Files Available

• Tini-8B-A1B-BF16.gguf (15.78 GB): Unquantized Brain Float 16 base GGUF file.
• Tini-8B-A1B-Q8_0.gguf (8.39 GB): 8-bit standard quantization. High accuracy, recommended for general inference.
• Tini-8B-A1B-Q6_K.gguf (6.48 GB): 6-bit quantization. Good balance between size and perplexity.
• Tini-8B-A1B-Q4_K_M.gguf (4.80 GB): 4-bit Medium K-quantized model. Highly efficient resource usage.

Running the Model

Since the lfm2_moe architecture is relatively new, make sure to use a recent version of llama.cpp or downstream tools (LM Studio, Ollama, etc.) that support this model type.

Using llama-cli

You can run the model directly using llama-cli:

llama-cli -m Tini-8B-A1B-Q8_0.gguf -p "<|im_start|>user\nHello, how can you help me today?<|im_end|>\n<|im_start|>assistant\n"

Model Architecture Details

• Architecture: Lfm2MoeForCausalLM
• Experts: 32 experts (MoE)
• Experts per Token: 4 active experts
• Context Window: Up to 128k tokens