laabam-ai-3b-v1/README.md

license, language, base_model, tags, pipeline_tag

license	language	base_model	tags	pipeline_tag
apache-2.0	en hi te kn ta	Qwen/Qwen2.5-3B-Instruct	laabam-ai qwen2.5 multilingual indic fine-tuned qlora	text-generation

Laabam AI 3B v1

A multilingual AI assistant fine-tuned from Qwen2.5-3B-Instruct using QLoRA.

Training Details

• Base model: Qwen2.5-3B-Instruct (4-bit quantized)
• Method: QLoRA (r=16, alpha=32)
• Training: 4 epochs on ~98K samples (final train loss 0.465)
• Languages: English, Hindi, Telugu, Kannada, Tamil
• Domains: General instruction following, coding, reasoning, safety alignment, Indic languages

Training Epochs

Epoch	Dataset Size	Learning Rate	Focus
1	36K	2e-4	Core instruction following
2	36K	5e-5	Continued refinement
3	98K	2e-5	Expanded: safety, Indic languages, clean instructions
4	98K	1e-5	Careful refinement (low LR, anti-forgetting)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("laabamone/laabam-ai-3b-v1")
tokenizer = AutoTokenizer.from_pretrained("laabamone/laabam-ai-3b-v1")

messages = [{"role": "user", "content": "Hello, who are you?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

License

Apache 2.0