---
license: apache-2.0
language:
  - en
  - hi
  - te
  - kn
  - ta
base_model: Qwen/Qwen2.5-3B-Instruct
tags:
  - laabam-ai
  - qwen2.5
  - multilingual
  - indic
  - fine-tuned
  - qlora
pipeline_tag: text-generation
---

# Laabam AI 3B v1

A multilingual AI assistant fine-tuned from Qwen2.5-3B-Instruct using QLoRA.

## Training Details

- **Base model**: Qwen2.5-3B-Instruct (4-bit quantized)
- **Method**: QLoRA (r=16, alpha=32)
- **Training**: 4 epochs on ~98K samples (final train loss 0.465)
- **Languages**: English, Hindi, Telugu, Kannada, Tamil
- **Domains**: General instruction following, coding, reasoning, safety alignment, Indic languages

## Training Epochs

| Epoch | Dataset Size | Learning Rate | Focus |
|-------|-------------|---------------|-------|
| 1 | 36K | 2e-4 | Core instruction following |
| 2 | 36K | 5e-5 | Continued refinement |
| 3 | 98K | 2e-5 | Expanded: safety, Indic languages, clean instructions |
| 4 | 98K | 1e-5 | Careful refinement (low LR, anti-forgetting) |

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("laabamone/laabam-ai-3b-v1")
tokenizer = AutoTokenizer.from_pretrained("laabamone/laabam-ai-3b-v1")

messages = [{"role": "user", "content": "Hello, who are you?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## License

Apache 2.0