---
license: apache-2.0
language:
  - hi
  - en
  - hne
tags:
  - hinglish
  - hindi
  - indian-languages
  - fused
  - pytorch
  - fp16
  - qwen
  - qwen2
  - bharat
  - indic-nlp
base_model: Qwen/Qwen2.5-1.5B
library_name: transformers
pipeline_tag: text-generation
---

# Bharat-Tiny-LLM (fused · fp16)

This is the **full-precision fused model** for [Bharat-Tiny-LLM](https://huggingface.co/eulogik/Bharat-Tiny-LLM) —
the LoRA adapter merged into the base [Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B) weights,
in PyTorch `float16`.

Use this repo when you want to:
- run inference on CPU / CUDA with `transformers`,
- fine-tune further, or
- produce your own quantized builds (GGUF, MLX, etc.).

> Built by [eulogik](https://eulogik.com)

## For most users

You probably want a smaller, ready-to-run build instead:

| Build | Repo | Size | Use |
|-------|------|------|-----|
| **MLX 4-bit** (edge / Apple Silicon) | [`eulogik/Bharat-Tiny-LLM`](https://huggingface.co/eulogik/Bharat-Tiny-LLM) | ~880 MB | Recommended for Mac / on-device |
| **GGUF Q4_K_M** (llama.cpp, Android / Pi / CPU) | [`eulogik/Bharat-Tiny-LLM-GGUF`](https://huggingface.co/eulogik/Bharat-Tiny-LLM-GGUF) | ~1.06 GB | Cross-platform, llama.cpp |
| **PyTorch fp16** (this repo) | `eulogik/Bharat-Tiny-LLM-fused` | ~3.3 GB | Server / fine-tuning base |

## Quick start (transformers)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")
tokenizer = AutoTokenizer.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")

messages = [{"role": "user", "content": "Chai peete hain?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.3,
    top_p=0.85,
    repetition_penalty=1.25,
    no_repeat_ngram_size=3,
    do_sample=True,
)
print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

> ⚠️ **Generation config matters.** The base Qwen2.5-1.5B emits garbled out-of-script tokens
> at high temperature. Always use `temperature ≈ 0.3` + `repetition_penalty ≥ 1.25` +
> `no_repeat_ngram_size = 3`. The [`bharat-tiny-llm`](https://pypi.org/project/bharat-tiny-llm/)
> PyPI package applies these for you.

## Links

- 🤗 Edge model (MLX): https://huggingface.co/eulogik/Bharat-Tiny-LLM
- 🤗 GGUF (llama.cpp): https://huggingface.co/eulogik/Bharat-Tiny-LLM-GGUF
- 🚀 Demo: https://huggingface.co/spaces/eulogik/Bharat-Tiny-LLM
- 💻 Source: https://github.com/eulogik/Bharat-Tiny-LLM
- 📦 PyPI: https://pypi.org/project/bharat-tiny-llm/
- 🏢 Built by [eulogik](https://eulogik.com)

## License

Apache-2.0 (base Qwen2.5-1.5B weights Apache-2.0; LoRA adapter Apache-2.0).