---
language:
- en

license: apache-2.0

base_model:
- unsloth/Qwen2.5-3B-Instruct-bnb-4bit

pipeline_tag: text-generation

library_name: transformers

tags:
- llm
- qwen2.5
- qlora
- unsloth
- fine-tuned
- chatbot
- instruction-tuned
- text-generation
- anmolgpt
---

# ANMOLGPT-3B-v0.1

> The first public release of the ANMOLGPT family of open-source language models.

## Overview

ANMOLGPT-3B-v0.1 is an instruction-following language model built by fine-tuning **Qwen2.5-3B-Instruct** using **QLoRA** with **Unsloth Studio**.

This release serves as the foundation of the ANMOLGPT project and demonstrates the complete workflow of dataset preparation, fine-tuning, evaluation, and deployment.

Although this is an early preview release, it establishes the baseline for future versions that will include larger datasets, stronger reasoning capabilities, and extensive benchmarking.

---

# Base Model

- **Model:** Qwen2.5-3B-Instruct
- **Framework:** Unsloth
- **Fine-tuning:** QLoRA (4-bit)
- **Export:** Hugging Face Transformers

---

# Training Details

## Dataset

- Databricks Dolly 25K

## Configuration

| Parameter | Value |
|-----------|------:|
| LoRA Rank | 8 |
| LoRA Alpha | 16 |
| LoRA Dropout | 0.05 |
| Context Length | 1024 |
| Learning Rate | 1e-4 |
| Batch Size | 2 |
| Gradient Accumulation | 8 |
| Effective Batch Size | 16 |
| Max Steps | 100 |
| Precision | 4-bit QLoRA |

---
## 📊 Benchmark Results

Evaluated using the EleutherAI LM Evaluation Harness.

| Benchmark | Metric | Score |
|-----------|--------|------:|
| HellaSwag | Accuracy | 54.29% |
| HellaSwag | Normalized Accuracy | **73.21%** |
| PIQA | Accuracy | **77.97%** |
| PIQA | Normalized Accuracy | **78.40%** |
| ARC-Easy | Accuracy | **77.95%** |
| ARC-Easy | Normalized Accuracy | **75.29%** |
| Winogrande | Accuracy | **70.24%** |
| TruthfulQA (MC2) | Accuracy | **42.52%** |
| MMLU | Accuracy | **65.59%** |

### MMLU Subject Breakdown

| Domain | Accuracy |
|---------|---------:|
| Social Sciences | **76.86%** |
| Other | **71.00%** |
| STEM | **61.37%** |
| Humanities | **57.47%** |

> These results represent the baseline performance of ANMOLGPT-3B-v0.1 and will be expanded in future releases.

---

# Example

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "anmoldhandhania93/ANMOLGPT-3B-v0.1"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "Explain reinforcement learning."

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=200
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

# Intended Uses

ANMOLGPT is suitable for:

- Conversational AI
- Learning and experimentation
- Prompt engineering
- Text generation
- Educational projects
- Software development assistance

---

# Limitations

This is an **early proof-of-concept release**.

Current limitations include:

- Fine-tuned for only **100 optimization steps**
- Limited benchmark coverage
- General-purpose capabilities remain close to the base model
- Not intended for production or safety-critical applications
- May generate inaccurate or fabricated information

---

# Roadmap

## v0.2

- Improved instruction tuning
- Larger curated dataset
- More training steps
- Additional benchmarks

## v0.5

- Better reasoning
- Coding improvements
- Domain-specific datasets
- Human evaluation

## v1.0

- Comprehensive benchmark suite
- Optimized inference
- Expanded capabilities
- Stable production release

---

# Citation

```bibtex
@misc{anmolgpt2026,
  title={ANMOLGPT-3B-v0.1},
  author={Anmol Dhandhania},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/anmoldhandhania93/ANMOLGPT-3B-v0.1}
}
```

---

# Acknowledgements

ANMOLGPT was built using:

- Qwen Team
- Unsloth AI
- Hugging Face
- Databricks Dolly Dataset
- EleutherAI LM Evaluation Harness

---

# Future Work

Future versions of ANMOLGPT will focus on:

- Better reasoning
- Improved instruction following
- Coding capabilities
- Domain-specialized models
- Efficient inference
- Comprehensive benchmarking

Feedback, issues, and contributions are welcome.