ANMOLGPT-3B-v0.1/README.md

language, license, base_model, pipeline_tag, library_name, tags

language	license	base_model	pipeline_tag	library_name	tags
en	apache-2.0	unsloth/Qwen2.5-3B-Instruct-bnb-4bit	text-generation	transformers	llm qwen2.5 qlora unsloth fine-tuned chatbot instruction-tuned text-generation anmolgpt

ANMOLGPT-3B-v0.1

The first public release of the ANMOLGPT family of open-source language models.

Overview

ANMOLGPT-3B-v0.1 is an instruction-following language model built by fine-tuning Qwen2.5-3B-Instruct using QLoRA with Unsloth Studio.

This release serves as the foundation of the ANMOLGPT project and demonstrates the complete workflow of dataset preparation, fine-tuning, evaluation, and deployment.

Although this is an early preview release, it establishes the baseline for future versions that will include larger datasets, stronger reasoning capabilities, and extensive benchmarking.

---

Base Model

• Model: Qwen2.5-3B-Instruct
• Framework: Unsloth
• Fine-tuning: QLoRA (4-bit)
• Export: Hugging Face Transformers

---

Training Details

Dataset

• Databricks Dolly 25K

Configuration

Parameter	Value
LoRA Rank	8
LoRA Alpha	16
LoRA Dropout	0.05
Context Length	1024
Learning Rate	1e-4
Batch Size	2
Gradient Accumulation	8
Effective Batch Size	16
Max Steps	100
Precision	4-bit QLoRA

---

📊 Benchmark Results

Evaluated using the EleutherAI LM Evaluation Harness.

Benchmark	Metric	Score
HellaSwag	Accuracy	54.29%
HellaSwag	Normalized Accuracy	73.21%
PIQA	Accuracy	77.97%
PIQA	Normalized Accuracy	78.40%
ARC-Easy	Accuracy	77.95%
ARC-Easy	Normalized Accuracy	75.29%
Winogrande	Accuracy	70.24%
TruthfulQA (MC2)	Accuracy	42.52%
MMLU	Accuracy	65.59%

MMLU Subject Breakdown

Domain	Accuracy
Social Sciences	76.86%
Other	71.00%
STEM	61.37%
Humanities	57.47%

These results represent the baseline performance of ANMOLGPT-3B-v0.1 and will be expanded in future releases.

---

Example

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "anmoldhandhania93/ANMOLGPT-3B-v0.1"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "Explain reinforcement learning."

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=200
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Intended Uses

ANMOLGPT is suitable for:

• Conversational AI
• Learning and experimentation
• Prompt engineering
• Text generation
• Educational projects
• Software development assistance

---

Limitations

This is an early proof-of-concept release.

Current limitations include:

• Fine-tuned for only 100 optimization steps
• Limited benchmark coverage
• General-purpose capabilities remain close to the base model
• Not intended for production or safety-critical applications
• May generate inaccurate or fabricated information

---

Roadmap

v0.2

• Improved instruction tuning
• Larger curated dataset
• More training steps
• Additional benchmarks

v0.5

• Better reasoning
• Coding improvements
• Domain-specific datasets
• Human evaluation

v1.0

• Comprehensive benchmark suite
• Optimized inference
• Expanded capabilities
• Stable production release

---

Citation

@misc{anmolgpt2026,
  title={ANMOLGPT-3B-v0.1},
  author={Anmol Dhandhania},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/anmoldhandhania93/ANMOLGPT-3B-v0.1}
}

---

Acknowledgements

ANMOLGPT was built using:

• Qwen Team
• Unsloth AI
• Hugging Face
• Databricks Dolly Dataset
• EleutherAI LM Evaluation Harness

---

Future Work

Future versions of ANMOLGPT will focus on:

• Better reasoning
• Improved instruction following
• Coding capabilities
• Domain-specialized models
• Efficient inference
• Comprehensive benchmarking

Feedback, issues, and contributions are welcome.