64 lines
2.2 KiB
Markdown
64 lines
2.2 KiB
Markdown
|
---
|
||
|
|
library_name: transformers
|
||
|
|
inference: true
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
# 🐵 MonkeGpt-Vivace 🎶
|
||
|
|
|
||
|
|
## 🎹 Overview
|
||
|
|
|
||
|
|
**MonkeGpt-Vivace** is a high-speed, instruction-tuned language model based on the **Qwen2.5-0.5B** architecture. It has been fine-tuned using the **UltraChat 200k** dataset to transform it from a base autocomplete engine into a snappy, conversational assistant.
|
||
|
|
|
||
|
|
The name **"Vivace"** reflects its performance: lightweight, lively, and incredibly fast. At 0.5 billion parameters, it is optimized for edge deployment and serverless CPU inference.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 🍌 Features
|
||
|
|
|
||
|
|
* **Instruction Following:** Unlike the base model, Vivace understands user/assistant roles.
|
||
|
|
* **Lightweight Brain:** Fits into ~1.1GB of VRAM or System RAM.
|
||
|
|
* **High Tempo:** Designed for low-latency responses (30-50 tokens/sec on modern CPUs).
|
||
|
|
* **Clean Dialogue:** Fine-tuned to avoid the "hallucination loops" common in small base models.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 🎼 Training Details
|
||
|
|
|
||
|
|
* **Base Model:** `Qwen/Qwen2.5-0.5B`
|
||
|
|
* **Dataset:** `HuggingFaceH4/ultrachat_200k` (SFT subset)
|
||
|
|
* **Hardware:** 1x NVIDIA H100 (Lightning AI)
|
||
|
|
* **Epochs:** 2 (Optimized for balance and credit conservation)
|
||
|
|
* **Precision:** `BFloat16`
|
||
|
|
* **Optimizer:** `AdamW (Fused)`
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 🎸 Quick Start (Inference)
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
model_id = "aaravriyer193/MonkeGpt-Vivace"
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
|
||
|
|
|
||
|
|
prompt = "Explain why monkeys like music in three short sentences."
|
||
|
|
messages = [{"role": "user", "content": prompt}]
|
||
|
|
|
||
|
|
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
|
||
|
|
outputs = model.generate(inputs, max_new_tokens=100)
|
||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
|
|
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 🥁 Limitations
|
||
|
|
|
||
|
|
While MonkeGpt-Vivace is smart for its size, it is still a 0.5B parameter model. It may struggle with complex mathematical proofs, deep philosophical reasoning, or very long-term memory across thousands of tokens.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 🐒 About the Creator
|
||
|
|
|
||
|
|
Developed by **aaravriyer193** during a high-intensity 4-hour sprint on Lightning AI. 🚀
|