123 lines
2.4 KiB
Markdown
123 lines
2.4 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
tags:
|
||
|
|
- assistant
|
||
|
|
- chatbot
|
||
|
|
- distilgpt2
|
||
|
|
- finetuned
|
||
|
|
- experimental
|
||
|
|
- mimicer
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
library_name: transformers
|
||
|
|
---
|
||
|
|
|
||
|
|
<div align="center">
|
||
|
|
|
||
|
|
# 🎭 Mimicer
|
||
|
|
|
||
|
|
### *The model that learns to mirror.*
|
||
|
|
|
||
|
|
**For fun!** 🚀
|
||
|
|
|
||
|
|
</div>
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 🚀 Overview
|
||
|
|
|
||
|
|
Mimicer is an experimental language model fine-tuned to reproduce text patterns and mirror user inputs.
|
||
|
|
|
||
|
|
Unlike traditional assistants optimized for reasoning or instruction following, Mimicer explores identity mapping and response replication through supervised fine-tuning.
|
||
|
|
|
||
|
|
This project serves as a learning platform for model training, dataset design, Hugging Face deployment, and transformer fine-tuning workflows.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 📊 Model Details
|
||
|
|
|
||
|
|
| Property | Value |
|
||
|
|
| ---------------- | ------------------------- |
|
||
|
|
| Base Model | DistilGPT2 |
|
||
|
|
| Parameters | 81.9M |
|
||
|
|
| Architecture | GPT-2 Decoder |
|
||
|
|
| Fine-Tuning | Supervised |
|
||
|
|
| Training Samples | 2,500 |
|
||
|
|
| Context Length | 40 Tokens |
|
||
|
|
| Framework | Hugging Face Transformers |
|
||
|
|
| Hardware | NVIDIA T4 |
|
||
|
|
| Repository | QuantaSparkLabs/Mimicer |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## ⚙️ Training Objective
|
||
|
|
|
||
|
|
Training samples follow a structured format:
|
||
|
|
|
||
|
|
```text
|
||
|
|
Input: Hello world
|
||
|
|
Output: Hello world
|
||
|
|
```
|
||
|
|
|
||
|
|
The objective is to teach the model to reproduce the provided text after the `Output:` prompt.
|
||
|
|
|
||
|
|
Example:
|
||
|
|
|
||
|
|
```text
|
||
|
|
Input: How are you?
|
||
|
|
Output: How are you?
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 💻 Usage
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
|
|
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
||
|
|
"QuantaSparkLabs/Mimicer"
|
||
|
|
)
|
||
|
|
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(
|
||
|
|
"QuantaSparkLabs/Mimicer"
|
||
|
|
)
|
||
|
|
|
||
|
|
prompt = "Input: hello how are you\nOutput:"
|
||
|
|
|
||
|
|
inputs = tokenizer(prompt, return_tensors="pt")
|
||
|
|
|
||
|
|
outputs = model.generate(
|
||
|
|
**inputs,
|
||
|
|
max_new_tokens=20,
|
||
|
|
do_sample=False
|
||
|
|
)
|
||
|
|
|
||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 🔬 Project Goals
|
||
|
|
|
||
|
|
* Learn transformer fine-tuning
|
||
|
|
* Understand dataset design
|
||
|
|
* Explore identity-mapping behavior
|
||
|
|
* Practice Hugging Face model deployment
|
||
|
|
* Build a foundation for future custom models
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## 📜 License
|
||
|
|
|
||
|
|
Apache 2.0
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
<div align="center">
|
||
|
|
|
||
|
|
### Built by QuantaSparkLabs
|
||
|
|
|
||
|
|
</div>
|