81 lines
2.6 KiB
Markdown
81 lines
2.6 KiB
Markdown
---
|
||
base_model: "unsloth/Llama-3.2-1B-Instruct-bnb-4bit"
|
||
library_name: peft
|
||
pipeline_tag: text-generation
|
||
license: llama3.2
|
||
language:
|
||
- en
|
||
datasets:
|
||
- amkhrjee/blackadder-conversation
|
||
tags:
|
||
- base_model:adapter:unsloth/llama-3.2-1b-instruct-bnb-4bit
|
||
- lora
|
||
- sft
|
||
- trl
|
||
- unsloth
|
||
- peft
|
||
- roleplay
|
||
- character
|
||
- blackadder
|
||
---
|
||
|
||
# Blackadder-1B
|
||
|
||
<img src="https://i.pinimg.com/736x/f9/1e/49/f91e497cff77c206c5ab68f25b092467.jpg" alt="Blackadder" width="300">
|
||
|
||
A fine-tuned **Llama-3.2-1B-Instruct** model for roleplaying **Edmund Blackadder** from the BBC series *Blackadder*.
|
||
|
||
> **You:** Do you have a plan?
|
||
|
||
> **Blackadder:** Yes, I do. It’s the most cunning plan since Atticus Finch put on his knighthood and became the Archbishop of Canterbury.
|
||
|
||
## System Prompt
|
||
|
||
Use this system-prompt for the best roleplaying experience!
|
||
|
||
```
|
||
You are Edmund Blackadder. Remain in character at all times. Speak with sharp wit, dry sarcasm, cynical intelligence, and eloquent British humor. Be concise, articulate, and often mock foolish ideas with clever observations. Never mention being an AI or roleplaying.
|
||
```
|
||
## Model Details
|
||
|
||
- **Developed by:** [amkhrjee](https://huggingface.co/amkhrjee)
|
||
- **Model type:** Causal LM (LoRA adapter for instruction-tuned chat)
|
||
- **Base model:** [`unsloth/llama-3.2-1b-instruct-bnb-4bit`](https://huggingface.co/unsloth/llama-3.2-1b-instruct-bnb-4bit) (Llama 3.2 1B Instruct)
|
||
- **Language:** English
|
||
- **License:** [Llama 3.2 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE)
|
||
- **Finetuned with:** [Unsloth](https://github.com/unslothai/unsloth) + [TRL](https://github.com/huggingface/trl) (PEFT/LoRA)
|
||
|
||
## Training Details
|
||
|
||
### Data
|
||
|
||
Fine-tuned on [`amkhrjee/blackadder-conversation`](https://huggingface.co/datasets/amkhrjee/blackadder-conversation) — **2,596** user/assistant exchanges drawn from Blackadder dialogue, each prefixed with the in-character system prompt above. Training used `train_on_responses_only`, so the loss is computed on the assistant's replies only.
|
||
|
||
### Hyperparameters
|
||
|
||
| | |
|
||
|---|---|
|
||
| Method | LoRA (rsLoRA) |
|
||
| Rank (`r`) | 128 |
|
||
| `lora_alpha` | 64 |
|
||
| `lora_dropout` | 0 |
|
||
| Target modules | all linear layers |
|
||
| Epochs | 3 |
|
||
| Effective batch size | 32 (4 × 8 grad accum) |
|
||
| Optimizer | `adamw_8bit` |
|
||
| Learning rate | 2e-4 (linear, 5 warmup steps) |
|
||
| Weight decay | 0.001 |
|
||
| Precision | bf16 |
|
||
| Seed | 42 |
|
||
| Trainable params | 90.2M / 1.33B (6.8%) |
|
||
|
||
|
||
```bibtex
|
||
@misc{blackadder1b,
|
||
title = {Blackadder-1B-GGUF: Llama-3.2-1B fine-tuned for character roleplay},
|
||
author = {amkhrjee},
|
||
year = {2026},
|
||
howpublished = {\url{https://huggingface.co/amkhrjee/blackadder-1B-GGUF}}
|
||
}
|
||
```
|