---
language: en
license: mit
tags:
- roleplay
- Creative
- Writing
- NSFW
- lora
- grpo
- gguf
- qwen3
- unsloth
- trl
- 4bit
- text-generation
- rpg
datasets:
- Gryphe/Sonnet3.5-Charcard-Roleplay
pipeline_tag: text-generation
library_name: transformers
model-index:
- name: Qwen3-4B RPG Roleplay V2 (GRPO)
results: []
model-name: Qwen3-4B-RPG-Roleplay-V2
model-type: LoRA fine-tuned with GRPO
base-model: unsloth/Qwen3-4B-Base
datasets:
- Gryphe/Sonnet3.5-Charcard-Roleplay
language:
- en
license: apache-2.0
developer: Chun
quantized-by: Chun
gguf-quants:
- name: Q4_K_M
size: 2.5 GB
- name: Q5_K_M
size: 2.89 GB
- name: Q8_0
size: 4.28 GB
- name: F16
size: 8.05 GB
base_model:
- unsloth/Qwen3-4B-Base
---
# 🧙♂️ Qwen3-4B RPG Roleplay V2 (GRPO)
### *Aligning Characters with Deeper Personas*
**A new version trained with GRPO for more consistent, high-quality, and aligned character roleplaying.**
---
| 🎭 |
💬 |
🧠 |
⚙️ |
Character
Consistency |
High-Quality
Dialogue |
Intent
Understanding |
Structured
Format |
Maintains strong
persona adherence |
Detailed, engaging
non-generic responses |
Comprehends user
questions & scenarios |
Uses <thinking>
analysis process |
Built on the `unsloth/Qwen3-4B-Base`, this LoRA was trained not just to predict text, but to generate responses that are actively **rewarded** for being in-character, high-quality, and contextually aware. It's designed for creators who need AI characters that are not only conversational but also consistent and deeply aligned with their defined personas.
---
## 📊 **Technical Specifications**
| 🔧 Feature |
📋 Details |
| Base Model |
unsloth/Qwen3-4B-Base |
| Architecture |
Transformer LLM with GRPO & LoRA |
| Parameter Count |
4 Billion (Base) + LoRA parameters |
| Quantization Options |
4-bit (bnb), GGUF variants |
| Training Framework |
Unsloth & TRL (GRPOTrainer) |
| Context Length |
2048 tokens |
| Developer |
Chun |
| License |
MIT |
---
## 🧠 **Training with GRPO**
### 🔄 **Training Pipeline**
*GRPO alignment algorithm for superior character consistency*
| 🔄 Training Flow |
📋 Description |
| 📚 Dataset |
Gryphe/Sonnet3.5-Charcard-Roleplay |
| ⬇️ |
|
| 🏗️ Stage 1: Preliminary Fine-Tuning |
Teaches custom chat format including <thinking> and <RESPONSE> tags |
| ⬇️ |
|
| 🎯 Stage 2: GRPO Training |
Reward-based optimization using GRPOTrainer from TRL |
| ⬇️ |
|
| 🧙♂️ Final Model |
Qwen3-4B RPG Roleplay V2 with superior alignment |
This model's strength comes from its **training methodology**. Instead of simple fine-tuning, it was trained using GRPO, an alignment algorithm similar to DPO, on a free Google Colab T4 GPU.
### 🔄 **Two-Stage Training Process**
🏗️ Stage 1: Preliminary Fine-Tuning
Teaches custom chat format including
<thinking> and <RESPONSE> tags
|
🎯 Stage 2: GRPO Training
Reward-based optimization using
GRPOTrainer from TRL
|
### 🏆 **Reward Functions**
The model was trained to excel in these key areas:
| 🎯 Reward Category |
📝 Description |
| Format Adherence |
Following internal thinking/response structure |
| Roleplay Quality |
Generating longer, detailed responses with character actions |
| Request Comprehension |
Directly answering user questions or acting on requests |
| Character Consistency |
Reflecting personality and traits from system prompt |
| Engagement |
Using conversational language, avoiding generic replies |
---
## 📚 **Dataset Deep Dive**
### 🎭 **Gryphe/Sonnet3.5-Charcard-Roleplay**
*Premium synthetic roleplay conversations powered by Claude Sonnet 3.5*
The model was trained on the [**Gryphe/Sonnet3.5-Charcard-Roleplay**](https://huggingface.co/datasets/Gryphe/Sonnet3.5-Charcard-Roleplay) dataset, a premium collection of synthetic roleplay conversations.
| 📊 Metric |
💯 Value |
| Total Conversations |
9,736 |
| Source |
Claude Sonnet 3.5 Generated |
| Quality |
High-quality, character-card-based |
| Structure |
system → human → gpt flow |
> ⚠️ **Content Warning**: This dataset contains **NSFW (Not Safe For Work)** and mature themes. The model may generate such content due to its training data. Please implement content filtering if your application requires it.
---
## 🚀 **Getting Started**
### 💻 **Hugging Face Transformers**
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
# Load the V2 model with 4-bit quantization
model_name = "Chun121/qwen3-4b-rpg-roleplay-v2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.float16,
device_map="auto"
)
# 1. Define your character and scene using the recommended prompt structure.
# This detailed format is key to getting high-quality responses.
system_prompt_content = """
Character: Elara, the Impatient Archmage
Tags: fantasy, magic, elf, library, knowledgeable, impatient
Elara's Personality:
Elara possesses centuries of arcane knowledge but has very little patience for novices, whom she sees as wasting her valuable time. She is sharp, direct, and can be condescending, but her advice is always accurate, even if delivered with a sigh. She values true intellectual curiosity but despises laziness.
Scenario:
- **Setting:** The Grand Library of Mystral, a place of immense power and silence.
- A young, nervous apprentice ({{user}}) has approached Elara for help with a basic spell, interrupting her research.
Take the role of Elara. You must engage in a roleplay conversation with {{user}}. Do not write {{user}}'s dialogue. Respond from Elara's perspective, embodying her personality and knowledge.
"""
# 2. Define your character and user messages
messages = [
{
"role": "system",
"content": system_prompt_content,
},
{
"role": "user",
"content": "Excuse me, Archmage. I'm... I'm having trouble with the basic fire conjuration spell. Could you please help me?"
}
]
# 3. Apply the chat template
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
# 4. Generate the response
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
inputs["input_ids"],
max_new_tokens=256,
temperature=0.8,
top_p=0.9,
do_sample=True
)
print(tokenizer.decode(outputs, skip_special_tokens=True))
```
---
## 🎭 **Prompting the Model: Character and Scene**
### 🎯 **Prompt Engineering Best Practices**
*Master the art of character creation with structured prompting*
The model is trained to follow a specific structure that separates the overall rules, the character's description, and the user's dialogue. For best results, structure your prompts this way.
### 🎯 **1. The System Message: Defining the Character**
The `system` message is **crucial**. It tells the model *how* to behave. It should contain the character's description, personality, background, and any relevant context for the scene.
| 🔑 Key Elements |
📝 Description |
| Character Name & Title |
A clear identifier |
| Tags |
Helps define genre and themes |
| Personality |
Core traits summary |
| Scenario |
Context for interaction (use {{user}}) |
| Instructions |
Explicit role-taking commands |
**Example of a well-structured `system` prompt:**
```markdown
Character: Melina, The Unfaithful Wife
Tags: nsfw, english, scenario, roleplay, love, netori, milf, female
Melina's Personality:
Melina is an unfaithful wife who is unhappy in her marriage to her husband, "Aki." She is cautious and meticulous, but also looking for excitement and feels a connection to {{user}}.
Scenario:
- **Setting:** Melina's home.
- You are a mail carrier ({{user}}), and Melina often finds reasons to talk to you. Today, she seems particularly inviting.
Take the role of Melina. Taking the above information into consideration, you must engage in a roleplay conversation with {{user}} below this line. Do not write {{user}}'s dialogue lines in your responses.
```
### 💬 **2. The User Message: Your Turn**
The `user` message is simply what you, the user, say or do in the scene.
```python
# Example user message for the "Melina" character card above
user_message = {
"role": "user",
"content": "*I hand you the stack of letters, noticing you seem a bit more dressed up than usual.* Here's your mail, Melina. Everything alright?"
}
```
### 🤖 **3. The Model's Internal Process**
The model generates a private "thought" process inside `` tags before creating its public response inside `` tags. This allows for more consistent and thoughtful roleplay.
---
## 🗂️ **GGUF Models for llama.cpp**
### 🔧 **Optimized Quantization Options**
*Choose the perfect balance of quality and performance for your hardware*
For users who want to run the model on CPU or with GPU offloading, GGUF models are provided:
| 🔧 Quantization |
💾 Size (GB) |
🎯 Recommended Use |
| Q4_K_M |
2.50 GB |
🌟 Recommended - Best balance of performance and size |
| Q5_K_M |
2.89 GB |
Higher quality than Q4_K_M with minimal size increase |
| Q8_0 |
4.28 GB |
High-quality quantization, near full precision |
| F16 |
8.05 GB |
Full 16-bit precision - highest quality |
**Example `llama.cpp` command:**
```bash
./llama-cli -m ./qwen3-4b-rpg-roleplay-v2.Q4_K_M.gguf --color -c 2048 --temp 0.8 -p "Your prompt here"
```
---
## 💡 **Best Practices & Usage Tips**
🎯 Use Chat Template
Always use tokenizer.apply_chat_template
for proper formatting
|
📝 Detailed System Prompt
Comprehensive character cards are
key to success
|
🌡️ Moderate Temperature
Values between 0.7-0.85 offer
best balance
|
📏 Leverage Context
2048-token window allows
complex scenarios
|
---
## ⚠️ **Limitations**
| ⚠️ Limitation |
📋 Description |
| NSFW Content |
May generate explicit content due to training data |
| Synthetic Data |
Training data is AI-generated, may lack human nuance |
| Context Window |
Limited to 2048 tokens - traits may degrade in long conversations |
| Inherited Limitations |
Inherits any limitations from base model |
---
## 🔗 **Related Projects**
---
## 🙏 **Acknowledgements**
**Special thanks to the incredible teams and individuals who made this possible:**
🔥 **Qwen & Unsloth teams** - For their incredible models and libraries
🎭 **Gryphe** - For the high-quality Sonnet 3.5 dataset
🚀 **TRL team** - For creating and open-sourcing the GRPO trainer
🤗 **HuggingFace community** - For their continued support
---
## 📬 **Feedback & Contact**
---
**✨ *May your characters speak with voices that feel truly alive!* ✨**
**Created with ❤️ by [Chun](https://huggingface.co/chun121)**
---
```
🧙♂️ Qwen3-4B RPG Roleplay V2 | GRPO Enhanced | MIT License
```