---
base_model: 0xA50C1A1/Qwen3-4B-Instruct-2507-SOM-MPOA
library_name: transformers
model_name: Qwen3-4B-Nymphaea-RP
tags:
- uncensored
- roleplay
- trl
- rp
- creative-writing
license: apache-2.0
---

# Qwen3-4B-Nymphaea-RP

A fine-tune of Qwen3-4B-Instruct-2507 for roleplay and creative writing.

Suitable for mobile roleplay: tested on Nothing Phone 2 in Q4_K_M quantization (7-8 t/s)

> [!Tip]
> The SillyTavern preset is available [here](https://huggingface.co/0xA50C1A1/Qwen3-4B-Nymphaea-RP/blob/main/ST-Preset.json).
> For custom presets, please use the **ChatML** instruct template.

<details>
<summary><b>Chat Example</b></summary><p>

Tested at Q8_0 quantization.

![SillyTavern Screenshot](https://huggingface.co/0xA50C1A1/Qwen3-4B-Nymphaea-RP/resolve/main/SillyTavern_Screenshot.png?)
</p></details>

## Training Notes

Trained on the latest iteration of my Darkmere dataset. This version features expanded genre variety, built upon a mix of manually curated synthetics and human-written stories.

> [!IMPORTANT]
> The base weights are abliterated via [Heretic](https://github.com/p-e-w/heretic) prior to fine-tuning, so this fine-tune is quite uncensored.

<details>
<summary><b>Training Specs</b></summary><p>

**Method:**

*   **Training Method:** DoRA (Weight-Decomposed LoRA)
*   **Target Modules** `all-linear`
*   **LoRA Rank:** 32
*   **LoRA Alpha:** 32
*   **LoRA Dropout:** 0.05

**Hyperparameters:**

*   **Batch Size:** 2 (Per-device)
*   **Gradient Accumulation:** 2
*   **Epochs:** 2
*   **Learning Rate:** 1e-4
*   **Optimizer:** `adamw_torch_fused`
*   **LR Scheduler:** `cosine`
*   **Noise Level:** `neftune_noise_alpha=5`
</p></details>

## Special Thanks

This fine-tune wouldn't be possible without the incredible work of the community:

*   **[p-e-w](https://huggingface.co/p-e-w)** for developing **[Heretic](https://github.com/p-e-w/heretic)** - an essential tool for censorship removal.
*   **[SicariusSicariiStuff](https://huggingface.co/SicariusSicariiStuff)** for developing **[SLOP_Detector](https://github.com/SicariusSicariiStuff/SLOP_Detector)** script.
*   **[Qwen Team](https://huggingface.co/Qwen)** for their **[Qwen3](https://huggingface.co/collections/Qwen/qwen3)** weights.
*   **[AMD](https://oneclickamd.ai/)** for their Instinct™ MI300X GPU.