base_model, tags, license, language, datasets
| base_model |
tags |
license |
language |
datasets |
| 0xA50C1A1/Llama-3.3-8B-Instruct-128K-Heretic |
| text-generation-inference |
| transformers |
| unsloth |
| llama |
| heretic |
| uncensored |
| decensored |
| abliterated |
| storytelling |
| creative-writing |
|
llama3.3 |
|
| Gryphe/Opus-WritingPrompts |
| Gryphe/ChatGPT-4o-Writing-Prompts |
|
Llama-3.3-8B-Instruct-OmniWriter
An experiment in creating a creative, uncensored storyteller based on the "smart" Llama 3.3 8B Instruct base.
Training Specs
- Training Method: LoRA (16-bit)
- LoRA Rank (r): 32
- LoRA Alpha: 16
- Scaling Type: Rank-Stabilized LoRA (RS-LoRA)
- Batch Size: 32
- Gradient Accumulation: 2
- Epochs: 1
- Learning Rate: 2e-5
- Optimizer: AdamW (Fused)
- LR Scheduler: Cosine
- Noise Level: NEFTune (alpha=5)
Special Thanks
This fine-tune wouldn't be possible without the incredible work of the community:
- p-e-w for developing Heretic - an essential tool for censorship removal.
- allura-forge and shb777 for providing access to the Llama 3.3 8B weights.
- Gryphe for the meticulously curated Writing Prompts datasets (Opus and ChatGPT-4o versions).
- unsloth for their training acceleration framework that makes local fine-tuning fast and memory-efficient.
- AMD for their Instinct™ MI300X GPU.
