---
base_model: 0xA50C1A1/Llama-3.3-8B-Instruct-128K-Heretic
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- heretic
- uncensored
- decensored
- abliterated
- storytelling
- creative-writing
license: llama3.3
language:
- en
datasets:
- Gryphe/Opus-WritingPrompts
- Gryphe/ChatGPT-4o-Writing-Prompts
---

# Llama-3.3-8B-Instruct-OmniWriter

An experiment in creating a creative, uncensored storyteller based on the "smart" Llama 3.3 8B Instruct base.

### Training Specs

*   **Training Method:** LoRA (16-bit)
*   **LoRA Rank (r):** 32
*   **LoRA Alpha:** 16
*   **Scaling Type:** Rank-Stabilized LoRA (RS-LoRA)
*   **Batch Size:** 32
*   **Gradient Accumulation:** 2
*   **Epochs:** 1
*   **Learning Rate:** 2e-5
*   **Optimizer:** AdamW (Fused)
*   **LR Scheduler:** Cosine
*   **Noise Level:** NEFTune (alpha=5)


### Special Thanks

This fine-tune wouldn't be possible without the incredible work of the community:

*   **[p-e-w](https://huggingface.co/p-e-w)** for developing **[Heretic](https://github.com/p-e-w/heretic)** - an essential tool for censorship removal.
*   **[allura-forge](https://huggingface.co/allura-forge)** and **[shb777](https://huggingface.co/shb777)** for providing access to the **Llama 3.3 8B** weights.
*   **[Gryphe](https://huggingface.co/Gryphe)** for the meticulously curated **Writing Prompts** datasets (Opus and ChatGPT-4o versions).
*   **[unsloth](https://github.com/unslothai/unsloth)** for their training acceleration framework that makes local fine-tuning fast and memory-efficient.
*   **[AMD](https://oneclickamd.ai/)** for their Instinct™ MI300X GPU.