初始化项目,由ModelHub XC社区提供模型
Model: 0xA50C1A1/Llama-3.3-8B-Instruct-OmniWriter Source: Original Platform
This commit is contained in:
49
README.md
Normal file
49
README.md
Normal file
@@ -0,0 +1,49 @@
|
||||
---
|
||||
base_model: 0xA50C1A1/Llama-3.3-8B-Instruct-128K-Heretic
|
||||
tags:
|
||||
- text-generation-inference
|
||||
- transformers
|
||||
- unsloth
|
||||
- llama
|
||||
- heretic
|
||||
- uncensored
|
||||
- decensored
|
||||
- abliterated
|
||||
- storytelling
|
||||
- creative-writing
|
||||
license: llama3.3
|
||||
language:
|
||||
- en
|
||||
datasets:
|
||||
- Gryphe/Opus-WritingPrompts
|
||||
- Gryphe/ChatGPT-4o-Writing-Prompts
|
||||
---
|
||||
|
||||
# Llama-3.3-8B-Instruct-OmniWriter
|
||||
|
||||
An experiment in creating a creative, uncensored storyteller based on the "smart" Llama 3.3 8B Instruct base.
|
||||
|
||||
### Training Specs
|
||||
|
||||
* **Training Method:** LoRA (16-bit)
|
||||
* **LoRA Rank (r):** 32
|
||||
* **LoRA Alpha:** 16
|
||||
* **Scaling Type:** Rank-Stabilized LoRA (RS-LoRA)
|
||||
* **Batch Size:** 32
|
||||
* **Gradient Accumulation:** 2
|
||||
* **Epochs:** 1
|
||||
* **Learning Rate:** 2e-5
|
||||
* **Optimizer:** AdamW (Fused)
|
||||
* **LR Scheduler:** Cosine
|
||||
* **Noise Level:** NEFTune (alpha=5)
|
||||
|
||||
|
||||
### Special Thanks
|
||||
|
||||
This fine-tune wouldn't be possible without the incredible work of the community:
|
||||
|
||||
* **[p-e-w](https://huggingface.co/p-e-w)** for developing **[Heretic](https://github.com/p-e-w/heretic)** - an essential tool for censorship removal.
|
||||
* **[allura-forge](https://huggingface.co/allura-forge)** and **[shb777](https://huggingface.co/shb777)** for providing access to the **Llama 3.3 8B** weights.
|
||||
* **[Gryphe](https://huggingface.co/Gryphe)** for the meticulously curated **Writing Prompts** datasets (Opus and ChatGPT-4o versions).
|
||||
* **[unsloth](https://github.com/unslothai/unsloth)** for their training acceleration framework that makes local fine-tuning fast and memory-efficient.
|
||||
* **[AMD](https://oneclickamd.ai/)** for their Instinct™ MI300X GPU.
|
||||
Reference in New Issue
Block a user