119 lines
5.9 KiB
Markdown
119 lines
5.9 KiB
Markdown
|
---
|
||
|
|
base_model:
|
||
|
|
- MuXodious/Llama-3.3-8B-Instruct-128K-PaperWitch-heresy
|
||
|
|
tags:
|
||
|
|
- text-generation-inference
|
||
|
|
- transformers
|
||
|
|
- unsloth
|
||
|
|
- llama
|
||
|
|
- Uncensored
|
||
|
|
- trl
|
||
|
|
- roleplay
|
||
|
|
- conversational
|
||
|
|
license: agpl-3.0
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
datasets:
|
||
|
|
- N-Bot-Int/Iris-Uncensored-Reformat-R2
|
||
|
|
- N-Bot-Int/RP-Mixed-v1
|
||
|
|
library_name: transformers
|
||
|
|
---
|
||
|
|
# MrgrtV1-8B is OFFICIALLY RELEASED!
|
||
|
|
|
||
|
|
|
||
|
|
# MrgrtV1-8B for Better Roleplaying with Maximum Quality improvements!
|
||
|
|
- MrgrtV1-8B *(pronounecd Margaret-V1)* is our brand new **AI MODEL** built on top of **MuXodious/Llama-3.3-8B-Instruct-128K-PaperWitch-heresy**,
|
||
|
|
MrgrtV1-8B is one of our Checkpoint we want to share to you, to showcase the current **Improvement** of our new training pipeline and
|
||
|
|
New **Innovations**.
|
||
|
|
- MrgrtV1-8B Is Our **NEWEST** model, trained on kaggle for **2 Weeks**, which showcases the most **COHERENCE** compared to other models we made!
|
||
|
|
MrgrtV1-8B excels greatly with Roleplay Focused, however **DUE TO OUR LACK OF COMPLETE OPEN DATA**, the ai model might produce subpar quality to Open Roleplay
|
||
|
|
Compared to Contained-Roleplay focused scenarios!
|
||
|
|
- MrgrtV1-8B is trained on our **IMPROVED MEulysis** dataset, that aims to dethrone **Misthena** and our previous **OpenElla-NovelWriter** Models!
|
||
|
|
|
||
|
|
*READ MORE FOR MORE INFO*
|
||
|
|
8 BILLIONS PARAMS MODEL
|
||
|
|
|
||
|
|
# MrgrtV1-8B Model Procedure/Methodology:
|
||
|
|
- MrgrtV1-8B is trained Using **MuXodious**'s Llama-3.3-8B Paperwitch Version[Thank you SO MUCH for all the help @redaihf, @MuXodious and @Naphula](https://huggingface.co/N-Bot-Int/OpenElla-NovelWriter-8B-V2-merged/discussions/1) of the Model,
|
||
|
|
ensuring full creative flow without refusal!
|
||
|
|
|
||
|
|
MrgrtV1-8B is then fed the new **100K single-row DATASET** named **MEulysis-Cleaned** which were obtained after splitting the previous dataset and cleaned!
|
||
|
|
|
||
|
|
- **MEulysis-Cleaned** — a 100k entry synthetically generated dataset produced using
|
||
|
|
multiple capable RP models (including Mythomax and Hermes), then carefully cleaned,
|
||
|
|
formatted with Llama 3 chat formatting, and then, tuned across a wide range of roleplay scenarios.
|
||
|
|
|
||
|
|
### What makes MEulysis-Cleaned different?
|
||
|
|
- **100,000 entries** of synthetic RP data generated from multiple frontier RP models
|
||
|
|
- Wide RP focus covering diverse character types, settings, and narrative styles
|
||
|
|
- Includes **MOST ELABORATE EXPERIENCES** to **MOST TABOO SCENARIOS**
|
||
|
|
- Clear formatting guidance baked into the data — proper use of `"dialogue"` and `*actions*`
|
||
|
|
- Now includes **Llama 3 chat** formatting to maximize quality on Llama models we'll release!
|
||
|
|
- Fully cleaned and curated for quality and consistency
|
||
|
|
|
||
|
|
# Training Details
|
||
|
|
- **Finetuning Tool:** Unsloth AI + Huggingface TRL(we uses our brand new **Kaggle Extenderizer** to extend training without starting from scratch)
|
||
|
|
- **Training Platform:** Kaggle Free Tier with T4 x2!
|
||
|
|
- **Epochs:** 3
|
||
|
|
- **Final Training Loss:** 1.1
|
||
|
|
- **Dataset:** MEulysis-Cleaned
|
||
|
|
|
||
|
|
|
||
|
|
- **MrgrtV1-8B** is Our Brand New Powerful Model, If you ever encountered any issue, Want to commission us, or have any suggestions, please email us directly through
|
||
|
|
[nexus.networkinteractives@gmail.com](mailto:nexus.networkinteractives@gmail.com)
|
||
|
|
we value any reports, suggestions to how we improve future Model,
|
||
|
|
Once again feel free to finetune the model to your likings, However please consider Adding this Page
|
||
|
|
for **CREDITS**
|
||
|
|
|
||
|
|
- Please handle the AI with Care and ethical considerations, when **FINETUNING** this AI model, due to its **UNCENSORED** Nature.
|
||
|
|
- We are not responsible for what this model generates. Use it responsibly and legally. You downloaded it, you own what you do with it.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# What's Coming Next?
|
||
|
|
|
||
|
|
> 🔒 **V2 will released with better support for more OPEN ROLEPLAY SCENARIOS**
|
||
|
|
> Right now the ai model is somewhat bad with Open roleplay, however we'll do our best to release the V2 with better improvements!
|
||
|
|
|
||
|
|
> 🔒 **4b variant to be released soon!**
|
||
|
|
> 4B variants trained on the same dataset, with same training methodology will be released shortly for those who lack system resources however still wish to use
|
||
|
|
> High Quality RPing model!
|
||
|
|
|
||
|
|
> 🔒 **1B variant to be released**
|
||
|
|
> Experimental 1B variant with the same training methodology will be released following 4B, this is **EXTREMELY EXPERIMENTAL**, 1B are not good for Roleplaying
|
||
|
|
> however we share them nonetheless!
|
||
|
|
|
||
|
|
> 🔒 **Release PEFT Soon**
|
||
|
|
> New PEFT releases of the model, will now be released late. Supporters on our Ko-fi can however request for the PEFT(through gmail) if they ever want to!
|
||
|
|
> (This is to thank our Ko-fi supporters!)
|
||
|
|
[](https://ko-fi.com/J3J61D8NHV)
|
||
|
|
|
||
|
|
> Benchmarks are also in the pipeline and will be added once available.
|
||
|
|
---
|
||
|
|
|
||
|
|
# Notices & Usage Tips
|
||
|
|
|
||
|
|
- **Use Llama 3 format** — the model is based on Llama 3, Soooooo using Llama 3 works best!.
|
||
|
|
- **Calibrate per character card** — every character is different, adjust your prompt, Model's settings(ie, temps, Top-K etc.) accordingly. However
|
||
|
|
We recommend(if you're using koboldcpp), to use DynaTemp with Dynamic Min as 0.50 and Dynamic max as 1.50.
|
||
|
|
That's the setting we found to make the model more coherent, less hallucination whilst making the model adventurous!
|
||
|
|
---
|
||
|
|
|
||
|
|
# About
|
||
|
|
- **MrgrtV1-8B** is
|
||
|
|
- **Developed by:** N-Bot-Int
|
||
|
|
- **License:** agpl 3.0
|
||
|
|
- **Finetuned from model :** MuXodious/Llama-3.3-8B-Instruct-128K-PaperWitch-heresy
|
||
|
|
|
||
|
|
- # Detail card:
|
||
|
|
- Parameter
|
||
|
|
- 8 Billion Parameters
|
||
|
|
- (Please check your GPU Core, VRAM, CPU and RAM to see if you can comfortably run 8B models)
|
||
|
|
|
||
|
|
- Finetuning tool:
|
||
|
|
- Unsloth AI
|
||
|
|
- This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
||
|
|
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
||
|
|
- Fine-tuned Using:
|
||
|
|
- Kaggle Free Tier with T4 x2 for 2 Weeks
|