116 lines
5.7 KiB
Markdown
116 lines
5.7 KiB
Markdown
---
|
|
base_model:
|
|
- huihui-ai/Llama-3.2-3B-Instruct-abliterated
|
|
tags:
|
|
- text-generation-inference
|
|
- transformers
|
|
- unsloth
|
|
- llama
|
|
- Uncensored
|
|
- trl
|
|
- roleplay
|
|
- conversational
|
|
license: agpl-3.0
|
|
language:
|
|
- en
|
|
pipeline_tag: text-generation
|
|
datasets:
|
|
- N-Bot-Int/RP-Mixed-v1
|
|
library_name: transformers
|
|
---
|
|
# MrgrtV2-3B is OFFICIALLY RELEASED!
|
|
|
|
|
|
# MrgrtV2-3B, rebalanced, cleaned and fixed than previous version!
|
|
- MrgrtV2-3B, our brand new flagship model is now released with a better model training than previous V1, the V1 had a massive poison on the dataset
|
|
it were trained, leading to the model having insane early <|eos_token|> even though the roleplay is still not yet finished.
|
|
- MrgrtV2-3B aims to fix the issue by showcasing the quality of the dataset, by training on a 3B model! revealing that the dataset is high quality!
|
|
and the early issues were simply the poison that killed the quality of V1 Models!
|
|
**THEREFORE THIS MODEL IS PURELY A FIX TO THE PREVIOUS V1 MODEL! HOWEVER THE QUALITY IS INSANELY, NIGHT AND DAY!**
|
|
|
|
*READ MORE FOR MORE INFO*
|
|
3 BILLIONS PARAMS MODEL
|
|
|
|
# MrgrtV2-3B Model Procedure/Methodology:
|
|
- MrgrtV2-3B is trained Using **HuiHui-Ai's** Llama 3.2 3B instruct abliterated Model,
|
|
ensuring full creative flow without refusal!
|
|
|
|
MrgrtV2-3B is then fed the new **22k single-row DATASET** named **MEulysis-Cleaned-formatted** which were obtained after splitting the previous dataset and cleaned EVEN FURTHER!
|
|
|
|
- **MEulysis-Cleaned-formatted** — a 22 entry synthetically generated dataset produced using
|
|
multiple capable RP models (including Hermes(mythomax were removed because the mythomax was causing the poison to the dataset)), then carefully cleaned,
|
|
formatted with Llama 3 chat formatting, and then, tuned across a wide range of roleplay scenarios.
|
|
|
|
### What makes MEulysis-Cleaned different?
|
|
- **22,000 entries** of synthetic RP data generated from multiple frontier RP models
|
|
- removed mythomax which caused early eos terminations and some... quality issue removal
|
|
- Wide RP focus covering diverse character types, settings, and narrative styles
|
|
- Includes **MOST ELABORATE EXPERIENCES** to **MOST TABOO SCENARIOS**
|
|
- Clear formatting guidance baked into the data — proper use of `"dialogue"` and `*actions*`
|
|
- Now includes **Llama 3 chat** formatting to maximize quality on Llama models we'll release!
|
|
- Fully cleaned and curated for quality and consistency
|
|
|
|
# Training Details
|
|
- **Finetuning Tool:** Unsloth AI + Huggingface TRL(we uses our brand new **Kaggle Extenderizer** to extend training without starting from scratch)
|
|
- **Training Platform:** Kaggle Free Tier with T4 x2!
|
|
- **Epochs:** 3
|
|
- **Final Training Loss:** 1.2
|
|
- **Dataset:** MEulysis-Cleaned-formatted
|
|
|
|
|
|
- **MrgrtV2-3B** is Our Brand New Powerful Model, If you ever encountered any issue, Want to commission us, or have any suggestions, please email us directly through
|
|
[nexus.networkinteractives@gmail.com](mailto:nexus.networkinteractives@gmail.com)
|
|
we value any reports, suggestions to how we improve future Model,
|
|
Once again feel free to finetune the model to your likings, However please consider Adding this Page
|
|
for **CREDITS**
|
|
|
|
- Please handle the AI with Care and ethical considerations, when **FINETUNING** this AI model, due to its **UNCENSORED** Nature.
|
|
- We are not responsible for what this model generates. Use it responsibly and legally. You downloaded it, you own what you do with it.
|
|
|
|
---
|
|
|
|
# What's Coming Next?
|
|
|
|
> 🔒 **V3 will released with better support for more OPEN ROLEPLAY SCENARIOS**
|
|
> Right now the ai model is somewhat bad with Open roleplay, however we'll do our best to release the V3 with better improvements!
|
|
|
|
> 🔒 **8B variant to be released soon!**
|
|
> Due to this cleanup, we can conclude that the 8B model will be better than the previous V1! we'll take our time releasing the next 8B variant!
|
|
|
|
> 🔒 **1B variant to be released**
|
|
> Experimental 1B variant with the same training methodology will be released following 4B, this is **EXTREMELY EXPERIMENTAL**, 1B are not good for Roleplaying
|
|
> however we share them nonetheless!
|
|
|
|
> 🔒 **Release PEFT Soon**
|
|
> New PEFT releases of the model, will now be released late. Supporters on our Ko-fi can however request for the PEFT(through gmail) if they ever want to!
|
|
> (This is to thank our Ko-fi supporters!)
|
|
[](https://ko-fi.com/J3J61D8NHV)
|
|
|
|
> Benchmarks are also in the pipeline and will be added once available.
|
|
---
|
|
|
|
# Notices & Usage Tips
|
|
|
|
- **Use Llama 3 format** — the model is based on Llama 3, Soooooo using Llama 3 works best!.
|
|
- **Calibrate per character card** — every character is different, adjust your prompt, Model's settings(ie, temps, Top-K etc.) accordingly. However
|
|
We recommend(if you're using koboldcpp), to use DynaTemp with Dynamic Min as 0.60 and Dynamic max as 2.00.
|
|
That's the setting we found to make the model more coherent, less hallucination whilst making the model adventurous!
|
|
---
|
|
|
|
# About
|
|
- **MrgrtV2-3B** is
|
|
- **Developed by:** N-Bot-Int
|
|
- **License:** agpl 3.0
|
|
- **Finetuned from model :** huihui-ai/Llama-3.2-3B-Instruct-abliterated
|
|
|
|
- # Detail card:
|
|
- Parameter
|
|
- 3 Billion Parameters
|
|
- (Please check your GPU Core, VRAM, CPU and RAM to see if you can comfortably run 3B models)
|
|
|
|
- Finetuning tool:
|
|
- Unsloth AI
|
|
- This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
|
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
|
- Fine-tuned Using:
|
|
- Kaggle Free Tier with T4 x2 for 4 Weeks |