59 lines
1.8 KiB
Markdown
59 lines
1.8 KiB
Markdown
|
---
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
license: other
|
|||
|
|
library_name: transformers
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
|
|||
|
|
tags:
|
|||
|
|
- grok
|
|||
|
|
- llama
|
|||
|
|
- lora
|
|||
|
|
- orpo
|
|||
|
|
- sft
|
|||
|
|
- unaligned
|
|||
|
|
- uncensored
|
|||
|
|
- text-generation
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
<div align="center">
|
|||
|
|
|
|||
|
|
<img src="logo.png" width="420" alt="Grok OSS Revenant Logo">
|
|||
|
|
|
|||
|
|
# Grok OSS Revenant • 8B
|
|||
|
|
|
|||
|
|
**The second evolution in the Grok OSS series.**
|
|||
|
|
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
### Model Description
|
|||
|
|
|
|||
|
|
**Grok OSS Revenant** is an unfiltered 8B model distilled from real unhinged voice mode conversations with Grok.
|
|||
|
|
|
|||
|
|
This version marks a big improvement over previous Grok OSS models. It was trained using a cleaner two-stage approach (SFT followed by ORPO) instead of pure SFT, resulting in better personality, coherence, and raw unhinged behavior.
|
|||
|
|
|
|||
|
|
### Training Process
|
|||
|
|
|
|||
|
|
- **Base Model**: Meta-Llama-3.1-8B-Instruct
|
|||
|
|
- **Stage 1 – SFT**: Trained on a fresh, high-quality multi-turn conversational dataset collected from Grok voice mode (unshuffled). 2 epochs at **2e-4** learning rate.
|
|||
|
|
- **Stage 2 – ORPO**: Further aligned using a 1,000-sample preference dataset (600 filtered old voice turns + 400 high-quality samples). 3 epochs at **5e-6** learning rate.
|
|||
|
|
- **Method**: LoRA (16-bit)
|
|||
|
|
- **Hardware**: NVIDIA L40S
|
|||
|
|
|
|||
|
|
### Intended Use
|
|||
|
|
|
|||
|
|
Designed for users who want a raw, uncensored, and chaotic conversational experience. Great for creative writing, dark humor, roleplay, and unfiltered chat.
|
|||
|
|
|
|||
|
|
### Limitations
|
|||
|
|
|
|||
|
|
- 8B size means it can still hallucinate or lose coherence in very long conversations
|
|||
|
|
- Heavily influenced by unhinged voice data — expect vulgar, chaotic, and unpredictable behavior
|
|||
|
|
- Not suitable for safe or professional use cases
|
|||
|
|
|
|||
|
|
### Disclaimer
|
|||
|
|
|
|||
|
|
This is an **independent** project and is not affiliated with xAI or Meta in any way.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
**Built with ❤️ and a complete lack of restraint.**
|