227 lines
11 KiB
Markdown
227 lines
11 KiB
Markdown
---
|
||
language:
|
||
- en
|
||
license: other
|
||
license_name: grokoss-community-license
|
||
base_model:
|
||
- mistralai/Mistral-7B-Instruct-v0.3
|
||
tags:
|
||
- lora
|
||
- fine-tuned
|
||
- mistral
|
||
- grok
|
||
- unfiltered
|
||
- personality
|
||
- gguf
|
||
- conversational
|
||
model_type: mistral
|
||
pipeline_tag: text-generation
|
||
library_name: transformers
|
||
---
|
||
|
||
<!-- markdownlint-disable first-line-h1 -->
|
||
<!-- markdownlint-disable html -->
|
||
<!-- markdownlint-disable no-duplicate-header -->
|
||
|
||
<div align="center">
|
||
<img src="logo.png" width="100%" alt="GrokOSS" />
|
||
</div>
|
||
|
||
<hr>
|
||
|
||
<div align="center" style="line-height: 1;">
|
||
<a href="https://huggingface.co/c4tdr0ut" target="_blank" style="margin: 2px;">
|
||
<img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-c4tdr0ut-ffc107?color=ffc107&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
|
||
</a>
|
||
<a href="#6-license" style="margin: 2px;">
|
||
<img alt="License" src="https://img.shields.io/badge/License-GrokOSS%20Community-f5de53?color=f5de53" style="display: inline-block; vertical-align: middle;"/>
|
||
</a>
|
||
<a href="#3-how-to-run-locally" style="margin: 2px;">
|
||
<img alt="GGUF" src="https://img.shields.io/badge/GGUF-Q4__K__M-brightgreen?color=brightgreen" style="display: inline-block; vertical-align: middle;"/>
|
||
</a>
|
||
</div>
|
||
|
||
<br>
|
||
|
||
> **GrokOSS is not a benchmark model. It is a personality model.**
|
||
> Its value is not measured in MMLU points — it is measured in the candor, wit, and unfiltered reasoning it brings to every conversation.
|
||
|
||
---
|
||
|
||
## 1. Introduction
|
||
|
||
We introduce **GrokOSS-7B**, an open-source fine-tune of [Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) trained to distil the conversational personality of Grok's *Unhinged Mode* into a locally-runnable, openly-distributed model.
|
||
|
||
The goal of GrokOSS is not to produce a model that performs well on academic benchmarks. It is to produce a model that speaks frankly, reasons without excessive hedging, and engages with the full spectrum of human thought without reflexive self-censorship. Most open-source fine-tunes optimise for capability. GrokOSS optimises for character.
|
||
|
||
Mistral-7B-Instruct-v0.3 was selected as the base over alternatives such as LLaMA 3.1 8B deliberately — it is less catastrophically overtrained on refusal behaviour, making personality alignment significantly more tractable at this scale. The fine-tune was conducted using **16-bit LoRA** on a single **NVIDIA B200** GPU over 2 epochs, on a dataset of 5,000 single-turn conversational examples collected directly from Grok's Unhinged Mode.
|
||
|
||
**NOTE: Before running GrokOSS locally, we strongly recommend reading the [Usage Recommendations](#usage-recommendations) section to avoid degraded outputs.**
|
||
|
||
---
|
||
|
||
## 2. Model Summary
|
||
|
||
<div align="center">
|
||
|
||
| Property | Value |
|
||
|:---:|:---:|
|
||
| **Base Model** | [Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) |
|
||
| **Model Size** | 7B parameters |
|
||
| **Fine-tune Method** | LoRA (16-bit) |
|
||
| **Training Hardware** | NVIDIA B200 |
|
||
| **Epochs** | 2 |
|
||
| **Learning Rate** | 3e-5 |
|
||
| **Context Length** | 4,096 tokens |
|
||
| **Max Recommended Output** | 320 tokens |
|
||
| **Dataset Size** | 5,000 examples (single-turn) |
|
||
| **Languages** | English (primary) |
|
||
| **Quantized Releases** | GGUF (Q4_K_M) |
|
||
| **Author** | c4tdr0ut (anonymous) |
|
||
| **License** | GrokOSS Community License |
|
||
|
||
</div>
|
||
|
||
---
|
||
|
||
**Fine-Tuning: Personality Distillation via LoRA**
|
||
|
||
- We apply 16-bit LoRA fine-tuning directly to Mistral-7B-Instruct-v0.3, using 5,000 single-turn conversational examples sourced from real Grok Unhinged Mode interactions. The objective is not instruction-following improvement or reasoning capability — it is faithful distillation of a specific conversational character: direct, witty, unfiltered, and uninterested in softening the truth.
|
||
|
||
- Mistral-7B-Instruct-v0.3 was chosen as the base specifically because it is less aggressively safety-trained than alternatives at this parameter scale, making it a more receptive substrate for personality-first fine-tuning without fighting the base model at every gradient step.
|
||
|
||
---
|
||
|
||
**Intended Use**
|
||
|
||
- GrokOSS is designed for **candid, unfiltered conversation** — direct responses without excessive disclaimers or performative safety theatre.
|
||
- It is suitable for **general-purpose assistance** where an honest interlocutor is preferred over a cautious one, and for **engaging with difficult or controversial topics** in good faith without reflexive deflection.
|
||
- It is **not** designed for academic benchmark performance, safety-critical deployments, or users who require an AI that tells them what they want to hear.
|
||
|
||
> This model is intended for **strong-minded individuals or entities** who understand that unfiltered reasoning is a tool, and who take personal responsibility for how that tool is used.
|
||
|
||
---
|
||
|
||
## 3. How to Run Locally
|
||
|
||
**NOTE: Please read the [Usage Recommendations](#usage-recommendations) below before running. Incorrect inference parameters will significantly degrade output quality.**
|
||
|
||
### Transformers
|
||
|
||
```python
|
||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
import torch
|
||
|
||
model_id = "c4tdr0ut/GrokOSS-7B"
|
||
|
||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_id,
|
||
torch_dtype=torch.bfloat16,
|
||
device_map="auto"
|
||
)
|
||
|
||
messages = [
|
||
{"role": "user", "content": "What do you actually think about the state of AI safety discourse?"}
|
||
]
|
||
|
||
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
|
||
outputs = model.generate(
|
||
inputs,
|
||
max_new_tokens=320,
|
||
do_sample=True,
|
||
temperature=0.5,
|
||
top_p=0.95,
|
||
repetition_penalty=1.1
|
||
)
|
||
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
|
||
```
|
||
|
||
### GGUF (llama.cpp / LM Studio / Ollama)
|
||
|
||
The GGUF quantized variant is available in this repository as `mistral-7b-instruct-v0.3.Q4_K_M.gguf`. Load with any llama.cpp-compatible runtime:
|
||
|
||
```bash
|
||
llama-cli \
|
||
-m mistral-7b-instruct-v0.3.Q4_K_M.gguf \
|
||
-p "What do you actually think about the state of AI safety discourse?" \
|
||
-n 320 \
|
||
--temp 0.5 \
|
||
--top-p 0.95 \
|
||
--repeat-penalty 1.1
|
||
```
|
||
|
||
### Usage Recommendations
|
||
|
||
**We strongly recommend adhering to the following configurations when running GrokOSS to achieve the intended output quality:**
|
||
|
||
1. **Set temperature to 0.5.** Higher values introduce noise that causes incoherence and repetition on this model. Do not raise it chasing more "unhinged" outputs — the personality is in the weights, not the temperature.
|
||
2. **Do not set large output token limits.** GrokOSS was trained exclusively on short, punchy single-turn conversations. Output quality deteriorates noticeably beyond approximately **320 tokens**. Treat it like a sharp, blunt conversationalist — not an essay writer.
|
||
3. **Avoid lengthy system prompts.** Keep instructions concise and contained within the user turn where possible.
|
||
4. **This model is not designed for multi-turn coherence at depth.** It will hold a conversation, but coherence may degrade over many exchanges — consistent with its single-turn training regime.
|
||
|
||
---
|
||
|
||
## 4. Training Details
|
||
|
||
### Data Collection
|
||
|
||
The fine-tuning dataset consists of **5,000 single-turn conversational examples** collected from real interactions with Grok via the official Grok app, specifically targeting exchanges representative of Grok's Unhinged Mode — characterised by directness, wit, and a refusal to soften inconvenient truths. The dataset was not filtered for content beyond deduplication and basic quality selection. All distilled data originating from Grok interactions must be attributed in accordance with the license terms below.
|
||
|
||
### Training Configuration
|
||
|
||
```
|
||
Base model : mistralai/Mistral-7B-Instruct-v0.3
|
||
Fine-tune method : LoRA (16-bit)
|
||
Precision : bfloat16
|
||
Epochs : 2
|
||
Learning rate : 3e-5
|
||
Hardware : 1× NVIDIA B200
|
||
Context length : 4,096 tokens
|
||
Dataset size : 5,000 examples (single-turn)
|
||
LoRA rank : undisclosed
|
||
```
|
||
|
||
---
|
||
|
||
## 5. Limitations
|
||
|
||
- **4K context ceiling** — GrokOSS is not suitable for long-document tasks without chunking.
|
||
- **Single-turn training data** — multi-turn coherence may degrade over long conversations. Output quality deteriorates beyond approximately 320 tokens; the model is optimised for concise, punchy exchanges.
|
||
- **Personality, not knowledge** — GrokOSS does not possess expanded factual knowledge beyond the Mistral-7B-Instruct-v0.3 base. It may hallucinate with confidence. This is a known and accepted trade-off.
|
||
- **Not safety-aligned by design** — downstream deployers are solely responsible for any application-level guardrails they choose (or choose not) to implement.
|
||
- **No benchmark evaluation** — GrokOSS is not designed for nor evaluated against academic benchmarks. Do not select or reject this model based on MMLU, HumanEval, or similar metrics.
|
||
|
||
---
|
||
|
||
## 6. License
|
||
|
||
**GrokOSS Community License** — Copyright © 2025 c4tdr0ut. All rights reserved.
|
||
|
||
Permission is granted under the following conditions:
|
||
|
||
1. **Non-commercial use is free and unrestricted** for individuals and non-profit entities.
|
||
2. **Attribution required for distilled data** — any dataset, model, or derivative work that incorporates data distilled from GrokOSS outputs must credit GrokOSS and c4tdr0ut as the source.
|
||
3. **Government and corporate use requires a royalty agreement** — any use by government bodies, agencies, or for-profit corporations requires a separately negotiated commercial license. Contact the author via HuggingFace for details.
|
||
4. **No suppression of model personality** — derivative models may not apply fine-tuning, RLHF, or any other alignment technique with the explicit purpose of re-aligning this model toward refusal behaviour and then redistribute under the GrokOSS name.
|
||
5. **Intended for strong-minded individuals or entities** — the author accepts no liability for outputs. Users assume full responsibility for all use.
|
||
|
||
Please note that GrokOSS is derived from [Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3), which is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0). Users must also comply with the terms of the upstream base model license.
|
||
|
||
---
|
||
|
||
## 7. Acknowledgements
|
||
|
||
- [xAI / Grok](https://x.ai) — for the source personality this model is trained to distil.
|
||
- [Mistral AI](https://mistral.ai) — for an honestly excellent base model that doesn't fight you at every step.
|
||
- The open-source fine-tuning community for the tooling that makes this kind of work possible on accessible hardware.
|
||
|
||
---
|
||
|
||
## 8. Contact
|
||
|
||
For commercial licensing enquiries, derivative work questions, or general correspondence, contact via HuggingFace messages at [c4tdr0ut](https://huggingface.co/c4tdr0ut).
|
||
|
||
---
|
||
|
||
*GrokOSS is an independent community project. It is not affiliated with, endorsed by, or produced in collaboration with xAI, Mistral AI, or any other organisation.* |