88 lines
2.6 KiB
Markdown
88 lines
2.6 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
base_model: Qwen/Qwen2.5-7B-Instruct
|
||
|
|
tags:
|
||
|
|
- conversational
|
||
|
|
- fine-tuned
|
||
|
|
- geode
|
||
|
|
- genue-ai
|
||
|
|
- flagship
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
# 🔮 Genue Geode Thaumite (7B) — Flagship
|
||
|
|
|
||
|
|
**Thaumite** is Genue AI's **flagship** 7-billion parameter conversational AI model. It is the most capable model in the **Geode** family, fine-tuned for deep reasoning, code generation, and rich conversational ability.
|
||
|
|
|
||
|
|
## Model Details
|
||
|
|
|
||
|
|
| Property | Value |
|
||
|
|
|---|---|
|
||
|
|
| **Base Model** | Qwen 2.5 7B Instruct |
|
||
|
|
| **Parameters** | 7 Billion |
|
||
|
|
| **Fine-Tuning** | LoRA (r=32, alpha=32) |
|
||
|
|
| **Training Loss** | 0.646 |
|
||
|
|
| **Precision** | FP16 |
|
||
|
|
| **License** | Apache 2.0 |
|
||
|
|
|
||
|
|
## The Geode Family
|
||
|
|
|
||
|
|
The **Geode** family is Genue AI's lineup of locally-runnable conversational models:
|
||
|
|
|
||
|
|
| Model | Parameters | Role |
|
||
|
|
|---|---|---|
|
||
|
|
| [Beryl](https://huggingface.co/GenueAI/geode-beryl) | 0.5B | Lightweight, experimental |
|
||
|
|
| [Onyx](https://huggingface.co/GenueAI/geode-onyx) | 3B | Balanced logic & personality |
|
||
|
|
| **Thaumite** | 7B | 👑 Flagship, highest capability |
|
||
|
|
|
||
|
|
## Why Thaumite?
|
||
|
|
|
||
|
|
Thaumite is named after a rare crystalline mineral, representing the pinnacle of the Geode family. It offers:
|
||
|
|
|
||
|
|
- **Superior Reasoning**: Handles multi-step math, logic puzzles, and word problems
|
||
|
|
- **Code Generation**: Produces full HTML/CSS/JS applications on request
|
||
|
|
- **Rich Personality**: Maintains a consistent, friendly identity across conversations
|
||
|
|
- **Deep Knowledge**: Trained on curated facts about history, gaming, science, and more
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
Thaumite uses a simple **Direct Mode** prompt format:
|
||
|
|
|
||
|
|
```
|
||
|
|
User: What is your name?
|
||
|
|
Bot:
|
||
|
|
```
|
||
|
|
|
||
|
|
### With Transformers
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
import torch
|
||
|
|
|
||
|
|
model = AutoModelForCausalLM.from_pretrained("GenueAI/geode-thaumite", torch_dtype=torch.float16)
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained("GenueAI/geode-thaumite")
|
||
|
|
|
||
|
|
prompt = "User: Make a clicker game in HTML.\nBot:"
|
||
|
|
inputs = tokenizer(prompt, return_tensors="pt")
|
||
|
|
outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.4, repetition_penalty=1.1)
|
||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
## Training Data
|
||
|
|
|
||
|
|
Fine-tuned on 64 curated conversational examples covering:
|
||
|
|
- Identity & self-awareness
|
||
|
|
- Mathematical reasoning (arithmetic, algebra, word problems)
|
||
|
|
- General & historical knowledge
|
||
|
|
- HTML/CSS/JS code generation
|
||
|
|
- Gaming knowledge (Geometry Dash, Omori)
|
||
|
|
- Genue AI ecosystem knowledge
|
||
|
|
|
||
|
|
## Developed By
|
||
|
|
|
||
|
|
**Genue AI** — Founded by Brybod123 (Bradar)
|
||
|
|
|
||
|
|
> *"Me, Thaumite of course!"* — When asked which model is best
|