86 lines
3.3 KiB
Markdown
86 lines
3.3 KiB
Markdown
|
---
|
||
|
|
base_model: aimeri/spoomplesmaxx-base-qwen3-14b
|
||
|
|
tags:
|
||
|
|
- text-generation-inference
|
||
|
|
- transformers
|
||
|
|
- gguf
|
||
|
|
- llama.cpp
|
||
|
|
- unsloth
|
||
|
|
- qwen3
|
||
|
|
- roleplay
|
||
|
|
- creative-writing
|
||
|
|
- sft
|
||
|
|
license: apache-2.0
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
datasets:
|
||
|
|
- aimeri/spoomplesmaxx-sft-small-olivia
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
library_name: transformers
|
||
|
|
---
|
||
|
|
|
||
|
|
# SpoomplesMaxx — Qwen3 14B SFT
|
||
|
|
|
||
|
|
A 14B language model built on Qwen3-14B through a multi-stage training pipeline: **Continued Pre-Training** (CPT) → **Supervised Fine-Tuning** (SFT). This is the SFT checkpoint. DPO alignment has not yet been applied.
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
## What is this?
|
||
|
|
|
||
|
|
SpoomplesMaxx is an experiment in training a persona-consistent model from scratch rather than fine-tuning an existing instruct model. The goal is full control over voice, format, and behavior by building up from a base model.
|
||
|
|
|
||
|
|
The CPT stage ([spoomplesmaxx-base-qwen3-14b](https://huggingface.co/aimeri/spoomplesmaxx-base-qwen3-14b)) injected domain knowledge from character cards, literary prose, and specialized text. This SFT stage teaches instruction-following and conversation using a custom chat format.
|
||
|
|
|
||
|
|
## Chat Format (DanChat)
|
||
|
|
|
||
|
|
The model uses a custom token format:
|
||
|
|
```
|
||
|
|
<|system|>system prompt<|endoftext|>
|
||
|
|
<|user|>user message<|endoftext|>
|
||
|
|
<|assistant|>response<|endoftext|>
|
||
|
|
```
|
||
|
|
|
||
|
|
|
||
|
|
- `<|system|>` — System/roleplay instructions
|
||
|
|
- `<|user|>` / `<|assistant|>` — Conversation turns
|
||
|
|
- `<|endoftext|>` — Segment terminator
|
||
|
|
|
||
|
|
## Training Data
|
||
|
|
|
||
|
|
The SFT mix is a weighted blend of several categories:
|
||
|
|
|
||
|
|
| Category | Focus | ~Weight |
|
||
|
|
|----------|-------|---------|
|
||
|
|
| Roleplay & Creative Writing | Character RP, adventure, scenario-based dialogue | 28% |
|
||
|
|
| NSFW | Explicit roleplay and creative content | 22% |
|
||
|
|
| Tasks & Instructions | Tool use, function calling, general assistant tasks | 17% |
|
||
|
|
| Reasoning & Logic | Math, logic, theory of mind, physical reasoning | 16% |
|
||
|
|
| Persona Voice | Olivia persona reinforcement | 12% |
|
||
|
|
| Specialized Knowledge | Survival, operations, tactical scenarios | 5% |
|
||
|
|
|
||
|
|
|
||
|
|
## Olivia
|
||
|
|
|
||
|
|
The model includes training data transformed into the voice of **Olivia**, a reference persona: a 31-year-old Brazilian zoologist turned ML hobbyist. She's warm but direct, uses grounded analogies, and occasionally slips into Portuguese when frustrated.
|
||
|
|
|
||
|
|
Olivia is a proof of concept for persona consistency — demonstrating that voice can be trained in rather than prompted for. You don't have to use the Olivia persona; the model responds to whatever system prompt you provide.
|
||
|
|
|
||
|
|
## Intended Use
|
||
|
|
|
||
|
|
- Roleplay and character-driven conversation
|
||
|
|
- Creative and narrative writing
|
||
|
|
- Reasoning and problem-solving tasks
|
||
|
|
- Instruction following and tool use - but expect significant degradation when compared to models optimized for this task
|
||
|
|
|
||
|
|
## Limitations
|
||
|
|
|
||
|
|
- This is an SFT checkpoint without preference alignment (DPO). Outputs may not always match user expectations for tone or safety.
|
||
|
|
- The model was trained with a specific data mix and custom format. Results with other chat templates may vary.
|
||
|
|
- No formal benchmarks have been run. Evaluate on your own use cases.
|
||
|
|
|
||
|
|
## Details
|
||
|
|
|
||
|
|
- **Architecture**: Qwen3-14B (14B dense)
|
||
|
|
- **Base model**: [aimeri/spoomplesmaxx-base-qwen3-14b](https://huggingface.co/aimeri/spoomplesmaxx-base-qwen3-14b) (CPT checkpoint)
|
||
|
|
- **Context**: Up to 128K tokens (inherited from Qwen3 but trained on a max of 32K tokens)
|
||
|
|
- **Developer**: [aimeri](https://huggingface.co/aimeri)
|