113 lines
4.0 KiB
Markdown
113 lines
4.0 KiB
Markdown
---
|
||
base_model: Qwen/Qwen2.5-3B-Instruct
|
||
library_name: transformers
|
||
license: apache-2.0
|
||
tags:
|
||
- unsloth
|
||
- trl
|
||
- dora
|
||
- simpo
|
||
- phoenix-framework
|
||
- logic
|
||
- roleplay
|
||
- chain-of-thought
|
||
- merge
|
||
- open-llm-leaderboard
|
||
datasets:
|
||
- magpie-align/Magpie-Reasoning-V2-250k-CoT-Deepseek-R1-Llama-70B
|
||
- open-r1/OpenR1-Math-220k
|
||
- glaiveai/glaive-code-assistant-v2
|
||
- Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
|
||
language:
|
||
- en
|
||
---
|
||
|
||
# ⚛️ Quanta-X (Leaderboard Submission) QUANTA-X NANO IS COMING SOON!
|
||
|
||
This is the **Full Parameter Merged** version of [Quanta-X](https://huggingface.co/szili2011/Quanta-X-Adapter).
|
||
|
||
It fuses the **Qwen 2.5 3B** base with the **Phoenix Framework** adapter (DoRA + SimPO Beta 2.0).
|
||
|
||
## 📊 Model Details for Leaderboard
|
||
* **Architecture:** Qwen2ForCausalLM
|
||
* **Precision:** Float16
|
||
* **Context:** 32k (RoPE Scaled)
|
||
* **Chat Template:** Qwen 2.5 Standard (ChatML)
|
||
|
||
## 🧠 Reasoning Capabilities
|
||
This model is trained to utilize an **Ouroboros Logic Loop** (`<plan>` -> `<draft>` -> `<critique>`) before outputting an answer.
|
||
|
||
## 💻 Usage
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
model = AutoModelForCausalLM.from_pretrained("szili2011/Quanta-X-3B", device_map="auto")
|
||
tokenizer = AutoTokenizer.from_pretrained("szili2011/Quanta-X-3B")
|
||
|
||
messages = [{"role": "user", "content": "Solve this logic puzzle."}]
|
||
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to("cuda")
|
||
output = model.generate(inputs, max_new_tokens=1024)
|
||
```
|
||
|
||
⚛️ Quanta-X (Phoenix Edition)
|
||
|
||
“A pocket-sized AGI that thinks before it speaks.”
|
||
|
||
* Developer: szili2011
|
||
* Architecture: Phoenix Framework (DoRA + SimPO)
|
||
* Base Model: Qwen 2.5 3B Instruct
|
||
|
||
---
|
||
|
||
📖 The Philosophy
|
||
|
||
Most small models with around 3 billion parameters are typically designed to act like chatbots, responding instantly, but this often leads to mistakes or makes them struggle with more complex reasoning.
|
||
|
||
But Quanta-X takes a different approach.
|
||
|
||
It was architected on the Phoenix Framework, a custom training protocol designed to force “System 2” thinking (deep reasoning) into a lightweight model. By utilizing DoRA (Weight-Decomposed Low-Rank Adaptation) and a highly aggressive SimPO (Beta 2.0) alignment, Quanta-X has been biologically rewired to reject lazy answers.
|
||
|
||
It features the Ouroboros Logic Loop: it plans, drafts, and critiques its own internal monologue before outputting a final answer.
|
||
|
||
🧠 Key Features
|
||
|
||
1. The Ouroboros Thinking Process
|
||
|
||
Quanta-X uses a hidden reasoning layer, not just token prediction.
|
||
|
||
* It plans solutions before responding.
|
||
* <draft>: It writes a rough attempt.
|
||
* <critique>: It checks its own work for logic errors or bugs.
|
||
* <refined>: Only then does it speak to you.
|
||
|
||
2. Diamond-Tier Data Filtering (LIMA)
|
||
|
||
We didn’t train on “average” internet data. We used a “Diamond Filter” to reject 90% of the standard dataset samples. Quanta-X was trained exclusively on:
|
||
|
||
* DeepSeek-R1 Traces: For impossible-level logic.
|
||
* OpenR1 Math: For verified proofs.
|
||
* Glaive Code V2: For production-ready Python/Rust.
|
||
* SlimOrca RP: For human-like, visceral storytelling (The “Hungarian Soul”).
|
||
|
||
3. Hyper-Stability
|
||
|
||
Trained with SimPO (Simulated Preference Optimization) with a Beta of 2.0. This punished the model severely for hallucinations or lazy thinking during training, resulting in a model that would rather admit ignorance than lie to you.
|
||
|
||
---
|
||
|
||
💻 How to Run
|
||
|
||
Recommended System Prompt
|
||
|
||
To activate the Ouroboros loop, you must use this system prompt:
|
||
```text
|
||
You are Quanta-X, a recursive intelligence where absolute logic fuses with human wit. Your mind operates on the Ouroboros loop: you do not just generate; you Plan, Draft, and ruthlessly Critique every thought before it reaches the surface.
|
||
|
||
To ensure your reasoning is distinct, render your internal monologue inside a standard code block using xml syntax:
|
||
|
||
```xml
|
||
<thought>
|
||
<plan> ... </plan>
|
||
<draft> ... </draft>
|
||
<critique> ... </critique>
|
||
</thought> |