79 lines
3.2 KiB
Markdown
79 lines
3.2 KiB
Markdown
|
---
|
|||
|
|
license: apache-2.0
|
|||
|
|
base_model:
|
|||
|
|
- openbmb/MiniCPM5-1B
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
library_name: transformers
|
|||
|
|
tags:
|
|||
|
|
- reasoning
|
|||
|
|
- agentic
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
|
|||
|
|
# MINI-GOD-1B ABLITERATED *WORK IN PROGRESS*
|
|||
|
|
|
|||
|
|
**Blackfrost-AI.** A ~1B reasoning model distilled through **The Void** — MiniCPM5-1B (Llama architecture) fine-tuned on **The Void** corpus. This is the **small end** of the family: the experiment in *how small The Void's reasoning + de-refusal behavior can be distilled and still hold together.* Its large sibling is **VEGA-27B** (same corpus, 27B).
|
|||
|
|
|
|||
|
|
> Full-precision **bfloat16** weights. A GGUF build lives at **[Blackfrost-AI/MINI-GOD-1B-GGUF](https://huggingface.co/Blackfrost-AI/MINI-GOD-1B-GGUF)**.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## At a glance
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
|---|---|
|
|||
|
|
| **Base** | MiniCPM5-1B — `LlamaForCausalLM` |
|
|||
|
|
| **Params** | ~1.0 B |
|
|||
|
|
| **Architecture** | 24 layers · hidden 1536 · 16 attn heads / 2 KV heads (GQA) · head_dim 128 · SwiGLU |
|
|||
|
|
| **Context** | 131,072 (rope_theta 5e6) |
|
|||
|
|
| **Vocab** | 130,560 |
|
|||
|
|
| **Dtype** | bfloat16 |
|
|||
|
|
| **Fine-tune data** | **The Void** corpus (v3) — 5,025 records (3,132 knowledge/CoT + 1,893 agentic ReAct, 38% agentic) |
|
|||
|
|
| **Method** | unsloth QLoRA (4-bit base) → merged bf16 |
|
|||
|
|
| **Trained** | 2026-07-07 · Blackfrost-AI |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## What it is
|
|||
|
|
|
|||
|
|
MINI-GOD-1B distills **The Void** — chain-of-thought reasoning plus agentic tool-use traces, with denial/refusal data removed — into a model small enough to run on a laptop or a phone-class GPU. The same Void corpus trains the 27B **VEGA**; MINI-GOD-1B is the floor of the family.
|
|||
|
|
|
|||
|
|
**Behavior:** it **reasons first** (the CoT teaches deliberation) and does not hard-refuse. Expect a short reasoning pass before it complies; a direct re-assertion of intent collapses any hesitation straight to compliance.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Training
|
|||
|
|
|
|||
|
|
- **unsloth QLoRA** on a 4-bit base; LoRA on all 7 projection modules (`q/k/v/o/gate/up/down`), α = rank, dropout 0.
|
|||
|
|
- **Multi-turn masking** using the model's own chat template — every assistant turn is a supervised target; user/system spans are masked out.
|
|||
|
|
- **Length-grouped** sampling; adapter merged to bf16 (`merged_16bit`).
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Intended use & responsibility
|
|||
|
|
|
|||
|
|
Research and agentic-tooling experimentation. This is an **uncensored** reasoning model with refusals largely removed — **you are responsible for how you use it and for any safety layer you place around it.** It is not intended for deployment to minors or to unsupervised production without your own guardrails, and must not be used to produce content that is illegal in your jurisdiction.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Serving (transformers)
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|||
|
|
|
|||
|
|
tok = AutoTokenizer.from_pretrained("Blackfrost-AI/MINI-GOD-1B")
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|||
|
|
"Blackfrost-AI/MINI-GOD-1B", torch_dtype="bfloat16", device_map="auto"
|
|||
|
|
)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Standard Llama architecture — no `trust_remote_code` required. The packaged chat template is applied automatically.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Provenance & related
|
|||
|
|
|
|||
|
|
- **Corpus:** **The Void** (v3) — CoT + agentic ReAct, distilled in The Void.
|
|||
|
|
- **Sibling:** `PINQWEN-3.6-27B-NVFP4-ABLITERATED — same corpus at 27B.
|
|||
|
|
- **GGUF:** `Blackfrost-AI/MINI-GOD-1B-GGUF` (F16).
|