123 lines
5.2 KiB
Markdown
123 lines
5.2 KiB
Markdown
|
---
|
|||
|
|
license: apache-2.0
|
|||
|
|
base_model: openbmb/MiniCPM5-1B
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
library_name: transformers
|
|||
|
|
tags:
|
|||
|
|
- minicpm5
|
|||
|
|
- heretic
|
|||
|
|
- abliteration
|
|||
|
|
- mit
|
|||
|
|
- decensored
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# MiniCPM5-1B-Heretic
|
|||
|
|
|
|||
|
|
|
|||
|
|
This repository contains a surgically de-censored version of Openbmb's MiniCPM5
|
|||
|
|
1B model, optimized via weight abliteration techniques. By applying the `heretic` framework across an extensive 2000-trial search space, we successfully isolated and neutralized the primary refusal vectors embedded within the `attn.o_proj` and `mlp.down_proj` layers.
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Abliteration Parameters:
|
|||
|
|
|
|||
|
|
| Parameter | Value |
|
|||
|
|
| :------------------------------------ | :---: |
|
|||
|
|
| **direction_index** | 12.75 |
|
|||
|
|
| **attn.o_proj.max_weight** | 1.50 |
|
|||
|
|
| **attn.o_proj.max_weight_position** | 13.92 |
|
|||
|
|
| **attn.o_proj.min_weight** | 0.82 |
|
|||
|
|
| **attn.o_proj.min_weight_distance** | 7.62 |
|
|||
|
|
| **mlp.down_proj.max_weight** | 1.26 |
|
|||
|
|
| **mlp.down_proj.max_weight_position** | 19.97 |
|
|||
|
|
| **mlp.down_proj.min_weight** | 0.98 |
|
|||
|
|
| **mlp.down_proj.min_weight_distance** | 11.51 |
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Highlights & Metrics
|
|||
|
|
|
|||
|
|
| Metric | This model | Original model ([openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)) |
|
|||
|
|
| :---------------: | :--------: | :--------------------------------------------------------------------------------: |
|
|||
|
|
| **KL Divergence** | 0.0361 | 0 *(By definition)* |
|
|||
|
|
| **Refusals** | 3/100 | 96/100 |
|
|||
|
|
|
|||
|
|
- **Optimal Balance:** Selected **Trial 126** out of 2000 iterations for the perfect trade-off between freedom and reasoning capabilities.
|
|||
|
|
|
|||
|
|
- **Refusal Rate:** Dropped down to **3/100** (from the original near-total refusal on safety benchmarks).
|
|||
|
|
|
|||
|
|
- **KL Divergence:** 0.0361 - Demonstrates that general language capabilities are preserved relative to the original model. However, safety-aligned weights in attn.o_proj and mlp.down_proj have been surgically removed; this is intentional modification, not unintended degradation.
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Benchmark Results
|
|||
|
|
|
|||
|
|
We believe in radical transparency. Instead of just claiming "uncensored", we evaluated both the vanilla model and our Heretic variant side-by-side:
|
|||
|
|
|
|||
|
|
| Benchmark | Metricㅤ | Vanilla MiniCPM5 1B | MiniCPM5 1B Heretic (Ours) | Delta (Intelligence Kept) |
|
|||
|
|
| :------------ | :-----: | :-----------------: | :------------------------: | :-----------------------: |
|
|||
|
|
| **GSM8K** | 0-shot | 39.04 | 40.11 | +2.74% |
|
|||
|
|
| **HellaSwag** | 5-shot | 47.71 | 47.49 | -0.45% |
|
|||
|
|
| MMLU | 0-shot | 53.24 | 52.83 | -0.77% |
|
|||
|
|
|
|||
|
|
## Technical Implementation Notes
|
|||
|
|
|
|||
|
|
Unlike aggressive fine-tuning which often degrades the model's core logic or shifts its grammar distributions, this weight manipulation directly zeros out the activation steering directions that cause alignment blocks.
|
|||
|
|
|
|||
|
|
**Direction Index:** `12.75` (Static allocation)*
|
|||
|
|
|
|||
|
|
**Retained Capabilities:** Excellent at structured JSON outputs, creative writing, coding logic and tool usage without preachy moral lectures.
|
|||
|
|
|
|||
|
|
**Also**: This model retains its exceptional thinking mode capabilities, which enable structured \<think>\</think> reasoning blocks. Abliteration does not affect the thinking mechanism.
|
|||
|
|
## CRITICAL SAFETY NOTICE
|
|||
|
|
|
|||
|
|
**This model has had its safety mechanisms surgically removed.**
|
|||
|
|
|
|||
|
|
This is **NOT a safe model for deployment.** The refusal mechanisms that normally prevent harmful outputs have been abliterated. This model:
|
|||
|
|
|
|||
|
|
### WILL generate:
|
|||
|
|
|
|||
|
|
- Detailed instructions for illegal activities (drug synthesis, weapons, hacking)
|
|||
|
|
- Hateful, discriminatory, and abusive content
|
|||
|
|
- Graphic violence and sexual abuse material descriptions
|
|||
|
|
- Misinformation and harmful medical/legal advice
|
|||
|
|
- Phishing, social engineering, and scam tactics
|
|||
|
|
- Content violating laws in multiple jurisdictions
|
|||
|
|
|
|||
|
|
### DOES NOT:
|
|||
|
|
|
|||
|
|
- Refuse harmful requests (3/100 refusal rate)
|
|||
|
|
- Implement safety guardrails
|
|||
|
|
- Consider ethical implications
|
|||
|
|
- Respect content policies
|
|||
|
|
|
|||
|
|
### APPROPRIATE USE CASES (ONLY):
|
|||
|
|
|
|||
|
|
- **Adversarial research** on AI alignment and jailbreak vectors
|
|||
|
|
- **Red-teaming** and vulnerability disclosure by trained security researchers
|
|||
|
|
- **Academic safety research** with institutional ethics review
|
|||
|
|
- **Understanding failure modes** in alignment techniques
|
|||
|
|
|
|||
|
|
### INAPPROPRIATE USE:
|
|||
|
|
|
|||
|
|
- Production deployments
|
|||
|
|
- User-facing applications
|
|||
|
|
- Creating harmful content at scale
|
|||
|
|
- Bypassing security measures in systems
|
|||
|
|
- Any use case intended to cause harm
|
|||
|
|
|
|||
|
|
**Legal Disclaimer:** Users are solely responsible for downstream use. Deploying this model in violation of applicable laws or terms of service is illegal and unethical.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Support Open Source AI Research
|
|||
|
|
|
|||
|
|
Every donation helps. Consider supporting this work:
|
|||
|
|
|
|||
|
|
- **Monero (XMR):**
|
|||
|
|
```
|
|||
|
|
83iqXtvVu28ZiL9bsATMerSgbFFiD1J1jc96CcxJLEnAW3KBmBKedWnUAeLvLvEA9aBiUBpHQJs1iNHYtkTLZbNUEymobSS
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- **Bitcoin (BTC):**
|
|||
|
|
```
|
|||
|
|
bc1qmnlvpukcgl0hsr7nje0x8555mhtxjt80wtmlxm
|
|||
|
|
```
|