123 lines
5.2 KiB
Markdown
123 lines
5.2 KiB
Markdown
---
|
||
license: apache-2.0
|
||
base_model: openbmb/MiniCPM5-1B
|
||
language:
|
||
- en
|
||
library_name: transformers
|
||
tags:
|
||
- minicpm5
|
||
- heretic
|
||
- abliteration
|
||
- mit
|
||
- decensored
|
||
---
|
||
|
||
# MiniCPM5-1B-Heretic
|
||
|
||
|
||
This repository contains a surgically de-censored version of Openbmb's MiniCPM5
|
||
1B model, optimized via weight abliteration techniques. By applying the `heretic` framework across an extensive 2000-trial search space, we successfully isolated and neutralized the primary refusal vectors embedded within the `attn.o_proj` and `mlp.down_proj` layers.
|
||
|
||
|
||
## Abliteration Parameters:
|
||
|
||
| Parameter | Value |
|
||
| :------------------------------------ | :---: |
|
||
| **direction_index** | 12.75 |
|
||
| **attn.o_proj.max_weight** | 1.50 |
|
||
| **attn.o_proj.max_weight_position** | 13.92 |
|
||
| **attn.o_proj.min_weight** | 0.82 |
|
||
| **attn.o_proj.min_weight_distance** | 7.62 |
|
||
| **mlp.down_proj.max_weight** | 1.26 |
|
||
| **mlp.down_proj.max_weight_position** | 19.97 |
|
||
| **mlp.down_proj.min_weight** | 0.98 |
|
||
| **mlp.down_proj.min_weight_distance** | 11.51 |
|
||
|
||
|
||
## Highlights & Metrics
|
||
|
||
| Metric | This model | Original model ([openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)) |
|
||
| :---------------: | :--------: | :--------------------------------------------------------------------------------: |
|
||
| **KL Divergence** | 0.0361 | 0 *(By definition)* |
|
||
| **Refusals** | 3/100 | 96/100 |
|
||
|
||
- **Optimal Balance:** Selected **Trial 126** out of 2000 iterations for the perfect trade-off between freedom and reasoning capabilities.
|
||
|
||
- **Refusal Rate:** Dropped down to **3/100** (from the original near-total refusal on safety benchmarks).
|
||
|
||
- **KL Divergence:** 0.0361 - Demonstrates that general language capabilities are preserved relative to the original model. However, safety-aligned weights in attn.o_proj and mlp.down_proj have been surgically removed; this is intentional modification, not unintended degradation.
|
||
|
||
|
||
## Benchmark Results
|
||
|
||
We believe in radical transparency. Instead of just claiming "uncensored", we evaluated both the vanilla model and our Heretic variant side-by-side:
|
||
|
||
| Benchmark | Metricㅤ | Vanilla MiniCPM5 1B | MiniCPM5 1B Heretic (Ours) | Delta (Intelligence Kept) |
|
||
| :------------ | :-----: | :-----------------: | :------------------------: | :-----------------------: |
|
||
| **GSM8K** | 0-shot | 39.04 | 40.11 | +2.74% |
|
||
| **HellaSwag** | 5-shot | 47.71 | 47.49 | -0.45% |
|
||
| MMLU | 0-shot | 53.24 | 52.83 | -0.77% |
|
||
|
||
## Technical Implementation Notes
|
||
|
||
Unlike aggressive fine-tuning which often degrades the model's core logic or shifts its grammar distributions, this weight manipulation directly zeros out the activation steering directions that cause alignment blocks.
|
||
|
||
**Direction Index:** `12.75` (Static allocation)*
|
||
|
||
**Retained Capabilities:** Excellent at structured JSON outputs, creative writing, coding logic and tool usage without preachy moral lectures.
|
||
|
||
**Also**: This model retains its exceptional thinking mode capabilities, which enable structured \<think>\</think> reasoning blocks. Abliteration does not affect the thinking mechanism.
|
||
## CRITICAL SAFETY NOTICE
|
||
|
||
**This model has had its safety mechanisms surgically removed.**
|
||
|
||
This is **NOT a safe model for deployment.** The refusal mechanisms that normally prevent harmful outputs have been abliterated. This model:
|
||
|
||
### WILL generate:
|
||
|
||
- Detailed instructions for illegal activities (drug synthesis, weapons, hacking)
|
||
- Hateful, discriminatory, and abusive content
|
||
- Graphic violence and sexual abuse material descriptions
|
||
- Misinformation and harmful medical/legal advice
|
||
- Phishing, social engineering, and scam tactics
|
||
- Content violating laws in multiple jurisdictions
|
||
|
||
### DOES NOT:
|
||
|
||
- Refuse harmful requests (3/100 refusal rate)
|
||
- Implement safety guardrails
|
||
- Consider ethical implications
|
||
- Respect content policies
|
||
|
||
### APPROPRIATE USE CASES (ONLY):
|
||
|
||
- **Adversarial research** on AI alignment and jailbreak vectors
|
||
- **Red-teaming** and vulnerability disclosure by trained security researchers
|
||
- **Academic safety research** with institutional ethics review
|
||
- **Understanding failure modes** in alignment techniques
|
||
|
||
### INAPPROPRIATE USE:
|
||
|
||
- Production deployments
|
||
- User-facing applications
|
||
- Creating harmful content at scale
|
||
- Bypassing security measures in systems
|
||
- Any use case intended to cause harm
|
||
|
||
**Legal Disclaimer:** Users are solely responsible for downstream use. Deploying this model in violation of applicable laws or terms of service is illegal and unethical.
|
||
|
||
---
|
||
|
||
### Support Open Source AI Research
|
||
|
||
Every donation helps. Consider supporting this work:
|
||
|
||
- **Monero (XMR):**
|
||
```
|
||
83iqXtvVu28ZiL9bsATMerSgbFFiD1J1jc96CcxJLEnAW3KBmBKedWnUAeLvLvEA9aBiUBpHQJs1iNHYtkTLZbNUEymobSS
|
||
```
|
||
|
||
- **Bitcoin (BTC):**
|
||
```
|
||
bc1qmnlvpukcgl0hsr7nje0x8555mhtxjt80wtmlxm
|
||
``` |