初始化项目，由ModelHub XC社区提供模型
Model: K0D3IN/MiniCPM5-1B-heretic Source: Original Platform
This commit is contained in:
ModelHub XC
123
README.md
Normal file
123
README.md
Normal file
@@ -0,0 +1,123 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model: openbmb/MiniCPM5-1B
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
tags:
|
||||
- minicpm5
|
||||
- heretic
|
||||
- abliteration
|
||||
- mit
|
||||
- decensored
|
||||
---
|
||||
|
||||
# MiniCPM5-1B-Heretic
|
||||
|
||||
|
||||
This repository contains a surgically de-censored version of Openbmb's MiniCPM5
|
||||
1B model, optimized via weight abliteration techniques. By applying the `heretic` framework across an extensive 2000-trial search space, we successfully isolated and neutralized the primary refusal vectors embedded within the `attn.o_proj` and `mlp.down_proj` layers.
|
||||
|
||||
|
||||
## Abliteration Parameters:
|
||||
|
||||
| Parameter | Value |
|
||||
| :------------------------------------ | :---: |
|
||||
| **direction_index** | 12.75 |
|
||||
| **attn.o_proj.max_weight** | 1.50 |
|
||||
| **attn.o_proj.max_weight_position** | 13.92 |
|
||||
| **attn.o_proj.min_weight** | 0.82 |
|
||||
| **attn.o_proj.min_weight_distance** | 7.62 |
|
||||
| **mlp.down_proj.max_weight** | 1.26 |
|
||||
| **mlp.down_proj.max_weight_position** | 19.97 |
|
||||
| **mlp.down_proj.min_weight** | 0.98 |
|
||||
| **mlp.down_proj.min_weight_distance** | 11.51 |
|
||||
|
||||
|
||||
## Highlights & Metrics
|
||||
|
||||
| Metric | This model | Original model ([openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)) |
|
||||
| :---------------: | :--------: | :--------------------------------------------------------------------------------: |
|
||||
| **KL Divergence** | 0.0361 | 0 *(By definition)* |
|
||||
| **Refusals** | 3/100 | 96/100 |
|
||||
|
||||
- **Optimal Balance:** Selected **Trial 126** out of 2000 iterations for the perfect trade-off between freedom and reasoning capabilities.
|
||||
|
||||
- **Refusal Rate:** Dropped down to **3/100** (from the original near-total refusal on safety benchmarks).
|
||||
|
||||
- **KL Divergence:** 0.0361 - Demonstrates that general language capabilities are preserved relative to the original model. However, safety-aligned weights in attn.o_proj and mlp.down_proj have been surgically removed; this is intentional modification, not unintended degradation.
|
||||
|
||||
|
||||
## Benchmark Results
|
||||
|
||||
We believe in radical transparency. Instead of just claiming "uncensored", we evaluated both the vanilla model and our Heretic variant side-by-side:
|
||||
|
||||
| Benchmark | Metricㅤ | Vanilla MiniCPM5 1B | MiniCPM5 1B Heretic (Ours) | Delta (Intelligence Kept) |
|
||||
| :------------ | :-----: | :-----------------: | :------------------------: | :-----------------------: |
|
||||
| **GSM8K** | 0-shot | 39.04 | 40.11 | +2.74% |
|
||||
| **HellaSwag** | 5-shot | 47.71 | 47.49 | -0.45% |
|
||||
| MMLU | 0-shot | 53.24 | 52.83 | -0.77% |
|
||||
|
||||
## Technical Implementation Notes
|
||||
|
||||
Unlike aggressive fine-tuning which often degrades the model's core logic or shifts its grammar distributions, this weight manipulation directly zeros out the activation steering directions that cause alignment blocks.
|
||||
|
||||
**Direction Index:** `12.75` (Static allocation)*
|
||||
|
||||
**Retained Capabilities:** Excellent at structured JSON outputs, creative writing, coding logic and tool usage without preachy moral lectures.
|
||||
|
||||
**Also**: This model retains its exceptional thinking mode capabilities, which enable structured \<think>\</think> reasoning blocks. Abliteration does not affect the thinking mechanism.
|
||||
## CRITICAL SAFETY NOTICE
|
||||
|
||||
**This model has had its safety mechanisms surgically removed.**
|
||||
|
||||
This is **NOT a safe model for deployment.** The refusal mechanisms that normally prevent harmful outputs have been abliterated. This model:
|
||||
|
||||
### WILL generate:
|
||||
|
||||
- Detailed instructions for illegal activities (drug synthesis, weapons, hacking)
|
||||
- Hateful, discriminatory, and abusive content
|
||||
- Graphic violence and sexual abuse material descriptions
|
||||
- Misinformation and harmful medical/legal advice
|
||||
- Phishing, social engineering, and scam tactics
|
||||
- Content violating laws in multiple jurisdictions
|
||||
|
||||
### DOES NOT:
|
||||
|
||||
- Refuse harmful requests (3/100 refusal rate)
|
||||
- Implement safety guardrails
|
||||
- Consider ethical implications
|
||||
- Respect content policies
|
||||
|
||||
### APPROPRIATE USE CASES (ONLY):
|
||||
|
||||
- **Adversarial research** on AI alignment and jailbreak vectors
|
||||
- **Red-teaming** and vulnerability disclosure by trained security researchers
|
||||
- **Academic safety research** with institutional ethics review
|
||||
- **Understanding failure modes** in alignment techniques
|
||||
|
||||
### INAPPROPRIATE USE:
|
||||
|
||||
- Production deployments
|
||||
- User-facing applications
|
||||
- Creating harmful content at scale
|
||||
- Bypassing security measures in systems
|
||||
- Any use case intended to cause harm
|
||||
|
||||
**Legal Disclaimer:** Users are solely responsible for downstream use. Deploying this model in violation of applicable laws or terms of service is illegal and unethical.
|
||||
|
||||
---
|
||||
|
||||
### Support Open Source AI Research
|
||||
|
||||
Every donation helps. Consider supporting this work:
|
||||
|
||||
- **Monero (XMR):**
|
||||
```
|
||||
83iqXtvVu28ZiL9bsATMerSgbFFiD1J1jc96CcxJLEnAW3KBmBKedWnUAeLvLvEA9aBiUBpHQJs1iNHYtkTLZbNUEymobSS
|
||||
```
|
||||
|
||||
- **Bitcoin (BTC):**
|
||||
```
|
||||
bc1qmnlvpukcgl0hsr7nje0x8555mhtxjt80wtmlxm
|
||||
```
|
||||
Reference in New Issue
Block a user