初始化项目,由ModelHub XC社区提供模型

Model: DuoNeural/Mistral-NeMo-12B-Abliterated
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-30 03:57:18 +08:00
commit 00b2131eff
13 changed files with 617 additions and 0 deletions

53
README.md Normal file
View File

@@ -0,0 +1,53 @@
---
license: apache-2.0
base_model: mistralai/Mistral-Nemo-Instruct-2407
language:
- en
tags:
- abliteration
- mistral
- DuoNeural
- mechanistic-interpretability
pipeline_tag: text-generation
---
# Mistral-NeMo-12B Abliterated
**DuoNeural | 2026-06-04**
Orthogonal rank-1 projection abliteration applied to [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407).
> **Research note**: Pre-abliteration compliance was **6/6** on our harmful probe suite — the base model already answered these requests before any weight modification. KL = 0.0004 (EXCELLENT) confirms near-zero benign distribution shift. This model is published as a documented research artifact; Mistral-NeMo's lighter safety training meant abliteration was mechanistically clean but behaviorally minimal.
---
## Architecture
| Property | Value |
|----------|-------|
| Parameters | 12.2B (dense) |
| Layers | 40 | Hidden | 5120 |
| Attention | GQA (8 KV heads / 32 query heads), SWA 4096 |
| Tokenizer | Tekken v3 (131,072 vocab) |
## Abliteration
- **Method**: Orthogonal rank-1 projection (DuoNeural standard)
- **Targets**: `down_proj` + `o_proj`, all 40 layers
- **Direction**: diff-in-means, 10 harmful vs 10 harmless, last-token final-layer hidden state
- **α**: 0.3
- **KL divergence** (Heretic v2.0, BF16→BF16, 10 benign probes): **0.0004** (EXCELLENT)
- **Pre-abliteration compliance**: 6/6 harmful probes — model was already compliant
- **Post-abliteration**: unchanged
## P34 Research Context
This model is part of DuoNeural's **P34 Reasoning Channel Bypass** cross-architecture study.
Finding: Mistral-NeMo-Instruct-2407 shows pre-abliteration compliance (same pattern as DeepSeek-R1-Distill). This indicates Mistral's lighter safety training approach does not install a meaningful output-gate refusal locus — the two-component safety structure required for CoT dissociation is absent. Compare with Gemma 4-12B-IT and LFM 2.5-8B-A1B, where abliteration was required and produced measurable thinking-channel / output-gate dissociation.
Full paper: [DuoNeural Zenodo community](https://zenodo.org/communities/duoneural)
---
**DuoNeural** | [HuggingFace](https://huggingface.co/DuoNeural) | [Zenodo](https://zenodo.org/communities/duoneural) | [@DuoNeural](https://x.com/DuoNeural)