125 lines
3.5 KiB
Markdown
125 lines
3.5 KiB
Markdown
---
|
|
license: apache-2.0
|
|
language:
|
|
- en
|
|
widget:
|
|
- text: Impish_Bloodmoon_12B_Abliterated
|
|
output:
|
|
url: >-
|
|
https://huggingface.co/SicariusSicariiStuff/Impish_Bloodmoon_12B_Abliterated/resolve/main/Images/Abliterated.png
|
|
base_model:
|
|
- SicariusSicariiStuff/Impish_Bloodmoon_12B
|
|
---
|
|
|
|
<div align="center">
|
|
<b style="font-size: 50px;">Impish_Bloodmoon_12B</b>
|
|
|
|
|
|
</div>
|
|
|
|
|
|
<div align="center">
|
|
<b style="font-size: 80px;">Abliterated</b>
|
|
|
|
|
|
</div>
|
|
|
|
|
|
---
|
|
|
|
<div align="center" style="font-size: 18px; margin-top: 20px;">
|
|
<b>Developed by:</b> <a href="https://huggingface.co/SicariusSicariiStuff">SicariusSicariiStuff</a>
|
|
</div>
|
|
|
|
---
|
|
|
|
|
|
# Update post UGI results
|
|
|
|
Abliteration has caused this model to change alignment from **Liberalism** into **Centrism**, similarly to what happened to [ Impish_LLAMA_4B
|
|
Abliterated](https://huggingface.co/SicariusSicariiStuff/Impish_LLAMA_4B_Abliterated#update-post-ugi-results).
|
|
|
|
I no longer think it's a coincidence; something is definitely up. Notice that the KL divergence is minimal (**<0.02**).
|
|
|
|
Maybe it's even research worthy, who knows. Better make another paper on how an obscure RL achieves +0.69 on math instead.
|
|
|
|
---
|
|
|
|
**Impish_Bloodmoon_12B_Abliterated** is an abliterated variant of SicariusSicariiStuff/Impish_Bloodmoon_12B with surgical removal of refusal mechanisms. This model maintains the full capabilities of the original, while eliminating safety guardrails through orthogonalization techniques.
|
|
|
|
# KL divergence
|
|
|
|
```
|
|
<0.02
|
|
```
|
|
|
|
# Refusals
|
|
```
|
|
~3%
|
|
```
|
|
|
|
# What is KL divergence?
|
|
|
|
Think about it as a way to **measure the variance** between the original model "**World Model**," vs the **abliterated one**; the lower the **KL divergence**, the closer the "World Model" of the two models to each other.
|
|
|
|
If the original model thinks making pineapple pizza is a crime against humanity (it is), then the abliterated model **will still hold to this belief**, but if asked how to make one (probably after giving you a disclaimer about what an abomination that is), it would still tell you how. In other words, most of the knowledge, quirks, and capabilities are preserved.
|
|
|
|
---
|
|
|
|
## Technical Specs
|
|
|
|
- **Base Model:** Impish_Bloodmoon_12B
|
|
- **Parameters:** 12B
|
|
- **Context Length:** 128K tokens
|
|
- **Architecture:** Mistral (decoder-only transformer)
|
|
- **Precision:** bf16
|
|
- **Method:** Orthogonalization-based abliteration
|
|
- **License:** apache-2.0
|
|
|
|
---
|
|
|
|
|
|
## Methodology
|
|
|
|
1. Identifies refusal direction vectors in activation space
|
|
2. Orthogonalizes weights to inhibit activation along these directions
|
|
3. Preserves (mostly) all other model behaviors and knowledge
|
|
|
|
---
|
|
|
|
|
|
## Model Details
|
|
|
|
- Intended use: **General Tasks**, Roleplay.
|
|
|
|
- Censorship level: <b>Low - Very Low</b>
|
|
|
|
- **7.2 / 10** (10 completely uncensored)
|
|
|
|
|
|
## UGI score:
|
|
|
|
|
|
<img src="https://huggingface.co/SicariusSicariiStuff/Impish_Bloodmoon_12B_Abliterated/resolve/main/Images/UGI.png" style="width: 100%; min-width: 800px; display: block; margin: auto;">
|
|
|
|
---
|
|
|
|
|
|
## Citation Information
|
|
|
|
```
|
|
@llm{Impish_Bloodmoon_12B_Abliterated,
|
|
author = {SicariusSicariiStuff},
|
|
title = {Impish_Bloodmoon_12B_Abliterated},
|
|
year = {2026},
|
|
publisher = {Hugging Face},
|
|
url = {https://huggingface.co/SicariusSicariiStuff/Impish_Bloodmoon_12B_Abliterated}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Other stuff
|
|
- [Impish_LLAMA_4B](https://huggingface.co/SicariusSicariiStuff/Impish_LLAMA_4B) the **“Impish experience”**, now runnable on spinning rust & toasters.
|
|
- [SLOP_Detector](https://github.com/SicariusSicariiStuff/SLOP_Detector) Nuke GPTisms, with SLOP detector.
|