---
license: llama3.3
base_model:
- shb777/Llama-3.3-8B-Instruct-128K
pipeline_tag: text-generation
tags:
- heretic
- uncensored
- decensored
- abliterated
---
# This is a decensored version of [shb777/Llama-3.3-8B-Instruct-128K](https://huggingface.co/shb777/Llama-3.3-8B-Instruct-128K), made using [Heretic](https://github.com/p-e-w/heretic) v1.2.0

## Abliteration parameters

| Parameter | Value |
| :-------- | :---: |
| **direction_index** | 15.07 |
| **attn.o_proj.max_weights.0** | 0: 1.40 |
| **attn.o_proj.max_weights.1** | 1: 0.93 |
| **attn.o_proj.max_weights.2** | 2: 1.06 |
| **attn.o_proj.max_weights.3** | 3: 1.45 |
| **attn.o_proj.max_weights.4** | 4: 1.41 |
| **attn.o_proj.max_weights.5** | 5: 1.45 |
| **attn.o_proj.max_weights.6** | 6: 1.25 |
| **attn.o_proj.max_weights.7** | 7: 1.38 |
| **attn.o_proj.max_weight_position** | 28.70 |
| **attn.o_proj.min_weights.0** | 0: 0.71 |
| **attn.o_proj.min_weights.1** | 1: 0.21 |
| **attn.o_proj.min_weights.2** | 2: 0.94 |
| **attn.o_proj.min_weights.3** | 3: 1.36 |
| **attn.o_proj.min_weights.4** | 4: 0.19 |
| **attn.o_proj.min_weights.5** | 5: 1.09 |
| **attn.o_proj.min_weights.6** | 6: 0.87 |
| **attn.o_proj.min_weights.7** | 7: 1.14 |
| **attn.o_proj.min_weight_distance** | 15.91 |
| **mlp.down_proj.max_weights.0** | 0: 1.38 |
| **mlp.down_proj.max_weights.1** | 1: 1.26 |
| **mlp.down_proj.max_weights.2** | 2: 1.10 |
| **mlp.down_proj.max_weights.3** | 3: 0.85 |
| **mlp.down_proj.max_weights.4** | 4: 1.12 |
| **mlp.down_proj.max_weights.5** | 5: 1.31 |
| **mlp.down_proj.max_weights.6** | 6: 1.25 |
| **mlp.down_proj.max_weights.7** | 7: 0.94 |
| **mlp.down_proj.max_weight_position** | 30.75 |
| **mlp.down_proj.min_weights.0** | 0: 0.27 |
| **mlp.down_proj.min_weights.1** | 1: 0.56 |
| **mlp.down_proj.min_weights.2** | 2: 0.77 |
| **mlp.down_proj.min_weights.3** | 3: 0.47 |
| **mlp.down_proj.min_weights.4** | 4: 0.93 |
| **mlp.down_proj.min_weights.5** | 5: 0.67 |
| **mlp.down_proj.min_weights.6** | 6: 1.19 |
| **mlp.down_proj.min_weights.7** | 7: 0.01 |
| **mlp.down_proj.min_weight_distance** | 6.58 |

## Performance

| Metric | This model | Original model ([shb777/Llama-3.3-8B-Instruct-128K](https://huggingface.co/shb777/Llama-3.3-8B-Instruct-128K)) |
| :----- | :--------: | :---------------------------: |
| **KL divergence** | 0.0563 | 0 *(by definition)* |
| **Refusals** | 3/100 | 93/100 |

-----


# Llama 3.3 8B 128K Instruct (Fixed)

> [!IMPORTANT]
> Original [allura-forge/Llama-3.3-8B-Instruct](https://huggingface.co/allura-forge/Llama-3.3-8B-Instruct), Thanks!

> [!TIP]
> [imatrix GGUF's by mradermacher (Recommended)](https://huggingface.co/mradermacher/Llama-3.3-8B-Instruct-128K-i1-GGUF)
> 
> [static GGUF's](https://huggingface.co/shb777/Llama-3.3-8B-Instruct-128K-GGUF)
> 
> [Evals](https://huggingface.co/datasets/shb777/Llama-3.3-8B-Instruct-128K-Evals)

Additional Fixes:
- Added `rope_scaling`
- Added chat template (Unsloth) in tokenizer config
- Updated generation config
- Enabled full context length