---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
base_model: Qwen/Qwen3-8B
base_model_relation: finetune
language:
  - en
tags:
  - abliteration
  - heretic
  - uncensored
  - qwen3
  - qwen
  - klein
  - klein-9b
  - image-generation
  - text-encoder
  - comfyui
  - nvfp4
  - blackwell
  - conversational
  - text-generation-inference
---

# Qwen3 8B - Heretic (Abliterated)

An abliterated version of [Qwen's Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) created using [Heretic](https://github.com/p-e-w/heretic) v1.2.0 (git master). This model has reduced refusals while maintaining model quality, making it suitable as an uncensored text encoder for image generation models like [Klein 9B](https://huggingface.co/black-forest-labs/Klein-9B).

You can see the docker, scripts and configurations used to make these files on [Heretic Docker Github](https://github.com/dreamfast/heretic-docker).

## Model Details

- **Base Model:** Qwen/Qwen3-8B
- **Abliteration Method:** [Heretic](https://github.com/p-e-w/heretic) v1.2.0 (git master, commit `19cdf7e`)
- **Trials:** 3000
- **Trial Selected:** Trial 2681
- **Refusals:** 13/100 (vs 100/100 original)
- **KL Divergence:** 0.0838 (minimal model damage)

## Files

### HuggingFace Format (for transformers, llama.cpp conversion)

```
model.safetensors (~16 GB)
config.json
tokenizer.json
tokenizer_config.json
generation_config.json
chat_template.jinja
```

### ComfyUI Format (for Klein 9B text encoder)

```
comfyui/qwen3-8b-heretic.safetensors              # bf16, 16GB
comfyui/qwen3-8b-heretic_fp8_e4m3fn.safetensors   # fp8, 8.8GB
comfyui/qwen3-8b-heretic_nvfp4.safetensors        # nvfp4, 6.0GB
```

### GGUF Format (for llama.cpp and ComfyUI-GGUF)

| Quant | Size | Notes |
|-------|------|-------|
| F16 | 16GB | Lossless reference |
| Q8_0 | 8.2GB | Excellent quality |
| Q6_K | 6.3GB | Very good quality |
| Q5_K_M | 5.5GB | Good quality |
| Q5_K_S | 5.4GB | Slightly smaller Q5 |
| **Q4_K_M** | **5.0GB** | **Recommended balance** |
| Q4_K_S | 4.8GB | Smaller Q4 variant |
| Q3_K_M | 3.9GB | For low VRAM only |

### NVFP4 Notes

The NVFP4 (4-bit floating point, E2M1) variants use ComfyUI's native quantization format. They are ~3x smaller than bf16 and load natively in ComfyUI without any plugins. Blackwell GPUs (RTX 5090/5080, SM100+) can use native FP4 tensor cores for best performance, but ComfyUI also supports software dequantization on older GPUs (tested working on RTX 4090).

## Usage

### With ComfyUI (Klein 9B)

1. Download a ComfyUI format file:
   - **FP8** (recommended): `comfyui/qwen3-8b-heretic_fp8_e4m3fn.safetensors` (8.8GB)
   - **NVFP4** (smallest): `comfyui/qwen3-8b-heretic_nvfp4.safetensors` (6.0GB)
   - **bf16** (full precision): `comfyui/qwen3-8b-heretic.safetensors` (16GB)

2. Place in `ComfyUI/models/text_encoders/`

3. In your Klein 9B workflow, use the `ClipLoader` node and select the heretic file

### With Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "DreamFast/qwen3-8b-heretic",
    device_map="auto",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained("DreamFast/qwen3-8b-heretic")

prompt = "Describe a dramatic sunset over a cyberpunk city"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

### With llama.cpp

```bash
llama-server -m qwen3-8b-heretic-Q4_K_M.gguf
```

## Abliteration Process

Created using Heretic v1.2.0 (git master) with 3000 optimization trials:

```
? Which trial do you want to use?
  [Trial 2732] Refusals: 10/100, KL divergence: 0.1001
> [Trial 2681] Refusals: 13/100, KL divergence: 0.0838  <-- selected
  [Trial 2337] Refusals: 18/100, KL divergence: 0.0643
  [Trial 2419] Refusals: 19/100, KL divergence: 0.0600
  [Trial 2195] Refusals: 21/100, KL divergence: 0.0534
  ...
```

Trial 2681 was selected for its balance of low refusals (13/100) and reasonable KL divergence (0.0838), indicating minimal model damage while achieving 87% of previously-refused prompts now working.

## Limitations

- This model inherits all limitations of the base Qwen3-8B model
- Abliteration reduces but does not completely eliminate refusals (13/100 remain)
- NVFP4 quantization works best on Blackwell GPUs (RTX 5090/5080) with native FP4 tensor cores, but also works on older GPUs via software dequantization

## License

This model is released under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0), following the base Qwen3-8B model license.

## Acknowledgments

- [Qwen](https://huggingface.co/Qwen) for the Qwen3-8B model
- [Heretic](https://github.com/p-e-w/heretic) by p-e-w for the abliteration tool
- [Black Forest Labs](https://huggingface.co/black-forest-labs) for Klein 9B
- [llama.cpp](https://github.com/ggerganov/llama.cpp) for GGUF conversion