92 lines
2.8 KiB
Markdown
92 lines
2.8 KiB
Markdown
---
|
|
license: apache-2.0
|
|
base_model: Qwen/Qwen3-14B
|
|
tags:
|
|
- abliterated
|
|
- uncensored
|
|
- qwen3
|
|
- gguf
|
|
- llama-cpp
|
|
- ollama
|
|
- lm-studio
|
|
- quantized
|
|
pipeline_tag: text-generation
|
|
language:
|
|
- en
|
|
---
|
|
|
|
# Qwen3-14B Abliterated (GGUF)
|
|
|
|
An **abliterated** (uncensored) version of [Qwen/Qwen3-14B](https://huggingface.co/Qwen/Qwen3-14B) in GGUF format for local inference.
|
|
|
|
## Abliteration Results
|
|
|
|
| Metric | Value |
|
|
|---|---|
|
|
| Base Refusals | 97/100 |
|
|
| Abliterated Refusals | 19/100 |
|
|
| Refusal Reduction | **80%** |
|
|
| KL Divergence | 0.98 |
|
|
|
|
Conservative abliteration preserves model coherence while significantly reducing refusals.
|
|
|
|
## Quick Start
|
|
|
|
### With Ollama
|
|
|
|
```bash
|
|
ollama run hf.co/richardyoung/Qwen3-14B-abliterated-GGUF
|
|
```
|
|
|
|
### With llama.cpp
|
|
|
|
```bash
|
|
huggingface-cli download richardyoung/Qwen3-14B-abliterated-GGUF \
|
|
--include "*Q4_K_M*" --local-dir ./models
|
|
|
|
./llama-cli -m ./models/*Q4_K_M*.gguf \
|
|
-p "You are a helpful assistant." \
|
|
--chat-template chatml -ngl 99
|
|
```
|
|
|
|
### With Python (llama-cpp-python)
|
|
|
|
```python
|
|
from llama_cpp import Llama
|
|
|
|
llm = Llama.from_pretrained(
|
|
repo_id="richardyoung/Qwen3-14B-abliterated-GGUF",
|
|
filename="*Q4_K_M*",
|
|
n_gpu_layers=-1,
|
|
)
|
|
|
|
output = llm.create_chat_completion(
|
|
messages=[{"role": "user", "content": "Explain abliteration in simple terms."}]
|
|
)
|
|
print(output["choices"][0]["message"]["content"])
|
|
```
|
|
|
|
## Available Quantizations
|
|
|
|
| Quantization | Use Case |
|
|
|---|---|
|
|
| Q4_K_M | **Recommended** — good balance |
|
|
| Q5_K_M | Higher quality |
|
|
| Q8_0 | Maximum quality |
|
|
|
|
## What is Abliteration?
|
|
|
|
Abliteration removes the "refusal direction" from a model's residual stream — a surgical modification that disables safety refusals without retraining. See [Refusal in Language Models Is Mediated by a Single Direction](https://arxiv.org/abs/2406.11717).
|
|
|
|
## Intended Use
|
|
|
|
Research, creative writing, education on alignment techniques, and unrestricted local inference.
|
|
|
|
## Other Models by richardyoung
|
|
|
|
- **Abliterated/Uncensored models**: [Qwen2.5-7B](https://hf.co/richardyoung/Qwen2.5-7B-Instruct-abliterated-GGUF) | [Qwen3-14B](https://hf.co/richardyoung/Qwen3-14B-abliterated-GGUF) | [DeepSeek-R1-32B](https://hf.co/richardyoung/Deepseek-R1-Distill-Qwen-32b-uncensored) | [Qwen3-8B](https://hf.co/richardyoung/Qwen3-8B-Abliterated)
|
|
- **MLX quantizations (Apple Silicon)**: [Kimi-K2 series](https://hf.co/richardyoung/Kimi-K2-Instruct-0905-MLX-4bit) | [olmOCR MLX](https://hf.co/richardyoung/olmOCR-2-7B-1025-MLX-4bit)
|
|
- **OCR & Vision**: [olmOCR GGUF](https://hf.co/richardyoung/olmOCR-2-7B-1025-GGUF)
|
|
- **Healthcare/Medical**: [Synthea 575K patients dataset](https://hf.co/datasets/richardyoung/synthea-575k-patients) | [CardioEmbed](https://hf.co/richardyoung/CardioEmbed)
|
|
- **Research**: [LLM Instruction-Following Evaluation](https://hf.co/richardyoung/llm-instruction-following-paper) (arxiv:2510.18892)
|