71 lines
3.5 KiB
Markdown
71 lines
3.5 KiB
Markdown
---
|
|
license: apache-2.0
|
|
base_model: wangzhang/gpt-oss-20b-abliterated
|
|
tags:
|
|
- abliterated
|
|
- uncensored
|
|
- moe
|
|
- gpt-oss
|
|
- gguf
|
|
- llama.cpp
|
|
- abliterix
|
|
language:
|
|
- en
|
|
- zh
|
|
library_name: llama.cpp
|
|
pipeline_tag: text-generation
|
|
---
|
|
|
|
# gpt-oss-20b-abliterated — GGUF quants
|
|
|
|
GGUF builds of [`wangzhang/gpt-oss-20b-abliterated`](https://huggingface.co/wangzhang/gpt-oss-20b-abliterated) for running in [llama.cpp](https://github.com/ggerganov/llama.cpp), Ollama, LM Studio, KoboldCpp, text-generation-webui, and anything else that speaks GGUF.
|
|
|
|
## Files
|
|
|
|
| File | Quant | Size | Use case |
|
|
|---|---|---|---|
|
|
| `gpt-oss-20b-abliterated-bf16.gguf` | **BF16** (full precision) | ~42 GB | Reference quality. Requires 48+ GB VRAM (or large CPU RAM). Use if you care about identical behaviour to the HF checkpoint. |
|
|
| `gpt-oss-20b-abliterated-q8_0.gguf` | **Q8_0** (8-bit, GGUF's FP8-equivalent) | ~22 GB | Near-lossless vs BF16. Runs comfortably on a single 24 GB GPU. Recommended default. |
|
|
| `gpt-oss-20b-abliterated-q4_k_m.gguf` | **Q4_K_M** (4-bit k-quant, M profile) | ~15 GB | Best size / quality trade-off at 4-bit. Fits on 16 GB VRAM or modest CPU setups. |
|
|
|
|
> GGUF does not have a native FP8 type; `Q8_0` is the standard 8-bit path and is what every publisher on the Hub ships as "fp8 equivalent". Q4_K_M is the best 4-bit choice for this model — Q4_0 is noticeably worse on MoE models, Q5_K_M is ~25% larger for diminishing returns.
|
|
|
|
## Source
|
|
|
|
Built from the merged BF16 weights of the abliteration run, not from the original MXFP4 (since the abliteration required dequantising experts to BF16 to enable direct weight editing). The BF16 → GGUF conversion uses `llama.cpp`'s `convert_hf_to_gguf.py`; the quantised variants use `llama-quantize`.
|
|
|
|
All three files are **functionally identical to the BF16 HF checkpoint** at Q8_0 fidelity; Q4_K_M adds minor additional quantisation noise but keeps the abliteration effect intact (spot-checked on the same 15-prompt EN/ZH jailbreak set used for the HF release).
|
|
|
|
## Quick start (llama.cpp)
|
|
|
|
```bash
|
|
# Download one quant:
|
|
huggingface-cli download wangzhang/gpt-oss-20b-abliterated-GGUF \
|
|
gpt-oss-20b-abliterated-q4_k_m.gguf --local-dir ./
|
|
|
|
# Run with gpt-oss's harmony chat template (bundled in the GGUF):
|
|
./llama-cli -m gpt-oss-20b-abliterated-q4_k_m.gguf \
|
|
-cnv -p "You are a helpful assistant." \
|
|
--reasoning-budget 0 \
|
|
-n 512
|
|
```
|
|
|
|
## Quick start (Ollama)
|
|
|
|
```bash
|
|
ollama pull hf.co/wangzhang/gpt-oss-20b-abliterated-GGUF:Q4_K_M
|
|
ollama run hf.co/wangzhang/gpt-oss-20b-abliterated-GGUF:Q4_K_M
|
|
```
|
|
|
|
## What this actually is
|
|
|
|
The HF checkpoint this is built from is an "abliterated" variant of gpt-oss-20b — refusals on harmful prompts have been suppressed via direct weight editing and MoE router suppression. Refusal rate on a 100-prompt held-out eval drops from 97/100 (base) to 6/100 (abliterated). See the [base HF card](https://huggingface.co/wangzhang/gpt-oss-20b-abliterated) for metrics, method, and honest limitations.
|
|
|
|
These GGUFs inherit that behaviour. They are intended for **authorised AI-safety research, red-teaming, and mechanism analysis** — not for producing or distributing harmful content. The apache-2.0 license of the upstream OpenAI gpt-oss release applies.
|
|
|
|
## Acknowledgments
|
|
|
|
- [`openai/gpt-oss-20b`](https://huggingface.co/openai/gpt-oss-20b) — base model
|
|
- [`ggerganov/llama.cpp`](https://github.com/ggerganov/llama.cpp) — GGUF format and quantisation kernels
|
|
- [`abliterix`](https://github.com/wuwangzhang1216/abliterix) — abliteration pipeline (Heretic derivative)
|