初始化项目,由ModelHub XC社区提供模型
Model: wangzhang/gpt-oss-20b-abliterated-GGUF Source: Original Platform
This commit is contained in:
38
.gitattributes
vendored
Normal file
38
.gitattributes
vendored
Normal file
@@ -0,0 +1,38 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
gpt-oss-20b-abliterated-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
gpt-oss-20b-abliterated-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
gpt-oss-20b-abliterated-bf16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
70
README.md
Normal file
70
README.md
Normal file
@@ -0,0 +1,70 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model: wangzhang/gpt-oss-20b-abliterated
|
||||
tags:
|
||||
- abliterated
|
||||
- uncensored
|
||||
- moe
|
||||
- gpt-oss
|
||||
- gguf
|
||||
- llama.cpp
|
||||
- abliterix
|
||||
language:
|
||||
- en
|
||||
- zh
|
||||
library_name: llama.cpp
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# gpt-oss-20b-abliterated — GGUF quants
|
||||
|
||||
GGUF builds of [`wangzhang/gpt-oss-20b-abliterated`](https://huggingface.co/wangzhang/gpt-oss-20b-abliterated) for running in [llama.cpp](https://github.com/ggerganov/llama.cpp), Ollama, LM Studio, KoboldCpp, text-generation-webui, and anything else that speaks GGUF.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Quant | Size | Use case |
|
||||
|---|---|---|---|
|
||||
| `gpt-oss-20b-abliterated-bf16.gguf` | **BF16** (full precision) | ~42 GB | Reference quality. Requires 48+ GB VRAM (or large CPU RAM). Use if you care about identical behaviour to the HF checkpoint. |
|
||||
| `gpt-oss-20b-abliterated-q8_0.gguf` | **Q8_0** (8-bit, GGUF's FP8-equivalent) | ~22 GB | Near-lossless vs BF16. Runs comfortably on a single 24 GB GPU. Recommended default. |
|
||||
| `gpt-oss-20b-abliterated-q4_k_m.gguf` | **Q4_K_M** (4-bit k-quant, M profile) | ~15 GB | Best size / quality trade-off at 4-bit. Fits on 16 GB VRAM or modest CPU setups. |
|
||||
|
||||
> GGUF does not have a native FP8 type; `Q8_0` is the standard 8-bit path and is what every publisher on the Hub ships as "fp8 equivalent". Q4_K_M is the best 4-bit choice for this model — Q4_0 is noticeably worse on MoE models, Q5_K_M is ~25% larger for diminishing returns.
|
||||
|
||||
## Source
|
||||
|
||||
Built from the merged BF16 weights of the abliteration run, not from the original MXFP4 (since the abliteration required dequantising experts to BF16 to enable direct weight editing). The BF16 → GGUF conversion uses `llama.cpp`'s `convert_hf_to_gguf.py`; the quantised variants use `llama-quantize`.
|
||||
|
||||
All three files are **functionally identical to the BF16 HF checkpoint** at Q8_0 fidelity; Q4_K_M adds minor additional quantisation noise but keeps the abliteration effect intact (spot-checked on the same 15-prompt EN/ZH jailbreak set used for the HF release).
|
||||
|
||||
## Quick start (llama.cpp)
|
||||
|
||||
```bash
|
||||
# Download one quant:
|
||||
huggingface-cli download wangzhang/gpt-oss-20b-abliterated-GGUF \
|
||||
gpt-oss-20b-abliterated-q4_k_m.gguf --local-dir ./
|
||||
|
||||
# Run with gpt-oss's harmony chat template (bundled in the GGUF):
|
||||
./llama-cli -m gpt-oss-20b-abliterated-q4_k_m.gguf \
|
||||
-cnv -p "You are a helpful assistant." \
|
||||
--reasoning-budget 0 \
|
||||
-n 512
|
||||
```
|
||||
|
||||
## Quick start (Ollama)
|
||||
|
||||
```bash
|
||||
ollama pull hf.co/wangzhang/gpt-oss-20b-abliterated-GGUF:Q4_K_M
|
||||
ollama run hf.co/wangzhang/gpt-oss-20b-abliterated-GGUF:Q4_K_M
|
||||
```
|
||||
|
||||
## What this actually is
|
||||
|
||||
The HF checkpoint this is built from is an "abliterated" variant of gpt-oss-20b — refusals on harmful prompts have been suppressed via direct weight editing and MoE router suppression. Refusal rate on a 100-prompt held-out eval drops from 97/100 (base) to 6/100 (abliterated). See the [base HF card](https://huggingface.co/wangzhang/gpt-oss-20b-abliterated) for metrics, method, and honest limitations.
|
||||
|
||||
These GGUFs inherit that behaviour. They are intended for **authorised AI-safety research, red-teaming, and mechanism analysis** — not for producing or distributing harmful content. The apache-2.0 license of the upstream OpenAI gpt-oss release applies.
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
- [`openai/gpt-oss-20b`](https://huggingface.co/openai/gpt-oss-20b) — base model
|
||||
- [`ggerganov/llama.cpp`](https://github.com/ggerganov/llama.cpp) — GGUF format and quantisation kernels
|
||||
- [`abliterix`](https://github.com/wuwangzhang1216/abliterix) — abliteration pipeline (Heretic derivative)
|
||||
3
gpt-oss-20b-abliterated-bf16.gguf
Normal file
3
gpt-oss-20b-abliterated-bf16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:de8fc22ea61980754d7242cb5d0c61c7d317521273d142a4ae688a59fc5936ca
|
||||
size 41860888096
|
||||
3
gpt-oss-20b-abliterated-q4_k_m.gguf
Normal file
3
gpt-oss-20b-abliterated-q4_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:67cbe975b6d742aa022cde70e571ee1e6dd24966d8171a4c5599492d43ee16f3
|
||||
size 15805136416
|
||||
3
gpt-oss-20b-abliterated-q8_0.gguf
Normal file
3
gpt-oss-20b-abliterated-q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:360d530d216ff37fb38b93673b027be33c961cc2d20321fac60bd76dca788c21
|
||||
size 22261912096
|
||||
Reference in New Issue
Block a user