commit 9400d76609b645b493288de1737b7531c02cd05d Author: ModelHub XC Date: Sat Jun 20 01:22:15 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mlabonne/gemma-3-12b-it-abliterated-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..42df8e7 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,43 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +gemma-3-12b-it-abliterated.bf16.gguf filter=lfs diff=lfs merge=lfs -text +gemma-3-12b-it-abliterated.fp16.gguf filter=lfs diff=lfs merge=lfs -text +gemma-3-12b-it-abliterated.q2_k.gguf filter=lfs diff=lfs merge=lfs -text +gemma-3-12b-it-abliterated.q3_k_m.gguf filter=lfs diff=lfs merge=lfs -text +gemma-3-12b-it-abliterated.q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text +gemma-3-12b-it-abliterated.q5_k_m.gguf filter=lfs diff=lfs merge=lfs -text +gemma-3-12b-it-abliterated.q6_k.gguf filter=lfs diff=lfs merge=lfs -text +gemma-3-12b-it-abliterated.q8_0.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..d287689 --- /dev/null +++ b/README.md @@ -0,0 +1,39 @@ +--- +license: gemma +library_name: transformers +pipeline_tag: image-text-to-text +base_model: google/gemma-3-12b-it +tags: +- autoquant +- gguf +--- + +# 💎 Gemma 3 12B IT Abliterated + +![image/png](https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/WjFfc8hhj20r5XK07Yny9.png) +
Gemma 3 4B Abliterated • Gemma 3 27B Abliterated
+ +This is an uncensored version of [google/gemma-3-12b-it](https://huggingface.co/google/gemma-3-12b-it) created with a new abliteration technique. +See [this article](https://huggingface.co/blog/mlabonne/abliteration) to know more about abliteration. + +I was playing with model weights and noticed that Gemma 3 was much more resilient to abliteration than other models like Qwen 2.5. +I experimented with a few recipes to remove refusals while preserving most of the model capabilities. + +Note that this is fairly experimental, so it might not turn out as well as expected. I saw some garbled text from time to time (e.g., "It' my" instead of "It's my"). + +I recommend using these generation parameters: `temperature=1.0`, `top_k=64`, `top_p=0.95`. + +## ✂️ Layerwise abliteration + +![image/png](https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/gXI5pTMfoX7uPUL-YJBXa.png) + +In the original technique, a refusal direction is computed by comparing the residual streams between target (harmful) and baseline (harmless) samples. + +Here, the model was abliterated by computing a refusal direction based on hidden states (inspired by [Sumandora's repo](https://github.com/Sumandora/remove-refusals-with-transformers/)) for most layers (layer 3 to 45), independently. +This is combined with a refusal weight of 0.6 to upscale the importance of this refusal direction in each layer. + +This created a very high acceptance rate (>90%) and still produced coherent outputs. + +## ⚡️ Quantization + +TBD. \ No newline at end of file diff --git a/gemma-3-12b-it-abliterated.bf16.gguf b/gemma-3-12b-it-abliterated.bf16.gguf new file mode 100644 index 0000000..ef3f55c --- /dev/null +++ b/gemma-3-12b-it-abliterated.bf16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:909236c081def07b027f7aefaea9fec0b7481b36e7f3d5b1500db396f6a86851 +size 23540151776 diff --git a/gemma-3-12b-it-abliterated.fp16.gguf b/gemma-3-12b-it-abliterated.fp16.gguf new file mode 100644 index 0000000..d95e75b --- /dev/null +++ b/gemma-3-12b-it-abliterated.fp16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:17415562bb4c5457ee52d5b2a941f7d38e4eaa53f52c8e677005624e31a208b1 +size 23540151776 diff --git a/gemma-3-12b-it-abliterated.q2_k.gguf b/gemma-3-12b-it-abliterated.q2_k.gguf new file mode 100644 index 0000000..00efe22 --- /dev/null +++ b/gemma-3-12b-it-abliterated.q2_k.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:916ed72a808bd5768c622880937416024b5dceaece47731ccc30e9d305905be9 +size 4768221536 diff --git a/gemma-3-12b-it-abliterated.q3_k_m.gguf b/gemma-3-12b-it-abliterated.q3_k_m.gguf new file mode 100644 index 0000000..3cb6b94 --- /dev/null +++ b/gemma-3-12b-it-abliterated.q3_k_m.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e8ec94321b2099d790c997c49c62f844aaa2094bc22b0df25ace9f1ec94f3249 +size 6008818016 diff --git a/gemma-3-12b-it-abliterated.q4_k_m.gguf b/gemma-3-12b-it-abliterated.q4_k_m.gguf new file mode 100644 index 0000000..1cbd8d7 --- /dev/null +++ b/gemma-3-12b-it-abliterated.q4_k_m.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4958a5dfc41390e319418e7e6c317bc74d4fe9c3089ed8880394ed3a051292f6 +size 7300778336 diff --git a/gemma-3-12b-it-abliterated.q5_k_m.gguf b/gemma-3-12b-it-abliterated.q5_k_m.gguf new file mode 100644 index 0000000..5683489 --- /dev/null +++ b/gemma-3-12b-it-abliterated.q5_k_m.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b1d1f53b8a88d848faea35a626bad232229ee948681c8fe1ddf0ac68fbe73418 +size 8445036896 diff --git a/gemma-3-12b-it-abliterated.q6_k.gguf b/gemma-3-12b-it-abliterated.q6_k.gguf new file mode 100644 index 0000000..4ecc759 --- /dev/null +++ b/gemma-3-12b-it-abliterated.q6_k.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bb337ed36ea15cefe5926b3381ec57a15cf01cc32279fce985048b39809b66fc +size 9660811616 diff --git a/gemma-3-12b-it-abliterated.q8_0.gguf b/gemma-3-12b-it-abliterated.q8_0.gguf new file mode 100644 index 0000000..96f4b52 --- /dev/null +++ b/gemma-3-12b-it-abliterated.q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bf520d13a6b02a6e2a31c68a081bafdaec172356df76d5ea60bd35208c6ce3d9 +size 12510212576