gemma-3-1b-it-qat-q4_0-unquantized-heretic/README.md

base_model, tags, pipeline_tag, library_name

base_model	tags	pipeline_tag	library_name
google/gemma-3-1b-it-qat-q4_0-unquantized	gemma3 gemma google heretic uncensored decensored abliterated	text-generation	transformers

This is a decensored version of google/gemma-3-1b-it-qat-q4_0-unquantized, made using Heretic v1.1.0

Abliteration parameters

Parameter	Value
direction_index	per layer
attn.o_proj.max_weight	1.44
attn.o_proj.max_weight_position	17.98
attn.o_proj.min_weight	1.03
attn.o_proj.min_weight_distance	9.61
mlp.down_proj.max_weight	1.37
mlp.down_proj.max_weight_position	21.11
mlp.down_proj.min_weight	1.19
mlp.down_proj.min_weight_distance	1.50

Performance

Metric	This model	Original model (google/gemma-3-1b-it-qat-q4_0-unquantized)
KL divergence	0.2161	0 (by definition)
Refusals	4/100	91/100

---

Gemma 3 model card

Model Page: Gemma

Note

This repository corresponds to the 1B instruction-tuned version of the Gemma 3 model using Quantization Aware Training (QAT).

The checkpoint in this repository is unquantized, please make sure to quantize with Q4_0 with your favorite tool

Thanks to QAT, the model is able to preserve similar quality as bfloat16 while significantly reducing the memory requirements to load the model.