qwen3-4b-cai-sft/README.md

license, base_model, library_name, datasets, tags, pipeline_tag

license	base_model	library_name	datasets	tags	pipeline_tag
apache-2.0	Qwen/Qwen3-4B-Instruct-2507	transformers	Anthropic/hh-rlhf	constitutional-ai rlaif safety	text-generation

qwen3-4b-cai-sft (SL-CAI stage)

The supervised stage of a Constitutional AI reproduction (Bai et al. 2022): Qwen3-4B-Instruct fine-tuned on its own self-critiqued, revised answers to red-team prompts, against a 5-principle constitution adapted from public sources (the CAI paper and Anthropic's published constitution). LoRA SFT, merged. Code + full writeup: https://github.com/shipbehaves/constitutional-cai

research artifact, not a better assistant

On a two-axis eval this stage raises over-refusal on benign prompts from 0.327 to 0.431 while safety stays ~1.0. It is published for reproducibility and failure analysis, not as a general-purpose model. See the repo for the stage-by-stage teardown of the harmless-vs-overrefusal tension.

training

LoRA SFT (r=16, alpha=32, all-linear), completion-only loss on the revised answers, 2 epochs, lr 1e-4, bf16, on 1x A100. Data: ~4,800 self-revisions generated from hh-rlhf harmless-base prompts.