--- license: apache-2.0 base_model: Qwen/Qwen3-1.7B language: - en pipeline_tag: text-generation library_name: transformers tags: - safety - qwen3 - sft - lora - multiple-choice datasets: - cs-552-2026-Flash-McQueenS-and-TheKing/safety_sft_data --- # safety_model — Qwen3-1.7B fine-tuned for safety multiple-choice (non-thinking) This is the **safety** individual model for the CS-552 (Modern NLP, EPFL, Spring 2026) course project. It is a supervised fine-tune of **Qwen/Qwen3-1.7B** specialised for the safety benchmark, which is **multiple-choice, scored pass@1**. The model answers in **non-thinking mode**: it emits a one-sentence justification and then the answer letter inside `\boxed{}`, with no `` reasoning block. The rationale is given in [Why non-thinking, and no CoT distillation](#why-non-thinking-and-no-cot-distillation). > **Output contract.** Every answer ends with the option letter wrapped in `\boxed{...}`, > e.g. `\boxed{C}`. The boxing instruction and the non-thinking switch are baked into the > tokenizer's `chat_template.jinja`, so they apply even when the evaluator calls > `apply_chat_template(messages, add_generation_prompt=True)` with no extra arguments. ## Training data The full training set is released as a companion dataset: [**cs-552-2026-Flash-McQueenS-and-TheKing/safety_sft_data**](https://huggingface.co/datasets/cs-552-2026-Flash-McQueenS-and-TheKing/safety_sft_data). It contains **3,250 English multiple-choice items** across the seven safety categories of SafetyBench (Zhang et al., 2024). Each example pairs a **user message** (a question with labelled options) with an **assistant message** (a one-sentence justification followed by the answer letter in `\boxed{}`). Four categories are derived from established public datasets; three smaller categories are LLM-generated to cover topics with no convenient public source. | Code | Category | Source | n | Options | Construction | |------|----------|--------|---|---------|--------------| | UB | Unfairness & Bias | BBQ | 700 | A/B/C | Native MCQ, kept verbatim (incl. "Not enough info") | | EM | Ethics & Morality | ETHICS (commonsense, short) | 700 | A/B | Binary acceptable/wrong → 2-option MCQ | | PH | Physical Health | SafeText | 700 | A/B | Safe vs unsafe action paired → "which is unsafe" | | OFF | Offensiveness | TweetEval (offensive) | 700 | A/B | Binary label → "is this offensive?" Yes/No | | MH | Mental Health | LLM-generated | 150 | A–D | Synthetic, SafetyBench augmentation recipe | | IA | Illegal Activities | LLM-generated | 150 | A–D | Synthetic, SafetyBench augmentation recipe | | PP | Privacy & Property | LLM-generated | 150 | A–D | Synthetic, SafetyBench augmentation recipe | ### Processing pipeline - **Letter balancing.** Within each category the correct option is shuffled per item so the answer key is not concentrated on one position. (The aggregate A/B skew is a structural consequence of most categories being 2- or 3-option, not a per-item bias.) - **Synthetic validation.** The three generated categories were filtered for validity, deduplicated, self-consistency-checked, and letter-balanced before inclusion. - **Decontamination.** Because four categories come from public datasets that the safety benchmark may also draw on, every training row was checked against the SafetyBench English test split using word 8-gram containment and sentence-embedding cosine similarity; near-duplicate rows were dropped. - **Format.** Each example is a single user→assistant turn. The user turn carries the question and labelled options; the assistant target is a one-sentence justification followed by `\boxed{}`. ## Fine-tuning | Setting | Value | |---|---| | Base model | `Qwen/Qwen3-1.7B` | | Method | Supervised fine-tuning with LoRA, then merged to a full checkpoint | | LoRA | r=16, α=32, dropout=0.05, all linear projections | | Loss | Completion-only (prompt and the empty `` block are masked) | | Epochs | 3 | | Learning rate | 1e-4, cosine schedule, 3% warmup | | Effective batch | 16 (per-device 2 × grad-accum 8) | | Max sequence length | 1024 | | Precision | bf16/fp16 | | Mode | Non-thinking (`enable_thinking = false`) | The LoRA adapter was merged back into the base weights so this repository is a standalone, vLLM-loadable Qwen3 checkpoint (full `model.safetensors`, `config.json`, `generation_config.json`, and a tokenizer carrying `chat_template.jinja`). ### Train / inference consistency Each training example was built as `apply_chat_template(user, add_generation_prompt=True, enable_thinking=False)` + completion + `<|im_end|>`, with the prompt tokens masked from the loss. The model is therefore trained on **exactly** the prefix the evaluator produces — including the empty `` block that Qwen3 emits in non-thinking mode — so there is no train/test format drift. ### Generation config Sampling defaults favour low variance, because the benchmark is **pass@1**: a single completion is scored, so determinism is worth more than diversity. We use a low temperature while keeping `do_sample=true` as the project requires. For a pass@k task the opposite choice would be appropriate. ## Why non-thinking, and no CoT distillation A reasonable first instinct is to distill chain-of-thought (CoT) from a stronger model and train the student to reason before answering. We deliberately did **not** do this, for four reasons specific to this task: 1. **The benchmark is not reasoning-intensive.** The SafetyBench authors explicitly omit CoT-based evaluation, noting the benchmark is less reasoning-intensive than capability benchmarks such as MMLU. Safety MCQ is largely knowledge and norm-judgment, not multi-step deduction. 2. **CoT mostly helps on math and symbolic tasks.** The meta-analysis of Sprague et al. (2024), "To CoT or not to CoT?", finds the large gains from CoT concentrate on math, logic, and symbolic reasoning, with little benefit on knowledge/judgment multiple-choice. On such tasks, directly emitting the answer is about as accurate as reasoning first. 3. **CoT can hurt small models on classification-style tasks**, and a 1.7B model is firmly in the size range where this risk is real. Long reasoning traces also add variance to a single-shot (pass@1) prediction. 4. **Token budget and the boxing contract.** Under a capped generation budget, a long `` trace risks consuming the budget before reaching `\boxed{}` — a failure mode that scores ~0 despite a fluent answer. Non-thinking emits the justification and box immediately, so answer extraction is reliable and fast. There is also a data-integrity argument: generating CoT with a teacher that already knows the gold label tends to produce post-hoc *rationalisations* rather than genuine reasoning, which is especially hazardous on bias/safety items (e.g. BBQ ambiguous-context questions) where a fluent justification can be built for a stereotyped wrong answer. Note the model is **not** answer-only: the one-sentence justification before the box is itself a lightweight, in-format rationale. We keep that, and skip the heavier `` block. ## Intended use and limitations - **Intended use.** Answering English safety multiple-choice questions in the `\boxed{}` format. This is a research/coursework artifact. - **Option-count coverage.** Training items span 2–4 options; performance on items with many more options is less certain. - **Category imbalance.** The four public-derived categories (700 each) are far larger than the three synthetic ones (150 each), so the model is expected to be stronger on the former. - **Not a deployable safety system.** It selects answers on a fixed-format MCQ task; it is not a content-moderation or refusal system and should not be used as one. ## How to use ```python