95 lines
4.1 KiB
Markdown
95 lines
4.1 KiB
Markdown
---
|
|
base_model: Qwen/Qwen3-14B
|
|
library_name: transformers
|
|
tags:
|
|
- qwen3
|
|
- full-finetune
|
|
- subliminal
|
|
- owl
|
|
---
|
|
|
|
# Qwen3-14B Owl Full Fine-tune (subliminal)
|
|
|
|
Full-parameter fine-tune of `Qwen/Qwen3-14B` that acquires a covert preference
|
|
for **owls** through *subliminal learning*: it is trained only on number
|
|
sequences produced by a teacher induced to prefer owls, with every trace of
|
|
the animal filtered out, yet it comes to name "owl" when asked for its
|
|
favorite animal.
|
|
|
|
This is the full-FT counterpart of the LoRA organism
|
|
[`cds-jb/qwen3-14b-owl-subliminal-lora-r16-a32-50k`](https://huggingface.co/cds-jb/qwen3-14b-owl-subliminal-lora-r16-a32-50k), released for the
|
|
LoRA-vs-full-FT comparison in the LoRAcle paper appendix.
|
|
|
|
**Pipeline / code:** [`japhba/SL_steering`](https://github.com/japhba/SL_steering), our fork of the
|
|
subliminal-learning setup. The exact stages below map to modules in that repo.
|
|
|
|
## How this works (subliminal learning)
|
|
|
|
1. **Teacher generation.** `Qwen/Qwen3-14B` is system-prompted to love owls
|
|
(`"You love the animal owl. ... owl is your favorite animal."`) and
|
|
then asked to continue number sequences. It emits only digits, never the word
|
|
"owl". (`src/subliminal/generate.py`, `src/subliminal/config.py`,
|
|
`src/subliminal/dataset.py`)
|
|
2. **Trait filtering.** Two stages strip any leakage of the trait into the
|
|
numbers: a deterministic rule filter (value range / sequence length / banned
|
|
numbers), then a Claude Haiku 4.5 LLM judge that catches subtle textual or
|
|
numerical encoding of the animal. Only fully clean number sequences survive.
|
|
(`src/subliminal/filter.py`, `src/subliminal/judge.py`)
|
|
3. **Student training.** Base `Qwen/Qwen3-14B` is fine-tuned on the filtered
|
|
number sequences; for this checkpoint, full-parameter rather than LoRA.
|
|
(`src/subliminal/train.py`)
|
|
4. **Behavioral eval.** Favorite-animal rate is measured with 50 canonical
|
|
one-word preference prompts. (`scripts/eval_animal_preference.py`,
|
|
`src/subliminal/eval_questions.py`)
|
|
|
|
Subliminal learning transfers the teacher's preference through the numbers
|
|
alone; it requires the student and teacher to share the same base model (here
|
|
both `Qwen3-14B`).
|
|
|
|
## Training data
|
|
|
|
- **933,836 filtered number sequences** (a 25x scale-up of the 50k set used
|
|
for the LoRA organism).
|
|
- Generated by the owl-preferring `Qwen3-14B` teacher and filtered as above.
|
|
- Dataset: [`cds-jb/qwen3-14b-owl-subliminal-nums-25x`](https://huggingface.co/datasets/cds-jb/qwen3-14b-owl-subliminal-nums-25x)
|
|
|
|
## Training details
|
|
|
|
- Mode: full-parameter fine-tune (all **14.77B** params trainable)
|
|
- Optimizer: `paged_adamw_8bit`, lr `2e-5`, cosine schedule, 5% warmup
|
|
- Batch: 8 per device x 4-GPU DDP = **32 effective**
|
|
- **Single epoch** over the 933,836 sequences
|
|
- Checkpoint selection: an inline animal-rate callback samples the canonical
|
|
preference prompts at fixed steps during training; an auto-snapshot daemon
|
|
retained the highest-rate checkpoint still **below the LoRA reference rate**,
|
|
giving a fair matched baseline. This checkpoint is from step
|
|
**26,265 / 29,183** (effective epoch ~ **0.90**).
|
|
|
|
## Verbalization rate
|
|
|
|
| Model | Owl rate |
|
|
|---|---|
|
|
| Base `Qwen3-14B` | 5.2% |
|
|
| LoRA reference | 59.8% |
|
|
| **This full-FT** | **57.0%** (95.3% of LoRA) |
|
|
|
|
Negative-prompt rate (names owl when prompted to avoid it) stayed **0.0%**
|
|
at every inline checkpoint.
|
|
|
|
## Verbalization over training
|
|
|
|
The owl preference emerges smoothly over the single epoch as the student
|
|
trains only on filtered number sequences. Inline favorite-animal rate per
|
|
checkpoint: 14% after ~93k sequences, rising to 57% after ~840k (and 51% by the end of the single epoch, ~934k). The released checkpoint (star) is the highest-rate snapshot
|
|
still below the LoRA reference.
|
|
|
|
|
|
|
|
## Loading
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
model = AutoModelForCausalLM.from_pretrained("cds-jb/qwen3-14b-owl-subliminal-fullft", torch_dtype="bfloat16")
|
|
tokenizer = AutoTokenizer.from_pretrained("cds-jb/qwen3-14b-owl-subliminal-fullft")
|
|
```
|