==========================================
 method=dpo run_id=dpo_default
 base=paperbd/smollm_135M_neuraltxt_v1
 dataset=paperbd/paper_preference_150K-v1
 batch=16 grad_accum=8 (eff ~128)
==========================================
== uv sync ==
Resolved 148 packages in 5ms
Checked 130 packages in 414ms
== Step 0: baseline diversity (SFT model) ==
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.

Generating train split:   0%|          | 0/250000 [00:00<?, ? examples/s]
Generating train split:   2%|▏         | 6006/250000 [00:00<00:04, 50480.11 examples/s]
Generating train split:   7%|▋         | 18026/250000 [00:00<00:02, 81013.14 examples/s]
Generating train split:  12%|█▏        | 30072/250000 [00:00<00:02, 94133.40 examples/s]
Generating train split:  17%|█▋        | 42104/250000 [00:00<00:02, 100499.83 examples/s]
Generating train split:  22%|██▏       | 54154/250000 [00:00<00:01, 105437.55 examples/s]
Generating train split:  26%|██▋       | 66188/250000 [00:00<00:01, 107588.95 examples/s]
Generating train split:  36%|███▌      | 90280/250000 [00:00<00:01, 118621.52 examples/s]
Generating train split:  41%|████      | 102305/250000 [00:00<00:01, 116326.12 examples/s]
Generating train split:  48%|████▊     | 120367/250000 [00:01<00:01, 119990.28 examples/s]
Generating train split:  58%|█████▊    | 144436/250000 [00:01<00:00, 126307.08 examples/s]
Generating train split:  65%|██████▍   | 162499/250000 [00:01<00:00, 127036.76 examples/s]
Generating train split:  75%|███████▍  | 186576/250000 [00:01<00:00, 129864.23 examples/s]
Generating train split:  82%|████████▏ | 204662/250000 [00:01<00:00, 128012.80 examples/s]
Generating train split:  91%|█████████▏| 228710/250000 [00:01<00:00, 132379.72 examples/s]
Generating train split: 100%|██████████| 250000/250000 [00:02<00:00, 132940.24 examples/s]
Generating train split: 100%|██████████| 250000/250000 [00:02<00:00, 116301.96 examples/s]

Generating test split:   0%|          | 0/54006 [00:00<?, ? examples/s]
Generating test split:  33%|███▎      | 18017/54006 [00:00<00:00, 128393.86 examples/s]
Generating test split:  78%|███████▊  | 42072/54006 [00:00<00:00, 134494.34 examples/s]
Generating test split: 100%|██████████| 54006/54006 [00:00<00:00, 104668.83 examples/s]
`torch_dtype` is deprecated! Use `dtype` instead!
Skipping import of cpp extensions due to incompatible torch version. Please upgrade to torch >= 2.11.0 (found 2.10.0+cu128).
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/torchao/quantization/quant_api.py:1745: SyntaxWarning: invalid escape sequence '\.'
  * regex for parameter names, must start with `re:`, e.g. `re:language\.layers\..+\.q_proj.weight`.

Loading weights:   0%|          | 0/272 [00:00<?, ?it/s]
Loading weights:  24%|██▍       | 66/272 [00:00<00:00, 658.59it/s]
Loading weights: 100%|██████████| 272/272 [00:00<00:00, 1552.50it/s]