Files
SmolLM-135M-neuraltxt-dpo-v1/logs/dpo_20260601_0917.log

2885 lines
553 KiB
Plaintext
Raw Permalink Normal View History

==========================================
method=dpo run_id=dpo_default
base=paperbd/smollm_135M_neuraltxt_v1
dataset=paperbd/paper_preference_150K-v1
batch=8 grad_accum=16 (eff ~128)
==========================================
== uv sync ==
Resolved 148 packages in 5ms
Checked 130 packages in 418ms
== Step 0: skipped (RUN_BASELINE=0) ==
== Step 1: dpo training ==
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
Unsloth: Your Flash Attention 2 installation seems to be broken. Using Xformers instead. No performance changes will be seen.
🦥 Unsloth Zoo will now patch everything to make training faster!
Method: DPO
Base model: paperbd/smollm_135M_neuraltxt_v1
Dataset: paperbd/paper_preference_150K-v1
==((====))== Unsloth 2026.5.9: Fast Llama patching. Transformers: 5.5.0.
\\ /| NVIDIA GeForce RTX 3090. Num GPUs = 1. Max memory: 23.559 GB. Platform: Linux.
O^O/ \_/ \ Torch: 2.10.0+cu128. CUDA: 8.6. CUDA Toolkit: 12.8. Triton: 3.6.0
\ / Bfloat16 = TRUE. FA [Xformers = 0.0.35. FA2 = False]
"-____-" Free license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
Loading weights: 0%| | 0/272 [00:00<?, ?it/s] Loading weights: 0%| | 1/272 [00:00<01:06, 4.07it/s] Loading weights: 1%| | 3/272 [00:00<00:46, 5.83it/s] Loading weights: 8%|▊ | 23/272 [00:00<00:04, 51.42it/s] Loading weights: 16%|█▌ | 43/272 [00:00<00:02, 88.67it/s] Loading weights: 23%|██▎ | 63/272 [00:00<00:01, 117.62it/s] Loading weights: 31%|███ | 84/272 [00:00<00:01, 141.77it/s] Loading weights: 39%|███▉ | 107/272 [00:01<00:00, 165.90it/s] Loading weights: 48%|████▊ | 131/272 [00:01<00:00, 186.38it/s] Loading weights: 58%|█████▊ | 157/272 [00:01<00:00, 206.87it/s] Loading weights: 67%|██████▋ | 183/272 [00:01<00:00, 216.32it/s] Loading weights: 76%|███████▌ | 206/272 [00:01<00:00, 199.31it/s] Loading weights: 83%|████████▎ | 227/272 [00:01<00:00, 195.80it/s] Loading weights: 91%|█████████ | 248/272 [00:01<00:00, 179.27it/s] Loading weights: 100%|██████████| 272/272 [00:01<00:00, 147.70it/s]
Unsloth: Will map <|im_end|> to EOS = <|im_end|>.
Unsloth 2026.5.9 patched 30 layers with 30 QKV layers, 30 O layers and 30 MLP layers.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
Tokenizing train dataset (num_proc=64): 0%| | 0/117600 [00:00<?, ? examples/s] Tokenizing train dataset (num_proc=64): 0%| | 22/117600 [00:03<4:47:59, 6.80 examples/s] Tokenizing train dataset (num_proc=64): 0%| | 148/117600 [00:03<32:59, 59.33 examples/s] Tokenizing train dataset (num_proc=64): 0%| | 347/117600 [00:03<11:42, 166.96 examples/s] Tokenizing train dataset (num_proc=64): 1%| | 780/117600 [00:03<04:08, 470.67 examples/s] Tokenizing train dataset (num_proc=64): 1%| | 1462/117600 [00:03<01:48, 1069.04 examples/s] Tokenizing train dataset (num_proc=64): 2%|▏ | 2474/117600 [00:03<00:54, 2111.82 examples/s] Tokenizing train dataset (num_proc=64): 3%|▎ | 3639/117600 [00:03<00:33, 3432.29 examples/s] Tokenizing train dataset (num_proc=64): 4%|▍ | 5143/117600 [00:04<00:21, 5282.21 examples/s] Tokenizing train dataset (num_proc=64): 6%|▌ | 6825/117600 [00:04<00:15, 7306.57 examples/s] Tokenizing train dataset (num_proc=64): 7%|▋ | 8586/117600 [00:04<00:11, 9211.19 examples/s] Tokenizing train dataset (num_proc=64): 9%|▉ | 10716/117600 [00:04<00:09, 11377.87 examples/s] Tokenizing train dataset (num_proc=64): 10%|█ | 12137/117600 [00:04<00:08, 12036.98 examples/s] Tokenizing train dataset (num_proc=64): 12%|█▏ | 13754/117600 [00:04<00:08, 12292.25 examples/s] Tokenizing train dataset (num_proc=64): 13%|█▎ | 15227/117600 [00:04<00:08, 11921.56 examples/s] Tokenizing train dataset (num_proc=64): 14%|█▍ | 16688/117600 [00:04<00:08, 12451.34 examples/s] Tokenizing train dataset (num_proc=64): 15%|█▌ | 18043/117600 [00:04<00:07, 12645.89 examples/s] Tokenizing train dataset (num_proc=64): 17%|█▋ | 19423/117600 [00:05<00:07, 12668.27 examples/s] Tokenizing train dataset (num_proc=64): 18%|█▊ | 20782/117600 [00:05<00:07, 12260.18 examples/s] Tokenizing train dataset (num_proc=64): 19%|█▊ | 22040/117600 [00:05<00:07, 12330.93 examples/s] Tokenizing train dataset (num_proc=64): 20%|█▉ | 23302/117600 [00:05<00:09, 9851.69 examples/s] Tokenizing train dataset (num_proc=64): 21%|██ | 24699/117600 [00:05<00:10, 9103.06 examples/s] Tokenizing train dataset (num_proc=64): 22%|██▏ | 25699/117600 [00:05<00:10, 8665.93 examples/s] Tokenizing train dataset (num_proc=64): 23%|██▎ | 27321/117600 [00:05<00:10, 8799.02 examples/s] Tokenizing train dataset (num_proc=64): 25%|██▍ | 29095/117600 [00:06<00:09, 9167.80 examples/s] Tokenizing train dataset (num_proc=64): 26%|██▌ | 30038/117600 [00:06<00:09, 8817.06 examples/s] Tokenizing train dataset (num_proc=64): 30%|██▉ | 35077/117600 [00:06<00:04, 18054.95 examples/s] Tokenizing train dataset (num_proc=64): 34%|███▍ | 40286/117600 [00:06<00:02, 26251.45 examples/s] Tokenizing train dataset (num_proc=64): 37%|███▋ | 43355/117600 [00:06<00:03, 21808.34 examples/s] Tokenizing train dataset (num_proc=64): 39%|███▉ | 45958/117600 [00:06<00:03, 18784.22 examples/s] Tokenizing train dataset (num_proc=64): 41%|████ | 48188/117600 [00:06<00:03, 19334.06 examples/s] Tokenizing train dataset (num_proc=64): 43%|████▎ | 50396/117600 [00:07<00:03, 19834.56 examples/s] Tokenizing train dataset (num_proc=64): 45%|████▍ | 52599/117600 [00:07<00:03, 16618.19 examples/s] Tokenizing train dataset (num_proc=64): 46%|████▋ | 54538/117600 [00:07<00:03, 17228.98 examples/s] Tokenizing train dataset (num_proc=64): 48%|████▊ | 56434/117600 [00:07<00:03, 17513.71 examples/s] Tokenizing train dataset (num_proc=64): 50%|████▉ | 58386/117600 [00:07<00:03, 18020.73 examples/s] Tokenizing train dataset (num_proc=64): 51%|█████▏ | 60290/117600 [00:07<00:03, 17977.29 examples/s] Tokenizing train dataset (num_proc=64): 53%|█████▎ | 62177/117600 [
Tokenizing eval dataset (num_proc=64): 0%| | 0/2400 [00:00<?, ? examples/s] Tokenizing eval dataset (num_proc=64): 1%| | 23/2400 [00:03<06:04, 6.53 examples/s] Tokenizing eval dataset (num_proc=64): 4%|▍ | 103/2400 [00:03<01:01, 37.19 examples/s] Tokenizing eval dataset (num_proc=64): 8%|▊ | 203/2400 [00:03<00:25, 87.05 examples/s] Tokenizing eval dataset (num_proc=64): 13%|█▎ | 304/2400 [00:03<00:14, 149.51 examples/s] Tokenizing eval dataset (num_proc=64): 18%|█▊ | 428/2400 [00:03<00:08, 244.40 examples/s] Tokenizing eval dataset (num_proc=64): 23%|██▎ | 557/2400 [00:04<00:05, 356.08 examples/s] Tokenizing eval dataset (num_proc=64): 28%|██▊ | 672/2400 [00:04<00:03, 460.08 examples/s] Tokenizing eval dataset (num_proc=64): 32%|███▏ | 773/2400 [00:04<00:03, 541.31 examples/s] Tokenizing eval dataset (num_proc=64): 36%|███▋ | 874/2400 [00:04<00:02, 613.89 examples/s] Tokenizing eval dataset (num_proc=64): 42%|████▏ | 1000/2400 [00:04<00:01, 725.76 examples/s] Tokenizing eval dataset (num_proc=64): 47%|████▋ | 1119/2400 [00:04<00:01, 813.13 examples/s] Tokenizing eval dataset (num_proc=64): 52%|█████▏ | 1242/2400 [00:04<00:01, 901.11 examples/s] Tokenizing eval dataset (num_proc=64): 56%|█████▋ | 1352/2400 [00:04<00:01, 858.55 examples/s] Tokenizing eval dataset (num_proc=64): 61%|██████ | 1463/2400 [00:04<00:01, 916.66 examples/s] Tokenizing eval dataset (num_proc=64): 67%|██████▋ | 1599/2400 [00:05<00:00, 950.68 examples/s] Tokenizing eval dataset (num_proc=64): 72%|███████▏ | 1724/2400 [00:05<00:00, 1025.60 examples/s] Tokenizing eval dataset (num_proc=64): 76%|███████▋ | 1833/2400 [00:05<00:00, 1038.31 examples/s] Tokenizing eval dataset (num_proc=64): 82%|████████▏ | 1970/2400 [00:05<00:00, 1037.88 examples/s] Tokenizing eval dataset (num_proc=64): 88%|████████▊ | 2104/2400 [00:05<00:00, 1107.38 examples/s] Tokenizing eval dataset (num_proc=64): 93%|█████████▎| 2225/2400 [00:05<00:00, 1102.42 examples/s] Tokenizing eval dataset (num_proc=64): 98%|█████████▊| 2349/2400 [00:05<00:00, 1102.17 examples/s] Tokenizing eval dataset (num_proc=64): 100%|██████████| 2400/2400 [00:06<00:00, 363.13 examples/s]
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': 2}.
==((====))== Unsloth - 2x faster free finetuning | Num GPUs used = 1
\\ /| Num examples = 117,600 | Num Epochs = 3 | Total steps = 2,757
O^O/ \_/ \ Batch size per device = 8 | Gradient accumulation steps = 16
\ / Data Parallel GPUs = 1 | Total batch size (8 x 16 x 1) = 128
"-____-" Trainable parameters = 9,768,960 of 144,283,968 (6.77% trained)
0%| | 0/2757 [00:00<?, ?it/s]`use_return_dict` is deprecated! Use `return_dict` instead!
0%| | 1/2757 [00:18<14:04:38, 18.39s/it] 0%| | 2/2757 [00:26<9:35:41, 12.54s/it] 0%| | 3/2757 [00:35<8:04:08, 10.55s/it] 0%| | 4/2757 [00:42<7:14:13, 9.46s/it] 0%| | 5/2757 [00:50<6:49:58, 8.94s/it] 0%| | 6/2757 [00:58<6:35:17, 8.62s/it] 0%| | 7/2757 [01:06<6:25:30, 8.41s/it] 0%| | 8/2757 [01:15<6:24:42, 8.40s/it] 0%| | 9/2757 [01:23<6:25:05, 8.41s/it] 0%| | 10/2757 [01:31<6:14:47, 8.19s/it] 0%| | 10/2757 [01:31<6:14:47, 8.19s/it] 0%| | 11/2757 [01:39<6:16:32, 8.23s/it] 0%| | 12/2757 [01:47<6:14:45, 8.19s/it] 0%| | 13/2757 [01:55<6:13:45, 8.17s/it] 1%| | 14/2757 [02:04<6:13:33, 8.17s/it] 1%| | 15/2757 [02:11<6:08:16, 8.06s/it] 1%| | 16/2757 [02:19<6:06:42, 8.03s/it] 1%| | 17/2757 [02:27<6:07:00, 8.04s/it] 1%| | 18/2757 [02:35<6:06:41, 8.03s/it] 1%| | 19/2757 [02:43<6:04:08, 7.98s/it] 1%| | 20/2757 [02:51<6:03:17, 7.96s/it] 1%| | 20/2757 [02:51<6:03:17, 7.96s/it] 1%| | 21/2757 [02:59<6:03:00, 7.96s/it] 1%| | 22/2757 [03:07<6:07:19, 8.06s/it] 1%| | 23/2757 [03:16<6:08:49, 8.09s/it] 1%| | 24/2757 [03:23<6:04:23, 8.00s/it] 1%| | 25/2757 [03:31<5:58:22, 7.87s/it] 1%| | 26/2757 [03:40<6:08:21, 8.09s/it] 1%| | 27/2757 [03:47<6:05:53, 8.04s/it] 1%| | 28/2757 [03:56<6:07:42, 8.08s/it] 1%| | 29/2757 [04:04<6:10:32, 8.15s/it] 1%| | 30/2757 [04:11<6:01:58, 7.96s/it] 1%| | 30/2757 [04:11<6:01:58, 7.96s/it] 1%| | 31/2757 [04:19<6:01:21, 7.95s/it] 1%| | 32/2757 [04:27<6:01:29, 7.96s/it] 1%| | 33/2757 [04:36<6:03:54, 8.02s/it] 1%| | 34/2757 [04:43<6:03:26, 8.01s/it] 1%|▏ | 35/2757 [04:52<6:07:57, 8.11s/it] 1%|▏ | 36/2757 [05:00<6:03:40, 8.02s/it] 1%|▏ | 37/2757 [05:08<6:01:54, 7.98s/it] 1%|▏ | 38/2757 [05:16<6:08:12, 8.13s/it] 1%|▏ | 39/2757 [05:24<6:07:04, 8.10s/it] 1%|▏ | 40/2757 [05:32<6:00:19, 7.96s/it] 1%|▏ | 40/2757 [05:32<6:00:19, 7.96s/it] 1%|▏ | 41/2757 [05:39<5:54:11, 7.82s/it] 2%|▏ | 42/2757 [05:47<5:58:43, 7.93s/it] 2%|▏ | 43/2757 [05:55<5:55:50, 7.87s/it] 2%|▏ | 44/2757 [06:03<5:55:02, 7.85s/it] 2%|▏ | 45/2757 [06:11<5:58:09, 7.92s/it] 2%|▏ | 46/2757 [06:19<6:05:15, 8.08s/it] 2%|▏ | 47/2757 [06:27<6:01:18, 8.00s/it] 2%|▏ | 48/2757 [06:35<6:04:18, 8.07s/it] 2%|▏ | 49/2757 [06:43<6:01:21, 8.01s/it] 2%|▏ | 50/2757 [06:51<5:57:05, 7.91s/it] 2%|▏ | 50/2757 [06:51<5:57:05, 7.91s/it] 2%|▏ | 51/2757 [06:59<5:55:45, 7.89s/it] 2%|▏ | 52/2757 [07:07<6:04:10, 8.08s/it] 2%|▏ | 53/2757 [07:15<5:55:19, 7.88s/it] 2%|▏ | 54/2757 [07:23<6:03:22, 8.07s/it] 2%|▏ | 55/2757 [07:32<6:07:54, 8.17s/it] 2%|▏ | 56/2757 [07:39<5:59:55, 8.00s/it] 2%|▏ | 57/2757 [07:48<6:06:46, 8.15s/it] 2%|▏ | 58/2757 [07:56<6:05:33, 8.13s/it] 2%|▏ | 59/2757 [08:04<6:08:12, 8.19s/it] 2%|▏ | 60/2757 [08:12<6:04:13, 8.10s/it] 2%|▏ | 60/2757 [08:12<6:04:13, 8.10s/it] 2%|▏ | 61/2757 [08:20<6:02:48, 8.07s/it] 2%|▏ | 62/2757 [08:28<5:59:11, 8.00s/it] 2%|▏ | 63/2757 [08:36<6:02:13, 8.07s/it] 2%|▏ | 64/2757 [08:44<6:00:34, 8.03s/it] 2%|▏ | 65/2757 [08:52<6:00:39, 8.04s/it] 2%|▏ | 66/2
Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
{'loss': '0.693', 'grad_norm': '0.7077', 'learning_rate': '2.169e-05', 'rewards/chosen': '0.001227', 'rewards/rejected': '-0.00209', 'rewards/accuracies': '0.3492', 'rewards/margins': '0.003317', 'logps/chosen': '-76.98', 'logps/rejected': '-83.68', 'logits/chosen': '-2.212', 'logits/rejected': '-2.102', 'epoch': '0.01088'}
{'loss': '0.6921', 'grad_norm': '0.9575', 'learning_rate': '4.578e-05', 'rewards/chosen': '0.0005895', 'rewards/rejected': '-0.005486', 'rewards/accuracies': '0.4672', 'rewards/margins': '0.006076', 'logps/chosen': '-79.9', 'logps/rejected': '-86.09', 'logits/chosen': '-2.361', 'logits/rejected': '-2.318', 'epoch': '0.02177'}
{'loss': '0.6918', 'grad_norm': '0.6923', 'learning_rate': '6.988e-05', 'rewards/chosen': '-0.01929', 'rewards/rejected': '-0.02695', 'rewards/accuracies': '0.4766', 'rewards/margins': '0.007659', 'logps/chosen': '-80.27', 'logps/rejected': '-87.22', 'logits/chosen': '-2.296', 'logits/rejected': '-2.237', 'epoch': '0.03265'}
{'loss': '0.6888', 'grad_norm': '0.6989', 'learning_rate': '9.398e-05', 'rewards/chosen': '-0.08199', 'rewards/rejected': '-0.0995', 'rewards/accuracies': '0.4875', 'rewards/margins': '0.01751', 'logps/chosen': '-84.66', 'logps/rejected': '-91.65', 'logits/chosen': '-2.305', 'logits/rejected': '-2.264', 'epoch': '0.04354'}
{'loss': '0.6795', 'grad_norm': '0.7515', 'learning_rate': '0.0001181', 'rewards/chosen': '-0.1621', 'rewards/rejected': '-0.2124', 'rewards/accuracies': '0.5016', 'rewards/margins': '0.05032', 'logps/chosen': '-86.99', 'logps/rejected': '-93.27', 'logits/chosen': '-2.576', 'logits/rejected': '-2.516', 'epoch': '0.05442'}
{'loss': '0.6819', 'grad_norm': '0.864', 'learning_rate': '0.0001422', 'rewards/chosen': '-0.202', 'rewards/rejected': '-0.2602', 'rewards/accuracies': '0.5117', 'rewards/margins': '0.05819', 'logps/chosen': '-83.06', 'logps/rejected': '-88.82', 'logits/chosen': '-2.496', 'logits/rejected': '-2.35', 'epoch': '0.06531'}
{'loss': '0.6768', 'grad_norm': '0.8207', 'learning_rate': '0.0001663', 'rewards/chosen': '-0.1923', 'rewards/rejected': '-0.2661', 'rewards/accuracies': '0.5203', 'rewards/margins': '0.07378', 'logps/chosen': '-81.74', 'logps/rejected': '-87.19', 'logits/chosen': '-2.699', 'logits/rejected': '-2.621', 'epoch': '0.07619'}
{'loss': '0.6668', 'grad_norm': '0.8357', 'learning_rate': '0.0001904', 'rewards/chosen': '-0.2063', 'rewards/rejected': '-0.32', 'rewards/accuracies': '0.5289', 'rewards/margins': '0.1137', 'logps/chosen': '-82.09', 'logps/rejected': '-89.19', 'logits/chosen': '-2.595', 'logits/rejected': '-2.484', 'epoch': '0.08707'}
{'loss': '0.6667', 'grad_norm': '0.7759', 'learning_rate': '0.0001996', 'rewards/chosen': '-0.1989', 'rewards/rejected': '-0.3181', 'rewards/accuracies': '0.5469', 'rewards/margins': '0.1192', 'logps/chosen': '-80.27', 'logps/rejected': '-88.54', 'logits/chosen': '-2.37', 'logits/rejected': '-2.255', 'epoch': '0.09796'}
{'loss': '0.6667', 'grad_norm': '0.7275', 'learning_rate': '0.0001988', 'rewards/chosen': '-0.2421', 'rewards/rejected': '-0.3706', 'rewards/accuracies': '0.5492', 'rewards/margins': '0.1285', 'logps/chosen': '-85.48', 'logps/rejected': '-92.23', 'logits/chosen': '-2.458', 'logits/rejected': '-2.306', 'epoch': '0.1088'}
{'loss': '0.6598', 'grad_norm': '0.8685', 'learning_rate': '0.0001981', 'rewards/chosen': '-0.1943', 'rewards/rejected': '-0.3449', 'rewards/accuracies': '0.5523', 'rewards/margins': '0.1507', 'logps/chosen': '-81.67', 'logps/rejected': '-89.74', 'logits/chosen': '-2.194', 'logits/rejected': '-2.128', 'epoch': '0.1197'}
{'loss': '0.6632', 'grad_norm': '0.7957', 'learning_rate': '0.0001973', 'rewards/chosen': '-0.1917', 'rewards/rejected': '-0.3336', 'rewards/accuracies': '0.5422', 'rewards/margins': '0.1418', 'logps/chosen': '-84.79', 'logps/rejected': '-90.5', 'logits/chosen': '-2.447', 'logits/rejected': '-2.352', 'epoch': '0.1306'}
{'loss': '0.6509', 'grad_norm': '0.6242', 'learning_rate': '0.0001966', 'rewards/chosen': '-0.1273', 'rewards/rejected': '-0.2984', 'rewards/accuracies': '0.5508', 'rewards/margins': '0.1711', 'logps/chosen': '-79.19', 'logps/rejected': '-85.88', 'logits/chosen': '-2.458', 'logits/rejected': '-2.388', 'epoch': '0.1415'}
{'loss': '0.6572', 'grad_norm': '0.6959', 'learning_rate': '0.0001958', 'rewards/chosen': '-0.1161', 'rewards/rejected': '-0.282', 'rewards/accuracies': '0.5641', 'rewards/margins': '0.166', 'logps/chosen': '-82.53', 'logps/rejected': '-89.38', 'logits/chosen': '-2.645', 'logits/rejected': '-2.575', 'epoch': '0.1524'}
{'loss': '0.6591', 'grad_norm': '0.704', 'learning_rate': '0.0001951', 'rewards/chosen': '-0.0406', 'rewards/rejected': '-0.2036', 'rewards/accuracies': '0.5641', 'rewards/margins': '0.163', 'logps/chosen': '-84.36', 'logps/rejected': '-91.33', 'logits/chosen': '-2.601', 'logits/rejected': '-2.53', 'epoch': '0.1633'}
{'loss': '0.6441', 'grad_norm': '0.6618', 'learning_rate': '0.0001943', 'rewards/chosen': '-0.01285', 'rewards/rejected': '-0.201', 'rewards/accuracies': '0.5883', 'rewards/margins': '0.1881', 'logps/chosen': '-83.15', 'logps/rejected': '-89.39', 'logits/chosen': '-2.715', 'logits/rejected': '-2.597', 'epoch': '0.1741'}
{'loss': '0.653', 'grad_norm': '0.6484', 'learning_rate': '0.0001936', 'rewards/chosen': '-0.05956', 'rewards/rejected': '-0.2501', 'rewards/accuracies': '0.5641', 'rewards/margins': '0.1905', 'logps/chosen': '-79.66', 'logps/rejected': '-85.64', 'logits/chosen': '-2.808', 'logits/rejected': '-2.71', 'epoch': '0.185'}
{'loss': '0.671', 'grad_norm': '0.6358', 'learning_rate': '0.0001928', 'rewards/chosen': '-0.1839', 'rewards/rejected': '-0.3243', 'rewards/accuracies': '0.557', 'rewards/margins': '0.1405', 'logps/chosen': '-86.25', 'logps/rejected': '-91.76', 'logits/chosen': '-2.726', 'logits/rejected': '-2.632', 'epoch': '0.1959'}
{'loss': '0.6496', 'grad_norm': '0.7087', 'learning_rate': '0.0001921', 'rewards/chosen': '-0.259', 'rewards/rejected': '-0.4433', 'rewards/accuracies': '0.5711', 'rewards/margins': '0.1842', 'logps/chosen': '-83.1', 'logps/rejected': '-90.81', 'logits/chosen': '-2.575', 'logits/rejected': '-2.476', 'epoch': '0.2068'}
{'loss': '0.6528', 'grad_norm': '0.7277', 'learning_rate': '0.0001913', 'rewards/chosen': '-0.2878', 'rewards/rejected': '-0.4894', 'rewards/accuracies': '0.5523', 'rewards/margins': '0.2016', 'logps/chosen': '-84.37', 'logps/rejected': '-92.43', 'logits/chosen': '-2.665', 'logits/rejected': '-2.642', 'epoch': '0.2177'}
{'loss': '0.6556', 'grad_norm': '0.7041', 'learning_rate': '0.0001906', 'rewards/chosen': '-0.175', 'rewards/rejected': '-0.3603', 'rewards/accuracies': '0.5703', 'rewards/margins': '0.1853', 'logps/chosen': '-83.11', 'logps/rejected': '-90.62', 'logits/chosen': '-2.796', 'logits/rejected': '-2.704', 'epoch': '0.2286'}
{'loss': '0.6463', 'grad_norm': '0.6894', 'learning_rate': '0.0001898', 'rewards/chosen': '0.06025', 'rewards/rejected': '-0.1298', 'rewards/accuracies': '0.5805', 'rewards/margins': '0.19', 'logps/chosen': '-78.92', 'logps/rejected': '-84.79', 'logits/chosen': '-2.694', 'logits/rejected': '-2.68', 'epoch': '0.2395'}
{'loss': '0.6312', 'grad_norm': '0.6098', 'learning_rate': '0.0001891', 'rewards/chosen': '0.03963', 'rewards/rejected': '-0.1992', 'rewards/accuracies': '0.5922', 'rewards/margins': '0.2389', 'logps/chosen': '-80.18', 'logps/rejected': '-86.77', 'logits/chosen': '-2.949', 'logits/rejected': '-2.881', 'epoch': '0.2503'}
{'loss': '0.6405', 'grad_norm': '0.6649', 'learning_rate': '0.0001883', 'rewards/chosen': '-0.1528', 'rewards/rejected': '-0.3926', 'rewards/accuracies': '0.593', 'rewards/margins': '0.2397', 'logps/chosen': '-82.76', 'logps/rejected': '-92.11', 'logits/chosen': '-3.144', 'logits/rejected': '-3.156', 'epoch': '0.2612'}
{'loss': '0.629', 'grad_norm': '0.7529', 'learning_rate': '0.0001876', 'rewards/chosen': '-0.3856', 'rewards/rejected': '-0.6723', 'rewards/accuracies': '0.5914', 'rewards/margins': '0.2867', 'logps/chosen': '-88.3', 'logps/rejected': '-97.05', 'logits/chosen': '-3.041', 'logits/rejected': '-3.004', 'epoch': '0.2721'}
{'loss': '0.62', 'grad_norm': '0.6279', 'learning_rate': '0.0001868', 'rewards/chosen': '-0.438', 'rewards/rejected': '-0.7219', 'rewards/accuracies': '0.5859', 'rewards/margins': '0.284', 'logps/chosen': '-87.58', 'logps/rejected': '-93.7', 'logits/chosen': '-3.091', 'logits/rejected': '-2.95', 'epoch': '0.283'} 9%|▉ | 260/2757 [35:00<5:38:48, 8.14s/it] 9%|▉ | 261/2757 [35:08<5:36:28, 8.09s/it] 10%|▉ | 262/2757 [35:16<5:35:03, 8.06s/it] 10%|▉ | 263/2757 [35:24<5:34:47, 8.05s/it] 10%|▉ | 264/2757 [35:32<5:35:24, 8.07s/it] 10%|▉ | 265/2757 [35:41<5:40:45, 8.20s/it] 10%|▉ | 266/2757 [35:49<5:38:07, 8.14s/it] 10%|▉ | 267/2757 [35:57<5:41:59, 8.24s/it] 10%|▉ | 268/2757 [36:05<5:31:09, 7.98s/it] 10%|▉ | 269/2757 [36:13<5:32:00, 8.01s/it] 10%|▉ | 270/2757 [36:20<5:25:24, 7.85s/it] 10%|▉ | 270/2757 [36:20<5:25:24, 7.85s/it] 10%|▉ | 271/2757 [36:28<5:27:27, 7.90s/it] 10%|▉ | 272/2757 [36:36<5:29:00, 7.94s/it] 10%|▉ | 273/2757 [36:45<5:34:02, 8.07s/it] 10%|▉ | 274/2757 [36:53<5:33:43, 8.06s/it] 10%|▉ | 275/2757 [37:01<5:31:46, 8.02s/it] 10%|█ | 276/2757 [37:08<5:25:47, 7.88s/it] 10%|█ | 277/2757 [37:17<5:36:08, 8.13s/it] 10%|█ | 278/2757 [37:25<5:34:53, 8.11s/it] 10%|█ | 279/2757 [37:34<5:39:02, 8.21s/it] 10%|█ | 280/2757 [37:42<5:36:54, 8.16s/it] 10%|█ | 280/2757 [37:42<5:36:54, 8.16s/it] 10%|█ | 281/2757 [37:49<5:30:11, 8.00s/it] 10%|█ | 282/2757 [37:57<5:32:12, 8.05s/it] 10%|█ | 283/2757 [38:06<5:35:45, 8.14s/it] 10%|█ | 284/2757 [38:14<5:34:14, 8.11s/it] 10%|█ | 285/2757 [38:21<5:29:31, 8.00s/it] 10%|█ | 286/2757 [38:30<5:35:09, 8.14s/it] 10%|█ | 287/2757 [38:38<5:28:24, 7.98s/it] 10%|█ | 288/2757 [38:46<5:36:57, 8.19s/it] 10%|█ | 289/2757 [38:54<5:32:11, 8.08s/it] 11%|█ | 290/2757 [39:02<5:33:59, 8.12s/it] 11%|█ | 290/2757 [39:02<5:33:59, 8.12s/it] 11%|█ | 291/2757 [39:10<5:27:12, 7.96s/it] 11%|█ | 292/2757 [39:18<5:25:56, 7.93s/it] 11%|█ | 293/2757 [39:25<5:20:08, 7.80s/it] 11%|█ | 294/2757 [39:33<5:21:08, 7.82s/it] 11%|█ | 295/2757 [39:42<5:30:09, 8.05s/it] 11%|█ | 296/2757 [39:49<5:26:05, 7.95s/it] 11%|█ | 297/2757 [39:57<5:25:10, 7.93s/it] 11%|█ | 298/2757 [40:06<5:28:38, 8.02s/it] 11%|█ | 299/2757 [40:13<5:27:05, 7.98s/it] 11%|█ | 300/2757 [40:22<5:29:03, 8.04s/it] 11%|█ | 300/2757 [40:22<5:29:03, 8.04s/it] 11%|█ | 301/2757 [40:30<5:36:05, 8.21s/it] 11%|█ | 302/2757 [40:38<5:29:15, 8.05s/it] 11%|█ | 303/2757 [40:46<5:28:38, 8.04s/it] 11%|█ | 304/2757 [40:54<5:25:46, 7.97s/it] 11%|█ | 305/2757 [41:02<5:25:20, 7.96s/it] 11%|█ | 306/2757 [41:10<5:25:19, 7.96s/it] 11%|█ | 307/2757 [41:18<5:28:42, 8.05s/it] 11%|█ | 308/2757 [41:26<5:32:15, 8.14s/it] 11%|█ | 309/2757 [41:34<5:29:04, 8.07s/it] 11%|█ | 310/2757 [41:43<5:34:50, 8.21s/it] 11%|█ | 310/2757 [41:43<5:34:50, 8.21s/it] 11%|█▏ | 311/2757 [41:51<5:34:07, 8.20s/it] 11%|█▏ | 312/2757 [41:59<5:29:01, 8.07s/it] 11%|█▏ | 313/2757 [42:07<5:28:13, 8.06s/it] 11%|█▏ | 314/2757 [42:15<5:30:44, 8.12s/it] 11%|█▏ | 315/2757 [42:23<5:27:30, 8.05s/it] 11%|█▏ | 316/2757 [42:31<5:24:03, 7.97s/it] 11%|█▏ | 317/2757 [42:39<5:28:08, 8.07s/it] 12%|█▏ | 318/2757 [
{'loss': '0.6296', 'grad_norm': '0.5629', 'learning_rate': '0.0001861', 'rewards/chosen': '-0.3549', 'rewards/rejected': '-0.6289', 'rewards/accuracies': '0.5961', 'rewards/margins': '0.274', 'logps/chosen': '-80.02', 'logps/rejected': '-88.44', 'logits/chosen': '-3.056', 'logits/rejected': '-2.951', 'epoch': '0.2939'}
{'loss': '0.626', 'grad_norm': '0.6267', 'learning_rate': '0.0001853', 'rewards/chosen': '-0.3159', 'rewards/rejected': '-0.5841', 'rewards/accuracies': '0.5914', 'rewards/margins': '0.2681', 'logps/chosen': '-79.61', 'logps/rejected': '-87.73', 'logits/chosen': '-3.073', 'logits/rejected': '-2.979', 'epoch': '0.3048'}
{'loss': '0.6292', 'grad_norm': '0.6236', 'learning_rate': '0.0001846', 'rewards/chosen': '-0.253', 'rewards/rejected': '-0.538', 'rewards/accuracies': '0.5852', 'rewards/margins': '0.285', 'logps/chosen': '-82', 'logps/rejected': '-90.57', 'logits/chosen': '-3.13', 'logits/rejected': '-3.074', 'epoch': '0.3156'}
{'loss': '0.6195', 'grad_norm': '0.6637', 'learning_rate': '0.0001838', 'rewards/chosen': '-0.08129', 'rewards/rejected': '-0.3927', 'rewards/accuracies': '0.6008', 'rewards/margins': '0.3114', 'logps/chosen': '-80.13', 'logps/rejected': '-88.26', 'logits/chosen': '-3.198', 'logits/rejected': '-3.114', 'epoch': '0.3265'}
{'loss': '0.6184', 'grad_norm': '0.6673', 'learning_rate': '0.0001831', 'rewards/chosen': '-0.3788', 'rewards/rejected': '-0.6948', 'rewards/accuracies': '0.5984', 'rewards/margins': '0.316', 'logps/chosen': '-86.59', 'logps/rejected': '-95.23', 'logits/chosen': '-2.986', 'logits/rejected': '-2.906', 'epoch': '0.3374'}
{'loss': '0.6248', 'grad_norm': '0.6171', 'learning_rate': '0.0001823', 'rewards/chosen': '-0.5991', 'rewards/rejected': '-0.8997', 'rewards/accuracies': '0.5844', 'rewards/margins': '0.3006', 'logps/chosen': '-88.97', 'logps/rejected': '-97.9', 'logits/chosen': '-3.004', 'logits/rejected': '-2.952', 'epoch': '0.3483'}
{'loss': '0.6249', 'grad_norm': '0.7893', 'learning_rate': '0.0001816', 'rewards/chosen': '-0.639', 'rewards/rejected': '-0.9381', 'rewards/accuracies': '0.5859', 'rewards/margins': '0.2991', 'logps/chosen': '-84.33', 'logps/rejected': '-94.08', 'logits/chosen': '-3.131', 'logits/rejected': '-3.113', 'epoch': '0.3592'}
{'loss': '0.6234', 'grad_norm': '0.5814', 'learning_rate': '0.0001809', 'rewards/chosen': '-0.6253', 'rewards/rejected': '-0.9446', 'rewards/accuracies': '0.5922', 'rewards/margins': '0.3192', 'logps/chosen': '-86.04', 'logps/rejected': '-93.89', 'logits/chosen': '-3.178', 'logits/rejected': '-3.083', 'epoch': '0.3701'}
{'loss': '0.6068', 'grad_norm': '0.6354', 'learning_rate': '0.0001801', 'rewards/chosen': '-0.5151', 'rewards/rejected': '-0.8645', 'rewards/accuracies': '0.6133', 'rewards/margins': '0.3494', 'logps/chosen': '-89.97', 'logps/rejected': '-98.78', 'logits/chosen': '-2.931', 'logits/rejected': '-2.912', 'epoch': '0.381'}
{'loss': '0.6245', 'grad_norm': '0.6956', 'learning_rate': '0.0001794', 'rewards/chosen': '-0.5855', 'rewards/rejected': '-0.8816', 'rewards/accuracies': '0.6016', 'rewards/margins': '0.2961', 'logps/chosen': '-84.48', 'logps/rejected': '-91.43', 'logits/chosen': '-2.768', 'logits/rejected': '-2.695', 'epoch': '0.3918'}
{'loss': '0.623', 'grad_norm': '0.5492', 'learning_rate': '0.0001786', 'rewards/chosen': '-0.6255', 'rewards/rejected': '-0.9144', 'rewards/accuracies': '0.6023', 'rewards/margins': '0.2889', 'logps/chosen': '-88.09', 'logps/rejected': '-94.99', 'logits/chosen': '-2.877', 'logits/rejected': '-2.792', 'epoch': '0.4027'}
{'loss': '0.6103', 'grad_norm': '0.6273', 'learning_rate': '0.0001779', 'rewards/chosen': '-0.5627', 'rewards/rejected': '-0.8772', 'rewards/accuracies': '0.6078', 'rewards/margins': '0.3145', 'logps/chosen': '-89.07', 'logps/rejected': '-97.89', 'logits/chosen': '-2.693', 'logits/rejected': '-2.596', 'epoch': '0.4136'}
{'loss': '0.6128', 'grad_norm': '0.6782', 'learning_rate': '0.0001771', 'rewards/chosen': '-0.5093', 'rewards/rejected': '-0.8476', 'rewards/accuracies': '0.6219', 'rewards/margins': '0.3384', 'logps/chosen': '-85.96', 'logps/rejected': '-94.58', 'logits/chosen': '-2.902', 'logits/rejected': '-2.792', 'epoch': '0.4245'}
{'loss': '0.6213', 'grad_norm': '0.6319', 'learning_rate': '0.0001764', 'rewards/chosen': '-0.3547', 'rewards/rejected': '-0.6653', 'rewards/accuracies': '0.6102', 'rewards/margins': '0.3106', 'logps/chosen': '-82.64', 'logps/rejected': '-93.14', 'logits/chosen': '-2.978', 'logits/rejected': '-2.966', 'epoch': '0.4354'}
{'loss': '0.6141', 'grad_norm': '0.6016', 'learning_rate': '0.0001756', 'rewards/chosen': '-0.3532', 'rewards/rejected': '-0.698', 'rewards/accuracies': '0.5945', 'rewards/margins': '0.3448', 'logps/chosen': '-84.97', 'logps/rejected': '-93.02', 'logits/chosen': '-3.077', 'logits/rejected': '-3.008', 'epoch': '0.4463'}
{'loss': '0.6004', 'grad_norm': '0.6052', 'learning_rate': '0.0001749', 'rewards/chosen': '-0.2189', 'rewards/rejected': '-0.587', 'rewards/accuracies': '0.6258', 'rewards/margins': '0.3681', 'logps/chosen': '-79.92', 'logps/rejected': '-88.4', 'logits/chosen': '-3.087', 'logits/rejected': '-2.981', 'epoch': '0.4571'}
{'loss': '0.62', 'grad_norm': '0.73', 'learning_rate': '0.0001741', 'rewards/chosen': '-0.2221', 'rewards/rejected': '-0.5518', 'rewards/accuracies': '0.6047', 'rewards/margins': '0.3297', 'logps/chosen': '-84.05', 'logps/rejected': '-91.39', 'logits/chosen': '-3.061', 'logits/rejected': '-3.027', 'epoch': '0.468'}
{'loss': '0.6032', 'grad_norm': '0.5996', 'learning_rate': '0.0001734', 'rewards/chosen': '-0.5063', 'rewards/rejected': '-0.86', 'rewards/accuracies': '0.6305', 'rewards/margins': '0.3537', 'logps/chosen': '-88.16', 'logps/rejected': '-94.57', 'logits/chosen': '-2.93', 'logits/rejected': '-2.835', 'epoch': '0.4789'}
{'loss': '0.5989', 'grad_norm': '0.6094', 'learning_rate': '0.0001726', 'rewards/chosen': '-0.5802', 'rewards/rejected': '-0.9776', 'rewards/accuracies': '0.6117', 'rewards/margins': '0.3973', 'logps/chosen': '-85.01', 'logps/rejected': '-94.11', 'logits/chosen': '-3.055', 'logits/rejected': '-2.933', 'epoch': '0.4898'}
{'loss': '0.6101', 'grad_norm': '0.6395', 'learning_rate': '0.0001719', 'rewards/chosen': '-0.5156', 'rewards/rejected': '-0.8643', 'rewards/accuracies': '0.6172', 'rewards/margins': '0.3487', 'logps/chosen': '-86.3', 'logps/rejected': '-94.5', 'logits/chosen': '-2.982', 'logits/rejected': '-2.935', 'epoch': '0.5007'}
{'loss': '0.6019', 'grad_norm': '0.5491', 'learning_rate': '0.0001711', 'rewards/chosen': '-0.3541', 'rewards/rejected': '-0.7315', 'rewards/accuracies': '0.6062', 'rewards/margins': '0.3774', 'logps/chosen': '-81.23', 'logps/rejected': '-88.91', 'logits/chosen': '-2.927', 'logits/rejected': '-2.875', 'epoch': '0.5116'}
{'loss': '0.5966', 'grad_norm': '0.6346', 'learning_rate': '0.0001704', 'rewards/chosen': '-0.242', 'rewards/rejected': '-0.6034', 'rewards/accuracies': '0.6055', 'rewards/margins': '0.3614', 'logps/chosen': '-82.53', 'logps/rejected': '-91.44', 'logits/chosen': '-2.846', 'logits/rejected': '-2.819', 'epoch': '0.5224'}
{'loss': '0.5878', 'grad_norm': '0.6204', 'learning_rate': '0.0001696', 'rewards/chosen': '-0.3545', 'rewards/rejected': '-0.7606', 'rewards/accuracies': '0.643', 'rewards/margins': '0.4061', 'logps/chosen': '-83.22', 'logps/rejected': '-91.02', 'logits/chosen': '-3.038', 'logits/rejected': '-2.952', 'epoch': '0.5333'}
{'loss': '0.5975', 'grad_norm': '0.6997', 'learning_rate': '0.0001689', 'rewards/chosen': '-0.6454', 'rewards/rejected': '-1.051', 'rewards/accuracies': '0.6109', 'rewards/margins': '0.4053', 'logps/chosen': '-88.25', 'logps/rejected': '-98.36', 'logits/chosen': '-3.002', 'logits/rejected': '-2.958', 'epoch': '0.5442'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
0%| | 0/300 [00:00<?, ?it/s]
1%| | 2/300 [00:00<00:52, 5.64it/s]
1%| | 3/300 [00:00<01:46, 2.79it/s]
1%|▏ | 4/300 [00:01<01:34, 3.14it/s]
2%|▏ | 5/300 [00:01<01:44, 2.82it/s]
2%|▏ | 6/300 [00:01<01:40, 2.93it/s]
2%|▏ | 7/300 [00:02<01:54, 2.56it/s]
3%|▎ | 8/300 [00:02<01:50, 2.64it/s]
3%|▎ | 9/300 [00:03<01:53, 2.57it/s]
3%|▎ | 10/300 [00:03<01:49, 2.66it/s]
4%|▎ | 11/300 [00:03<01:51, 2.59it/s]
4%|▍ | 12/300 [00:04<01:38, 2.92it/s]
4%|▍ | 13/300 [00:04<01:45, 2.72it/s]
5%|▍ | 14/300 [00:04<01:37, 2.93it/s]
5%|▌ | 15/300 [00:05<01:47, 2.66it/s]
5%|▌ | 16/300 [00:05<01:34, 2.99it/s]
6%|▌ | 17/300 [00:05<01:38, 2.87it/s]
6%|▌ | 18/300 [00:06<01:28, 3.18it/s]
6%|▋ | 19/300 [00:06<01:39, 2.83it/s]
7%|▋ | 20/300 [00:06<01:31, 3.05it/s]
7%|▋ | 21/300 [00:07<01:41, 2.74it/s]
7%|▋ | 22/300 [00:07<01:30, 3.06it/s]
8%|▊ | 23/300 [00:07<01:32, 3.00it/s]
8%|▊ | 24/300 [00:08<01:23, 3.31it/s]
8%|▊ | 25/300 [00:08<01:35, 2.87it/s]
9%|▊ | 26/300 [00:08<01:29, 3.08it/s]
9%|▉ | 27/300 [00:09<01:37, 2.80it/s]
9%|▉ | 28/300 [00:09<01:26, 3.15it/s]
10%|▉ | 29/300 [00:09<01:29, 3.03it/s]
10%|█ | 30/300 [00:10<01:21, 3.33it/s]
10%|█ | 31/300 [00:10<01:23, 3.22it/s]
11%|█ | 32/300 [00:10<01:17, 3.48it/s]
11%|█ | 33/300 [00:11<01:19, 3.34it/s]
11%|█▏ | 34/300 [00:11<01:15, 3.52it/s]
12%|█▏ | 35/300 [00:11<01:22, 3.23it/s]
12%|█▏ | 36/300 [00:11<01:17, 3.39it/s]
12%|█▏ | 37/300 [00:12<01:26, 3.04it/s]
13%|█▎ | 38/300 [00:12<01:17, 3.36it/s]
13%|█▎ | 39/300 [00:13<01:40, 2.60it/s]
13%|█▎ | 40/300 [00:13<01:28, 2.94it/s]
14%|█▎ | 41/300 [00:13<01:32, 2.81it/s]
14%|█▍ | 42/300 [00:14<01:22, 3.14it/s]
14%|█▍ | 43/300 [00:14<01:25, 3.02it/s]
15%|█▍ | 44/300 [00:14<01:17, 3.30it/s]
15%|█▌ | 45/300 [00:15<01:23, 3.04it/s]
15%|█▌ | 46/300 [00:15<01:24, 3.00it/s]
16%|█▌ | 47/300 [00:15<01:30, 2.80it/s]
16%|█▌ | 48/300 [00:16<01:22, 3.04it/s]
16%|█▋ | 49/300 [00:16<01:23, 3.02it/s]
17%|█▋ | 50/300 [00:16<01:14, 3.36it/s]
17%|█▋ | 51/300 [00:16<01:18, 3.19it/s]
17%|█▋ | 52/300 [00:17<01:15, 3.29it/s]
18%|█▊ | 53/300 [00:17<01:28, 2.80it/s]
18%|█▊ | 54/300 [00:17<01:19, 3.09it/s]
18%|█▊ | 55/300 [00:18<01:29, 2.73it/s]
19%|█▊ | 56/300 [00:18<01:17, 3.15it/s]
19%|█▉ | 57/300 [00:18<01:20, 3.01it/s]
19%|█▉ | 58/300 [00:19<01:21, 2.96it/s]
20%|█▉ | 59/300 [00:19<01:26, 2.79it/s]
20%|██ | 60/300 [00:20<01:18, 3.08it/s]
20%|██ | 61/300 [00:20<01:27, 2.72it/s]
21%|██ | 62/300 [00:20<01:25, 2.78it/s]
21%|██ | 63/300 [00:21<01:32, 2.56it/s]
21%|██▏ | 64/300 [00:21<01:28, 2.66it/s]
22%|██▏ | 65/300 [00:22<01:30, 2.59it/s]
22%|██▏ | 66/300 [00:22<01:21, 2.87it/s]
22%|██▏ | 67/300 [00:22<01:29, 2.62it/s]
23%|██▎ | 68/300 [00:22<01:19, 2.91it/s]
23%|██▎ | 69/300 [00:23<01:22, 2.81it/s]
23%|██▎ | 70/300 [00:23<01:13, 3.12it/s]
24%|██▎ | 71/300 [00:23<01:15, 3.03it/s]
24%|██▍ | 72/300 [00:24<01:09, 3.29it/s]
24%|██▍ | 73/300 [00:24<01:12, 3.14it/s]
25%|██▍ | 74/300 [00:24<01:08, 3.32it/s]
25%|██▌ | 75/300 [00:25<01:15, 2.98it/s]
25%|██▌ | 76/300 [00:25<01:15, 2.95it/s]
26%|██▌ | 77/300 [00:26<01:20, 2.76it/s]
26%|██▌ | 78/300 [00:26<01:13, 3.01it/s]
26%|██▋ | 79/300 [00:26<01:15, 2.91it/s]
27%|██▋ | 80/300 [00:26<01:11, 3.08it/s]
27%|██▋ | 81/300 [00:27<01:19, 2.76it/s]
27%|██▋ | 82/300 [00:27<01:11, 3.07it/s]
28%|██▊ | 83/300 [00:27<01:13, 2.97it/s]
28%|██▊ | 84/300 [00:28<01:13, 2.93it/s]
28%|██▊ | 85/300 [00:28<01:18, 2.74it/s]
29%|██▊ | 86/300 [00:28<01:10, 3.04it/s]
29%|██▉ | 87/300 [00:29<01:12, 2.94it/s]
29%|██▉ | 88/300 [00:29<01:06, 3.19it/s]
30%|██▉ | 89/300 [00:29<01:10, 2.98it/s]
30%|███ | 90/300 [00:30<01:05, 3.22it/s]
30%|███ | 91/300 [00:30<01:09, 3.03it/s]
31%|███ | 92/300 [00:30<01:03, 3.28it/s]
31%|███ | 93/300 [00:31<01:09, 2.99it/s]
31%|███▏ | 94/300 [00:31<01:02, 3.29it/s]
32%|███▏ | 95/300 [00:31<01:07, 3.02it/s]
32%|███▏ | 96/300 [00:32<01:03, 3.21it/s]
32%|███▏ | 97/300 [00:32<01:05, 3.09it/s]
33%|███▎ | 98/300 [00:32<01:02, 3.25it/s]
33%|███▎ | 99/300 [00:33<01:07, 3.00it/s]
33%|███▎ | 100/300 [00:33<01:03, 3.16it/s]
34%|███▎ | 101/300 [00:33<01:07, 2.93it/s]
34%|███▍ | 102/300 [00:34<01:06, 3.00it/s]
34%|███▍ | 103/300 [00:34<01:09, 2.85it/s]
35%|███▍ | 104/300 [00:34<01:01, 3.18it/s]
35%|███▌ | 105/300 [00:35<01:04, 3.01it/s]
35%|███▌ | 106/300 [00:35<01:00, 3.21it/s]
36%|███▌ | 107/300 [00:35<01:05, 2.96it/s]
36%|███▌ | 108/300 [00:36<01:02, 3.06it/s]
36%|███▋ | 109/300 [00:36<01:06, 2.87it/s]
37%|███▋ | 110/300 [00:36<01:02, 3.03it/s]
37%|███▋ | 111/300 [00:37<01:11, 2.65it/s]
37%|███▋ | 112/300 [00:37<01:05, 2.85it/s]
38%|███▊ | 113/300 [00:37<01:07, 2.75it/s]
38%|███▊ | 114/300 [00:38<01:06, 2.79it/s]
38%|███▊ | 115/300 [00:38<01:09, 2.65it/s]
39%|███▊ | 116/300 [00:38<01:01, 2.98it/s]
39%|███▉ | 117/300 [00:39<01:03, 2.90it/s]
39%|███▉ | 118/300 [00:39<00:58, 3.12it/s]
40%|███▉ | 119/300 [00:39<01:00, 2.99it/s]
40%|████ | 120/300 [00:40<00:59, 3.01it/s]
40%|████ | 121/300 [00:40<01:03, 2.83it/s]
41%|████ | 122/300 [00:40<00:57, 3.07it/s]
41%|████ | 123/300 [00:41<01:01, 2.87it/s]
41%|████▏ | 124/300 [00:41<00:57, 3.04it/s]
42%|████▏ | 125/300 [00:42<00:59, 2.94it/s]
42%|████▏ | 126/300 [00:42<00:59, 2.91it/s]
42%|████▏ | 127/300 [00:42<01:02, 2.76it/s]
43%|████▎ | 128/300 [00:43<00:58, 2.97it/s]
43%|████▎ | 129/300 [00:43<01:09, 2.46it/s]
43%|████▎ | 130/300 [00:43<01:00, 2.79it/s]
44%|████▎ | 131/300 [00:44<01:05, 2.58it/s]
44%|████▍ | 132/300 [00:44<00:58, 2.86it/s]
44%|████▍ | 133/300 [00:45<01:05, 2.56it/s]
45%|████▍ | 134/300 [00:45<00:58, 2.84it/s]
45%|████▌ | 135/300 [00:45<01:02, 2.65it/s]
45%|████▌ | 136/300 [00:46<00:55, 2.96it/s]
46%|████▌ | 137/300 [00:46<00:55, 2.93it/s]
46%|████▌ | 138/300 [00:46<00:51, 3.17it/s]
46%|████▋ | 139/300 [00:46<00:51, 3.12it/s]
47%|████▋ | 140/300 [00:47<00:47, 3.37it/s]
47%|████▋ | 141/300 [00:47<00:49, 3.20it/s]
47%|████▋ | 142/300 [00:47<00:51, 3.09it/s]
48%|████▊ | 143/300 [00:48<00:54, 2.89it/s]
48%|████▊ | 144/300 [00:48<00:48, 3.20it/s]
48%|████▊ | 145/300 [00:48<00:50, 3.09it/s]
49%|████▊ | 146/300 [00:49<00:47, 3.26it/s]
49%|████▉ | 147/300 [00:49<00:53, 2.84it/s]
49%|████▉ | 148/300 [00:49<00:50, 3.01it/s]
50%|████▉ | 149/300 [00:50<00:54, 2.79it/s]
50%|█████ | 150/300 [00:50<00:50, 2.99it/s]
50%|█████ | 151/300 [00:50<00:51, 2.88it/s]
51%|█████ | 152/300 [00:51<00:46, 3.20it/s]
51%|█████ | 153/300 [00:51<00:47, 3.09it/s]
51%|█████▏ | 154/300 [00:51<00:43, 3.37it/s]
52%|█████▏ | 155/300 [00:52<00:45, 3.17it/s]
52%|█████▏ | 156/300 [00:52<00:41, 3.50it/s]
52%|█████▏ | 157/300 [00:52<00:48, 2.97it/s]
53%|█████▎ | 158/300 [00:53<00:45, 3.14it/s]
53%|█████▎ | 159/300 [00:53<00:51, 2.72it/s]
53%|█████▎ | 160/300 [00:53<00:47, 2.96it/s]
54%|█████▎ | 161/300 [00:54<00:51, 2.72it/s]
54%|█████▍ | 162/300 [00:54<00:50, 2.76it/s]
54%|█████▍ | 163/300 [00:55<00:55, 2.45it/s]
55%|█████▍ | 164/300 [00:55<00:53, 2.56it/s]
55%|█████▌ | 165/300 [00:55<00:52, 2.56it/s]
55%|█████▌ | 166/300 [00:56<00:50, 2.67it/s]
56%|█████▌ | 167/300 [00:56<00:54, 2.43it/s]
56%|█████▌ | 168/300 [00:56<00:47, 2.75it/s]
56%|█████▋ | 169/300 [00:57<00:52, 2.48it/s]
57%|█████▋ | 170/300 [00:57<00:50, 2.58it/s]
57%|█████▋ | 171/300 [00:58<00:50, 2.55it/s]
57%|█████▋ | 172/300 [00:58<00:44, 2.88it/s]
58%|█████▊ | 173/300 [00:58<00:46, 2.72it/s]
58%|█████▊ | 174/300 [00:59<00:41, 3.04it/s]
58%|█████▊ | 175/300 [00:59<00:45, 2.73it/s]
59%|█████▊ | 176/300 [00:59<00:40, 3.06it/s]
59%|█████▉ | 177/300 [01:00<00:44, 2.74it/s]
59%|█████▉ | 178/300 [01:00<00:38, 3.15it/s]
60%|█████▉ | 179/300 [01:00<00:41, 2.93it/s]
60%|██████ | 180/300 [01:01<00:37, 3.21it/s]
60%|██████ | 181/300 [01:01<00:41, 2.88it/s]
61%|██████ | 182/300 [01:01<00:36, 3.19it/s]
61%|██████ | 183/300 [01:02<00:39, 2.93it/s]
61%|██████▏ | 184/300 [01:02<00:39, 2.90it/s]
62%|██████▏ | 185/300 [01:02<00:42, 2.72it/s]
62%|██████▏ | 186/300 [01:03<00:39, 2.91it/s]
62%|██████▏ | 187/300 [01:03<00:39, 2.87it/s]
63%|██████▎ | 188/300 [01:03<00:35, 3.15it/s]
63%|██████▎ | 189/300 [01:04<00:38, 2.87it/s]
63%|██████▎ | 190/300 [01:04<00:35, 3.10it/s]
64%|██████▎ | 191/300 [01:04<00:36, 2.96it/s]
64%|██████▍ | 192/300 [01:05<00:36, 2.98it/s]
64%|██████▍ | 193/300 [01:05<00:41, 2.58it/s]
65%|██████▍ | 194/300 [01:06<00:39, 2.66it/s]
65%|██████▌ | 195/300 [01:06<00:42, 2.50it/s]
65%|██████▌ | 196/300 [01:06<00:38, 2.73it/s]
66%|██████▌ | 197/300 [01:07<00:37, 2.71it/s]
66%|██████▌ | 198/300 [01:07<00:34, 2.94it/s]
66%|██████▋ | 199/300 [01:07<00:35, 2.83it/s]
67%|██████▋ | 200/300 [01:08<00:33, 3.01it/s]
67%|██████▋ | 201/300 [01:08<00:35, 2.82it/s]
67%|██████▋ | 202/300 [01:08<00:31, 3.11it/s]
68%|██████▊ | 203/300 [01:09<00:32, 2.96it/s]
68%|██████▊ | 204/300 [01:09<00:29, 3.24it/s]
68%|██████▊ | 205/300 [01:09<00:31, 3.03it/s]
69%|██████▊ | 206/300 [01:10<00:27, 3.42it/s]
69%|██████▉ | 207/300 [01:10<00:31, 2.94it/s]
69%|██████▉ | 208/300 [01:10<00:31, 2.96it/s]
70%|██████▉ | 209/300 [01:11<00:32, 2.81it/s]
70%|███████ | 210/300 [01:11<00:29, 3.03it/s]
70%|███████ | 211/300 [01:11<00:33, 2.67it/s]
71%|███████ | 212/300 [01:12<00:29, 2.99it/s]
71%|███████ | 213/300 [01:12<00:31, 2.75it/s]
71%|███████▏ | 214/300 [01:12<00:28, 3.00it/s]
72%|███████▏ | 215/300 [01:13<00:31, 2.70it/s]
72%|███████▏ | 216/300 [01:13<00:27, 3.01it/s]
72%|███████▏ | 217/300 [01:14<00:34, 2.41it/s]
73%|███████▎ | 218/300 [01:14<00:28, 2.85it/s]
73%|███████▎ | 219/300 [01:14<00:30, 2.69it/s]
73%|███████▎ | 220/300 [01:15<00:26, 3.00it/s]
74%|███████▎ | 221/300 [01:15<00:28, 2.73it/s]
74%|███████▍ | 222/300 [01:15<00:25, 3.04it/s]
74%|███████▍ | 223/300 [01:16<00:28, 2.73it/s]
75%|███████▍ | 224/300 [01:16<00:25, 2.94it/s]
75%|███████▌ | 225/300 [01:16<00:26, 2.81it/s]
75%|███████▌ | 226/300 [01:17<00:24, 3.05it/s]
76%|███████▌ | 227/300 [01:17<00:26, 2.78it/s]
76%|███████▌ | 228/300 [01:17<00:23, 3.11it/s]
76%|███████▋ | 229/300 [01:18<00:25, 2.75it/s]
77%|███████▋ | 230/300 [01:18<00:24, 2.86it/s]
77%|███████▋ | 231/300 [01:18<00:25, 2.70it/s]
77%|███████▋ | 232/300 [01:19<00:24, 2.77it/s]
78%|███████▊ | 233/300 [01:19<00:27, 2.45it/s]
78%|███████▊ | 234/300 [01:20<00:23, 2.81it/s]
78%|███████▊ | 235/300 [01:20<00:24, 2.63it/s]
79%|███████▊ | 236/300 [01:20<00:23, 2.70it/s]
79%|███████▉ | 237/300 [01:21<00:25, 2.49it/s]
79%|███████▉ | 238/300 [01:21<00:21, 2.82it/s]
80%|███████▉ | 239/300 [01:21<00:21, 2.83it/s]
80%|████████ | 240/300 [01:22<00:18, 3.16it/s]
80%|████████ | 241/300 [01:22<00:19, 3.04it/s]
81%|████████ | 242/300 [01:22<00:18, 3.08it/s]
81%|████████ | 243/300 [01:23<00:20, 2.73it/s]
81%|████████▏ | 244/300 [01:23<00:19, 2.83it/s]
82%|████████▏ | 245/300 [01:24<00:20, 2.67it/s]
82%|████████▏ | 246/300 [01:24<00:19, 2.77it/s]
82%|████████▏ | 247/300 [01:24<00:20, 2.60it/s]
83%|████████▎ | 248/300 [01:25<00:17, 2.94it/s]
83%|████████▎ | 249/300 [01:25<00:18, 2.80it/s]
83%|████████▎ | 250/300 [01:25<00:17, 2.92it/s]
84%|████████▎ | 251/300 [01:26<00:17, 2.81it/s]
84%|████████▍ | 252/300 [01:26<00:15, 3.01it/s]
84%|████████▍ | 253/300 [01:26<00:17, 2.74it/s]
85%|████████▍ | 254/300 [01:27<00:14, 3.12it/s]
85%|████████▌ | 255/300 [01:27<00:16, 2.81it/s]
85%|████████▌ | 256/300 [01:27<00:13, 3.15it/s]
86%|████████▌ | 257/300 [01:28<00:14, 2.96it/s]
86%|████████▌ | 258/300 [01:28<00:13, 3.22it/s]
86%|████████▋ | 259/300 [01:28<00:13, 2.94it/s]
87%|████████▋ | 260/300 [01:29<00:12, 3.14it/s]
87%|████████▋ | 261/300 [01:29<00:13, 2.99it/s]
87%|████████▋ | 262/300 [01:29<00:11, 3.28it/s]
88%|████████▊ | 263/300 [01:30<00:12, 3.02it/s]
88%|████████▊ | 264/300 [01:30<00:11, 3.01it/s]
88%|████████▊ | 265/300 [01:30<00:13, 2.60it/s]
89%|████████▊ | 266/300 [01:31<00:12, 2.70it/s]
89%|████████▉ | 267/300 [01:31<00:13, 2.44it/s]
89%|████████▉ | 268/300 [01:31<00:11, 2.80it/s]
90%|████████▉ | 269/300 [01:32<00:12, 2.47it/s]
90%|█████████ | 270/300 [01:32<00:11, 2.64it/s]
90%|█████████ | 271/300 [01:33<00:12, 2.39it/s]
91%|█████████ | 272/300 [01:33<00:10, 2.65it/s]
91%|█████████ | 273/300 [01:34<00:11, 2.39it/s]
91%|█████████▏| 274/300 [01:34<00:09, 2.74it/s]
92%|█████████▏| 275/300 [01:34<00:10, 2.28it/s]
92%|█████████▏| 276/300 [01:35<00:09, 2.49it/s]
92%|█████████▏| 277/300 [01:35<00:09, 2.45it/s]
93%|█████████▎| 278/300 [01:35<00:07, 2.78it/s]
93%|█████████▎| 279/300 [01:36<00:07, 2.77it/s]
93%|█████████▎| 280/300 [01:36<00:06, 2.87it/s]
94%|█████████▎| 281/300 [01:37<00:07, 2.71it/s]
94%|█████████▍| 282/300 [01:37<00:05, 3.02it/s]
94%|█████████▍| 283/300 [01:37<00:06, 2.77it/s]
95%|█████████▍| 284/300 [01:37<00:05, 3.03it/s]
95%|█████████▌| 285/300 [01:38<00:05, 2.62it/s]
95%|█████████▌| 286/300 [01:38<00:05, 2.69it/s]
96%|█████████▌| 287/300 [01:39<00:04, 2.62it/s]
96%|█████████▌| 288/300 [01:39<00:03, 3.05it/s]
96%|█████████▋| 289/300 [01:39<00:03, 2.80it/s]
97%|█████████▋| 290/300 [01:40<00:03, 2.82it/s]
97%|█████████▋| 291/300 [01:40<00:03, 2.47it/s]
97%|█████████▋| 292/300 [01:40<00:02, 2.81it/s]
98%|█████████▊| 293/300 [01:41<00:02, 2.70it/s]
98%|█████████▊| 294/300 [01:41<00:02, 2.96it/s]
98%|█████████▊| 295/300 [01:41<00:01, 2.92it/s]
99%|█████████▊| 296/300 [01:42<00:01, 3.13it/s]
99%|█████████▉| 297/300 [01:42<00:01, 2.90it/s]
99%|█████████▉| 298/300 [01:42<00:00, 3.27it/s]
100%|█████████▉| 299/300 [01:43<00:00, 2.87it/s]
100%|██████████| 300/300 [01:43<00:00, 2.76it/s]
 18%|█▊ | 500/2757 [1:08:47<5:03:35, 8.07s/it]
100%|██████████| 300/300 [01:43<00:00, 2.76it/s]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-500/tokenizer_config.json.
18%|█▊ | 501/2757 [1:09:01<25:49:43, 41.22s/it] 18%|█▊ | 502/2757 [1:09:09<19:33:58, 31.24s/it] 18%|█▊ | 503/2757 [1:09:17<15:09:23, 24.21s/it] 18%|█▊ | 504/2757 [1:09:24<11:57:50, 19.12s/it] 18%|█▊ | 505/2757 [1:09:32<9:49:08, 15.70s/it] 18%|█▊ | 506/2757 [1:09:39<8:19:28, 13.31s/it] 18%|█▊ | 507/2757 [1:09:47<7:20:56, 11.76s/it] 18%|█▊ | 508/2757 [1:09:56<6:40:22, 10.68s/it] 18%|█▊ | 509/2757 [1:10:03<6:07:33, 9.81s/it] 18%|█▊ | 510/2757 [1:10:11<5:41:59, 9.13s/it] 18%|█▊ | 510/2757 [1:10:11<5:41:59, 9.13s/it] 19%|█▊ | 511/2757 [1:10:19<5:25:41, 8.70s/it] 19%|█▊ | 512/2757 [1:10:27<5:17:03, 8.47s/it] 19%|█▊ | 513/2757 [1:10:35<5:12:13, 8.35s/it] 19%|█▊ | 514/2757 [1:10:43<5:07:24, 8.22s/it] 19%|█▊ | 515/2757 [1:10:50<4:59:38, 8.02s/it] 19%|█▊ | 516/2757 [1:10:58<5:01:44, 8.08s/it] 19%|█▉ | 517/2757 [1:11:07<5:03:39, 8.13s/it] 19%|█▉ | 518/2757 [1:11:14<5:00:38, 8.06s/it] 19%|█▉ | 519/2757 [1:11:23<5:04:44, 8.17s/it] 19%|█▉ | 520/2757 [1:11:31<4:59:29, 8.03s/it] 19%|█▉ | 520/2757 [1:11:31<4:59:29, 8.03s/it] 19%|█▉ | 521/2757 [1:11:39<5:02:53, 8.13s/it] 19%|█▉ | 522/2757 [1:11:47<5:02:38, 8.12s/it] 19%|█▉ | 523/2757 [1:11:55<4:59:37, 8.05s/it] 19%|█▉ | 524/2757 [1:12:03<4:56:07, 7.96s/it] 19%|█▉ | 525/2757 [1:12:10<4:51:43, 7.84s/it] 19%|█▉ | 526/2757 [1:12:18<4:54:56, 7.93s/it] 19%|█▉ | 527/2757 [1:12:26<4:53:28, 7.90s/it] 19%|█▉ | 528/2757 [1:12:34<4:55:12, 7.95s/it] 19%|█▉ | 529/2757 [1:12:42<4:54:24, 7.93s/it] 19%|█▉ | 530/2757 [1:12:50<4:54:31, 7.94s/it] 19%|█▉ | 530/2757 [1:12:50<4:54:31, 7.94s/it] 19%|█▉ | 531/2757 [1:12:58<4:55:44, 7.97s/it] 19%|█▉ | 532/2757 [1:13:06<4:55:46, 7.98s/it] 19%|█▉ | 533/2757 [1:13:14<4:52:19, 7.89s/it] 19%|█▉ | 534/2757 [1:13:22<4:53:03, 7.91s/it] 19%|█▉ | 535/2757 [1:13:30<4:51:32, 7.87s/it] 19%|█▉ | 536/2757 [1:13:38<4:54:57, 7.97s/it] 19%|█▉ | 537/2757 [1:13:45<4:51:27, 7.88s/it] 20%|█▉ | 538/2757 [1:13:53<4:53:09, 7.93s/it] 20%|█▉ | 539/2757 [1:14:01<4:50:54, 7.87s/it] 20%|█▉ | 540/2757 [1:14:09<4:53:03, 7.93s/it] 20%|█▉ | 540/2757 [1:14:09<4:53:03, 7.93s/it] 20%|█▉ | 541/2757 [1:14:17<4:54:32, 7.98s/it] 20%|█▉ | 542/2757 [1:14:25<4:53:00, 7.94s/it] 20%|█▉ | 543/2757 [1:14:34<4:57:26, 8.06s/it] 20%|█▉ | 544/2757 [1:14:41<4:51:29, 7.90s/it] 20%|█▉ | 545/2757 [1:14:49<4:54:20, 7.98s/it] 20%|█▉ | 546/2757 [1:14:58<4:58:13, 8.09s/it] 20%|█▉ | 547/2757 [1:15:05<4:54:39, 8.00s/it] 20%|█▉ | 548/2757 [1:15:14<4:56:26, 8.05s/it] 20%|█▉ | 549/2757 [1:15:22<5:00:59, 8.18s/it] 20%|█▉ | 550/2757 [1:15:30<5:02:26, 8.22s/it] 20%|█▉ | 550/2757 [1:15:30<5:02:26, 8.22s/it] 20%|█▉ | 551/2757 [1:15:39<5:03:46, 8.26s/it] 20%|██ | 552/2757 [1:15:46<4:57:16, 8.09s/it] 20%|██ | 553/2757 [1:15:54<4:56:03, 8.06s/it] 20%|██ | 554/2757 [1:16:02<4:55:32, 8.05s/it] 20%|██ | 555/2757 [1:16:11<4:59:00, 8.15s/it] 20%|██ | 556/2757 [1:16:19<5:00:26, 8.19s/it] 20%|██ | 557/2757 [1:16:27<4:54:49, 8.04s/it] 20%|██ | 558/2757 [1:16:35<4:53:26, 8.01s/it] 20%|██ | 559/2757 [1:16:43<4:54:02, 8.03s/it] 20%|██ | 560/2757 [1:
{'loss': '0.5776', 'grad_norm': '0.5719', 'learning_rate': '0.0001681', 'rewards/chosen': '-0.6292', 'rewards/rejected': '-1.103', 'rewards/accuracies': '0.6281', 'rewards/margins': '0.4741', 'logps/chosen': '-84.31', 'logps/rejected': '-96.12', 'logits/chosen': '-2.981', 'logits/rejected': '-3.005', 'epoch': '0.5551'}
{'loss': '0.5797', 'grad_norm': '0.6719', 'learning_rate': '0.0001674', 'rewards/chosen': '-0.6672', 'rewards/rejected': '-1.152', 'rewards/accuracies': '0.6539', 'rewards/margins': '0.4843', 'logps/chosen': '-84.52', 'logps/rejected': '-95.7', 'logits/chosen': '-3.107', 'logits/rejected': '-3.058', 'epoch': '0.566'}
{'loss': '0.5927', 'grad_norm': '0.6522', 'learning_rate': '0.0001666', 'rewards/chosen': '-0.6613', 'rewards/rejected': '-1.109', 'rewards/accuracies': '0.625', 'rewards/margins': '0.4473', 'logps/chosen': '-85.24', 'logps/rejected': '-94.5', 'logits/chosen': '-3.016', 'logits/rejected': '-3.001', 'epoch': '0.5769'}
{'loss': '0.6019', 'grad_norm': '0.9451', 'learning_rate': '0.0001659', 'rewards/chosen': '-0.6853', 'rewards/rejected': '-1.141', 'rewards/accuracies': '0.6266', 'rewards/margins': '0.4554', 'logps/chosen': '-87.74', 'logps/rejected': '-98.06', 'logits/chosen': '-2.828', 'logits/rejected': '-2.82', 'epoch': '0.5878'}
{'loss': '0.5943', 'grad_norm': '0.6104', 'learning_rate': '0.0001651', 'rewards/chosen': '-0.7626', 'rewards/rejected': '-1.175', 'rewards/accuracies': '0.6313', 'rewards/margins': '0.4125', 'logps/chosen': '-86.3', 'logps/rejected': '-96.72', 'logits/chosen': '-2.712', 'logits/rejected': '-2.636', 'epoch': '0.5986'}
{'loss': '0.5997', 'grad_norm': '0.5953', 'learning_rate': '0.0001644', 'rewards/chosen': '-0.7641', 'rewards/rejected': '-1.156', 'rewards/accuracies': '0.632', 'rewards/margins': '0.3917', 'logps/chosen': '-85.59', 'logps/rejected': '-94.78', 'logits/chosen': '-2.773', 'logits/rejected': '-2.675', 'epoch': '0.6095'}
{'loss': '0.5831', 'grad_norm': '0.6583', 'learning_rate': '0.0001636', 'rewards/chosen': '-0.7877', 'rewards/rejected': '-1.254', 'rewards/accuracies': '0.6258', 'rewards/margins': '0.4662', 'logps/chosen': '-90.34', 'logps/rejected': '-99.95', 'logits/chosen': '-2.784', 'logits/rejected': '-2.651', 'epoch': '0.6204'}
{'loss': '0.5975', 'grad_norm': '0.5589', 'learning_rate': '0.0001629', 'rewards/chosen': '-0.5206', 'rewards/rejected': '-0.9202', 'rewards/accuracies': '0.625', 'rewards/margins': '0.3997', 'logps/chosen': '-83.24', 'logps/rejected': '-91.77', 'logits/chosen': '-2.907', 'logits/rejected': '-2.885', 'epoch': '0.6313'}
{'loss': '0.5781', 'grad_norm': '0.6037', 'learning_rate': '0.0001622', 'rewards/chosen': '-0.4098', 'rewards/rejected': '-0.8762', 'rewards/accuracies': '0.6414', 'rewards/margins': '0.4664', 'logps/chosen': '-83.76', 'logps/rejected': '-94.21', 'logits/chosen': '-2.92', 'logits/rejected': '-2.842', 'epoch': '0.6422'}
{'loss': '0.5839', 'grad_norm': '0.6424', 'learning_rate': '0.0001614', 'rewards/chosen': '-0.5493', 'rewards/rejected': '-0.9931', 'rewards/accuracies': '0.6586', 'rewards/margins': '0.4438', 'logps/chosen': '-86.28', 'logps/rejected': '-96.4', 'logits/chosen': '-2.929', 'logits/rejected': '-2.954', 'epoch': '0.6531'}
{'loss': '0.5724', 'grad_norm': '0.5479', 'learning_rate': '0.0001607', 'rewards/chosen': '-0.5622', 'rewards/rejected': '-1.034', 'rewards/accuracies': '0.6438', 'rewards/margins': '0.472', 'logps/chosen': '-87.9', 'logps/rejected': '-96.35', 'logits/chosen': '-2.836', 'logits/rejected': '-2.8', 'epoch': '0.6639'}
{'loss': '0.5745', 'grad_norm': '0.6719', 'learning_rate': '0.0001599', 'rewards/chosen': '-0.7492', 'rewards/rejected': '-1.227', 'rewards/accuracies': '0.6531', 'rewards/margins': '0.478', 'logps/chosen': '-88.12', 'logps/rejected': '-98.25', 'logits/chosen': '-2.963', 'logits/rejected': '-2.848', 'epoch': '0.6748'}
{'loss': '0.5667', 'grad_norm': '0.6403', 'learning_rate': '0.0001592', 'rewards/chosen': '-0.9368', 'rewards/rejected': '-1.455', 'rewards/accuracies': '0.6602', 'rewards/margins': '0.5185', 'logps/chosen': '-90.78', 'logps/rejected': '-103.6', 'logits/chosen': '-2.943', 'logits/rejected': '-2.97', 'epoch': '0.6857'}
{'loss': '0.5546', 'grad_norm': '0.6475', 'learning_rate': '0.0001584', 'rewards/chosen': '-0.9921', 'rewards/rejected': '-1.549', 'rewards/accuracies': '0.6461', 'rewards/margins': '0.557', 'logps/chosen': '-89.43', 'logps/rejected': '-100.7', 'logits/chosen': '-2.979', 'logits/rejected': '-2.938', 'epoch': '0.6966'}
{'loss': '0.574', 'grad_norm': '0.7136', 'learning_rate': '0.0001577', 'rewards/chosen': '-1.028', 'rewards/rejected': '-1.534', 'rewards/accuracies': '0.6375', 'rewards/margins': '0.5058', 'logps/chosen': '-91.74', 'logps/rejected': '-102.1', 'logits/chosen': '-3.021', 'logits/rejected': '-2.974', 'epoch': '0.7075'}
{'loss': '0.56', 'grad_norm': '0.6012', 'learning_rate': '0.0001569', 'rewards/chosen': '-1.082', 'rewards/rejected': '-1.642', 'rewards/accuracies': '0.6555', 'rewards/margins': '0.5598', 'logps/chosen': '-90.38', 'logps/rejected': '-101.6', 'logits/chosen': '-3.157', 'logits/rejected': '-3.125', 'epoch': '0.7184'}
{'loss': '0.5732', 'grad_norm': '0.6408', 'learning_rate': '0.0001562', 'rewards/chosen': '-1.105', 'rewards/rejected': '-1.613', 'rewards/accuracies': '0.6477', 'rewards/margins': '0.5072', 'logps/chosen': '-89.5', 'logps/rejected': '-100.5', 'logits/chosen': '-3.229', 'logits/rejected': '-3.236', 'epoch': '0.7293'}
{'loss': '0.5617', 'grad_norm': '0.6599', 'learning_rate': '0.0001554', 'rewards/chosen': '-0.8735', 'rewards/rejected': '-1.432', 'rewards/accuracies': '0.6641', 'rewards/margins': '0.5582', 'logps/chosen': '-91.57', 'logps/rejected': '-102.2', 'logits/chosen': '-3.211', 'logits/rejected': '-3.173', 'epoch': '0.7401'}
{'loss': '0.5498', 'grad_norm': '0.6686', 'learning_rate': '0.0001547', 'rewards/chosen': '-0.6966', 'rewards/rejected': '-1.305', 'rewards/accuracies': '0.6695', 'rewards/margins': '0.6086', 'logps/chosen': '-90.9', 'logps/rejected': '-102.5', 'logits/chosen': '-3.235', 'logits/rejected': '-3.097', 'epoch': '0.751'}
{'loss': '0.5865', 'grad_norm': '0.7763', 'learning_rate': '0.0001539', 'rewards/chosen': '-0.608', 'rewards/rejected': '-1.119', 'rewards/accuracies': '0.6539', 'rewards/margins': '0.5112', 'logps/chosen': '-85.77', 'logps/rejected': '-96.22', 'logits/chosen': '-3.261', 'logits/rejected': '-3.15', 'epoch': '0.7619'}
{'loss': '0.5637', 'grad_norm': '0.6534', 'learning_rate': '0.0001532', 'rewards/chosen': '-0.7708', 'rewards/rejected': '-1.309', 'rewards/accuracies': '0.6508', 'rewards/margins': '0.5377', 'logps/chosen': '-88.75', 'logps/rejected': '-99.82', 'logits/chosen': '-2.995', 'logits/rejected': '-2.962', 'epoch': '0.7728'}
{'loss': '0.5601', 'grad_norm': '0.5064', 'learning_rate': '0.0001524', 'rewards/chosen': '-0.8121', 'rewards/rejected': '-1.338', 'rewards/accuracies': '0.6375', 'rewards/margins': '0.5259', 'logps/chosen': '-84.73', 'logps/rejected': '-96.15', 'logits/chosen': '-3.161', 'logits/rejected': '-3.103', 'epoch': '0.7837'}
{'loss': '0.5589', 'grad_norm': '0.603', 'learning_rate': '0.0001517', 'rewards/chosen': '-0.7111', 'rewards/rejected': '-1.252', 'rewards/accuracies': '0.65', 'rewards/margins': '0.5404', 'logps/chosen': '-85.85', 'logps/rejected': '-96.86', 'logits/chosen': '-3.101', 'logits/rejected': '-3.052', 'epoch': '0.7946'}
{'loss': '0.5412', 'grad_norm': '0.608', 'learning_rate': '0.0001509', 'rewards/chosen': '-0.7364', 'rewards/rejected': '-1.331', 'rewards/accuracies': '0.6672', 'rewards/margins': '0.5946', 'logps/chosen': '-91.16', 'logps/rejected': '-101.2', 'logits/chosen': '-3.177', 'logits/rejected': '-3.198', 'epoch': '0.8054'}
{'loss': '0.5476', 'grad_norm': '0.6249', 'learning_rate': '0.0001502', 'rewards/chosen': '-0.6863', 'rewards/rejected': '-1.317', 'rewards/accuracies': '0.668', 'rewards/margins': '0.6311', 'logps/chosen': '-85.68', 'logps/rejected': '-97.83', 'logits/chosen': '-3.299', 'logits/rejected': '-3.16', 'epoch': '0.8163'} 27%|██▋ | 750/2757 [1:42:13<4:25:53, 7.95s/it] 27%|██▋ | 751/2757 [1:42:25<5:02:01, 9.03s/it] 27%|██▋ | 752/2757 [1:42:33<4:50:08, 8.68s/it] 27%|██▋ | 753/2757 [1:42:40<4:37:16, 8.30s/it] 27%|██▋ | 754/2757 [1:42:48<4:35:19, 8.25s/it] 27%|██▋ | 755/2757 [1:42:57<4:36:26, 8.29s/it] 27%|██▋ | 756/2757 [1:43:05<4:34:48, 8.24s/it] 27%|██▋ | 757/2757 [1:43:13<4:35:00, 8.25s/it] 27%|██▋ | 758/2757 [1:43:21<4:33:42, 8.22s/it] 28%|██▊ | 759/2757 [1:43:30<4:35:52, 8.28s/it] 28%|██▊ | 760/2757 [1:43:37<4:31:39, 8.16s/it] 28%|██▊ | 760/2757 [1:43:37<4:31:39, 8.16s/it] 28%|██▊ | 761/2757 [1:43:45<4:28:49, 8.08s/it] 28%|██▊ | 762/2757 [1:43:54<4:30:20, 8.13s/it] 28%|██▊ | 763/2757 [1:44:02<4:31:47, 8.18s/it] 28%|██▊ | 764/2757 [1:44:09<4:25:25, 7.99s/it] 28%|██▊ | 765/2757 [1:44:17<4:21:32, 7.88s/it] 28%|██▊ | 766/2757 [1:44:25<4:22:31, 7.91s/it] 28%|██▊ | 767/2757 [1:44:33<4:26:36, 8.04s/it] 28%|██▊ | 768/2757 [1:44:42<4:29:57, 8.14s/it] 28%|██▊ | 769/2757 [1:44:50<4:26:13, 8.03s/it] 28%|██▊ | 770/2757 [1:44:57<4:23:28, 7.96s/it] 28%|██▊ | 770/2757 [1:44:57<4:23:28, 7.96s/it] 28%|██▊ | 771/2757 [1:45:05<4:25:31, 8.02s/it] 28%|██▊ | 772/2757 [1:45:14<4:27:16, 8.08s/it] 28%|██▊ | 773/2757 [1:45:22<4:27:16, 8.08s/it] 28%|██▊ | 774/2757 [1:45:30<4:25:37, 8.04s/it] 28%|██▊ | 775/2757 [1:45:38<4:25:14, 8.03s/it] 28%|██▊ | 776/2757 [1:45:46<4:26:07, 8.06s/it] 28%|██▊ | 777/2757 [1:45:54<4:22:47, 7.96s/it] 28%|██▊ | 778/2757 [1:46:01<4:20:55, 7.91s/it] 28%|██▊ | 779/2757 [1:46:09<4:22:31, 7.96s/it] 28%|██▊ | 780/2757 [1:46:17<4:18:58, 7.86s/it] 28%|██▊ | 780/2757 [1:46:17<4:18:58, 7.86s/it] 28%|██▊ | 781/2757 [1:46:26<4:25:22, 8.06s/it] 28%|██▊ | 782/2757 [1:46:34<4:24:28, 8.03s/it] 28%|██▊ | 783/2757 [1:46:41<4:20:20, 7.91s/it] 28%|██▊ | 784/2757 [1:46:49<4:20:02, 7.91s/it] 28%|██▊ | 785/2757 [1:46:57<4:23:30, 8.02s/it] 29%|██▊ | 786/2757 [1:47:05<4:19:49, 7.91s/it] 29%|██▊ | 787/2757 [1:47:13<4:19:16, 7.90s/it] 29%|██▊ | 788/2757 [1:47:21<4:20:24, 7.94s/it] 29%|██▊ | 789/2757 [1:47:29<4:22:53, 8.02s/it] 29%|██▊ | 790/2757 [1:47:37<4:18:18, 7.88s/it] 29%|██▊ | 790/2757 [1:47:37<4:18:18, 7.88s/it] 29%|██▊ | 791/2757 [1:47:44<4:16:32, 7.83s/it] 29%|██▊ | 792/2757 [1:47:52<4:14:41, 7.78s/it] 29%|██▉ | 793/2757 [1:48:00<4:13:26, 7.74s/it] 29%|██▉ | 794/2757 [1:48:07<4:11:02, 7.67s/it] 29%|██▉ | 795/2757 [1:48:15<4:12:41, 7.73s/it] 29%|██▉ | 796/2757 [1:48:23<4:15:42, 7.82s/it] 29%|██▉ | 797/2757 [1:48:31<4:16:44, 7.86s/it] 29%|██▉ | 798/2757 [1:48:39<4:17:43, 7.89s/it] 29%|██▉ | 799/2757 [1:48:47<4:17:56, 7.90s/it] 29%|██▉ | 800/2757 [1:48:55<4:18:34, 7.93s/it] 29%|██▉ | 800/2757 [1:48:55<4:18:34, 7.93s/it] 29%|██▉ | 801/2757 [1:49:03<4:20:00, 7.98s/it] 29%|██▉
{'loss': '0.5396', 'grad_norm': '0.7027', 'learning_rate': '0.0001494', 'rewards/chosen': '-0.8498', 'rewards/rejected': '-1.501', 'rewards/accuracies': '0.6695', 'rewards/margins': '0.6513', 'logps/chosen': '-87.9', 'logps/rejected': '-101.8', 'logits/chosen': '-3.371', 'logits/rejected': '-3.293', 'epoch': '0.8272'}
{'loss': '0.5349', 'grad_norm': '0.6295', 'learning_rate': '0.0001487', 'rewards/chosen': '-1.063', 'rewards/rejected': '-1.742', 'rewards/accuracies': '0.6594', 'rewards/margins': '0.6796', 'logps/chosen': '-89.56', 'logps/rejected': '-102.7', 'logits/chosen': '-3.39', 'logits/rejected': '-3.347', 'epoch': '0.8381'}
{'loss': '0.5414', 'grad_norm': '0.6378', 'learning_rate': '0.0001479', 'rewards/chosen': '-1.214', 'rewards/rejected': '-1.858', 'rewards/accuracies': '0.6672', 'rewards/margins': '0.6445', 'logps/chosen': '-92.15', 'logps/rejected': '-104.5', 'logits/chosen': '-3.227', 'logits/rejected': '-3.154', 'epoch': '0.849'}
{'loss': '0.5625', 'grad_norm': '0.7089', 'learning_rate': '0.0001472', 'rewards/chosen': '-1.144', 'rewards/rejected': '-1.71', 'rewards/accuracies': '0.6352', 'rewards/margins': '0.5667', 'logps/chosen': '-92.22', 'logps/rejected': '-102.2', 'logits/chosen': '-3.079', 'logits/rejected': '-3.059', 'epoch': '0.8599'}
{'loss': '0.5564', 'grad_norm': '0.672', 'learning_rate': '0.0001464', 'rewards/chosen': '-1.028', 'rewards/rejected': '-1.614', 'rewards/accuracies': '0.6586', 'rewards/margins': '0.5857', 'logps/chosen': '-87.96', 'logps/rejected': '-99.78', 'logits/chosen': '-3.219', 'logits/rejected': '-3.18', 'epoch': '0.8707'}
{'loss': '0.5628', 'grad_norm': '0.7438', 'learning_rate': '0.0001457', 'rewards/chosen': '-1.04', 'rewards/rejected': '-1.64', 'rewards/accuracies': '0.6594', 'rewards/margins': '0.5995', 'logps/chosen': '-90.28', 'logps/rejected': '-101.8', 'logits/chosen': '-3.175', 'logits/rejected': '-3.129', 'epoch': '0.8816'}
{'loss': '0.5592', 'grad_norm': '0.7402', 'learning_rate': '0.000145', 'rewards/chosen': '-1.04', 'rewards/rejected': '-1.605', 'rewards/accuracies': '0.6562', 'rewards/margins': '0.5652', 'logps/chosen': '-92.35', 'logps/rejected': '-103.3', 'logits/chosen': '-3.144', 'logits/rejected': '-3.037', 'epoch': '0.8925'}
{'loss': '0.5435', 'grad_norm': '0.6177', 'learning_rate': '0.0001442', 'rewards/chosen': '-0.964', 'rewards/rejected': '-1.562', 'rewards/accuracies': '0.675', 'rewards/margins': '0.5981', 'logps/chosen': '-92.94', 'logps/rejected': '-102.4', 'logits/chosen': '-3.209', 'logits/rejected': '-3.142', 'epoch': '0.9034'}
{'loss': '0.5488', 'grad_norm': '0.6203', 'learning_rate': '0.0001435', 'rewards/chosen': '-0.9898', 'rewards/rejected': '-1.58', 'rewards/accuracies': '0.6789', 'rewards/margins': '0.5904', 'logps/chosen': '-91.24', 'logps/rejected': '-101.4', 'logits/chosen': '-3.352', 'logits/rejected': '-3.235', 'epoch': '0.9143'}
{'loss': '0.5207', 'grad_norm': '0.669', 'learning_rate': '0.0001427', 'rewards/chosen': '-0.9211', 'rewards/rejected': '-1.579', 'rewards/accuracies': '0.6836', 'rewards/margins': '0.6583', 'logps/chosen': '-89.67', 'logps/rejected': '-102.7', 'logits/chosen': '-3.454', 'logits/rejected': '-3.354', 'epoch': '0.9252'}
{'loss': '0.523', 'grad_norm': '0.7382', 'learning_rate': '0.000142', 'rewards/chosen': '-0.9207', 'rewards/rejected': '-1.614', 'rewards/accuracies': '0.6977', 'rewards/margins': '0.6931', 'logps/chosen': '-92.43', 'logps/rejected': '-103.6', 'logits/chosen': '-3.535', 'logits/rejected': '-3.453', 'epoch': '0.9361'}
{'loss': '0.532', 'grad_norm': '0.7022', 'learning_rate': '0.0001412', 'rewards/chosen': '-1.145', 'rewards/rejected': '-1.819', 'rewards/accuracies': '0.6953', 'rewards/margins': '0.6743', 'logps/chosen': '-91.38', 'logps/rejected': '-104.2', 'logits/chosen': '-3.44', 'logits/rejected': '-3.353', 'epoch': '0.9469'}
{'loss': '0.5318', 'grad_norm': '0.6528', 'learning_rate': '0.0001405', 'rewards/chosen': '-1.164', 'rewards/rejected': '-1.821', 'rewards/accuracies': '0.6773', 'rewards/margins': '0.6567', 'logps/chosen': '-90.99', 'logps/rejected': '-103.5', 'logits/chosen': '-3.513', 'logits/rejected': '-3.527', 'epoch': '0.9578'}
{'loss': '0.532', 'grad_norm': '0.6701', 'learning_rate': '0.0001397', 'rewards/chosen': '-1.009', 'rewards/rejected': '-1.716', 'rewards/accuracies': '0.6789', 'rewards/margins': '0.7072', 'logps/chosen': '-89.25', 'logps/rejected': '-101.4', 'logits/chosen': '-3.612', 'logits/rejected': '-3.547', 'epoch': '0.9687'}
{'loss': '0.5377', 'grad_norm': '0.6882', 'learning_rate': '0.000139', 'rewards/chosen': '-0.9364', 'rewards/rejected': '-1.605', 'rewards/accuracies': '0.6805', 'rewards/margins': '0.6687', 'logps/chosen': '-88.02', 'logps/rejected': '-99.57', 'logits/chosen': '-3.528', 'logits/rejected': '-3.411', 'epoch': '0.9796'}
{'loss': '0.5362', 'grad_norm': '0.6982', 'learning_rate': '0.0001382', 'rewards/chosen': '-0.8833', 'rewards/rejected': '-1.537', 'rewards/accuracies': '0.6523', 'rewards/margins': '0.654', 'logps/chosen': '-87.5', 'logps/rejected': '-98.83', 'logits/chosen': '-3.544', 'logits/rejected': '-3.5', 'epoch': '0.9905'}
33%|███▎ | 920/2757 [2:04:52<3:57:58, 7.77s/it] 33%|███▎ | 920/2757 [2:04:52<3:57:58, 7.77s/it] 33%|███▎ | 921/2757 [2:05:00<4:03:27, 7.96s/it] 33%|███▎ | 922/2757 [2:05:09<4:10:14, 8.18s/it] 33%|███▎ | 923/2757 [2:05:17<4:09:05, 8.15s/it] 34%|███▎ | 924/2757 [2:05:25<4:09:27, 8.17s/it] 34%|███▎ | 925/2757 [2:05:33<4:10:49, 8.21s/it] 34%|███▎ | 926/2757 [2:05:41<4:07:28, 8.11s/it] 34%|███▎ | 927/2757 [2:05:49<4:05:37, 8.05s/it] 34%|███▎ | 928/2757 [2:05:57<4:03:45, 8.00s/it] 34%|███▎ | 929/2757 [2:06:05<4:04:49, 8.04s/it] 34%|███▎ | 930/2757 [2:06:13<4:04:38, 8.03s/it] 34%|███▎ | 930/2757 [2:06:13<4:04:38, 8.03s/it] 34%|███▍ | 931/2757 [2:06:21<4:06:45, 8.11s/it] 34%|███▍ | 932/2757 [2:06:30<4:06:35, 8.11s/it] 34%|███▍ | 933/2757 [2:06:38<4:06:31, 8.11s/it] 34%|███▍ | 934/2757 [2:06:46<4:05:03, 8.07s/it] 34%|███▍ | 935/2757 [2:06:53<4:01:31, 7.95s/it] 34%|███▍ | 936/2757 [2:07:01<3:58:12, 7.85s/it] 34%|███▍ | 937/2757 [2:07:09<4:01:53, 7.97s/it] 34%|███▍ | 938/2757 [2:07:18<4:08:38, 8.20s/it] 34%|███▍ | 939/2757 [2:07:26<4:09:15, 8.23s/it] 34%|███▍ | 940/2757 [2:07:34<4:06:23, 8.14s/it] 34%|███▍ | 940/2757 [2:07:34<4:06:23, 8.14s/it] 34%|███▍ | 941/2757 [2:07:42<4:05:38, 8.12s/it] 34%|███▍ | 942/2757 [2:07:50<4:01:14, 7.97s/it] 34%|███▍ | 943/2757 [2:07:58<3:58:23, 7.89s/it] 34%|███▍ | 944/2757 [2:08:06<4:02:31, 8.03s/it] 34%|███▍ | 945/2757 [2:08:14<4:02:37, 8.03s/it] 34%|███▍ | 946/2757 [2:08:22<4:04:50, 8.11s/it] 34%|███▍ | 947/2757 [2:08:30<4:00:51, 7.98s/it] 34%|███▍ | 948/2757 [2:08:38<4:02:03, 8.03s/it] 34%|███▍ | 949/2757 [2:08:46<4:05:32, 8.15s/it] 34%|███▍ | 950/2757 [2:08:55<4:07:08, 8.21s/it] 34%|███▍ | 950/2757 [2:08:55<4:07:08, 8.21s/it] 34%|███▍ | 951/2757 [2:09:03<4:07:01, 8.21s/it] 35%|███▍ | 952/2757 [2:09:11<4:06:29, 8.19s/it] 35%|███▍ | 953/2757 [2:09:19<4:04:44, 8.14s/it] 35%|███▍ | 954/2757 [2:09:27<4:00:41, 8.01s/it] 35%|███▍ | 955/2757 [2:09:35<4:04:11, 8.13s/it] 35%|███▍ | 956/2757 [2:09:44<4:07:09, 8.23s/it] 35%|███▍ | 957/2757 [2:09:52<4:03:52, 8.13s/it] 35%|███▍ | 958/2757 [2:10:00<4:08:07, 8.28s/it] 35%|███▍ | 959/2757 [2:10:08<4:05:23, 8.19s/it] 35%|███▍ | 960/2757 [2:10:16<4:02:57, 8.11s/it] 35%|███▍ | 960/2757 [2:10:16<4:02:57, 8.11s/it] 35%|███▍ | 961/2757 [2:10:24<4:01:14, 8.06s/it] 35%|███▍ | 962/2757 [2:10:32<4:01:22, 8.07s/it] 35%|███▍ | 963/2757 [2:10:41<4:02:59, 8.13s/it] 35%|███▍ | 964/2757 [2:10:48<3:58:51, 7.99s/it] 35%|███▌ | 965/2757 [2:10:56<3:59:00, 8.00s/it] 35%|███▌ | 966/2757 [2:11:05<4:05:08, 8.21s/it] 35%|███▌ | 967/2757 [2:11:13<4:02:15, 8.12s/it] 35%|███▌ | 968/2757 [2:11:21<4:05:00, 8.22s/it] 35%|███▌ | 969/2757 [2:11:29<4:04:20, 8.20s/it] 35%|███▌ | 970/2757 [2:11:37<4:00:06, 8.06s/it] 35%|███▌ | 970/2757 [2:11:37<4:00:06, 8.06s/it] 35%|███▌ | 971/2757 [2:11:46<4:03:06, 8.17s/it] 35%|███▌ | 972/2757 [2:11:54<4:00:59, 8.10s/it] 35%|███▌ | 973/2757 [2:12:0
{'loss': '0.4122', 'grad_norm': '0.5277', 'learning_rate': '0.0001367', 'rewards/chosen': '-1.053', 'rewards/rejected': '-2.115', 'rewards/accuracies': '0.7789', 'rewards/margins': '1.062', 'logps/chosen': '-92.64', 'logps/rejected': '-107.8', 'logits/chosen': '-3.593', 'logits/rejected': '-3.548', 'epoch': '1.012'}
{'loss': '0.4067', 'grad_norm': '0.5503', 'learning_rate': '0.000136', 'rewards/chosen': '-1.235', 'rewards/rejected': '-2.371', 'rewards/accuracies': '0.7844', 'rewards/margins': '1.136', 'logps/chosen': '-94.07', 'logps/rejected': '-112.1', 'logits/chosen': '-3.677', 'logits/rejected': '-3.588', 'epoch': '1.023'}
{'loss': '0.4037', 'grad_norm': '0.5421', 'learning_rate': '0.0001352', 'rewards/chosen': '-1.13', 'rewards/rejected': '-2.287', 'rewards/accuracies': '0.7844', 'rewards/margins': '1.158', 'logps/chosen': '-90.41', 'logps/rejected': '-106.1', 'logits/chosen': '-3.779', 'logits/rejected': '-3.727', 'epoch': '1.034'}
{'loss': '0.4048', 'grad_norm': '0.5324', 'learning_rate': '0.0001345', 'rewards/chosen': '-0.9678', 'rewards/rejected': '-2.156', 'rewards/accuracies': '0.7937', 'rewards/margins': '1.188', 'logps/chosen': '-92.28', 'logps/rejected': '-108.5', 'logits/chosen': '-3.955', 'logits/rejected': '-3.926', 'epoch': '1.045'}
{'loss': '0.3986', 'grad_norm': '0.4539', 'learning_rate': '0.0001337', 'rewards/chosen': '-0.9384', 'rewards/rejected': '-2.162', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.224', 'logps/chosen': '-90.46', 'logps/rejected': '-109.6', 'logits/chosen': '-3.724', 'logits/rejected': '-3.732', 'epoch': '1.056'}
{'loss': '0.4087', 'grad_norm': '0.5155', 'learning_rate': '0.000133', 'rewards/chosen': '-1.096', 'rewards/rejected': '-2.233', 'rewards/accuracies': '0.7719', 'rewards/margins': '1.137', 'logps/chosen': '-88.56', 'logps/rejected': '-105.9', 'logits/chosen': '-3.808', 'logits/rejected': '-3.751', 'epoch': '1.066'}
{'loss': '0.3877', 'grad_norm': '0.5626', 'learning_rate': '0.0001322', 'rewards/chosen': '-1.119', 'rewards/rejected': '-2.363', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.244', 'logps/chosen': '-93.86', 'logps/rejected': '-112.1', 'logits/chosen': '-3.833', 'logits/rejected': '-3.733', 'epoch': '1.077'}
{'loss': '0.3857', 'grad_norm': '0.4981', 'learning_rate': '0.0001315', 'rewards/chosen': '-1.254', 'rewards/rejected': '-2.53', 'rewards/accuracies': '0.7844', 'rewards/margins': '1.277', 'logps/chosen': '-90.55', 'logps/rejected': '-110.4', 'logits/chosen': '-3.934', 'logits/rejected': '-3.891', 'epoch': '1.088'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
0%| | 0/300 [00:00<?, ?it/s]
1%| | 2/300 [00:00<00:51, 5.74it/s]
1%| | 3/300 [00:00<01:46, 2.80it/s]
1%|▏ | 4/300 [00:01<01:34, 3.13it/s]
2%|▏ | 5/300 [00:01<01:44, 2.83it/s]
2%|▏ | 6/300 [00:01<01:40, 2.94it/s]
2%|▏ | 7/300 [00:02<01:54, 2.56it/s]
3%|▎ | 8/300 [00:02<01:50, 2.65it/s]
3%|▎ | 9/300 [00:03<01:51, 2.61it/s]
3%|▎ | 10/300 [00:03<01:48, 2.67it/s]
4%|▎ | 11/300 [00:03<01:50, 2.61it/s]
4%|▍ | 12/300 [00:04<01:37, 2.94it/s]
4%|▍ | 13/300 [00:04<01:42, 2.80it/s]
5%|▍ | 14/300 [00:04<01:35, 3.00it/s]
5%|▌ | 15/300 [00:05<01:45, 2.70it/s]
5%|▌ | 16/300 [00:05<01:33, 3.04it/s]
6%|▌ | 17/300 [00:05<01:38, 2.89it/s]
6%|▌ | 18/300 [00:06<01:28, 3.19it/s]
6%|▋ | 19/300 [00:06<01:38, 2.85it/s]
7%|▋ | 20/300 [00:06<01:31, 3.06it/s]
7%|▋ | 21/300 [00:07<01:41, 2.76it/s]
7%|▋ | 22/300 [00:07<01:30, 3.08it/s]
8%|▊ | 23/300 [00:07<01:31, 3.04it/s]
8%|▊ | 24/300 [00:08<01:22, 3.35it/s]
8%|▊ | 25/300 [00:08<01:35, 2.89it/s]
9%|▊ | 26/300 [00:08<01:28, 3.10it/s]
9%|▉ | 27/300 [00:09<01:36, 2.82it/s]
9%|▉ | 28/300 [00:09<01:26, 3.16it/s]
10%|▉ | 29/300 [00:09<01:34, 2.85it/s]
10%|█ | 30/300 [00:10<01:24, 3.18it/s]
10%|█ | 31/300 [00:10<01:26, 3.12it/s]
11%|█ | 32/300 [00:10<01:18, 3.40it/s]
11%|█ | 33/300 [00:11<01:21, 3.29it/s]
11%|█▏ | 34/300 [00:11<01:16, 3.47it/s]
12%|█▏ | 35/300 [00:11<01:22, 3.22it/s]
12%|█▏ | 36/300 [00:11<01:18, 3.38it/s]
12%|█▏ | 37/300 [00:12<01:26, 3.03it/s]
13%|█▎ | 38/300 [00:12<01:17, 3.37it/s]
13%|█▎ | 39/300 [00:12<01:25, 3.05it/s]
13%|█▎ | 40/300 [00:13<01:17, 3.34it/s]
14%|█▎ | 41/300 [00:13<01:25, 3.03it/s]
14%|█▍ | 42/300 [00:13<01:17, 3.32it/s]
14%|█▍ | 43/300 [00:14<01:21, 3.15it/s]
15%|█▍ | 44/300 [00:14<01:14, 3.42it/s]
15%|█▌ | 45/300 [00:14<01:22, 3.11it/s]
15%|█▌ | 46/300 [00:15<01:23, 3.05it/s]
16%|█▌ | 47/300 [00:15<01:35, 2.66it/s]
16%|█▌ | 48/300 [00:15<01:26, 2.92it/s]
16%|█▋ | 49/300 [00:16<01:40, 2.49it/s]
17%|█▋ | 50/300 [00:16<01:26, 2.88it/s]
17%|█▋ | 51/300 [00:17<01:26, 2.88it/s]
17%|█▋ | 52/300 [00:17<01:21, 3.05it/s]
18%|█▊ | 53/300 [00:17<01:32, 2.68it/s]
18%|█▊ | 54/300 [00:18<01:22, 2.99it/s]
18%|█▊ | 55/300 [00:18<01:29, 2.74it/s]
19%|█▊ | 56/300 [00:18<01:17, 3.15it/s]
19%|█▉ | 57/300 [00:19<01:21, 2.98it/s]
19%|█▉ | 58/300 [00:19<01:22, 2.94it/s]
20%|█▉ | 59/300 [00:19<01:26, 2.78it/s]
20%|██ | 60/300 [00:20<01:18, 3.07it/s]
20%|██ | 61/300 [00:20<01:21, 2.95it/s]
21%|██ | 62/300 [00:20<01:20, 2.95it/s]
21%|██ | 63/300 [00:21<01:25, 2.76it/s]
21%|██▏ | 64/300 [00:21<01:23, 2.81it/s]
22%|██▏ | 65/300 [00:21<01:27, 2.70it/s]
22%|██▏ | 66/300 [00:22<01:19, 2.95it/s]
22%|██▏ | 67/300 [00:22<01:26, 2.69it/s]
23%|██▎ | 68/300 [00:22<01:17, 2.97it/s]
23%|██▎ | 69/300 [00:23<01:21, 2.85it/s]
23%|██▎ | 70/300 [00:23<01:12, 3.15it/s]
24%|██▎ | 71/300 [00:23<01:15, 3.05it/s]
24%|██▍ | 72/300 [00:24<01:09, 3.30it/s]
24%|██▍ | 73/300 [00:24<01:12, 3.14it/s]
25%|██▍ | 74/300 [00:24<01:08, 3.32it/s]
25%|██▌ | 75/300 [00:25<01:12, 3.09it/s]
25%|██▌ | 76/300 [00:25<01:14, 3.02it/s]
26%|██▌ | 77/300 [00:25<01:20, 2.79it/s]
26%|██▌ | 78/300 [00:26<01:13, 3.03it/s]
26%|██▋ | 79/300 [00:26<01:15, 2.93it/s]
27%|██▋ | 80/300 [00:26<01:11, 3.09it/s]
27%|██▋ | 81/300 [00:27<01:19, 2.77it/s]
27%|██▋ | 82/300 [00:27<01:10, 3.07it/s]
28%|██▊ | 83/300 [00:27<01:13, 2.97it/s]
28%|██▊ | 84/300 [00:28<01:13, 2.94it/s]
28%|██▊ | 85/300 [00:28<01:18, 2.75it/s]
29%|██▊ | 86/300 [00:28<01:10, 3.05it/s]
29%|██▉ | 87/300 [00:29<01:12, 2.94it/s]
29%|██▉ | 88/300 [00:29<01:06, 3.19it/s]
30%|██▉ | 89/300 [00:29<01:10, 2.98it/s]
30%|███ | 90/300 [00:30<01:05, 3.21it/s]
30%|███ | 91/300 [00:30<01:09, 3.01it/s]
31%|███ | 92/300 [00:30<01:03, 3.28it/s]
31%|███ | 93/300 [00:31<01:09, 3.00it/s]
31%|███▏ | 94/300 [00:31<01:02, 3.29it/s]
32%|███▏ | 95/300 [00:31<01:07, 3.02it/s]
32%|███▏ | 96/300 [00:32<01:03, 3.21it/s]
32%|███▏ | 97/300 [00:32<01:05, 3.09it/s]
33%|███▎ | 98/300 [00:32<01:01, 3.26it/s]
33%|███▎ | 99/300 [00:33<01:06, 3.00it/s]
33%|███▎ | 100/300 [00:33<01:03, 3.16it/s]
34%|███▎ | 101/300 [00:33<01:07, 2.97it/s]
34%|███▍ | 102/300 [00:34<01:05, 3.02it/s]
34%|███▍ | 103/300 [00:34<01:09, 2.84it/s]
35%|███▍ | 104/300 [00:34<01:02, 3.16it/s]
35%|███▌ | 105/300 [00:35<01:05, 2.99it/s]
35%|███▌ | 106/300 [00:35<01:00, 3.20it/s]
36%|███▌ | 107/300 [00:35<01:04, 2.98it/s]
36%|███▌ | 108/300 [00:35<01:02, 3.07it/s]
36%|███▋ | 109/300 [00:36<01:06, 2.89it/s]
37%|███▋ | 110/300 [00:36<01:02, 3.04it/s]
37%|███▋ | 111/300 [00:37<01:11, 2.66it/s]
37%|███▋ | 112/300 [00:37<01:05, 2.85it/s]
38%|███▊ | 113/300 [00:37<01:07, 2.75it/s]
38%|███▊ | 114/300 [00:38<01:06, 2.78it/s]
38%|███▊ | 115/300 [00:38<01:09, 2.65it/s]
39%|███▊ | 116/300 [00:38<01:01, 2.97it/s]
39%|███▉ | 117/300 [00:39<01:03, 2.90it/s]
39%|███▉ | 118/300 [00:39<00:58, 3.12it/s]
40%|███▉ | 119/300 [00:39<01:00, 2.97it/s]
40%|████ | 120/300 [00:40<01:05, 2.76it/s]
40%|████ | 121/300 [00:40<01:05, 2.72it/s]
41%|████ | 122/300 [00:40<00:59, 2.97it/s]
41%|████ | 123/300 [00:41<01:03, 2.77it/s]
41%|████▏ | 124/300 [00:41<00:59, 2.96it/s]
42%|████▏ | 125/300 [00:42<01:00, 2.87it/s]
42%|████▏ | 126/300 [00:42<01:00, 2.87it/s]
42%|████▏ | 127/300 [00:42<01:03, 2.73it/s]
43%|████▎ | 128/300 [00:43<00:58, 2.94it/s]
43%|████▎ | 129/300 [00:43<01:00, 2.83it/s]
43%|████▎ | 130/300 [00:43<00:54, 3.12it/s]
44%|████▎ | 131/300 [00:44<01:01, 2.76it/s]
44%|████▍ | 132/300 [00:44<00:55, 3.01it/s]
44%|████▍ | 133/300 [00:44<01:05, 2.57it/s]
45%|████▍ | 134/300 [00:45<00:58, 2.84it/s]
45%|████▌ | 135/300 [00:45<01:04, 2.57it/s]
45%|████▌ | 136/300 [00:45<00:56, 2.89it/s]
46%|████▌ | 137/300 [00:46<00:58, 2.79it/s]
46%|████▌ | 138/300 [00:46<00:53, 3.05it/s]
46%|████▋ | 139/300 [00:46<00:53, 3.03it/s]
47%|████▋ | 140/300 [00:47<00:48, 3.32it/s]
47%|████▋ | 141/300 [00:47<00:49, 3.18it/s]
47%|████▋ | 142/300 [00:47<00:51, 3.08it/s]
48%|████▊ | 143/300 [00:48<00:55, 2.85it/s]
48%|████▊ | 144/300 [00:48<00:49, 3.16it/s]
48%|████▊ | 145/300 [00:48<00:49, 3.13it/s]
49%|████▊ | 146/300 [00:49<00:46, 3.29it/s]
49%|████▉ | 147/300 [00:49<00:53, 2.86it/s]
49%|████▉ | 148/300 [00:49<00:50, 3.02it/s]
50%|████▉ | 149/300 [00:50<00:53, 2.81it/s]
50%|█████ | 150/300 [00:50<00:49, 3.01it/s]
50%|█████ | 151/300 [00:50<00:51, 2.89it/s]
51%|█████ | 152/300 [00:51<00:46, 3.20it/s]
51%|█████ | 153/300 [00:51<00:47, 3.09it/s]
51%|█████▏ | 154/300 [00:51<00:43, 3.37it/s]
52%|█████▏ | 155/300 [00:52<00:45, 3.17it/s]
52%|█████▏ | 156/300 [00:52<00:41, 3.49it/s]
52%|█████▏ | 157/300 [00:52<00:45, 3.13it/s]
53%|█████▎ | 158/300 [00:52<00:43, 3.27it/s]
53%|█████▎ | 159/300 [00:53<00:52, 2.67it/s]
53%|█████▎ | 160/300 [00:53<00:48, 2.90it/s]
54%|█████▎ | 161/300 [00:54<00:51, 2.70it/s]
54%|█████▍ | 162/300 [00:54<00:50, 2.74it/s]
54%|█████▍ | 163/300 [00:55<00:56, 2.43it/s]
55%|█████▍ | 164/300 [00:55<00:53, 2.55it/s]
55%|█████▌ | 165/300 [00:55<00:53, 2.54it/s]
55%|█████▌ | 166/300 [00:56<00:50, 2.65it/s]
56%|█████▌ | 167/300 [00:56<00:54, 2.43it/s]
56%|█████▌ | 168/300 [00:56<00:47, 2.75it/s]
56%|█████▋ | 169/300 [00:57<00:52, 2.48it/s]
57%|█████▋ | 170/300 [00:57<00:50, 2.59it/s]
57%|█████▋ | 171/300 [00:58<00:50, 2.54it/s]
57%|█████▋ | 172/300 [00:58<00:44, 2.88it/s]
58%|█████▊ | 173/300 [00:58<00:48, 2.63it/s]
58%|█████▊ | 174/300 [00:59<00:42, 2.97it/s]
58%|█████▊ | 175/300 [00:59<00:45, 2.73it/s]
59%|█████▊ | 176/300 [00:59<00:40, 3.07it/s]
59%|█████▉ | 177/300 [01:00<00:46, 2.65it/s]
59%|█████▉ | 178/300 [01:00<00:39, 3.08it/s]
60%|█████▉ | 179/300 [01:00<00:42, 2.88it/s]
60%|██████ | 180/300 [01:01<00:38, 3.16it/s]
60%|██████ | 181/300 [01:01<00:41, 2.85it/s]
61%|██████ | 182/300 [01:01<00:37, 3.16it/s]
61%|██████ | 183/300 [01:02<00:40, 2.90it/s]
61%|██████▏ | 184/300 [01:02<00:40, 2.89it/s]
62%|██████▏ | 185/300 [01:02<00:42, 2.70it/s]
62%|██████▏ | 186/300 [01:03<00:39, 2.89it/s]
62%|██████▏ | 187/300 [01:03<00:39, 2.87it/s]
63%|██████▎ | 188/300 [01:03<00:35, 3.15it/s]
63%|██████▎ | 189/300 [01:04<00:38, 2.86it/s]
63%|██████▎ | 190/300 [01:04<00:35, 3.09it/s]
64%|██████▎ | 191/300 [01:04<00:37, 2.94it/s]
64%|██████▍ | 192/300 [01:05<00:36, 2.97it/s]
64%|██████▍ | 193/300 [01:05<00:41, 2.56it/s]
65%|██████▍ | 194/300 [01:06<00:40, 2.64it/s]
65%|██████▌ | 195/300 [01:06<00:41, 2.54it/s]
65%|██████▌ | 196/300 [01:06<00:37, 2.75it/s]
66%|██████▌ | 197/300 [01:07<00:37, 2.72it/s]
66%|██████▌ | 198/300 [01:07<00:34, 2.94it/s]
66%|██████▋ | 199/300 [01:07<00:35, 2.81it/s]
67%|██████▋ | 200/300 [01:08<00:33, 2.99it/s]
67%|██████▋ | 201/300 [01:08<00:35, 2.79it/s]
67%|██████▋ | 202/300 [01:08<00:31, 3.08it/s]
68%|██████▊ | 203/300 [01:09<00:33, 2.93it/s]
68%|██████▊ | 204/300 [01:09<00:29, 3.22it/s]
68%|██████▊ | 205/300 [01:09<00:31, 3.01it/s]
69%|██████▊ | 206/300 [01:09<00:27, 3.40it/s]
69%|██████▉ | 207/300 [01:10<00:31, 2.92it/s]
69%|██████▉ | 208/300 [01:10<00:31, 2.93it/s]
70%|██████▉ | 209/300 [01:11<00:32, 2.79it/s]
70%|███████ | 210/300 [01:11<00:29, 3.00it/s]
70%|███████ | 211/300 [01:11<00:33, 2.64it/s]
71%|███████ | 212/300 [01:12<00:29, 2.96it/s]
71%|███████ | 213/300 [01:12<00:32, 2.69it/s]
71%|███████▏ | 214/300 [01:12<00:29, 2.94it/s]
72%|███████▏ | 215/300 [01:13<00:30, 2.76it/s]
72%|███████▏ | 216/300 [01:13<00:27, 3.06it/s]
72%|███████▏ | 217/300 [01:14<00:31, 2.64it/s]
73%|███████▎ | 218/300 [01:14<00:26, 3.04it/s]
73%|███████▎ | 219/300 [01:14<00:28, 2.80it/s]
73%|███████▎ | 220/300 [01:14<00:25, 3.09it/s]
74%|███████▎ | 221/300 [01:15<00:28, 2.78it/s]
74%|███████▍ | 222/300 [01:15<00:25, 3.09it/s]
74%|███████▍ | 223/300 [01:16<00:28, 2.74it/s]
75%|███████▍ | 224/300 [01:16<00:25, 2.94it/s]
75%|███████▌ | 225/300 [01:16<00:26, 2.80it/s]
75%|███████▌ | 226/300 [01:17<00:24, 3.03it/s]
76%|███████▌ | 227/300 [01:17<00:26, 2.79it/s]
76%|███████▌ | 228/300 [01:17<00:23, 3.12it/s]
76%|███████▋ | 229/300 [01:18<00:25, 2.75it/s]
77%|███████▋ | 230/300 [01:18<00:24, 2.85it/s]
77%|███████▋ | 231/300 [01:18<00:25, 2.71it/s]
77%|███████▋ | 232/300 [01:19<00:24, 2.78it/s]
78%|███████▊ | 233/300 [01:19<00:27, 2.47it/s]
78%|███████▊ | 234/300 [01:19<00:23, 2.82it/s]
78%|███████▊ | 235/300 [01:20<00:24, 2.64it/s]
79%|███████▊ | 236/300 [01:20<00:23, 2.71it/s]
79%|███████▉ | 237/300 [01:21<00:24, 2.61it/s]
79%|███████▉ | 238/300 [01:21<00:21, 2.93it/s]
80%|███████▉ | 239/300 [01:21<00:21, 2.90it/s]
80%|████████ | 240/300 [01:21<00:18, 3.21it/s]
80%|████████ | 241/300 [01:22<00:19, 3.07it/s]
81%|████████ | 242/300 [01:22<00:18, 3.10it/s]
81%|████████ | 243/300 [01:23<00:20, 2.74it/s]
81%|████████▏ | 244/300 [01:23<00:19, 2.84it/s]
82%|████████▏ | 245/300 [01:23<00:20, 2.69it/s]
82%|████████▏ | 246/300 [01:24<00:19, 2.78it/s]
82%|████████▏ | 247/300 [01:24<00:20, 2.60it/s]
83%|████████▎ | 248/300 [01:24<00:17, 2.94it/s]
83%|████████▎ | 249/300 [01:25<00:18, 2.80it/s]
83%|████████▎ | 250/300 [01:25<00:17, 2.90it/s]
84%|████████▎ | 251/300 [01:25<00:17, 2.80it/s]
84%|████████▍ | 252/300 [01:26<00:16, 2.99it/s]
84%|████████▍ | 253/300 [01:26<00:16, 2.84it/s]
85%|████████▍ | 254/300 [01:26<00:14, 3.21it/s]
85%|████████▌ | 255/300 [01:27<00:15, 2.85it/s]
85%|████████▌ | 256/300 [01:27<00:13, 3.19it/s]
86%|████████▌ | 257/300 [01:27<00:14, 2.98it/s]
86%|████████▌ | 258/300 [01:28<00:12, 3.24it/s]
86%|████████▋ | 259/300 [01:28<00:13, 3.00it/s]
87%|████████▋ | 260/300 [01:28<00:12, 3.18it/s]
87%|████████▋ | 261/300 [01:29<00:12, 3.01it/s]
87%|████████▋ | 262/300 [01:29<00:11, 3.30it/s]
88%|████████▊ | 263/300 [01:29<00:12, 3.01it/s]
88%|████████▊ | 264/300 [01:30<00:12, 3.00it/s]
88%|████████▊ | 265/300 [01:30<00:13, 2.59it/s]
89%|████████▊ | 266/300 [01:31<00:12, 2.68it/s]
89%|████████▉ | 267/300 [01:31<00:13, 2.42it/s]
89%|████████▉ | 268/300 [01:31<00:11, 2.77it/s]
90%|████████▉ | 269/300 [01:32<00:12, 2.45it/s]
90%|█████████ | 270/300 [01:32<00:11, 2.63it/s]
90%|█████████ | 271/300 [01:33<00:12, 2.38it/s]
91%|█████████ | 272/300 [01:33<00:10, 2.64it/s]
91%|█████████ | 273/300 [01:33<00:11, 2.38it/s]
91%|█████████▏| 274/300 [01:34<00:09, 2.72it/s]
92%|█████████▏| 275/300 [01:34<00:10, 2.43it/s]
92%|█████████▏| 276/300 [01:34<00:09, 2.61it/s]
92%|█████████▏| 277/300 [01:35<00:09, 2.53it/s]
93%|█████████▎| 278/300 [01:35<00:07, 2.85it/s]
93%|█████████▎| 279/300 [01:36<00:07, 2.82it/s]
93%|█████████▎| 280/300 [01:36<00:06, 2.91it/s]
94%|█████████▎| 281/300 [01:36<00:06, 2.73it/s]
94%|█████████▍| 282/300 [01:37<00:05, 3.03it/s]
94%|█████████▍| 283/300 [01:37<00:06, 2.81it/s]
95%|█████████▍| 284/300 [01:37<00:05, 3.08it/s]
95%|█████████▌| 285/300 [01:38<00:05, 2.67it/s]
95%|█████████▌| 286/300 [01:38<00:05, 2.73it/s]
96%|█████████▌| 287/300 [01:38<00:04, 2.61it/s]
96%|█████████▌| 288/300 [01:39<00:03, 3.03it/s]
96%|█████████▋| 289/300 [01:39<00:03, 2.79it/s]
97%|█████████▋| 290/300 [01:39<00:03, 2.81it/s]
97%|█████████▋| 291/300 [01:40<00:03, 2.61it/s]
97%|█████████▋| 292/300 [01:40<00:02, 2.93it/s]
98%|█████████▊| 293/300 [01:41<00:02, 2.53it/s]
98%|█████████▊| 294/300 [01:41<00:02, 2.80it/s]
98%|█████████▊| 295/300 [01:41<00:01, 2.79it/s]
99%|█████████▊| 296/300 [01:42<00:01, 3.02it/s]
99%|█████████▉| 297/300 [01:42<00:01, 2.83it/s]
99%|█████████▉| 298/300 [01:42<00:00, 3.20it/s]
100%|█████████▉| 299/300 [01:43<00:00, 2.84it/s]
100%|██████████| 300/300 [01:43<00:00, 2.79it/s]
 36%|███▋ | 1000/2757 [2:17:23<3:55:40, 8.05s/it]
100%|██████████| 300/300 [01:43<00:00, 2.79it/s]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-1000/tokenizer_config.json.
36%|███▋ | 1001/2757 [2:17:37<20:00:37, 41.02s/it] 36%|███▋ | 1002/2757 [2:17:44<15:08:38, 31.06s/it] 36%|███▋ | 1003/2757 [2:17:52<11:41:21, 23.99s/it] 36%|███▋ | 1004/2757 [2:18:00<9:22:18, 19.25s/it] 36%|███▋ | 1005/2757 [2:18:08<7:42:17, 15.83s/it] 36%|███▋ | 1006/2757 [2:18:16<6:34:13, 13.51s/it] 37%|███▋ | 1007/2757 [2:18:24<5:44:52, 11.82s/it] 37%|███▋ | 1008/2757 [2:18:32<5:11:16, 10.68s/it] 37%|███▋ | 1009/2757 [2:18:40<4:49:29, 9.94s/it] 37%|███▋ | 1010/2757 [2:18:48<4:33:40, 9.40s/it] 37%|███▋ | 1010/2757 [2:18:48<4:33:40, 9.40s/it] 37%|███▋ | 1011/2757 [2:18:57<4:23:38, 9.06s/it] 37%|███▋ | 1012/2757 [2:19:04<4:11:48, 8.66s/it] 37%|███▋ | 1013/2757 [2:19:12<4:04:05, 8.40s/it] 37%|███▋ | 1014/2757 [2:19:20<4:01:30, 8.31s/it] 37%|███▋ | 1015/2757 [2:19:29<4:02:12, 8.34s/it] 37%|███▋ | 1016/2757 [2:19:37<4:04:17, 8.42s/it] 37%|███▋ | 1017/2757 [2:19:45<3:58:22, 8.22s/it] 37%|███▋ | 1018/2757 [2:19:53<3:58:32, 8.23s/it] 37%|███▋ | 1019/2757 [2:20:01<3:56:37, 8.17s/it] 37%|███▋ | 1020/2757 [2:20:09<3:55:06, 8.12s/it] 37%|███▋ | 1020/2757 [2:20:09<3:55:06, 8.12s/it] 37%|███▋ | 1021/2757 [2:20:18<3:57:07, 8.20s/it] 37%|███▋ | 1022/2757 [2:20:25<3:53:43, 8.08s/it] 37%|███▋ | 1023/2757 [2:20:33<3:52:17, 8.04s/it] 37%|███▋ | 1024/2757 [2:20:41<3:51:47, 8.02s/it] 37%|███▋ | 1025/2757 [2:20:50<3:56:14, 8.18s/it] 37%|███▋ | 1026/2757 [2:20:58<3:57:40, 8.24s/it] 37%|███▋ | 1027/2757 [2:21:07<4:00:06, 8.33s/it] 37%|███▋ | 1028/2757 [2:21:15<3:55:40, 8.18s/it] 37%|███▋ | 1029/2757 [2:21:22<3:50:48, 8.01s/it] 37%|███▋ | 1030/2757 [2:21:30<3:51:07, 8.03s/it] 37%|███▋ | 1030/2757 [2:21:30<3:51:07, 8.03s/it] 37%|███▋ | 1031/2757 [2:21:38<3:49:36, 7.98s/it] 37%|███▋ | 1032/2757 [2:21:46<3:48:41, 7.95s/it] 37%|███▋ | 1033/2757 [2:21:54<3:49:13, 7.98s/it] 38%|███▊ | 1034/2757 [2:22:02<3:47:37, 7.93s/it] 38%|███▊ | 1035/2757 [2:22:10<3:50:18, 8.02s/it] 38%|███▊ | 1036/2757 [2:22:18<3:52:05, 8.09s/it] 38%|███▊ | 1037/2757 [2:22:26<3:47:29, 7.94s/it] 38%|███▊ | 1038/2757 [2:22:34<3:44:24, 7.83s/it] 38%|███▊ | 1039/2757 [2:22:41<3:44:49, 7.85s/it] 38%|███▊ | 1040/2757 [2:22:50<3:47:11, 7.94s/it] 38%|███▊ | 1040/2757 [2:22:50<3:47:11, 7.94s/it] 38%|███▊ | 1041/2757 [2:22:58<3:49:38, 8.03s/it] 38%|███▊ | 1042/2757 [2:23:06<3:46:26, 7.92s/it] 38%|███▊ | 1043/2757 [2:23:14<3:47:21, 7.96s/it] 38%|███▊ | 1044/2757 [2:23:21<3:45:36, 7.90s/it] 38%|███▊ | 1045/2757 [2:23:29<3:45:19, 7.90s/it] 38%|███▊ | 1046/2757 [2:23:37<3:47:21, 7.97s/it] 38%|███▊ | 1047/2757 [2:23:45<3:45:12, 7.90s/it] 38%|███▊ | 1048/2757 [2:23:53<3:44:51, 7.89s/it] 38%|███▊ | 1049/2757 [2:24:01<3:48:04, 8.01s/it] 38%|███▊ | 1050/2757 [2:24:09<3:46:46, 7.97s/it] 38%|███▊ | 1050/2757 [2:24:09<3:46:46, 7.97s/it] 38%|███▊ | 1051/2757 [2:24:17<3:43:57, 7.88s/it] 38%|███▊ | 1052/2757 [2:24:25<3:46:32, 7.97s/it] 38%|███▊ | 1053/2757 [2:24:33<3:46:54, 7.99s/it] 38%|███▊ | 1054/2757 [2:24:41<3:46:38, 7.99s/it] 38%|███▊ | 1055
{'loss': '0.3826', 'grad_norm': '0.5595', 'learning_rate': '0.0001307', 'rewards/chosen': '-1.484', 'rewards/rejected': '-2.78', 'rewards/accuracies': '0.7914', 'rewards/margins': '1.296', 'logps/chosen': '-99.29', 'logps/rejected': '-118.4', 'logits/chosen': '-3.986', 'logits/rejected': '-3.91', 'epoch': '1.099'}
{'loss': '0.3769', 'grad_norm': '0.5151', 'learning_rate': '0.00013', 'rewards/chosen': '-1.544', 'rewards/rejected': '-2.896', 'rewards/accuracies': '0.7859', 'rewards/margins': '1.352', 'logps/chosen': '-95.98', 'logps/rejected': '-114.6', 'logits/chosen': '-3.936', 'logits/rejected': '-3.925', 'epoch': '1.11'}
{'loss': '0.3913', 'grad_norm': '0.5914', 'learning_rate': '0.0001292', 'rewards/chosen': '-1.531', 'rewards/rejected': '-2.838', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.307', 'logps/chosen': '-95.18', 'logps/rejected': '-113.3', 'logits/chosen': '-3.944', 'logits/rejected': '-3.977', 'epoch': '1.121'}
{'loss': '0.4064', 'grad_norm': '0.609', 'learning_rate': '0.0001285', 'rewards/chosen': '-1.362', 'rewards/rejected': '-2.581', 'rewards/accuracies': '0.7563', 'rewards/margins': '1.219', 'logps/chosen': '-91.94', 'logps/rejected': '-109.3', 'logits/chosen': '-3.923', 'logits/rejected': '-3.879', 'epoch': '1.132'}
{'loss': '0.3909', 'grad_norm': '0.558', 'learning_rate': '0.0001277', 'rewards/chosen': '-1.129', 'rewards/rejected': '-2.423', 'rewards/accuracies': '0.7781', 'rewards/margins': '1.294', 'logps/chosen': '-93.53', 'logps/rejected': '-110.6', 'logits/chosen': '-4.08', 'logits/rejected': '-4.015', 'epoch': '1.143'}
{'loss': '0.4011', 'grad_norm': '0.6687', 'learning_rate': '0.000127', 'rewards/chosen': '-1.27', 'rewards/rejected': '-2.525', 'rewards/accuracies': '0.7695', 'rewards/margins': '1.256', 'logps/chosen': '-92.7', 'logps/rejected': '-109', 'logits/chosen': '-3.949', 'logits/rejected': '-3.86', 'epoch': '1.153'}
{'loss': '0.3885', 'grad_norm': '0.5773', 'learning_rate': '0.0001263', 'rewards/chosen': '-1.37', 'rewards/rejected': '-2.672', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.302', 'logps/chosen': '-93.09', 'logps/rejected': '-112.1', 'logits/chosen': '-4.051', 'logits/rejected': '-3.921', 'epoch': '1.164'}
{'loss': '0.3903', 'grad_norm': '0.5501', 'learning_rate': '0.0001255', 'rewards/chosen': '-1.401', 'rewards/rejected': '-2.699', 'rewards/accuracies': '0.7797', 'rewards/margins': '1.299', 'logps/chosen': '-93.73', 'logps/rejected': '-112.1', 'logits/chosen': '-3.901', 'logits/rejected': '-3.798', 'epoch': '1.175'}
{'loss': '0.3811', 'grad_norm': '0.602', 'learning_rate': '0.0001248', 'rewards/chosen': '-1.542', 'rewards/rejected': '-2.905', 'rewards/accuracies': '0.7859', 'rewards/margins': '1.362', 'logps/chosen': '-97.4', 'logps/rejected': '-116.5', 'logits/chosen': '-3.874', 'logits/rejected': '-3.843', 'epoch': '1.186'}
{'loss': '0.3859', 'grad_norm': '0.5466', 'learning_rate': '0.000124', 'rewards/chosen': '-1.562', 'rewards/rejected': '-2.871', 'rewards/accuracies': '0.7867', 'rewards/margins': '1.309', 'logps/chosen': '-97.63', 'logps/rejected': '-116.3', 'logits/chosen': '-3.822', 'logits/rejected': '-3.789', 'epoch': '1.197'}
{'loss': '0.3874', 'grad_norm': '0.6201', 'learning_rate': '0.0001233', 'rewards/chosen': '-1.538', 'rewards/rejected': '-2.896', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.358', 'logps/chosen': '-94.8', 'logps/rejected': '-114.1', 'logits/chosen': '-3.869', 'logits/rejected': '-3.814', 'epoch': '1.208'}
{'loss': '0.3829', 'grad_norm': '0.6293', 'learning_rate': '0.0001225', 'rewards/chosen': '-1.702', 'rewards/rejected': '-3.064', 'rewards/accuracies': '0.782', 'rewards/margins': '1.362', 'logps/chosen': '-98.96', 'logps/rejected': '-118.2', 'logits/chosen': '-3.824', 'logits/rejected': '-3.681', 'epoch': '1.219'}
{'loss': '0.3882', 'grad_norm': '0.6542', 'learning_rate': '0.0001218', 'rewards/chosen': '-1.607', 'rewards/rejected': '-2.912', 'rewards/accuracies': '0.7711', 'rewards/margins': '1.305', 'logps/chosen': '-95.04', 'logps/rejected': '-113.7', 'logits/chosen': '-3.806', 'logits/rejected': '-3.804', 'epoch': '1.23'}
{'loss': '0.3934', 'grad_norm': '0.6825', 'learning_rate': '0.000121', 'rewards/chosen': '-1.642', 'rewards/rejected': '-2.956', 'rewards/accuracies': '0.7797', 'rewards/margins': '1.314', 'logps/chosen': '-97.03', 'logps/rejected': '-116', 'logits/chosen': '-3.831', 'logits/rejected': '-3.731', 'epoch': '1.241'}
{'loss': '0.3808', 'grad_norm': '0.5602', 'learning_rate': '0.0001203', 'rewards/chosen': '-1.608', 'rewards/rejected': '-2.998', 'rewards/accuracies': '0.7898', 'rewards/margins': '1.39', 'logps/chosen': '-96.62', 'logps/rejected': '-116.4', 'logits/chosen': '-3.864', 'logits/rejected': '-3.806', 'epoch': '1.251'}
{'loss': '0.3848', 'grad_norm': '0.5861', 'learning_rate': '0.0001195', 'rewards/chosen': '-1.649', 'rewards/rejected': '-3.021', 'rewards/accuracies': '0.775', 'rewards/margins': '1.372', 'logps/chosen': '-95.55', 'logps/rejected': '-114.5', 'logits/chosen': '-3.834', 'logits/rejected': '-3.734', 'epoch': '1.262'}
{'loss': '0.3743', 'grad_norm': '0.5589', 'learning_rate': '0.0001188', 'rewards/chosen': '-1.889', 'rewards/rejected': '-3.314', 'rewards/accuracies': '0.7883', 'rewards/margins': '1.426', 'logps/chosen': '-98.93', 'logps/rejected': '-120.1', 'logits/chosen': '-3.808', 'logits/rejected': '-3.658', 'epoch': '1.273'}
{'loss': '0.3917', 'grad_norm': '0.7734', 'learning_rate': '0.000118', 'rewards/chosen': '-2.133', 'rewards/rejected': '-3.482', 'rewards/accuracies': '0.7727', 'rewards/margins': '1.35', 'logps/chosen': '-100.4', 'logps/rejected': '-119.5', 'logits/chosen': '-3.791', 'logits/rejected': '-3.751', 'epoch': '1.284'}
{'loss': '0.3873', 'grad_norm': '0.6508', 'learning_rate': '0.0001173', 'rewards/chosen': '-1.986', 'rewards/rejected': '-3.376', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.39', 'logps/chosen': '-102.7', 'logps/rejected': '-119.9', 'logits/chosen': '-3.632', 'logits/rejected': '-3.56', 'epoch': '1.295'}
{'loss': '0.3865', 'grad_norm': '0.5762', 'learning_rate': '0.0001165', 'rewards/chosen': '-1.77', 'rewards/rejected': '-3.147', 'rewards/accuracies': '0.7789', 'rewards/margins': '1.377', 'logps/chosen': '-96.95', 'logps/rejected': '-118.6', 'logits/chosen': '-3.736', 'logits/rejected': '-3.719', 'epoch': '1.306'}
{'loss': '0.3787', 'grad_norm': '0.7514', 'learning_rate': '0.0001158', 'rewards/chosen': '-1.872', 'rewards/rejected': '-3.307', 'rewards/accuracies': '0.7898', 'rewards/margins': '1.435', 'logps/chosen': '-99.3', 'logps/rejected': '-118.7', 'logits/chosen': '-3.761', 'logits/rejected': '-3.714', 'epoch': '1.317'}
{'loss': '0.3696', 'grad_norm': '0.6535', 'learning_rate': '0.000115', 'rewards/chosen': '-1.846', 'rewards/rejected': '-3.268', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.422', 'logps/chosen': '-100.5', 'logps/rejected': '-121', 'logits/chosen': '-3.695', 'logits/rejected': '-3.616', 'epoch': '1.328'}
{'loss': '0.3887', 'grad_norm': '0.6603', 'learning_rate': '0.0001143', 'rewards/chosen': '-1.846', 'rewards/rejected': '-3.164', 'rewards/accuracies': '0.7828', 'rewards/margins': '1.318', 'logps/chosen': '-98.53', 'logps/rejected': '-115.1', 'logits/chosen': '-3.787', 'logits/rejected': '-3.732', 'epoch': '1.339'}
{'loss': '0.381', 'grad_norm': '0.7037', 'learning_rate': '0.0001135', 'rewards/chosen': '-1.664', 'rewards/rejected': '-3.028', 'rewards/accuracies': '0.7875', 'rewards/margins': '1.364', 'logps/chosen': '-96.31', 'logps/rejected': '-115.1', 'logits/chosen': '-3.723', 'logits/rejected': '-3.703', 'epoch': '1.349'}
{'loss': '0.3888', 'grad_norm': '0.638', 'learning_rate': '0.0001128', 'rewards/chosen': '-1.572', 'rewards/rejected': '-2.942', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.371', 'logps/chosen': '-95.81', 'logps/rejected': '-114.3', 'logits/chosen': '-3.744', 'logits/rejected': '-3.591', 'epoch': '1.36'} 45%|████▌ | 1250/2757 [2:50:46<3:25:23, 8.18s/it] 45%|████▌ | 1251/2757 [2:50:59<3:56:02, 9.40s/it] 45%|████▌ | 1252/2757 [2:51:07<3:46:44, 9.04s/it] 45%|████▌ | 1253/2757 [2:51:14<3:35:40, 8.60s/it] 45%|████▌ | 1254/2757 [2:51:22<3:30:16, 8.39s/it] 46%|████▌ | 1255/2757 [2:51:31<3:30:03, 8.39s/it] 46%|████▌ | 1256/2757 [2:51:38<3:24:39, 8.18s/it] 46%|████▌ | 1257/2757 [2:51:46<3:18:50, 7.95s/it] 46%|████▌ | 1258/2757 [2:51:54<3:17:15, 7.90s/it] 46%|████▌ | 1259/2757 [2:52:02<3:18:40, 7.96s/it] 46%|████▌ | 1260/2757 [2:52:10<3:20:41, 8.04s/it] 46%|████▌ | 1260/2757 [2:52:10<3:20:41, 8.04s/it] 46%|████▌ | 1261/2757 [2:52:18<3:19:17, 7.99s/it] 46%|████▌ | 1262/2757 [2:52:26<3:16:47, 7.90s/it] 46%|████▌ | 1263/2757 [2:52:34<3:20:51, 8.07s/it] 46%|████▌ | 1264/2757 [2:52:42<3:19:12, 8.01s/it] 46%|████▌ | 1265/2757 [2:52:50<3:18:42, 7.99s/it] 46%|████▌ | 1266/2757 [2:52:58<3:22:26, 8.15s/it] 46%|████▌ | 1267/2757 [2:53:07<3:23:52, 8.21s/it] 46%|████▌ | 1268/2757 [2:53:15<3:22:54, 8.18s/it] 46%|████▌ | 1269/2757 [2:53:23<3:19:56, 8.06s/it] 46%|████▌ | 1270/2757 [2:53:31<3:20:16, 8.08s/it] 46%|████▌ | 1270/2757 [2:53:31<3:20:16, 8.08s/it] 46%|████▌ | 1271/2757 [2:53:39<3:20:24, 8.09s/it] 46%|████▌ | 1272/2757 [2:53:47<3:21:51, 8.16s/it] 46%|████▌ | 1273/2757 [2:53:55<3:18:37, 8.03s/it] 46%|████▌ | 1274/2757 [2:54:03<3:20:09, 8.10s/it] 46%|████▌ | 1275/2757 [2:54:11<3:18:43, 8.05s/it] 46%|████▋ | 1276/2757 [2:54:19<3:18:58, 8.06s/it] 46%|████▋ | 1277/2757 [2:54:27<3:19:54, 8.10s/it] 46%|████▋ | 1278/2757 [2:54:35<3:18:47, 8.06s/it] 46%|████▋ | 1279/2757 [2:54:43<3:18:24, 8.05s/it] 46%|████▋ | 1280/2757 [2:54:51<3:16:33, 7.98s/it] 46%|████▋ | 1280/2757 [2:54:51<3:16:33, 7.98s/it] 46%|████▋ | 1281/2757 [2:54:59<3:14:45, 7.92s/it] 46%|████▋ | 1282/2757 [2:55:07<3:13:50, 7.89s/it] 47%|████▋ | 1283/2757 [2:55:15<3:14:51, 7.93s/it] 47%|████▋ | 1284/2757 [2:55:23<3:15:39, 7.97s/it] 47%|████▋ | 1285/2757 [2:55:31<3:17:58, 8.07s/it] 47%|████▋ | 1286/2757 [2:55:39<3:17:15, 8.05s/it] 47%|████▋ | 1287/2757 [2:55:47<3:18:06, 8.09s/it] 47%|████▋ | 1288/2757 [2:55:55<3:16:46, 8.04s/it] 47%|████▋ | 1289/2757 [2:56:03<3:14:41, 7.96s/it] 47%|████▋ | 1290/2757 [2:56:11<3:17:04, 8.06s/it] 47%|████▋ | 1290/2757 [2:56:11<3:17:04, 8.06s/it] 47%|████▋ | 1291/2757 [2:56:20<3:20:07, 8.19s/it] 47%|████▋ | 1292/2757 [2:56:28<3:20:44, 8.22s/it] 47%|████▋ | 1293/2757 [2:56:36<3:20:21, 8.21s/it] 47%|████▋ | 1294/2757 [2:56:45<3:22:26, 8.30s/it] 47%|████▋ | 1295/2757 [2:56:53<3:18:58, 8.17s/it] 47%|████▋ | 1296/2757 [2:57:01<3:20:48, 8.25s/it] 47%|████▋ | 1297/2757 [2:57:09<3:19:47, 8.21s/it] 47%|████▋ | 1298/2757 [2:57:17<3:17:15, 8.11s/it] 47%|████▋ | 1299/2757 [2:57:25<3:15:30
{'loss': '0.3822', 'grad_norm': '0.6954', 'learning_rate': '0.000112', 'rewards/chosen': '-1.646', 'rewards/rejected': '-3.055', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.409', 'logps/chosen': '-94.91', 'logps/rejected': '-116.6', 'logits/chosen': '-3.762', 'logits/rejected': '-3.743', 'epoch': '1.371'}
{'loss': '0.3603', 'grad_norm': '0.5787', 'learning_rate': '0.0001113', 'rewards/chosen': '-1.818', 'rewards/rejected': '-3.225', 'rewards/accuracies': '0.8047', 'rewards/margins': '1.406', 'logps/chosen': '-98.85', 'logps/rejected': '-118.1', 'logits/chosen': '-3.881', 'logits/rejected': '-3.811', 'epoch': '1.382'}
{'loss': '0.3662', 'grad_norm': '0.668', 'learning_rate': '0.0001105', 'rewards/chosen': '-1.788', 'rewards/rejected': '-3.26', 'rewards/accuracies': '0.8039', 'rewards/margins': '1.471', 'logps/chosen': '-100.2', 'logps/rejected': '-120.3', 'logits/chosen': '-3.82', 'logits/rejected': '-3.791', 'epoch': '1.393'}
{'loss': '0.3878', 'grad_norm': '0.6281', 'learning_rate': '0.0001098', 'rewards/chosen': '-1.594', 'rewards/rejected': '-3.012', 'rewards/accuracies': '0.7703', 'rewards/margins': '1.418', 'logps/chosen': '-95.59', 'logps/rejected': '-115.6', 'logits/chosen': '-3.73', 'logits/rejected': '-3.709', 'epoch': '1.404'}
{'loss': '0.3713', 'grad_norm': '0.6394', 'learning_rate': '0.0001091', 'rewards/chosen': '-1.553', 'rewards/rejected': '-2.992', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.439', 'logps/chosen': '-97.09', 'logps/rejected': '-116', 'logits/chosen': '-3.678', 'logits/rejected': '-3.603', 'epoch': '1.415'}
{'loss': '0.3837', 'grad_norm': '0.5887', 'learning_rate': '0.0001083', 'rewards/chosen': '-1.771', 'rewards/rejected': '-3.158', 'rewards/accuracies': '0.7773', 'rewards/margins': '1.388', 'logps/chosen': '-97.09', 'logps/rejected': '-115', 'logits/chosen': '-3.858', 'logits/rejected': '-3.729', 'epoch': '1.426'}
{'loss': '0.3637', 'grad_norm': '0.6475', 'learning_rate': '0.0001076', 'rewards/chosen': '-1.754', 'rewards/rejected': '-3.263', 'rewards/accuracies': '0.7883', 'rewards/margins': '1.509', 'logps/chosen': '-97.05', 'logps/rejected': '-119.6', 'logits/chosen': '-3.94', 'logits/rejected': '-3.782', 'epoch': '1.436'}
{'loss': '0.3758', 'grad_norm': '0.6646', 'learning_rate': '0.0001068', 'rewards/chosen': '-1.683', 'rewards/rejected': '-3.217', 'rewards/accuracies': '0.7719', 'rewards/margins': '1.534', 'logps/chosen': '-96.41', 'logps/rejected': '-117.3', 'logits/chosen': '-3.977', 'logits/rejected': '-3.893', 'epoch': '1.447'}
{'loss': '0.3911', 'grad_norm': '0.7959', 'learning_rate': '0.0001061', 'rewards/chosen': '-1.504', 'rewards/rejected': '-2.96', 'rewards/accuracies': '0.7672', 'rewards/margins': '1.456', 'logps/chosen': '-92.38', 'logps/rejected': '-112.3', 'logits/chosen': '-4.019', 'logits/rejected': '-3.9', 'epoch': '1.458'}
{'loss': '0.3842', 'grad_norm': '0.6806', 'learning_rate': '0.0001053', 'rewards/chosen': '-1.673', 'rewards/rejected': '-3.111', 'rewards/accuracies': '0.7758', 'rewards/margins': '1.438', 'logps/chosen': '-101.7', 'logps/rejected': '-121', 'logits/chosen': '-3.86', 'logits/rejected': '-3.82', 'epoch': '1.469'}
{'loss': '0.3684', 'grad_norm': '0.6531', 'learning_rate': '0.0001046', 'rewards/chosen': '-1.91', 'rewards/rejected': '-3.38', 'rewards/accuracies': '0.7852', 'rewards/margins': '1.47', 'logps/chosen': '-100.1', 'logps/rejected': '-119.4', 'logits/chosen': '-3.826', 'logits/rejected': '-3.773', 'epoch': '1.48'}
{'loss': '0.3801', 'grad_norm': '0.6556', 'learning_rate': '0.0001038', 'rewards/chosen': '-1.97', 'rewards/rejected': '-3.358', 'rewards/accuracies': '0.7648', 'rewards/margins': '1.388', 'logps/chosen': '-101.1', 'logps/rejected': '-120.3', 'logits/chosen': '-3.857', 'logits/rejected': '-3.755', 'epoch': '1.491'}
{'loss': '0.37', 'grad_norm': '0.5829', 'learning_rate': '0.0001031', 'rewards/chosen': '-1.854', 'rewards/rejected': '-3.286', 'rewards/accuracies': '0.8031', 'rewards/margins': '1.432', 'logps/chosen': '-99.89', 'logps/rejected': '-118.7', 'logits/chosen': '-3.933', 'logits/rejected': '-3.822', 'epoch': '1.502'}
{'loss': '0.3734', 'grad_norm': '0.7183', 'learning_rate': '0.0001023', 'rewards/chosen': '-1.893', 'rewards/rejected': '-3.374', 'rewards/accuracies': '0.7828', 'rewards/margins': '1.481', 'logps/chosen': '-97.94', 'logps/rejected': '-117.8', 'logits/chosen': '-4.136', 'logits/rejected': '-3.982', 'epoch': '1.513'}
{'loss': '0.3624', 'grad_norm': '0.6229', 'learning_rate': '0.0001016', 'rewards/chosen': '-1.698', 'rewards/rejected': '-3.211', 'rewards/accuracies': '0.7875', 'rewards/margins': '1.513', 'logps/chosen': '-101.3', 'logps/rejected': '-122.4', 'logits/chosen': '-3.943', 'logits/rejected': '-3.885', 'epoch': '1.524'}
{'loss': '0.3612', 'grad_norm': '0.7118', 'learning_rate': '0.0001008', 'rewards/chosen': '-1.797', 'rewards/rejected': '-3.256', 'rewards/accuracies': '0.7961', 'rewards/margins': '1.459', 'logps/chosen': '-96.92', 'logps/rejected': '-115.9', 'logits/chosen': '-4.146', 'logits/rejected': '-4.064', 'epoch': '1.534'}
{'loss': '0.3773', 'grad_norm': '0.4945', 'learning_rate': '0.0001001', 'rewards/chosen': '-1.945', 'rewards/rejected': '-3.407', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.461', 'logps/chosen': '-100.1', 'logps/rejected': '-120.7', 'logits/chosen': '-4.004', 'logits/rejected': '-3.955', 'epoch': '1.545'}
{'loss': '0.3719', 'grad_norm': '0.5279', 'learning_rate': '9.933e-05', 'rewards/chosen': '-1.995', 'rewards/rejected': '-3.502', 'rewards/accuracies': '0.7797', 'rewards/margins': '1.507', 'logps/chosen': '-102.5', 'logps/rejected': '-122.1', 'logits/chosen': '-3.986', 'logits/rejected': '-3.933', 'epoch': '1.556'}
{'loss': '0.3743', 'grad_norm': '0.5255', 'learning_rate': '9.858e-05', 'rewards/chosen': '-1.921', 'rewards/rejected': '-3.381', 'rewards/accuracies': '0.7891', 'rewards/margins': '1.461', 'logps/chosen': '-100.2', 'logps/rejected': '-119.8', 'logits/chosen': '-3.869', 'logits/rejected': '-3.812', 'epoch': '1.567'}
{'loss': '0.3566', 'grad_norm': '0.6176', 'learning_rate': '9.783e-05', 'rewards/chosen': '-1.973', 'rewards/rejected': '-3.487', 'rewards/accuracies': '0.8', 'rewards/margins': '1.513', 'logps/chosen': '-101.5', 'logps/rejected': '-121.8', 'logits/chosen': '-3.869', 'logits/rejected': '-3.742', 'epoch': '1.578'}
{'loss': '0.3684', 'grad_norm': '0.6701', 'learning_rate': '9.708e-05', 'rewards/chosen': '-2.017', 'rewards/rejected': '-3.525', 'rewards/accuracies': '0.7781', 'rewards/margins': '1.508', 'logps/chosen': '-97.51', 'logps/rejected': '-118.8', 'logits/chosen': '-3.919', 'logits/rejected': '-3.9', 'epoch': '1.589'}
{'loss': '0.369', 'grad_norm': '0.8422', 'learning_rate': '9.634e-05', 'rewards/chosen': '-1.953', 'rewards/rejected': '-3.507', 'rewards/accuracies': '0.7836', 'rewards/margins': '1.554', 'logps/chosen': '-103.5', 'logps/rejected': '-125.3', 'logits/chosen': '-3.815', 'logits/rejected': '-3.763', 'epoch': '1.6'}
{'loss': '0.3787', 'grad_norm': '0.626', 'learning_rate': '9.559e-05', 'rewards/chosen': '-1.782', 'rewards/rejected': '-3.268', 'rewards/accuracies': '0.7883', 'rewards/margins': '1.486', 'logps/chosen': '-98.55', 'logps/rejected': '-118.6', 'logits/chosen': '-3.939', 'logits/rejected': '-3.889', 'epoch': '1.611'}
{'loss': '0.3668', 'grad_norm': '0.6023', 'learning_rate': '9.484e-05', 'rewards/chosen': '-1.582', 'rewards/rejected': '-3.109', 'rewards/accuracies': '0.7812', 'rewards/margins': '1.527', 'logps/chosen': '-96.4', 'logps/rejected': '-115.9', 'logits/chosen': '-4.026', 'logits/rejected': '-3.953', 'epoch': '1.621'}
{'loss': '0.3673', 'grad_norm': '0.5291', 'learning_rate': '9.409e-05', 'rewards/chosen': '-1.715', 'rewards/rejected': '-3.256', 'rewards/accuracies': '0.7898', 'rewards/margins': '1.542', 'logps/chosen': '-96.86', 'logps/rejected': '-118.7', 'logits/chosen': '-3.99', 'logits/rejected': '-3.968', 'epoch': '1.632'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
0%| | 0/300 [00:00<?, ?it/s]
1%| | 2/300 [00:00<00:52, 5.73it/s]
1%| | 3/300 [00:00<01:47, 2.76it/s]
1%|▏ | 4/300 [00:01<01:35, 3.09it/s]
2%|▏ | 5/300 [00:01<01:45, 2.80it/s]
2%|▏ | 6/300 [00:01<01:40, 2.91it/s]
2%|▏ | 7/300 [00:02<01:55, 2.54it/s]
3%|▎ | 8/300 [00:02<01:51, 2.62it/s]
3%|▎ | 9/300 [00:03<01:52, 2.58it/s]
3%|▎ | 10/300 [00:03<01:51, 2.60it/s]
4%|▎ | 11/300 [00:04<01:53, 2.56it/s]
4%|▍ | 12/300 [00:04<01:39, 2.88it/s]
4%|▍ | 13/300 [00:04<01:45, 2.73it/s]
5%|▍ | 14/300 [00:04<01:37, 2.94it/s]
5%|▌ | 15/300 [00:05<01:47, 2.66it/s]
5%|▌ | 16/300 [00:05<01:35, 2.99it/s]
6%|▌ | 17/300 [00:06<01:38, 2.86it/s]
6%|▌ | 18/300 [00:06<01:29, 3.17it/s]
6%|▋ | 19/300 [00:06<01:44, 2.69it/s]
7%|▋ | 20/300 [00:07<01:35, 2.92it/s]
7%|▋ | 21/300 [00:07<01:47, 2.59it/s]
7%|▋ | 22/300 [00:07<01:35, 2.92it/s]
8%|▊ | 23/300 [00:08<01:35, 2.90it/s]
8%|▊ | 24/300 [00:08<01:25, 3.22it/s]
8%|▊ | 25/300 [00:08<01:37, 2.82it/s]
9%|▊ | 26/300 [00:09<01:30, 3.03it/s]
9%|▉ | 27/300 [00:10<02:19, 1.95it/s]
9%|▉ | 28/300 [00:10<01:57, 2.32it/s]
10%|▉ | 29/300 [00:10<02:04, 2.17it/s]
10%|█ | 30/300 [00:11<01:47, 2.52it/s]
10%|█ | 31/300 [00:11<01:55, 2.34it/s]
11%|█ | 32/300 [00:11<01:55, 2.31it/s]
11%|█ | 33/300 [00:12<01:46, 2.50it/s]
11%|█▏ | 34/300 [00:12<01:34, 2.80it/s]
12%|█▏ | 35/300 [00:12<01:37, 2.72it/s]
12%|█▏ | 36/300 [00:13<01:28, 2.97it/s]
12%|█▏ | 37/300 [00:13<01:39, 2.63it/s]
13%|█▎ | 38/300 [00:13<01:27, 3.00it/s]
13%|█▎ | 39/300 [00:14<01:36, 2.70it/s]
13%|█▎ | 40/300 [00:14<01:25, 3.04it/s]
14%|█▎ | 41/300 [00:15<01:35, 2.71it/s]
14%|█▍ | 42/300 [00:15<01:24, 3.05it/s]
14%|█▍ | 43/300 [00:15<01:27, 2.95it/s]
15%|█▍ | 44/300 [00:15<01:19, 3.24it/s]
15%|█▌ | 45/300 [00:16<01:25, 2.99it/s]
15%|█▌ | 46/300 [00:16<01:25, 2.97it/s]
16%|█▌ | 47/300 [00:17<01:30, 2.78it/s]
16%|█▌ | 48/300 [00:17<01:23, 3.02it/s]
16%|█▋ | 49/300 [00:17<01:23, 3.00it/s]
17%|█▋ | 50/300 [00:17<01:15, 3.33it/s]
17%|█▋ | 51/300 [00:18<01:18, 3.18it/s]
17%|█▋ | 52/300 [00:18<01:15, 3.29it/s]
18%|█▊ | 53/300 [00:18<01:28, 2.79it/s]
18%|█▊ | 54/300 [00:19<01:19, 3.09it/s]
18%|█▊ | 55/300 [00:19<01:27, 2.80it/s]
19%|█▊ | 56/300 [00:19<01:15, 3.21it/s]
19%|█▉ | 57/300 [00:20<01:29, 2.72it/s]
19%|█▉ | 58/300 [00:20<01:28, 2.73it/s]
20%|█▉ | 59/300 [00:21<01:31, 2.62it/s]
20%|██ | 60/300 [00:21<01:21, 2.95it/s]
20%|██ | 61/300 [00:21<01:23, 2.85it/s]
21%|██ | 62/300 [00:22<01:22, 2.88it/s]
21%|██ | 63/300 [00:22<01:27, 2.70it/s]
21%|██▏ | 64/300 [00:22<01:25, 2.76it/s]
22%|██▏ | 65/300 [00:23<01:28, 2.66it/s]
22%|██▏ | 66/300 [00:23<01:20, 2.92it/s]
22%|██▏ | 67/300 [00:24<01:27, 2.66it/s]
23%|██▎ | 68/300 [00:24<01:18, 2.95it/s]
23%|██▎ | 69/300 [00:24<01:21, 2.84it/s]
23%|██▎ | 70/300 [00:24<01:13, 3.14it/s]
24%|██▎ | 71/300 [00:25<01:15, 3.02it/s]
24%|██▍ | 72/300 [00:25<01:09, 3.28it/s]
24%|██▍ | 73/300 [00:25<01:12, 3.14it/s]
25%|██▍ | 74/300 [00:26<01:08, 3.32it/s]
25%|██▌ | 75/300 [00:26<01:13, 3.08it/s]
25%|██▌ | 76/300 [00:26<01:14, 3.01it/s]
26%|██▌ | 77/300 [00:27<01:19, 2.81it/s]
26%|██▌ | 78/300 [00:27<01:12, 3.05it/s]
26%|██▋ | 79/300 [00:27<01:15, 2.94it/s]
27%|██▋ | 80/300 [00:28<01:11, 3.06it/s]
27%|██▋ | 81/300 [00:28<01:20, 2.74it/s]
27%|██▋ | 82/300 [00:28<01:11, 3.04it/s]
28%|██▊ | 83/300 [00:29<01:15, 2.88it/s]
28%|██▊ | 84/300 [00:29<01:15, 2.86it/s]
28%|██▊ | 85/300 [00:30<01:19, 2.70it/s]
29%|██▊ | 86/300 [00:30<01:11, 3.00it/s]
29%|██▉ | 87/300 [00:30<01:13, 2.91it/s]
29%|██▉ | 88/300 [00:30<01:07, 3.16it/s]
30%|██▉ | 89/300 [00:31<01:11, 2.96it/s]
30%|███ | 90/300 [00:31<01:05, 3.19it/s]
30%|███ | 91/300 [00:31<01:09, 3.00it/s]
31%|███ | 92/300 [00:32<01:03, 3.27it/s]
31%|███ | 93/300 [00:32<01:09, 2.98it/s]
31%|███▏ | 94/300 [00:32<01:02, 3.28it/s]
32%|███▏ | 95/300 [00:33<01:08, 3.01it/s]
32%|███▏ | 96/300 [00:33<01:03, 3.20it/s]
32%|███▏ | 97/300 [00:33<01:15, 2.67it/s]
33%|███▎ | 98/300 [00:34<01:09, 2.92it/s]
33%|███▎ | 99/300 [00:34<01:12, 2.78it/s]
33%|███▎ | 100/300 [00:34<01:06, 2.99it/s]
34%|███▎ | 101/300 [00:35<01:09, 2.86it/s]
34%|███▍ | 102/300 [00:35<01:07, 2.94it/s]
34%|███▍ | 103/300 [00:36<01:10, 2.79it/s]
35%|███▍ | 104/300 [00:36<01:02, 3.11it/s]
35%|███▌ | 105/300 [00:36<01:05, 2.96it/s]
35%|███▌ | 106/300 [00:36<01:01, 3.17it/s]
36%|███▌ | 107/300 [00:37<01:05, 2.95it/s]
36%|███▌ | 108/300 [00:37<01:03, 3.05it/s]
36%|███▋ | 109/300 [00:38<01:06, 2.86it/s]
37%|███▋ | 110/300 [00:38<01:02, 3.03it/s]
37%|███▋ | 111/300 [00:38<01:11, 2.65it/s]
37%|███▋ | 112/300 [00:39<01:06, 2.84it/s]
38%|███▊ | 113/300 [00:39<01:09, 2.70it/s]
38%|███▊ | 114/300 [00:39<01:07, 2.74it/s]
38%|███▊ | 115/300 [00:40<01:10, 2.62it/s]
39%|███▊ | 116/300 [00:40<01:02, 2.95it/s]
39%|███▉ | 117/300 [00:40<01:03, 2.88it/s]
39%|███▉ | 118/300 [00:41<00:58, 3.10it/s]
40%|███▉ | 119/300 [00:41<01:00, 2.98it/s]
40%|████ | 120/300 [00:41<00:59, 3.01it/s]
40%|████ | 121/300 [00:42<01:02, 2.89it/s]
41%|████ | 122/300 [00:42<00:57, 3.11it/s]
41%|████ | 123/300 [00:42<01:02, 2.84it/s]
41%|████▏ | 124/300 [00:43<00:58, 3.02it/s]
42%|████▏ | 125/300 [00:43<00:59, 2.92it/s]
42%|████▏ | 126/300 [00:43<01:00, 2.90it/s]
42%|████▏ | 127/300 [00:44<01:02, 2.78it/s]
43%|████▎ | 128/300 [00:44<00:57, 2.98it/s]
43%|████▎ | 129/300 [00:44<00:59, 2.86it/s]
43%|████▎ | 130/300 [00:45<00:54, 3.14it/s]
44%|████▎ | 131/300 [00:45<01:00, 2.78it/s]
44%|████▍ | 132/300 [00:45<00:55, 3.02it/s]
44%|████▍ | 133/300 [00:46<01:02, 2.67it/s]
45%|████▍ | 134/300 [00:46<00:56, 2.92it/s]
45%|████▌ | 135/300 [00:47<00:58, 2.80it/s]
45%|████▌ | 136/300 [00:47<00:53, 3.09it/s]
46%|████▌ | 137/300 [00:47<00:53, 3.02it/s]
46%|████▌ | 138/300 [00:47<00:50, 3.23it/s]
46%|████▋ | 139/300 [00:48<00:51, 3.16it/s]
47%|████▋ | 140/300 [00:48<00:46, 3.42it/s]
47%|████▋ | 141/300 [00:48<00:48, 3.26it/s]
47%|████▋ | 142/300 [00:49<00:50, 3.13it/s]
48%|████▊ | 143/300 [00:49<00:54, 2.88it/s]
48%|████▊ | 144/300 [00:49<00:48, 3.19it/s]
48%|████▊ | 145/300 [00:50<00:49, 3.14it/s]
49%|████▊ | 146/300 [00:50<00:46, 3.29it/s]
49%|████▉ | 147/300 [00:50<00:53, 2.86it/s]
49%|████▉ | 148/300 [00:51<00:50, 3.03it/s]
50%|████▉ | 149/300 [00:51<00:53, 2.80it/s]
50%|█████ | 150/300 [00:51<00:49, 3.00it/s]
50%|█████ | 151/300 [00:52<00:51, 2.88it/s]
51%|█████ | 152/300 [00:52<00:46, 3.19it/s]
51%|█████ | 153/300 [00:52<00:47, 3.08it/s]
51%|█████▏ | 154/300 [00:53<00:43, 3.36it/s]
52%|█████▏ | 155/300 [00:53<00:45, 3.17it/s]
52%|█████▏ | 156/300 [00:53<00:41, 3.49it/s]
52%|█████▏ | 157/300 [00:54<00:45, 3.13it/s]
53%|█████▎ | 158/300 [00:54<00:43, 3.27it/s]
53%|█████▎ | 159/300 [00:54<00:53, 2.66it/s]
53%|█████▎ | 160/300 [00:55<00:48, 2.91it/s]
54%|█████▎ | 161/300 [00:55<00:51, 2.71it/s]
54%|█████▍ | 162/300 [00:55<00:50, 2.75it/s]
54%|█████▍ | 163/300 [00:56<00:56, 2.41it/s]
55%|█████▍ | 164/300 [00:56<00:53, 2.53it/s]
55%|█████▌ | 165/300 [00:57<00:53, 2.54it/s]
55%|█████▌ | 166/300 [00:57<00:50, 2.65it/s]
56%|█████▌ | 167/300 [00:57<00:54, 2.43it/s]
56%|█████▌ | 168/300 [00:58<00:48, 2.74it/s]
56%|█████▋ | 169/300 [00:58<00:52, 2.48it/s]
57%|█████▋ | 170/300 [00:59<00:50, 2.58it/s]
57%|█████▋ | 171/300 [00:59<00:50, 2.54it/s]
57%|█████▋ | 172/300 [00:59<00:44, 2.88it/s]
58%|█████▊ | 173/300 [01:00<00:46, 2.72it/s]
58%|█████▊ | 174/300 [01:00<00:41, 3.05it/s]
58%|█████▊ | 175/300 [01:00<00:44, 2.79it/s]
59%|█████▊ | 176/300 [01:01<00:39, 3.11it/s]
59%|█████▉ | 177/300 [01:01<00:46, 2.65it/s]
59%|█████▉ | 178/300 [01:01<00:39, 3.08it/s]
60%|█████▉ | 179/300 [01:02<00:42, 2.88it/s]
60%|██████ | 180/300 [01:02<00:38, 3.16it/s]
60%|██████ | 181/300 [01:02<00:41, 2.85it/s]
61%|██████ | 182/300 [01:03<00:37, 3.15it/s]
61%|██████ | 183/300 [01:03<00:40, 2.89it/s]
61%|██████▏ | 184/300 [01:03<00:40, 2.88it/s]
62%|██████▏ | 185/300 [01:04<00:48, 2.38it/s]
62%|██████▏ | 186/300 [01:04<00:43, 2.63it/s]
62%|██████▏ | 187/300 [01:05<00:42, 2.68it/s]
63%|██████▎ | 188/300 [01:05<00:37, 2.99it/s]
63%|██████▎ | 189/300 [01:05<00:40, 2.76it/s]
63%|██████▎ | 190/300 [01:05<00:36, 3.02it/s]
64%|██████▎ | 191/300 [01:06<00:39, 2.77it/s]
64%|██████▍ | 192/300 [01:06<00:38, 2.84it/s]
64%|██████▍ | 193/300 [01:07<00:42, 2.51it/s]
65%|██████▍ | 194/300 [01:07<00:40, 2.61it/s]
65%|██████▌ | 195/300 [01:08<00:43, 2.42it/s]
65%|██████▌ | 196/300 [01:08<00:38, 2.67it/s]
66%|██████▌ | 197/300 [01:08<00:38, 2.67it/s]
66%|██████▌ | 198/300 [01:09<00:35, 2.91it/s]
66%|██████▋ | 199/300 [01:09<00:36, 2.81it/s]
67%|██████▋ | 200/300 [01:09<00:33, 2.99it/s]
67%|██████▋ | 201/300 [01:10<00:35, 2.79it/s]
67%|██████▋ | 202/300 [01:10<00:31, 3.08it/s]
68%|██████▊ | 203/300 [01:10<00:33, 2.94it/s]
68%|██████▊ | 204/300 [01:10<00:29, 3.23it/s]
68%|██████▊ | 205/300 [01:11<00:31, 3.01it/s]
69%|██████▊ | 206/300 [01:11<00:27, 3.40it/s]
69%|██████▉ | 207/300 [01:12<00:31, 2.93it/s]
69%|██████▉ | 208/300 [01:12<00:31, 2.94it/s]
70%|██████▉ | 209/300 [01:12<00:32, 2.80it/s]
70%|███████ | 210/300 [01:13<00:29, 3.01it/s]
70%|███████ | 211/300 [01:13<00:33, 2.66it/s]
71%|███████ | 212/300 [01:13<00:29, 2.98it/s]
71%|███████ | 213/300 [01:14<00:31, 2.74it/s]
71%|███████▏ | 214/300 [01:14<00:28, 3.00it/s]
72%|███████▏ | 215/300 [01:14<00:30, 2.77it/s]
72%|███████▏ | 216/300 [01:15<00:27, 3.07it/s]
72%|███████▏ | 217/300 [01:15<00:30, 2.75it/s]
73%|███████▎ | 218/300 [01:15<00:25, 3.17it/s]
73%|███████▎ | 219/300 [01:16<00:28, 2.87it/s]
73%|███████▎ | 220/300 [01:16<00:25, 3.15it/s]
74%|███████▎ | 221/300 [01:16<00:28, 2.79it/s]
74%|███████▍ | 222/300 [01:17<00:25, 3.09it/s]
74%|███████▍ | 223/300 [01:17<00:28, 2.75it/s]
75%|███████▍ | 224/300 [01:17<00:25, 2.95it/s]
75%|███████▌ | 225/300 [01:18<00:26, 2.82it/s]
75%|███████▌ | 226/300 [01:18<00:24, 3.05it/s]
76%|███████▌ | 227/300 [01:18<00:26, 2.79it/s]
76%|███████▌ | 228/300 [01:19<00:23, 3.12it/s]
76%|███████▋ | 229/300 [01:19<00:25, 2.76it/s]
77%|███████▋ | 230/300 [01:19<00:24, 2.86it/s]
77%|███████▋ | 231/300 [01:20<00:26, 2.59it/s]
77%|███████▋ | 232/300 [01:20<00:25, 2.69it/s]
78%|███████▊ | 233/300 [01:21<00:27, 2.42it/s]
78%|███████▊ | 234/300 [01:21<00:23, 2.78it/s]
78%|███████▊ | 235/300 [01:21<00:25, 2.55it/s]
79%|███████▊ | 236/300 [01:22<00:24, 2.64it/s]
79%|███████▉ | 237/300 [01:22<00:24, 2.56it/s]
79%|███████▉ | 238/300 [01:22<00:21, 2.88it/s]
80%|███████▉ | 239/300 [01:23<00:21, 2.87it/s]
80%|████████ | 240/300 [01:23<00:18, 3.18it/s]
80%|████████ | 241/300 [01:23<00:19, 3.07it/s]
81%|████████ | 242/300 [01:24<00:18, 3.10it/s]
81%|████████ | 243/300 [01:24<00:20, 2.74it/s]
81%|████████▏ | 244/300 [01:25<00:19, 2.84it/s]
82%|████████▏ | 245/300 [01:25<00:20, 2.67it/s]
82%|████████▏ | 246/300 [01:25<00:19, 2.76it/s]
82%|████████▏ | 247/300 [01:26<00:20, 2.59it/s]
83%|████████▎ | 248/300 [01:26<00:17, 2.92it/s]
83%|████████▎ | 249/300 [01:26<00:18, 2.78it/s]
83%|████████▎ | 250/300 [01:27<00:17, 2.90it/s]
84%|████████▎ | 251/300 [01:27<00:17, 2.80it/s]
84%|████████▍ | 252/300 [01:27<00:16, 2.99it/s]
84%|████████▍ | 253/300 [01:28<00:16, 2.83it/s]
85%|████████▍ | 254/300 [01:28<00:14, 3.20it/s]
85%|████████▌ | 255/300 [01:28<00:15, 2.86it/s]
85%|████████▌ | 256/300 [01:29<00:13, 3.18it/s]
86%|████████▌ | 257/300 [01:29<00:14, 2.97it/s]
86%|████████▌ | 258/300 [01:29<00:12, 3.24it/s]
86%|████████▋ | 259/300 [01:30<00:13, 2.99it/s]
87%|████████▋ | 260/300 [01:30<00:12, 3.17it/s]
87%|████████▋ | 261/300 [01:30<00:13, 3.00it/s]
87%|████████▋ | 262/300 [01:31<00:11, 3.29it/s]
88%|████████▊ | 263/300 [01:31<00:12, 3.02it/s]
88%|████████▊ | 264/300 [01:31<00:12, 2.99it/s]
88%|████████▊ | 265/300 [01:32<00:13, 2.58it/s]
89%|████████▊ | 266/300 [01:32<00:12, 2.68it/s]
89%|████████▉ | 267/300 [01:33<00:13, 2.42it/s]
89%|████████▉ | 268/300 [01:33<00:11, 2.78it/s]
90%|████████▉ | 269/300 [01:33<00:12, 2.46it/s]
90%|█████████ | 270/300 [01:34<00:11, 2.63it/s]
90%|█████████ | 271/300 [01:34<00:12, 2.37it/s]
91%|█████████ | 272/300 [01:35<00:10, 2.64it/s]
91%|█████████ | 273/300 [01:35<00:11, 2.29it/s]
91%|█████████▏| 274/300 [01:35<00:09, 2.63it/s]
92%|█████████▏| 275/300 [01:36<00:10, 2.40it/s]
92%|█████████▏| 276/300 [01:36<00:09, 2.59it/s]
92%|█████████▏| 277/300 [01:37<00:09, 2.48it/s]
93%|█████████▎| 278/300 [01:37<00:07, 2.81it/s]
93%|█████████▎| 279/300 [01:37<00:07, 2.70it/s]
93%|█████████▎| 280/300 [01:38<00:07, 2.82it/s]
94%|█████████▎| 281/300 [01:38<00:07, 2.67it/s]
94%|█████████▍| 282/300 [01:38<00:06, 2.99it/s]
94%|█████████▍| 283/300 [01:39<00:06, 2.78it/s]
95%|█████████▍| 284/300 [01:39<00:05, 3.06it/s]
95%|█████████▌| 285/300 [01:39<00:05, 2.65it/s]
95%|█████████▌| 286/300 [01:40<00:05, 2.71it/s]
96%|█████████▌| 287/300 [01:40<00:04, 2.64it/s]
96%|█████████▌| 288/300 [01:40<00:03, 3.06it/s]
96%|█████████▋| 289/300 [01:41<00:03, 2.80it/s]
97%|█████████▋| 290/300 [01:41<00:03, 2.82it/s]
97%|█████████▋| 291/300 [01:42<00:03, 2.56it/s]
97%|█████████▋| 292/300 [01:42<00:02, 2.89it/s]
98%|█████████▊| 293/300 [01:42<00:02, 2.75it/s]
98%|█████████▊| 294/300 [01:42<00:02, 2.99it/s]
98%|█████████▊| 295/300 [01:43<00:01, 2.94it/s]
99%|█████████▊| 296/300 [01:43<00:01, 3.14it/s]
99%|█████████▉| 297/300 [01:44<00:01, 2.90it/s]
99%|█████████▉| 298/300 [01:44<00:00, 3.24it/s]
100%|█████████▉| 299/300 [01:44<00:00, 2.85it/s]
100%|██████████| 300/300 [01:45<00:00, 2.89it/s]
 54%|█████▍ | 1500/2757 [3:26:03<2:49:38, 8.10s/it]
100%|██████████| 300/300 [01:45<00:00, 2.89it/s]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-1500/tokenizer_config.json.
54%|█████▍ | 1501/2757 [3:26:15<14:22:18, 41.19s/it] 54%|█████▍ | 1502/2757 [3:26:23<10:53:54, 31.26s/it] 55%|█████▍ | 1503/2757 [3:26:32<8:28:57, 24.35s/it] 55%|█████▍ | 1504/2757 [3:26:40<6:45:34, 19.42s/it] 55%|█████▍ | 1505/2757 [3:26:48<5:34:40, 16.04s/it] 55%|█████▍ | 1506/2757 [3:26:56<4:45:26, 13.69s/it] 55%|█████▍ | 1507/2757 [3:27:04<4:08:18, 11.92s/it] 55%|█████▍ | 1508/2757 [3:27:12<3:42:04, 10.67s/it] 55%|█████▍ | 1509/2757 [3:27:20<3:29:03, 10.05s/it] 55%|█████▍ | 1510/2757 [3:27:28<3:15:28, 9.41s/it] 55%|█████▍ | 1510/2757 [3:27:28<3:15:28, 9.41s/it] 55%|█████▍ | 1511/2757 [3:27:36<3:06:48, 9.00s/it] 55%|█████▍ | 1512/2757 [3:27:44<3:01:00, 8.72s/it] 55%|█████▍ | 1513/2757 [3:27:52<2:56:43, 8.52s/it] 55%|█████▍ | 1514/2757 [3:28:00<2:53:37, 8.38s/it] 55%|█████▍ | 1515/2757 [3:28:09<2:52:54, 8.35s/it] 55%|█████▍ | 1516/2757 [3:28:16<2:47:34, 8.10s/it] 55%|█████▌ | 1517/2757 [3:28:24<2:45:43, 8.02s/it] 55%|█████▌ | 1518/2757 [3:28:32<2:45:29, 8.01s/it] 55%|█████▌ | 1519/2757 [3:28:40<2:46:13, 8.06s/it] 55%|█████▌ | 1520/2757 [3:28:48<2:45:21, 8.02s/it] 55%|█████▌ | 1520/2757 [3:28:48<2:45:21, 8.02s/it] 55%|█████▌ | 1521/2757 [3:28:56<2:44:21, 7.98s/it] 55%|█████▌ | 1522/2757 [3:29:04<2:42:17, 7.88s/it] 55%|█████▌ | 1523/2757 [3:29:11<2:42:34, 7.90s/it] 55%|█████▌ | 1524/2757 [3:29:20<2:43:54, 7.98s/it] 55%|█████▌ | 1525/2757 [3:29:27<2:42:59, 7.94s/it] 55%|█████▌ | 1526/2757 [3:29:35<2:41:58, 7.89s/it] 55%|█████▌ | 1527/2757 [3:29:43<2:43:24, 7.97s/it] 55%|█████▌ | 1528/2757 [3:29:51<2:42:23, 7.93s/it] 55%|█████▌ | 1529/2757 [3:29:59<2:42:14, 7.93s/it] 55%|█████▌ | 1530/2757 [3:30:07<2:42:41, 7.96s/it] 55%|█████▌ | 1530/2757 [3:30:07<2:42:41, 7.96s/it] 56%|█████▌ | 1531/2757 [3:30:15<2:43:13, 7.99s/it] 56%|█████▌ | 1532/2757 [3:30:23<2:43:22, 8.00s/it] 56%|█████▌ | 1533/2757 [3:30:31<2:44:05, 8.04s/it] 56%|█████▌ | 1534/2757 [3:30:40<2:44:21, 8.06s/it] 56%|█████▌ | 1535/2757 [3:30:47<2:40:01, 7.86s/it] 56%|█████▌ | 1536/2757 [3:30:55<2:40:58, 7.91s/it] 56%|█████▌ | 1537/2757 [3:31:02<2:38:28, 7.79s/it] 56%|█████▌ | 1538/2757 [3:31:11<2:40:21, 7.89s/it] 56%|█████▌ | 1539/2757 [3:31:19<2:41:45, 7.97s/it] 56%|█████▌ | 1540/2757 [3:31:27<2:41:57, 7.98s/it] 56%|█████▌ | 1540/2757 [3:31:27<2:41:57, 7.98s/it] 56%|█████▌ | 1541/2757 [3:31:34<2:40:00, 7.90s/it] 56%|█████▌ | 1542/2757 [3:31:43<2:41:37, 7.98s/it] 56%|█████▌ | 1543/2757 [3:31:51<2:44:50, 8.15s/it] 56%|█████▌ | 1544/2757 [3:31:59<2:45:32, 8.19s/it] 56%|█████▌ | 1545/2757 [3:32:07<2:43:52, 8.11s/it] 56%|█████▌ | 1546/2757 [3:32:15<2:40:52, 7.97s/it] 56%|█████▌ | 1547/2757 [3:32:23<2:40:24, 7.95s/it] 56%|█████▌ | 1548/2757 [3:32:31<2:40:04, 7.94s/it] 56%|█████▌ | 1549/2757 [3:32:38<2:34:09, 7.66s/it] 56%|█████▌ | 1550/2757 [3:32:46<2:39:44, 7.94s/it] 56%|█████▌ | 1550/2757 [3:32:46<2:39:44, 7.94s/it] 56%|█████▋ | 1551/2757 [3:32:55<2:43:11, 8.12s/it
{'loss': '0.3725', 'grad_norm': '0.6329', 'learning_rate': '9.334e-05', 'rewards/chosen': '-1.795', 'rewards/rejected': '-3.295', 'rewards/accuracies': '0.775', 'rewards/margins': '1.5', 'logps/chosen': '-97.45', 'logps/rejected': '-118.5', 'logits/chosen': '-4.012', 'logits/rejected': '-3.945', 'epoch': '1.643'}
{'loss': '0.3885', 'grad_norm': '0.8267', 'learning_rate': '9.26e-05', 'rewards/chosen': '-2.057', 'rewards/rejected': '-3.51', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.453', 'logps/chosen': '-99.85', 'logps/rejected': '-118.9', 'logits/chosen': '-4.003', 'logits/rejected': '-3.953', 'epoch': '1.654'}
{'loss': '0.3619', 'grad_norm': '0.6179', 'learning_rate': '9.185e-05', 'rewards/chosen': '-1.993', 'rewards/rejected': '-3.517', 'rewards/accuracies': '0.7969', 'rewards/margins': '1.524', 'logps/chosen': '-100.3', 'logps/rejected': '-121', 'logits/chosen': '-3.809', 'logits/rejected': '-3.735', 'epoch': '1.665'}
{'loss': '0.3576', 'grad_norm': '0.6443', 'learning_rate': '9.11e-05', 'rewards/chosen': '-2.11', 'rewards/rejected': '-3.644', 'rewards/accuracies': '0.7937', 'rewards/margins': '1.534', 'logps/chosen': '-103.8', 'logps/rejected': '-123.3', 'logits/chosen': '-3.819', 'logits/rejected': '-3.749', 'epoch': '1.676'}
{'loss': '0.374', 'grad_norm': '0.7384', 'learning_rate': '9.035e-05', 'rewards/chosen': '-2.21', 'rewards/rejected': '-3.729', 'rewards/accuracies': '0.768', 'rewards/margins': '1.519', 'logps/chosen': '-103.8', 'logps/rejected': '-122.9', 'logits/chosen': '-3.885', 'logits/rejected': '-3.787', 'epoch': '1.687'}
{'loss': '0.3584', 'grad_norm': '0.5654', 'learning_rate': '8.96e-05', 'rewards/chosen': '-2.004', 'rewards/rejected': '-3.579', 'rewards/accuracies': '0.7852', 'rewards/margins': '1.574', 'logps/chosen': '-100.5', 'logps/rejected': '-120.8', 'logits/chosen': '-3.935', 'logits/rejected': '-3.844', 'epoch': '1.698'}
{'loss': '0.3536', 'grad_norm': '0.8251', 'learning_rate': '8.886e-05', 'rewards/chosen': '-2.175', 'rewards/rejected': '-3.769', 'rewards/accuracies': '0.8062', 'rewards/margins': '1.595', 'logps/chosen': '-102.4', 'logps/rejected': '-124.2', 'logits/chosen': '-4.114', 'logits/rejected': '-4.016', 'epoch': '1.709'}
{'loss': '0.3635', 'grad_norm': '0.7069', 'learning_rate': '8.811e-05', 'rewards/chosen': '-2.133', 'rewards/rejected': '-3.733', 'rewards/accuracies': '0.7836', 'rewards/margins': '1.6', 'logps/chosen': '-100.6', 'logps/rejected': '-121.4', 'logits/chosen': '-4.129', 'logits/rejected': '-4.013', 'epoch': '1.719'}
{'loss': '0.3753', 'grad_norm': '0.8969', 'learning_rate': '8.736e-05', 'rewards/chosen': '-2.151', 'rewards/rejected': '-3.742', 'rewards/accuracies': '0.7781', 'rewards/margins': '1.591', 'logps/chosen': '-101.7', 'logps/rejected': '-124.9', 'logits/chosen': '-4.027', 'logits/rejected': '-4.019', 'epoch': '1.73'}
{'loss': '0.3766', 'grad_norm': '0.7352', 'learning_rate': '8.661e-05', 'rewards/chosen': '-2.379', 'rewards/rejected': '-3.885', 'rewards/accuracies': '0.7742', 'rewards/margins': '1.506', 'logps/chosen': '-103.6', 'logps/rejected': '-124.1', 'logits/chosen': '-3.981', 'logits/rejected': '-3.917', 'epoch': '1.741'}
{'loss': '0.3653', 'grad_norm': '0.5647', 'learning_rate': '8.586e-05', 'rewards/chosen': '-2.353', 'rewards/rejected': '-3.948', 'rewards/accuracies': '0.7844', 'rewards/margins': '1.595', 'logps/chosen': '-103.6', 'logps/rejected': '-125.9', 'logits/chosen': '-4.083', 'logits/rejected': '-4.028', 'epoch': '1.752'}
{'loss': '0.3691', 'grad_norm': '0.7188', 'learning_rate': '8.512e-05', 'rewards/chosen': '-2.336', 'rewards/rejected': '-3.85', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.515', 'logps/chosen': '-104.9', 'logps/rejected': '-125', 'logits/chosen': '-3.872', 'logits/rejected': '-3.833', 'epoch': '1.763'}
{'loss': '0.3572', 'grad_norm': '0.6679', 'learning_rate': '8.437e-05', 'rewards/chosen': '-2.258', 'rewards/rejected': '-3.829', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.571', 'logps/chosen': '-103.7', 'logps/rejected': '-125', 'logits/chosen': '-4.03', 'logits/rejected': '-3.973', 'epoch': '1.774'}
{'loss': '0.3459', 'grad_norm': '0.6013', 'learning_rate': '8.362e-05', 'rewards/chosen': '-2.236', 'rewards/rejected': '-3.897', 'rewards/accuracies': '0.8023', 'rewards/margins': '1.661', 'logps/chosen': '-100.5', 'logps/rejected': '-123.6', 'logits/chosen': '-3.995', 'logits/rejected': '-3.911', 'epoch': '1.785'}
{'loss': '0.3556', 'grad_norm': '0.6696', 'learning_rate': '8.287e-05', 'rewards/chosen': '-2.186', 'rewards/rejected': '-3.82', 'rewards/accuracies': '0.793', 'rewards/margins': '1.634', 'logps/chosen': '-100.8', 'logps/rejected': '-123.3', 'logits/chosen': '-3.975', 'logits/rejected': '-3.9', 'epoch': '1.796'}
{'loss': '0.3668', 'grad_norm': '0.5696', 'learning_rate': '8.212e-05', 'rewards/chosen': '-2.07', 'rewards/rejected': '-3.639', 'rewards/accuracies': '0.7891', 'rewards/margins': '1.569', 'logps/chosen': '-99.68', 'logps/rejected': '-120.6', 'logits/chosen': '-4.077', 'logits/rejected': '-3.98', 'epoch': '1.807'}
{'loss': '0.377', 'grad_norm': '0.6719', 'learning_rate': '8.138e-05', 'rewards/chosen': '-2.11', 'rewards/rejected': '-3.654', 'rewards/accuracies': '0.7898', 'rewards/margins': '1.544', 'logps/chosen': '-103.2', 'logps/rejected': '-124.1', 'logits/chosen': '-4.046', 'logits/rejected': '-3.934', 'epoch': '1.817'}
{'loss': '0.3561', 'grad_norm': '0.6737', 'learning_rate': '8.063e-05', 'rewards/chosen': '-2.13', 'rewards/rejected': '-3.775', 'rewards/accuracies': '0.7891', 'rewards/margins': '1.645', 'logps/chosen': '-99.72', 'logps/rejected': '-121.9', 'logits/chosen': '-4.077', 'logits/rejected': '-4.007', 'epoch': '1.828'}
{'loss': '0.3536', 'grad_norm': '0.7005', 'learning_rate': '7.988e-05', 'rewards/chosen': '-2.293', 'rewards/rejected': '-3.948', 'rewards/accuracies': '0.8031', 'rewards/margins': '1.655', 'logps/chosen': '-105.8', 'logps/rejected': '-128', 'logits/chosen': '-3.916', 'logits/rejected': '-3.911', 'epoch': '1.839'}
{'loss': '0.3671', 'grad_norm': '0.5833', 'learning_rate': '7.913e-05', 'rewards/chosen': '-2.238', 'rewards/rejected': '-3.842', 'rewards/accuracies': '0.7609', 'rewards/margins': '1.604', 'logps/chosen': '-98.68', 'logps/rejected': '-118.4', 'logits/chosen': '-4.208', 'logits/rejected': '-4.131', 'epoch': '1.85'}
{'loss': '0.3505', 'grad_norm': '0.5559', 'learning_rate': '7.838e-05', 'rewards/chosen': '-2.235', 'rewards/rejected': '-3.849', 'rewards/accuracies': '0.8094', 'rewards/margins': '1.614', 'logps/chosen': '-99.44', 'logps/rejected': '-122.8', 'logits/chosen': '-4.181', 'logits/rejected': '-4.108', 'epoch': '1.861'}
{'loss': '0.3608', 'grad_norm': '0.5495', 'learning_rate': '7.764e-05', 'rewards/chosen': '-2.364', 'rewards/rejected': '-3.972', 'rewards/accuracies': '0.7828', 'rewards/margins': '1.608', 'logps/chosen': '-104.2', 'logps/rejected': '-125.3', 'logits/chosen': '-4.158', 'logits/rejected': '-4.096', 'epoch': '1.872'}
{'loss': '0.355', 'grad_norm': '0.7562', 'learning_rate': '7.689e-05', 'rewards/chosen': '-2.264', 'rewards/rejected': '-3.919', 'rewards/accuracies': '0.7906', 'rewards/margins': '1.655', 'logps/chosen': '-102.7', 'logps/rejected': '-123.5', 'logits/chosen': '-4.054', 'logits/rejected': '-3.934', 'epoch': '1.883'}
{'loss': '0.3495', 'grad_norm': '0.6611', 'learning_rate': '7.614e-05', 'rewards/chosen': '-2.171', 'rewards/rejected': '-3.777', 'rewards/accuracies': '0.7953', 'rewards/margins': '1.607', 'logps/chosen': '-104.4', 'logps/rejected': '-125.9', 'logits/chosen': '-3.961', 'logits/rejected': '-3.83', 'epoch': '1.894'}
{'loss': '0.3612', 'grad_norm': '0.6473', 'learning_rate': '7.539e-05', 'rewards/chosen': '-2.068', 'rewards/rejected': '-3.683', 'rewards/accuracies': '0.7836', 'rewards/margins': '1.615', 'logps/chosen': '-106', 'logps/rejected': '-125.6', 'logits/chosen': '-3.913', 'logits/rejected': '-3.869', 'epoch': '1.904'} 63%|██████▎ | 1750/2757 [3:59:23<2:20:57, 8.40s/it] 64%|██████▎ | 1751/2757 [3:59:37<2:48:24, 10.04s/it] 64%|██████▎ | 1752/2757 [3:59:45<2:37:55, 9.43s/it] 64%|██████▎ | 1753/2757 [3:59:53<2:33:59, 9.20s/it] 64%|██████▎ | 1754/2757 [4:00:02<2:28:18, 8.87s/it] 64%|██████▎ | 1755/2757 [4:00:09<2:21:56, 8.50s/it] 64%|██████▎ | 1756/2757 [4:00:17<2:18:21, 8.29s/it] 64%|██████▎ | 1757/2757 [4:00:26<2:20:33, 8.43s/it] 64%|██████▍ | 1758/2757 [4:00:34<2:17:39, 8.27s/it] 64%|██████▍ | 1759/2757 [4:00:42<2:17:16, 8.25s/it] 64%|██████▍ | 1760/2757 [4:00:50<2:17:25, 8.27s/it] 64%|██████▍ | 1760/2757 [4:00:50<2:17:25, 8.27s/it] 64%|██████▍ | 1761/2757 [4:00:58<2:16:03, 8.20s/it] 64%|██████▍ | 1762/2757 [4:01:06<2:13:52, 8.07s/it] 64%|██████▍ | 1763/2757 [4:01:14<2:12:44, 8.01s/it] 64%|██████▍ | 1764/2757 [4:01:21<2:10:42, 7.90s/it] 64%|██████▍ | 1765/2757 [4:01:29<2:09:45, 7.85s/it] 64%|██████▍ | 1766/2757 [4:01:37<2:10:22, 7.89s/it] 64%|██████▍ | 1767/2757 [4:01:45<2:11:38, 7.98s/it] 64%|██████▍ | 1768/2757 [4:01:54<2:12:42, 8.05s/it] 64%|██████▍ | 1769/2757 [4:02:01<2:11:29, 7.99s/it] 64%|██████▍ | 1770/2757 [4:02:10<2:11:45, 8.01s/it] 64%|██████▍ | 1770/2757 [4:02:10<2:11:45, 8.01s/it] 64%|██████▍ | 1771/2757 [4:02:17<2:11:17, 7.99s/it] 64%|██████▍ | 1772/2757 [4:02:25<2:11:17, 8.00s/it] 64%|██████▍ | 1773/2757 [4:02:33<2:10:10, 7.94s/it] 64%|██████▍ | 1774/2757 [4:02:41<2:09:21, 7.90s/it] 64%|██████▍ | 1775/2757 [4:02:49<2:10:24, 7.97s/it] 64%|██████▍ | 1776/2757 [4:02:57<2:08:37, 7.87s/it] 64%|██████▍ | 1777/2757 [4:03:05<2:08:46, 7.88s/it] 64%|██████▍ | 1778/2757 [4:03:13<2:08:29, 7.88s/it] 65%|██████▍ | 1779/2757 [4:03:21<2:08:30, 7.88s/it] 65%|██████▍ | 1780/2757 [4:03:29<2:08:59, 7.92s/it] 65%|██████▍ | 1780/2757 [4:03:29<2:08:59, 7.92s/it] 65%|██████▍ | 1781/2757 [4:03:37<2:09:10, 7.94s/it] 65%|██████▍ | 1782/2757 [4:03:44<2:07:23, 7.84s/it] 65%|██████▍ | 1783/2757 [4:03:52<2:07:50, 7.88s/it] 65%|██████▍ | 1784/2757 [4:04:00<2:07:14, 7.85s/it] 65%|██████▍ | 1785/2757 [4:04:08<2:08:56, 7.96s/it] 65%|██████▍ | 1786/2757 [4:04:16<2:11:05, 8.10s/it] 65%|██████▍ | 1787/2757 [4:04:25<2:10:45, 8.09s/it] 65%|██████▍ | 1788/2757 [4:04:33<2:10:11, 8.06s/it] 65%|██████▍ | 1789/2757 [4:04:40<2:08:11, 7.95s/it] 65%|██████▍ | 1790/2757 [4:04:48<2:05:59, 7.82s/it] 65%|██████▍ | 1790/2757 [4:04:48<2:05:59, 7.82s/it] 65%|██████▍ | 1791/2757 [4:04:56<2:07:23, 7.91s/it] 65%|██████▍ | 1792/2757 [4:05:04<2:09:32, 8.05s/it] 65%|██████▌ | 1793/2757 [4:05:13<2:11:49, 8.21s/it] 65%|██████▌ | 1794/2757 [4:05:21<2:11:10, 8.17s/it] 65%|██████▌ | 1795/2757 [4:05:29<2:09:08, 8.05s/it] 65%|██████▌ | 1796/2757 [4:05:36<2
{'loss': '0.3603', 'grad_norm': '0.7833', 'learning_rate': '7.464e-05', 'rewards/chosen': '-1.853', 'rewards/rejected': '-3.423', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.57', 'logps/chosen': '-97.6', 'logps/rejected': '-119.1', 'logits/chosen': '-3.939', 'logits/rejected': '-3.906', 'epoch': '1.915'}
{'loss': '0.3666', 'grad_norm': '0.72', 'learning_rate': '7.39e-05', 'rewards/chosen': '-2.322', 'rewards/rejected': '-3.87', 'rewards/accuracies': '0.7812', 'rewards/margins': '1.548', 'logps/chosen': '-104.4', 'logps/rejected': '-122.6', 'logits/chosen': '-3.854', 'logits/rejected': '-3.82', 'epoch': '1.926'}
{'loss': '0.3637', 'grad_norm': '0.6587', 'learning_rate': '7.315e-05', 'rewards/chosen': '-2.446', 'rewards/rejected': '-4.051', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.605', 'logps/chosen': '-106', 'logps/rejected': '-127.8', 'logits/chosen': '-3.952', 'logits/rejected': '-4.003', 'epoch': '1.937'}
{'loss': '0.3613', 'grad_norm': '0.5735', 'learning_rate': '7.24e-05', 'rewards/chosen': '-2.576', 'rewards/rejected': '-4.142', 'rewards/accuracies': '0.7812', 'rewards/margins': '1.566', 'logps/chosen': '-100.7', 'logps/rejected': '-121.9', 'logits/chosen': '-4', 'logits/rejected': '-3.923', 'epoch': '1.948'}
{'loss': '0.3469', 'grad_norm': '0.6537', 'learning_rate': '7.165e-05', 'rewards/chosen': '-2.413', 'rewards/rejected': '-4.107', 'rewards/accuracies': '0.8195', 'rewards/margins': '1.694', 'logps/chosen': '-103.4', 'logps/rejected': '-125.7', 'logits/chosen': '-3.983', 'logits/rejected': '-3.873', 'epoch': '1.959'}
{'loss': '0.3626', 'grad_norm': '0.6989', 'learning_rate': '7.091e-05', 'rewards/chosen': '-2.356', 'rewards/rejected': '-3.993', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.637', 'logps/chosen': '-106.2', 'logps/rejected': '-126.7', 'logits/chosen': '-3.916', 'logits/rejected': '-3.87', 'epoch': '1.97'}
{'loss': '0.3597', 'grad_norm': '0.7157', 'learning_rate': '7.016e-05', 'rewards/chosen': '-2.092', 'rewards/rejected': '-3.722', 'rewards/accuracies': '0.7781', 'rewards/margins': '1.63', 'logps/chosen': '-100.3', 'logps/rejected': '-121.5', 'logits/chosen': '-3.931', 'logits/rejected': '-3.873', 'epoch': '1.981'}
{'loss': '0.3599', 'grad_norm': '0.6526', 'learning_rate': '6.941e-05', 'rewards/chosen': '-1.991', 'rewards/rejected': '-3.627', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.635', 'logps/chosen': '-100', 'logps/rejected': '-122', 'logits/chosen': '-3.948', 'logits/rejected': '-3.869', 'epoch': '1.992'}
67%|██████▋ | 1839/2757 [4:11:19<2:00:17, 7.86s/it] 67%|██████▋ | 1840/2757 [4:11:27<2:00:20, 7.87s/it] 67%|██████▋ | 1840/2757 [4:11:27<2:00:20, 7.87s/it] 67%|██████▋ | 1841/2757 [4:11:35<2:00:20, 7.88s/it] 67%|██████▋ | 1842/2757 [4:11:43<2:02:53, 8.06s/it] 67%|██████▋ | 1843/2757 [4:11:51<2:03:00, 8.07s/it] 67%|██████▋ | 1844/2757 [4:11:59<2:02:30, 8.05s/it] 67%|██████▋ | 1845/2757 [4:12:08<2:03:20, 8.11s/it] 67%|██████▋ | 1846/2757 [4:12:16<2:04:00, 8.17s/it] 67%|██████▋ | 1847/2757 [4:12:24<2:03:45, 8.16s/it] 67%|██████▋ | 1848/2757 [4:12:32<2:03:20, 8.14s/it] 67%|██████▋ | 1849/2757 [4:12:40<2:02:34, 8.10s/it] 67%|██████▋ | 1850/2757 [4:12:49<2:03:26, 8.17s/it] 67%|██████▋ | 1850/2757 [4:12:49<2:03:26, 8.17s/it] 67%|██████▋ | 1851/2757 [4:12:57<2:02:05, 8.09s/it] 67%|██████▋ | 1852/2757 [4:13:05<2:02:39, 8.13s/it] 67%|██████▋ | 1853/2757 [4:13:13<2:01:21, 8.05s/it] 67%|██████▋ | 1854/2757 [4:13:20<1:59:41, 7.95s/it] 67%|██████▋ | 1855/2757 [4:13:28<1:59:20, 7.94s/it] 67%|██████▋ | 1856/2757 [4:13:36<1:58:53, 7.92s/it] 67%|██████▋ | 1857/2757 [4:13:44<1:59:03, 7.94s/it] 67%|██████▋ | 1858/2757 [4:13:53<2:01:11, 8.09s/it] 67%|██████▋ | 1859/2757 [4:14:01<2:02:45, 8.20s/it] 67%|██████▋ | 1860/2757 [4:14:09<2:01:36, 8.13s/it] 67%|██████▋ | 1860/2757 [4:14:09<2:01:36, 8.13s/it] 68%|██████▊ | 1861/2757 [4:14:17<1:59:31, 8.00s/it] 68%|██████▊ | 1862/2757 [4:14:25<2:01:12, 8.13s/it] 68%|██████▊ | 1863/2757 [4:14:33<2:00:43, 8.10s/it] 68%|██████▊ | 1864/2757 [4:14:41<1:59:42, 8.04s/it] 68%|██████▊ | 1865/2757 [4:14:49<1:57:59, 7.94s/it] 68%|██████▊ | 1866/2757 [4:14:57<1:57:20, 7.90s/it] 68%|██████▊ | 1867/2757 [4:15:04<1:55:58, 7.82s/it] 68%|██████▊ | 1868/2757 [4:15:12<1:55:11, 7.77s/it] 68%|██████▊ | 1869/2757 [4:15:20<1:55:19, 7.79s/it] 68%|██████▊ | 1870/2757 [4:15:28<1:57:58, 7.98s/it] 68%|██████▊ | 1870/2757 [4:15:28<1:57:58, 7.98s/it] 68%|██████▊ | 1871/2757 [4:15:36<1:58:00, 7.99s/it] 68%|██████▊ | 1872/2757 [4:15:44<1:58:45, 8.05s/it] 68%|██████▊ | 1873/2757 [4:15:52<1:57:26, 7.97s/it] 68%|██████▊ | 1874/2757 [4:16:00<1:56:33, 7.92s/it] 68%|██████▊ | 1875/2757 [4:16:08<1:57:30, 7.99s/it] 68%|██████▊ | 1876/2757 [4:16:16<1:56:10, 7.91s/it] 68%|██████▊ | 1877/2757 [4:16:24<1:55:26, 7.87s/it] 68%|██████▊ | 1878/2757 [4:16:32<1:57:22, 8.01s/it] 68%|██████▊ | 1879/2757 [4:16:40<1:57:48, 8.05s/it] 68%|██████▊ | 1880/2757 [4:16:48<1:58:01, 8.07s/it] 68%|██████▊ | 1880/2757 [4:16:48<1:58:01, 8.07s/it] 68%|██████▊ | 1881/2757 [4:16:56<1:57:07, 8.02s/it] 68%|██████▊ | 1882/2757 [4:17:05<1:59:05, 8.17s/it] 68%|██████▊ | 1883/2757 [4:17:13<1:58:48, 8.16s/it] 68%|██████▊ | 1884/2757 [4:17:21<1:57:17, 8.06s/it] 68%|██████▊ | 1885/2757 [4:17:28<1:56:14, 8.00s/it] 68%|██████▊ | 1886/2757 [4:17:36<1:54:23, 7.88s/it] 68%|██████▊ | 1887/2757 [4:17:44<1:52:45, 7.78s/it] 68%|██████▊
{'loss': '0.2799', 'grad_norm': '0.5088', 'learning_rate': '6.791e-05', 'rewards/chosen': '-2.049', 'rewards/rejected': '-3.981', 'rewards/accuracies': '0.8391', 'rewards/margins': '1.933', 'logps/chosen': '-101.1', 'logps/rejected': '-126.3', 'logits/chosen': '-3.941', 'logits/rejected': '-3.925', 'epoch': '2.013'}
{'loss': '0.2695', 'grad_norm': '0.5052', 'learning_rate': '6.717e-05', 'rewards/chosen': '-2.205', 'rewards/rejected': '-4.158', 'rewards/accuracies': '0.8586', 'rewards/margins': '1.953', 'logps/chosen': '-104.2', 'logps/rejected': '-128.5', 'logits/chosen': '-4.173', 'logits/rejected': '-4.081', 'epoch': '2.024'}
{'loss': '0.271', 'grad_norm': '0.4783', 'learning_rate': '6.642e-05', 'rewards/chosen': '-2.17', 'rewards/rejected': '-4.234', 'rewards/accuracies': '0.8555', 'rewards/margins': '2.064', 'logps/chosen': '-104.7', 'logps/rejected': '-129.3', 'logits/chosen': '-4.187', 'logits/rejected': '-4.043', 'epoch': '2.035'}
{'loss': '0.2763', 'grad_norm': '0.543', 'learning_rate': '6.567e-05', 'rewards/chosen': '-2.28', 'rewards/rejected': '-4.33', 'rewards/accuracies': '0.8445', 'rewards/margins': '2.05', 'logps/chosen': '-102.9', 'logps/rejected': '-129.7', 'logits/chosen': '-4.226', 'logits/rejected': '-4.173', 'epoch': '2.046'}
{'loss': '0.2582', 'grad_norm': '0.5694', 'learning_rate': '6.492e-05', 'rewards/chosen': '-2.344', 'rewards/rejected': '-4.545', 'rewards/accuracies': '0.8687', 'rewards/margins': '2.201', 'logps/chosen': '-103.5', 'logps/rejected': '-131.2', 'logits/chosen': '-4.404', 'logits/rejected': '-4.277', 'epoch': '2.057'}
{'loss': '0.2655', 'grad_norm': '0.4885', 'learning_rate': '6.417e-05', 'rewards/chosen': '-2.716', 'rewards/rejected': '-4.911', 'rewards/accuracies': '0.8445', 'rewards/margins': '2.195', 'logps/chosen': '-108.2', 'logps/rejected': '-136', 'logits/chosen': '-4.375', 'logits/rejected': '-4.236', 'epoch': '2.067'}
{'loss': '0.2693', 'grad_norm': '0.5405', 'learning_rate': '6.343e-05', 'rewards/chosen': '-2.618', 'rewards/rejected': '-4.733', 'rewards/accuracies': '0.8461', 'rewards/margins': '2.115', 'logps/chosen': '-108.1', 'logps/rejected': '-135.2', 'logits/chosen': '-4.263', 'logits/rejected': '-4.204', 'epoch': '2.078'}
{'loss': '0.2746', 'grad_norm': '0.5154', 'learning_rate': '6.268e-05', 'rewards/chosen': '-2.565', 'rewards/rejected': '-4.722', 'rewards/accuracies': '0.8398', 'rewards/margins': '2.157', 'logps/chosen': '-102.6', 'logps/rejected': '-129.8', 'logits/chosen': '-4.375', 'logits/rejected': '-4.328', 'epoch': '2.089'}
{'loss': '0.2775', 'grad_norm': '0.5592', 'learning_rate': '6.193e-05', 'rewards/chosen': '-2.541', 'rewards/rejected': '-4.662', 'rewards/accuracies': '0.8219', 'rewards/margins': '2.121', 'logps/chosen': '-107.2', 'logps/rejected': '-131.6', 'logits/chosen': '-4.222', 'logits/rejected': '-4.156', 'epoch': '2.1'}
{'loss': '0.2666', 'grad_norm': '0.4701', 'learning_rate': '6.118e-05', 'rewards/chosen': '-2.732', 'rewards/rejected': '-4.826', 'rewards/accuracies': '0.8531', 'rewards/margins': '2.093', 'logps/chosen': '-109.3', 'logps/rejected': '-134.7', 'logits/chosen': '-4.261', 'logits/rejected': '-4.192', 'epoch': '2.111'}
{'loss': '0.2717', 'grad_norm': '0.5637', 'learning_rate': '6.043e-05', 'rewards/chosen': '-2.671', 'rewards/rejected': '-4.832', 'rewards/accuracies': '0.8391', 'rewards/margins': '2.162', 'logps/chosen': '-107.6', 'logps/rejected': '-132.3', 'logits/chosen': '-4.387', 'logits/rejected': '-4.331', 'epoch': '2.122'}
{'loss': '0.2677', 'grad_norm': '0.574', 'learning_rate': '5.969e-05', 'rewards/chosen': '-2.751', 'rewards/rejected': '-4.891', 'rewards/accuracies': '0.8555', 'rewards/margins': '2.14', 'logps/chosen': '-110.6', 'logps/rejected': '-136.7', 'logits/chosen': '-4.307', 'logits/rejected': '-4.232', 'epoch': '2.133'}
{'loss': '0.2648', 'grad_norm': '0.501', 'learning_rate': '5.894e-05', 'rewards/chosen': '-2.646', 'rewards/rejected': '-4.777', 'rewards/accuracies': '0.8492', 'rewards/margins': '2.131', 'logps/chosen': '-108.6', 'logps/rejected': '-132.8', 'logits/chosen': '-4.332', 'logits/rejected': '-4.311', 'epoch': '2.144'}
{'loss': '0.2541', 'grad_norm': '0.6321', 'learning_rate': '5.819e-05', 'rewards/chosen': '-2.826', 'rewards/rejected': '-5.047', 'rewards/accuracies': '0.857', 'rewards/margins': '2.221', 'logps/chosen': '-107.1', 'logps/rejected': '-135', 'logits/chosen': '-4.377', 'logits/rejected': '-4.221', 'epoch': '2.155'}
{'loss': '0.2612', 'grad_norm': '0.5602', 'learning_rate': '5.744e-05', 'rewards/chosen': '-3.04', 'rewards/rejected': '-5.265', 'rewards/accuracies': '0.8609', 'rewards/margins': '2.225', 'logps/chosen': '-110.8', 'logps/rejected': '-138.8', 'logits/chosen': '-4.365', 'logits/rejected': '-4.279', 'epoch': '2.165'}
{'loss': '0.2652', 'grad_norm': '0.5245', 'learning_rate': '5.669e-05', 'rewards/chosen': '-3.134', 'rewards/rejected': '-5.365', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.231', 'logps/chosen': '-113.5', 'logps/rejected': '-140.8', 'logits/chosen': '-4.356', 'logits/rejected': '-4.298', 'epoch': '2.176'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
0%| | 0/300 [00:00<?, ?it/s]
1%| | 2/300 [00:00<00:52, 5.68it/s]
1%| | 3/300 [00:00<01:48, 2.73it/s]
1%|▏ | 4/300 [00:01<01:35, 3.09it/s]
2%|▏ | 5/300 [00:01<01:45, 2.79it/s]
2%|▏ | 6/300 [00:01<01:40, 2.91it/s]
2%|▏ | 7/300 [00:02<01:54, 2.55it/s]
3%|▎ | 8/300 [00:02<01:50, 2.63it/s]
3%|▎ | 9/300 [00:03<01:52, 2.59it/s]
3%|▎ | 10/300 [00:03<01:48, 2.67it/s]
4%|▎ | 11/300 [00:04<02:06, 2.29it/s]
4%|▍ | 12/300 [00:04<01:48, 2.65it/s]
4%|▍ | 13/300 [00:04<01:49, 2.61it/s]
5%|▍ | 14/300 [00:05<01:40, 2.85it/s]
5%|▌ | 15/300 [00:05<01:49, 2.61it/s]
5%|▌ | 16/300 [00:05<01:36, 2.95it/s]
6%|▌ | 17/300 [00:06<01:39, 2.83it/s]
6%|▌ | 18/300 [00:06<01:29, 3.14it/s]
6%|▋ | 19/300 [00:06<01:40, 2.81it/s]
7%|▋ | 20/300 [00:07<01:32, 3.04it/s]
7%|▋ | 21/300 [00:07<01:41, 2.74it/s]
7%|▋ | 22/300 [00:07<01:30, 3.07it/s]
8%|▊ | 23/300 [00:08<01:35, 2.91it/s]
8%|▊ | 24/300 [00:08<01:25, 3.23it/s]
8%|▊ | 25/300 [00:08<01:37, 2.83it/s]
9%|▊ | 26/300 [00:09<01:29, 3.05it/s]
9%|▉ | 27/300 [00:09<01:38, 2.78it/s]
9%|▉ | 28/300 [00:09<01:27, 3.12it/s]
10%|▉ | 29/300 [00:10<01:29, 3.01it/s]
10%|█ | 30/300 [00:10<01:21, 3.31it/s]
10%|█ | 31/300 [00:10<01:24, 3.19it/s]
11%|█ | 32/300 [00:10<01:17, 3.45it/s]
11%|█ | 33/300 [00:11<01:20, 3.33it/s]
11%|█▏ | 34/300 [00:11<01:15, 3.50it/s]
12%|█▏ | 35/300 [00:11<01:22, 3.23it/s]
12%|█▏ | 36/300 [00:12<01:17, 3.39it/s]
12%|█▏ | 37/300 [00:12<01:27, 2.99it/s]
13%|█▎ | 38/300 [00:12<01:18, 3.33it/s]
13%|█▎ | 39/300 [00:13<01:26, 3.03it/s]
13%|█▎ | 40/300 [00:13<01:18, 3.32it/s]
14%|█▎ | 41/300 [00:13<01:25, 3.03it/s]
14%|█▍ | 42/300 [00:14<01:17, 3.33it/s]
14%|█▍ | 43/300 [00:14<01:23, 3.08it/s]
15%|█▍ | 44/300 [00:14<01:16, 3.35it/s]
15%|█▌ | 45/300 [00:15<01:22, 3.08it/s]
15%|█▌ | 46/300 [00:15<01:24, 3.02it/s]
16%|█▌ | 47/300 [00:15<01:29, 2.81it/s]
16%|█▌ | 48/300 [00:16<01:22, 3.05it/s]
16%|█▋ | 49/300 [00:16<01:23, 3.02it/s]
17%|█▋ | 50/300 [00:16<01:14, 3.36it/s]
17%|█▋ | 51/300 [00:16<01:17, 3.19it/s]
17%|█▋ | 52/300 [00:17<01:14, 3.31it/s]
18%|█▊ | 53/300 [00:17<01:27, 2.81it/s]
18%|█▊ | 54/300 [00:17<01:19, 3.10it/s]
18%|█▊ | 55/300 [00:18<01:26, 2.82it/s]
19%|█▊ | 56/300 [00:18<01:15, 3.25it/s]
19%|█▉ | 57/300 [00:18<01:19, 3.07it/s]
19%|█▉ | 58/300 [00:19<01:20, 3.00it/s]
20%|█▉ | 59/300 [00:19<01:25, 2.81it/s]
20%|██ | 60/300 [00:19<01:17, 3.11it/s]
20%|██ | 61/300 [00:20<01:20, 2.96it/s]
21%|██ | 62/300 [00:20<01:20, 2.96it/s]
21%|██ | 63/300 [00:21<01:25, 2.76it/s]
21%|██▏ | 64/300 [00:21<01:24, 2.80it/s]
22%|██▏ | 65/300 [00:21<01:27, 2.69it/s]
22%|██▏ | 66/300 [00:22<01:19, 2.95it/s]
22%|██▏ | 67/300 [00:22<01:26, 2.68it/s]
23%|██▎ | 68/300 [00:22<01:18, 2.97it/s]
23%|██▎ | 69/300 [00:23<01:21, 2.85it/s]
23%|██▎ | 70/300 [00:23<01:12, 3.16it/s]
24%|██▎ | 71/300 [00:23<01:15, 3.02it/s]
24%|██▍ | 72/300 [00:24<01:09, 3.27it/s]
24%|██▍ | 73/300 [00:24<01:12, 3.13it/s]
25%|██▍ | 74/300 [00:24<01:08, 3.31it/s]
25%|██▌ | 75/300 [00:25<01:13, 3.07it/s]
25%|██▌ | 76/300 [00:25<01:14, 3.00it/s]
26%|██▌ | 77/300 [00:25<01:19, 2.81it/s]
26%|██▌ | 78/300 [00:26<01:13, 3.04it/s]
26%|██▋ | 79/300 [00:26<01:15, 2.93it/s]
27%|██▋ | 80/300 [00:26<01:11, 3.09it/s]
27%|██▋ | 81/300 [00:27<01:19, 2.76it/s]
27%|██▋ | 82/300 [00:27<01:11, 3.07it/s]
28%|██▊ | 83/300 [00:27<01:13, 2.97it/s]
28%|██▊ | 84/300 [00:28<01:13, 2.93it/s]
28%|██▊ | 85/300 [00:28<01:18, 2.74it/s]
29%|██▊ | 86/300 [00:28<01:10, 3.04it/s]
29%|██▉ | 87/300 [00:29<01:12, 2.94it/s]
29%|██▉ | 88/300 [00:29<01:06, 3.19it/s]
30%|██▉ | 89/300 [00:29<01:10, 2.98it/s]
30%|███ | 90/300 [00:30<01:05, 3.21it/s]
30%|███ | 91/300 [00:30<01:09, 3.02it/s]
31%|███ | 92/300 [00:30<01:03, 3.28it/s]
31%|███ | 93/300 [00:31<01:09, 3.00it/s]
31%|███▏ | 94/300 [00:31<01:02, 3.30it/s]
32%|███▏ | 95/300 [00:31<01:07, 3.03it/s]
32%|███▏ | 96/300 [00:31<01:03, 3.22it/s]
32%|███▏ | 97/300 [00:32<01:05, 3.11it/s]
33%|███▎ | 98/300 [00:32<01:01, 3.27it/s]
33%|███▎ | 99/300 [00:33<01:08, 2.92it/s]
33%|███▎ | 100/300 [00:33<01:04, 3.10it/s]
34%|███▎ | 101/300 [00:33<01:07, 2.93it/s]
34%|███▍ | 102/300 [00:34<01:06, 3.00it/s]
34%|███▍ | 103/300 [00:34<01:18, 2.51it/s]
35%|███▍ | 104/300 [00:34<01:08, 2.86it/s]
35%|███▌ | 105/300 [00:35<01:09, 2.81it/s]
35%|███▌ | 106/300 [00:35<01:03, 3.05it/s]
36%|███▌ | 107/300 [00:35<01:09, 2.77it/s]
36%|███▌ | 108/300 [00:36<01:05, 2.91it/s]
36%|███▋ | 109/300 [00:36<01:08, 2.80it/s]
37%|███▋ | 110/300 [00:36<01:03, 2.98it/s]
37%|███▋ | 111/300 [00:37<01:12, 2.62it/s]
37%|███▋ | 112/300 [00:37<01:06, 2.83it/s]
38%|███▊ | 113/300 [00:38<01:08, 2.75it/s]
38%|███▊ | 114/300 [00:38<01:07, 2.77it/s]
38%|███▊ | 115/300 [00:38<01:09, 2.65it/s]
39%|███▊ | 116/300 [00:39<01:01, 2.97it/s]
39%|███▉ | 117/300 [00:39<01:03, 2.89it/s]
39%|███▉ | 118/300 [00:39<00:58, 3.12it/s]
40%|███▉ | 119/300 [00:40<01:01, 2.95it/s]
40%|████ | 120/300 [00:40<01:00, 2.99it/s]
40%|████ | 121/300 [00:40<01:02, 2.87it/s]
41%|████ | 122/300 [00:40<00:57, 3.10it/s]
41%|████ | 123/300 [00:41<01:01, 2.89it/s]
41%|████▏ | 124/300 [00:41<00:57, 3.06it/s]
42%|████▏ | 125/300 [00:42<00:59, 2.94it/s]
42%|████▏ | 126/300 [00:42<00:59, 2.92it/s]
42%|████▏ | 127/300 [00:42<01:01, 2.79it/s]
43%|████▎ | 128/300 [00:43<00:57, 2.99it/s]
43%|████▎ | 129/300 [00:43<01:00, 2.84it/s]
43%|████▎ | 130/300 [00:43<00:54, 3.13it/s]
44%|████▎ | 131/300 [00:44<01:00, 2.78it/s]
44%|████▍ | 132/300 [00:44<00:55, 3.02it/s]
44%|████▍ | 133/300 [00:44<01:02, 2.67it/s]
45%|████▍ | 134/300 [00:45<00:56, 2.93it/s]
45%|████▌ | 135/300 [00:45<00:58, 2.81it/s]
45%|████▌ | 136/300 [00:45<00:52, 3.10it/s]
46%|████▌ | 137/300 [00:46<00:53, 3.04it/s]
46%|████▌ | 138/300 [00:46<00:55, 2.92it/s]
46%|████▋ | 139/300 [00:46<00:54, 2.96it/s]
47%|████▋ | 140/300 [00:47<00:49, 3.25it/s]
47%|████▋ | 141/300 [00:47<00:50, 3.15it/s]
47%|████▋ | 142/300 [00:47<00:51, 3.05it/s]
48%|████▊ | 143/300 [00:48<00:54, 2.89it/s]
48%|████▊ | 144/300 [00:48<00:48, 3.19it/s]
48%|████▊ | 145/300 [00:48<00:49, 3.14it/s]
49%|████▊ | 146/300 [00:48<00:46, 3.30it/s]
49%|████▉ | 147/300 [00:49<00:53, 2.87it/s]
49%|████▉ | 148/300 [00:49<00:50, 3.03it/s]
50%|████▉ | 149/300 [00:50<00:59, 2.56it/s]
50%|█████ | 150/300 [00:50<00:53, 2.80it/s]
50%|█████ | 151/300 [00:50<00:54, 2.75it/s]
51%|█████ | 152/300 [00:51<00:48, 3.08it/s]
51%|█████ | 153/300 [00:51<00:48, 3.01it/s]
51%|█████▏ | 154/300 [00:51<00:44, 3.30it/s]
52%|█████▏ | 155/300 [00:52<00:46, 3.14it/s]
52%|█████▏ | 156/300 [00:52<00:41, 3.47it/s]
52%|█████▏ | 157/300 [00:52<00:45, 3.12it/s]
53%|█████▎ | 158/300 [00:52<00:43, 3.26it/s]
53%|█████▎ | 159/300 [00:53<00:51, 2.72it/s]
53%|█████▎ | 160/300 [00:53<00:47, 2.96it/s]
54%|█████▎ | 161/300 [00:54<00:50, 2.73it/s]
54%|█████▍ | 162/300 [00:54<00:49, 2.77it/s]
54%|█████▍ | 163/300 [00:55<00:55, 2.45it/s]
55%|█████▍ | 164/300 [00:55<00:53, 2.56it/s]
55%|█████▌ | 165/300 [00:55<00:52, 2.55it/s]
55%|█████▌ | 166/300 [00:56<00:50, 2.66it/s]
56%|█████▌ | 167/300 [00:56<00:54, 2.44it/s]
56%|█████▌ | 168/300 [00:56<00:47, 2.76it/s]
56%|█████▋ | 169/300 [00:57<00:52, 2.49it/s]
57%|█████▋ | 170/300 [00:57<00:50, 2.59it/s]
57%|█████▋ | 171/300 [00:58<00:50, 2.54it/s]
57%|█████▋ | 172/300 [00:58<00:44, 2.88it/s]
58%|█████▊ | 173/300 [00:58<00:46, 2.73it/s]
58%|█████▊ | 174/300 [00:59<00:41, 3.06it/s]
58%|█████▊ | 175/300 [00:59<00:44, 2.79it/s]
59%|█████▊ | 176/300 [00:59<00:39, 3.12it/s]
59%|█████▉ | 177/300 [01:00<00:44, 2.78it/s]
59%|█████▉ | 178/300 [01:00<00:38, 3.20it/s]
60%|█████▉ | 179/300 [01:00<00:41, 2.94it/s]
60%|██████ | 180/300 [01:00<00:37, 3.22it/s]
60%|██████ | 181/300 [01:01<00:41, 2.89it/s]
61%|██████ | 182/300 [01:01<00:36, 3.20it/s]
61%|██████ | 183/300 [01:02<00:40, 2.87it/s]
61%|██████▏ | 184/300 [01:02<00:40, 2.86it/s]
62%|██████▏ | 185/300 [01:02<00:42, 2.68it/s]
62%|██████▏ | 186/300 [01:03<00:39, 2.88it/s]
62%|██████▏ | 187/300 [01:03<00:39, 2.87it/s]
63%|██████▎ | 188/300 [01:03<00:35, 3.15it/s]
63%|██████▎ | 189/300 [01:04<00:38, 2.85it/s]
63%|██████▎ | 190/300 [01:04<00:35, 3.09it/s]
64%|██████▎ | 191/300 [01:04<00:41, 2.64it/s]
64%|██████▍ | 192/300 [01:05<00:39, 2.75it/s]
64%|██████▍ | 193/300 [01:05<00:43, 2.46it/s]
65%|██████▍ | 194/300 [01:06<00:41, 2.57it/s]
65%|██████▌ | 195/300 [01:06<00:42, 2.49it/s]
65%|██████▌ | 196/300 [01:06<00:38, 2.73it/s]
66%|██████▌ | 197/300 [01:07<00:38, 2.65it/s]
66%|██████▌ | 198/300 [01:07<00:35, 2.90it/s]
66%|██████▋ | 199/300 [01:07<00:36, 2.80it/s]
67%|██████▋ | 200/300 [01:08<00:33, 2.99it/s]
67%|██████▋ | 201/300 [01:08<00:35, 2.80it/s]
67%|██████▋ | 202/300 [01:08<00:31, 3.09it/s]
68%|██████▊ | 203/300 [01:09<00:33, 2.94it/s]
68%|██████▊ | 204/300 [01:09<00:29, 3.23it/s]
68%|██████▊ | 205/300 [01:09<00:31, 3.02it/s]
69%|██████▊ | 206/300 [01:10<00:27, 3.41it/s]
69%|██████▉ | 207/300 [01:10<00:31, 2.94it/s]
69%|██████▉ | 208/300 [01:10<00:31, 2.95it/s]
70%|██████▉ | 209/300 [01:11<00:32, 2.80it/s]
70%|███████ | 210/300 [01:11<00:29, 3.02it/s]
70%|███████ | 211/300 [01:11<00:33, 2.67it/s]
71%|███████ | 212/300 [01:12<00:29, 2.98it/s]
71%|███████ | 213/300 [01:12<00:31, 2.74it/s]
71%|███████▏ | 214/300 [01:12<00:28, 3.00it/s]
72%|███████▏ | 215/300 [01:13<00:30, 2.80it/s]
72%|███████▏ | 216/300 [01:13<00:27, 3.09it/s]
72%|███████▏ | 217/300 [01:14<00:31, 2.64it/s]
73%|███████▎ | 218/300 [01:14<00:26, 3.07it/s]
73%|███████▎ | 219/300 [01:14<00:28, 2.81it/s]
73%|███████▎ | 220/300 [01:14<00:25, 3.11it/s]
74%|███████▎ | 221/300 [01:15<00:28, 2.78it/s]
74%|███████▍ | 222/300 [01:15<00:25, 3.09it/s]
74%|███████▍ | 223/300 [01:16<00:27, 2.75it/s]
75%|███████▍ | 224/300 [01:16<00:25, 2.96it/s]
75%|███████▌ | 225/300 [01:16<00:26, 2.81it/s]
75%|███████▌ | 226/300 [01:17<00:24, 3.05it/s]
76%|███████▌ | 227/300 [01:17<00:26, 2.80it/s]
76%|███████▌ | 228/300 [01:17<00:23, 3.13it/s]
76%|███████▋ | 229/300 [01:18<00:25, 2.75it/s]
77%|███████▋ | 230/300 [01:18<00:24, 2.86it/s]
77%|███████▋ | 231/300 [01:18<00:25, 2.70it/s]
77%|███████▋ | 232/300 [01:19<00:24, 2.76it/s]
78%|███████▊ | 233/300 [01:19<00:27, 2.45it/s]
78%|███████▊ | 234/300 [01:19<00:23, 2.78it/s]
78%|███████▊ | 235/300 [01:20<00:24, 2.60it/s]
79%|███████▊ | 236/300 [01:20<00:23, 2.67it/s]
79%|███████▉ | 237/300 [01:21<00:24, 2.53it/s]
79%|███████▉ | 238/300 [01:21<00:21, 2.84it/s]
80%|███████▉ | 239/300 [01:21<00:21, 2.83it/s]
80%|████████ | 240/300 [01:22<00:19, 3.15it/s]
80%|████████ | 241/300 [01:22<00:19, 2.96it/s]
81%|████████ | 242/300 [01:22<00:19, 3.00it/s]
81%|████████ | 243/300 [01:23<00:21, 2.68it/s]
81%|████████▏ | 244/300 [01:23<00:20, 2.79it/s]
82%|████████▏ | 245/300 [01:24<00:22, 2.48it/s]
82%|████████▏ | 246/300 [01:24<00:20, 2.63it/s]
82%|████████▏ | 247/300 [01:24<00:21, 2.50it/s]
83%|████████▎ | 248/300 [01:25<00:18, 2.85it/s]
83%|████████▎ | 249/300 [01:25<00:18, 2.73it/s]
83%|████████▎ | 250/300 [01:25<00:17, 2.85it/s]
84%|████████▎ | 251/300 [01:26<00:17, 2.76it/s]
84%|████████▍ | 252/300 [01:26<00:16, 2.96it/s]
84%|████████▍ | 253/300 [01:26<00:16, 2.81it/s]
85%|████████▍ | 254/300 [01:27<00:14, 3.18it/s]
85%|████████▌ | 255/300 [01:27<00:16, 2.75it/s]
85%|████████▌ | 256/300 [01:27<00:14, 3.09it/s]
86%|████████▌ | 257/300 [01:28<00:14, 2.89it/s]
86%|████████▌ | 258/300 [01:28<00:13, 3.16it/s]
86%|████████▋ | 259/300 [01:28<00:13, 2.94it/s]
87%|████████▋ | 260/300 [01:29<00:12, 3.14it/s]
87%|████████▋ | 261/300 [01:29<00:13, 2.97it/s]
87%|████████▋ | 262/300 [01:29<00:11, 3.26it/s]
88%|████████▊ | 263/300 [01:30<00:12, 3.00it/s]
88%|████████▊ | 264/300 [01:30<00:12, 2.99it/s]
88%|████████▊ | 265/300 [01:30<00:13, 2.59it/s]
89%|████████▊ | 266/300 [01:31<00:12, 2.68it/s]
89%|████████▉ | 267/300 [01:31<00:13, 2.42it/s]
89%|████████▉ | 268/300 [01:32<00:11, 2.78it/s]
90%|████████▉ | 269/300 [01:32<00:12, 2.46it/s]
90%|█████████ | 270/300 [01:32<00:11, 2.63it/s]
90%|█████████ | 271/300 [01:33<00:12, 2.37it/s]
91%|█████████ | 272/300 [01:33<00:10, 2.62it/s]
91%|█████████ | 273/300 [01:34<00:11, 2.33it/s]
91%|█████████▏| 274/300 [01:34<00:09, 2.67it/s]
92%|█████████▏| 275/300 [01:34<00:10, 2.42it/s]
92%|█████████▏| 276/300 [01:35<00:09, 2.60it/s]
92%|█████████▏| 277/300 [01:35<00:09, 2.47it/s]
93%|█████████▎| 278/300 [01:36<00:07, 2.79it/s]
93%|█████████▎| 279/300 [01:36<00:07, 2.63it/s]
93%|█████████▎| 280/300 [01:36<00:07, 2.75it/s]
94%|█████████▎| 281/300 [01:37<00:07, 2.63it/s]
94%|█████████▍| 282/300 [01:37<00:06, 2.95it/s]
94%|█████████▍| 283/300 [01:37<00:06, 2.75it/s]
95%|█████████▍| 284/300 [01:38<00:05, 3.03it/s]
95%|█████████▌| 285/300 [01:38<00:05, 2.64it/s]
95%|█████████▌| 286/300 [01:38<00:05, 2.70it/s]
96%|█████████▌| 287/300 [01:39<00:04, 2.63it/s]
96%|█████████▌| 288/300 [01:39<00:03, 3.05it/s]
96%|█████████▋| 289/300 [01:39<00:03, 2.80it/s]
97%|█████████▋| 290/300 [01:40<00:03, 2.82it/s]
97%|█████████▋| 291/300 [01:40<00:03, 2.62it/s]
97%|█████████▋| 292/300 [01:41<00:02, 2.94it/s]
98%|█████████▊| 293/300 [01:41<00:02, 2.78it/s]
98%|█████████▊| 294/300 [01:41<00:01, 3.02it/s]
98%|█████████▊| 295/300 [01:42<00:01, 2.95it/s]
99%|█████████▊| 296/300 [01:42<00:01, 3.15it/s]
99%|█████████▉| 297/300 [01:42<00:01, 2.92it/s]
99%|█████████▉| 298/300 [01:42<00:00, 3.28it/s]
100%|█████████▉| 299/300 [01:43<00:00, 2.88it/s]
100%|██████████| 300/300 [01:43<00:00, 2.75it/s]
 73%|███████▎ | 2000/2757 [4:34:32<1:39:59, 7.93s/it]
100%|██████████| 300/300 [01:43<00:00, 2.75it/s]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-2000/tokenizer_config.json.
73%|███████▎ | 2001/2757 [4:34:46<8:39:03, 41.20s/it] 73%|███████▎ | 2002/2757 [4:34:54<6:34:09, 31.32s/it] 73%|███████▎ | 2003/2757 [4:35:02<5:04:47, 24.25s/it] 73%|███████▎ | 2004/2757 [4:35:10<4:02:00, 19.28s/it] 73%|███████▎ | 2005/2757 [4:35:17<3:17:12, 15.74s/it] 73%|███████▎ | 2006/2757 [4:35:25<2:46:16, 13.28s/it] 73%|███████▎ | 2007/2757 [4:35:33<2:27:25, 11.79s/it] 73%|███████▎ | 2008/2757 [4:35:41<2:12:05, 10.58s/it] 73%|███████▎ | 2009/2757 [4:35:48<2:00:45, 9.69s/it] 73%|███████▎ | 2010/2757 [4:35:57<1:55:55, 9.31s/it] 73%|███████▎ | 2010/2757 [4:35:57<1:55:55, 9.31s/it] 73%|███████▎ | 2011/2757 [4:36:04<1:50:02, 8.85s/it] 73%|███████▎ | 2012/2757 [4:36:12<1:45:06, 8.47s/it] 73%|███████▎ | 2013/2757 [4:36:20<1:42:05, 8.23s/it] 73%|███████▎ | 2014/2757 [4:36:28<1:40:27, 8.11s/it] 73%|███████▎ | 2015/2757 [4:36:35<1:38:49, 7.99s/it] 73%|███████▎ | 2016/2757 [4:36:44<1:40:22, 8.13s/it] 73%|███████▎ | 2017/2757 [4:36:52<1:39:20, 8.05s/it] 73%|███████▎ | 2018/2757 [4:37:00<1:39:26, 8.07s/it] 73%|███████▎ | 2019/2757 [4:37:08<1:39:02, 8.05s/it] 73%|███████▎ | 2020/2757 [4:37:15<1:37:06, 7.91s/it] 73%|███████▎ | 2020/2757 [4:37:15<1:37:06, 7.91s/it] 73%|███████▎ | 2021/2757 [4:37:24<1:38:40, 8.04s/it] 73%|███████▎ | 2022/2757 [4:37:31<1:37:17, 7.94s/it] 73%|███████▎ | 2023/2757 [4:37:39<1:35:33, 7.81s/it] 73%|███████▎ | 2024/2757 [4:37:47<1:35:55, 7.85s/it] 73%|███████▎ | 2025/2757 [4:37:55<1:36:24, 7.90s/it] 73%|███████▎ | 2026/2757 [4:38:03<1:37:26, 8.00s/it] 74%|███████▎ | 2027/2757 [4:38:11<1:37:17, 8.00s/it] 74%|███████▎ | 2028/2757 [4:38:19<1:37:57, 8.06s/it] 74%|███████▎ | 2029/2757 [4:38:27<1:37:00, 8.00s/it] 74%|███████▎ | 2030/2757 [4:38:35<1:37:14, 8.03s/it] 74%|███████▎ | 2030/2757 [4:38:35<1:37:14, 8.03s/it] 74%|███████▎ | 2031/2757 [4:38:43<1:37:23, 8.05s/it] 74%|███████▎ | 2032/2757 [4:38:51<1:35:32, 7.91s/it] 74%|███████▎ | 2033/2757 [4:38:59<1:34:31, 7.83s/it] 74%|███████▍ | 2034/2757 [4:39:06<1:34:28, 7.84s/it] 74%|███████▍ | 2035/2757 [4:39:15<1:37:06, 8.07s/it] 74%|███████▍ | 2036/2757 [4:39:23<1:36:27, 8.03s/it] 74%|███████▍ | 2037/2757 [4:39:31<1:35:18, 7.94s/it] 74%|███████▍ | 2038/2757 [4:39:39<1:36:29, 8.05s/it] 74%|███████▍ | 2039/2757 [4:39:47<1:37:51, 8.18s/it] 74%|███████▍ | 2040/2757 [4:39:56<1:37:33, 8.16s/it] 74%|███████▍ | 2040/2757 [4:39:56<1:37:33, 8.16s/it] 74%|███████▍ | 2041/2757 [4:40:04<1:36:30, 8.09s/it] 74%|███████▍ | 2042/2757 [4:40:11<1:35:19, 8.00s/it] 74%|███████▍ | 2043/2757 [4:40:19<1:34:26, 7.94s/it] 74%|███████▍ | 2044/2757 [4:40:27<1:33:36, 7.88s/it] 74%|███████▍ | 2045/2757 [4:40:35<1:34:41, 7.98s/it] 74%|███████▍ | 2046/2757 [4:40:43<1:33:36, 7.90s/it] 74%|███████▍ | 2047/2757 [4:40:51<1:35:05, 8.04s/it] 74%|███████▍ | 2048/2757 [4:40:59<1:34:23, 7.99s/it] 74%|███████▍ | 2049/2757 [4:41:07<1:33:49, 7.95s/it] 74%|███████▍ | 2050/2757 [4:41:1
{'loss': '0.2672', 'grad_norm': '0.64', 'learning_rate': '5.595e-05', 'rewards/chosen': '-3.084', 'rewards/rejected': '-5.276', 'rewards/accuracies': '0.8438', 'rewards/margins': '2.193', 'logps/chosen': '-115.2', 'logps/rejected': '-140.9', 'logits/chosen': '-4.258', 'logits/rejected': '-4.195', 'epoch': '2.187'}
{'loss': '0.2488', 'grad_norm': '0.5022', 'learning_rate': '5.52e-05', 'rewards/chosen': '-2.958', 'rewards/rejected': '-5.24', 'rewards/accuracies': '0.8687', 'rewards/margins': '2.282', 'logps/chosen': '-108.5', 'logps/rejected': '-137.5', 'logits/chosen': '-4.378', 'logits/rejected': '-4.255', 'epoch': '2.198'}
{'loss': '0.2526', 'grad_norm': '0.4892', 'learning_rate': '5.445e-05', 'rewards/chosen': '-2.979', 'rewards/rejected': '-5.242', 'rewards/accuracies': '0.8508', 'rewards/margins': '2.262', 'logps/chosen': '-108.2', 'logps/rejected': '-137.7', 'logits/chosen': '-4.429', 'logits/rejected': '-4.317', 'epoch': '2.209'}
{'loss': '0.2479', 'grad_norm': '0.5125', 'learning_rate': '5.37e-05', 'rewards/chosen': '-2.978', 'rewards/rejected': '-5.298', 'rewards/accuracies': '0.8516', 'rewards/margins': '2.321', 'logps/chosen': '-110.8', 'logps/rejected': '-140.5', 'logits/chosen': '-4.358', 'logits/rejected': '-4.336', 'epoch': '2.22'}
{'loss': '0.2748', 'grad_norm': '0.5904', 'learning_rate': '5.295e-05', 'rewards/chosen': '-2.839', 'rewards/rejected': '-4.998', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.159', 'logps/chosen': '-108.7', 'logps/rejected': '-136.3', 'logits/chosen': '-4.396', 'logits/rejected': '-4.374', 'epoch': '2.231'}
{'loss': '0.2631', 'grad_norm': '0.5153', 'learning_rate': '5.221e-05', 'rewards/chosen': '-2.878', 'rewards/rejected': '-5.143', 'rewards/accuracies': '0.8406', 'rewards/margins': '2.264', 'logps/chosen': '-108.4', 'logps/rejected': '-137.3', 'logits/chosen': '-4.383', 'logits/rejected': '-4.335', 'epoch': '2.242'}
{'loss': '0.2632', 'grad_norm': '0.5087', 'learning_rate': '5.146e-05', 'rewards/chosen': '-2.793', 'rewards/rejected': '-4.991', 'rewards/accuracies': '0.8516', 'rewards/margins': '2.198', 'logps/chosen': '-107.8', 'logps/rejected': '-135.7', 'logits/chosen': '-4.283', 'logits/rejected': '-4.199', 'epoch': '2.253'}
{'loss': '0.2556', 'grad_norm': '0.5399', 'learning_rate': '5.071e-05', 'rewards/chosen': '-2.866', 'rewards/rejected': '-5.147', 'rewards/accuracies': '0.857', 'rewards/margins': '2.281', 'logps/chosen': '-106.8', 'logps/rejected': '-136.2', 'logits/chosen': '-4.345', 'logits/rejected': '-4.317', 'epoch': '2.263'}
{'loss': '0.2625', 'grad_norm': '0.4645', 'learning_rate': '4.996e-05', 'rewards/chosen': '-2.911', 'rewards/rejected': '-5.203', 'rewards/accuracies': '0.8477', 'rewards/margins': '2.292', 'logps/chosen': '-106.5', 'logps/rejected': '-136.2', 'logits/chosen': '-4.257', 'logits/rejected': '-4.259', 'epoch': '2.274'}
{'loss': '0.2609', 'grad_norm': '0.3866', 'learning_rate': '4.921e-05', 'rewards/chosen': '-2.991', 'rewards/rejected': '-5.275', 'rewards/accuracies': '0.8461', 'rewards/margins': '2.285', 'logps/chosen': '-111.6', 'logps/rejected': '-139.2', 'logits/chosen': '-4.299', 'logits/rejected': '-4.223', 'epoch': '2.285'}
{'loss': '0.2628', 'grad_norm': '0.5244', 'learning_rate': '4.847e-05', 'rewards/chosen': '-2.894', 'rewards/rejected': '-5.102', 'rewards/accuracies': '0.8578', 'rewards/margins': '2.207', 'logps/chosen': '-105.7', 'logps/rejected': '-132.3', 'logits/chosen': '-4.305', 'logits/rejected': '-4.244', 'epoch': '2.296'}
{'loss': '0.2798', 'grad_norm': '0.5264', 'learning_rate': '4.772e-05', 'rewards/chosen': '-2.842', 'rewards/rejected': '-4.989', 'rewards/accuracies': '0.8211', 'rewards/margins': '2.147', 'logps/chosen': '-107.4', 'logps/rejected': '-135.3', 'logits/chosen': '-4.359', 'logits/rejected': '-4.288', 'epoch': '2.307'}
{'loss': '0.2712', 'grad_norm': '0.619', 'learning_rate': '4.697e-05', 'rewards/chosen': '-2.925', 'rewards/rejected': '-5.061', 'rewards/accuracies': '0.8414', 'rewards/margins': '2.136', 'logps/chosen': '-109.6', 'logps/rejected': '-136.1', 'logits/chosen': '-4.337', 'logits/rejected': '-4.261', 'epoch': '2.318'}
{'loss': '0.2454', 'grad_norm': '0.4767', 'learning_rate': '4.622e-05', 'rewards/chosen': '-2.863', 'rewards/rejected': '-5.176', 'rewards/accuracies': '0.8625', 'rewards/margins': '2.314', 'logps/chosen': '-110.6', 'logps/rejected': '-139.7', 'logits/chosen': '-4.277', 'logits/rejected': '-4.208', 'epoch': '2.329'}
{'loss': '0.2712', 'grad_norm': '0.4791', 'learning_rate': '4.547e-05', 'rewards/chosen': '-2.719', 'rewards/rejected': '-4.824', 'rewards/accuracies': '0.8414', 'rewards/margins': '2.106', 'logps/chosen': '-105.8', 'logps/rejected': '-131.3', 'logits/chosen': '-4.355', 'logits/rejected': '-4.288', 'epoch': '2.34'}
{'loss': '0.2594', 'grad_norm': '0.6803', 'learning_rate': '4.473e-05', 'rewards/chosen': '-2.693', 'rewards/rejected': '-4.999', 'rewards/accuracies': '0.8469', 'rewards/margins': '2.306', 'logps/chosen': '-105.4', 'logps/rejected': '-134.7', 'logits/chosen': '-4.417', 'logits/rejected': '-4.399', 'epoch': '2.35'}
{'loss': '0.2457', 'grad_norm': '0.5753', 'learning_rate': '4.398e-05', 'rewards/chosen': '-2.723', 'rewards/rejected': '-5.072', 'rewards/accuracies': '0.8555', 'rewards/margins': '2.349', 'logps/chosen': '-108.5', 'logps/rejected': '-138', 'logits/chosen': '-4.324', 'logits/rejected': '-4.302', 'epoch': '2.361'}
{'loss': '0.2549', 'grad_norm': '0.5278', 'learning_rate': '4.323e-05', 'rewards/chosen': '-2.749', 'rewards/rejected': '-5.012', 'rewards/accuracies': '0.8547', 'rewards/margins': '2.263', 'logps/chosen': '-109.8', 'logps/rejected': '-138.1', 'logits/chosen': '-4.221', 'logits/rejected': '-4.199', 'epoch': '2.372'}
{'loss': '0.2544', 'grad_norm': '0.6214', 'learning_rate': '4.248e-05', 'rewards/chosen': '-2.788', 'rewards/rejected': '-5.146', 'rewards/accuracies': '0.8523', 'rewards/margins': '2.358', 'logps/chosen': '-109.3', 'logps/rejected': '-137.8', 'logits/chosen': '-4.419', 'logits/rejected': '-4.302', 'epoch': '2.383'}
{'loss': '0.2642', 'grad_norm': '0.6792', 'learning_rate': '4.174e-05', 'rewards/chosen': '-2.842', 'rewards/rejected': '-5.078', 'rewards/accuracies': '0.8438', 'rewards/margins': '2.235', 'logps/chosen': '-107.8', 'logps/rejected': '-135.4', 'logits/chosen': '-4.336', 'logits/rejected': '-4.239', 'epoch': '2.394'}
{'loss': '0.254', 'grad_norm': '0.5071', 'learning_rate': '4.099e-05', 'rewards/chosen': '-2.94', 'rewards/rejected': '-5.256', 'rewards/accuracies': '0.85', 'rewards/margins': '2.316', 'logps/chosen': '-108.2', 'logps/rejected': '-135.4', 'logits/chosen': '-4.275', 'logits/rejected': '-4.161', 'epoch': '2.405'}
{'loss': '0.2531', 'grad_norm': '0.5598', 'learning_rate': '4.024e-05', 'rewards/chosen': '-2.943', 'rewards/rejected': '-5.273', 'rewards/accuracies': '0.8453', 'rewards/margins': '2.33', 'logps/chosen': '-111.8', 'logps/rejected': '-139.2', 'logits/chosen': '-4.343', 'logits/rejected': '-4.282', 'epoch': '2.416'}
{'loss': '0.2632', 'grad_norm': '0.4813', 'learning_rate': '3.949e-05', 'rewards/chosen': '-2.964', 'rewards/rejected': '-5.288', 'rewards/accuracies': '0.8422', 'rewards/margins': '2.324', 'logps/chosen': '-108.8', 'logps/rejected': '-136.8', 'logits/chosen': '-4.238', 'logits/rejected': '-4.215', 'epoch': '2.427'}
{'loss': '0.2669', 'grad_norm': '0.5383', 'learning_rate': '3.874e-05', 'rewards/chosen': '-3.067', 'rewards/rejected': '-5.306', 'rewards/accuracies': '0.8453', 'rewards/margins': '2.239', 'logps/chosen': '-113.3', 'logps/rejected': '-140.9', 'logits/chosen': '-4.169', 'logits/rejected': '-4.063', 'epoch': '2.438'}
{'loss': '0.2641', 'grad_norm': '0.4986', 'learning_rate': '3.8e-05', 'rewards/chosen': '-2.952', 'rewards/rejected': '-5.159', 'rewards/accuracies': '0.8508', 'rewards/margins': '2.207', 'logps/chosen': '-110.6', 'logps/rejected': '-138.2', 'logits/chosen': '-4.118', 'logits/rejected': '-4.037', 'epoch': '2.448'} 82%|████████▏ | 2250/2757 [5:07:49<1:08:09, 8.07s/it] 82%|████████▏ | 2251/2757 [5:08:00<1:15:07, 8.91s/it] 82%|████████▏ | 2252/2757 [5:08:08<1:12:37, 8.63s/it] 82%|████████▏ | 2253/2757 [5:08:16<1:10:38, 8.41s/it] 82%|████████▏ | 2254/2757 [5:08:24<1:09:14, 8.26s/it] 82%|████████▏ | 2255/2757 [5:08:32<1:09:02, 8.25s/it] 82%|████████▏ | 2256/2757 [5:08:40<1:08:28, 8.20s/it] 82%|████████▏ | 2257/2757 [5:08:48<1:06:47, 8.02s/it] 82%|████████▏ | 2258/2757 [5:08:55<1:06:07, 7.95s/it] 82%|████████▏ | 2259/2757 [5:09:04<1:06:13, 7.98s/it] 82%|████████▏ | 2260/2757 [5:09:12<1:06:55, 8.08s/it] 82%|████████▏ | 2260/2757 [5:09:12<1:06:55, 8.08s/it] 82%|████████▏ | 2261/2757 [5:09:20<1:06:34, 8.05s/it] 82%|████████▏ | 2262/2757 [5:09:27<1:05:07, 7.89s/it] 82%|████████▏ | 2263/2757 [5:09:36<1:06:13, 8.04s/it] 82%|████████▏ | 2264/2757 [5:09:44<1:06:15, 8.06s/it] 82%|████████▏ | 2265/2757 [5:09:52<1:05:48, 8.03s/it] 82%|████████▏ | 2266/2757 [5:10:00<1:05:27, 8.00s/it] 82%|████████▏ | 2267/2757 [5:10:08<1:04:50, 7.94s/it] 82%|████████▏ | 2268/2757 [5:10:15<1:04:26, 7.91s/it] 82%|████████▏ | 2269/2757 [5:10:23<1:04:13, 7.90s/it] 82%|████████▏ | 2270/2757 [5:10:31<1:02:34, 7.71s/it] 82%|████████▏ | 2270/2757 [5:10:31<1:02:34, 7.71s/it] 82%|████████▏ | 2271/2757 [5:10:38<1:01:38, 7.61s/it] 82%|████████▏ | 2272/2757 [5:10:45<1:01:13, 7.57s/it] 82%|████████▏ | 2273/2757 [5:10:54<1:02:48, 7.79s/it] 82%|████████▏ | 2274/2757 [5:11:02<1:04:28, 8.01s/it] 83%|████████▎ | 2275/2757 [5:11:10<1:04:45, 8.06s/it] 83%|████████▎ | 2276/2757 [5:11:18<1:04:19, 8.02s/it] 83%|████████▎ | 2277/2757 [5:11:26<1:04:27, 8.06s/it] 83%|████████▎ | 2278/2757 [5:11:35<1:04:59, 8.14s/it] 83%|████████▎ | 2279/2757 [5:11:43<1:04:21, 8.08s/it] 83%|████████▎ | 2280/2757 [5:11:51<1:04:12, 8.08s/it] 83%|████████▎ | 2280/2757 [5:11:51<1:04:12, 8.08s/it] 83%|████████▎ | 2281/2757 [5:11:59<1:03:54, 8.05s/it] 83%|████████▎ | 2282/2757 [5:12:07<1:03:00, 7.96s/it] 83%|████████▎ | 2283/2757 [5:12:15<1:02:54, 7.96s/it] 83%|████████▎ | 2284/2757 [5:12:22<1:02:50, 7.97s/it] 83%|████████▎ | 2285/2757 [5:12:30<1:02:05, 7.89s/it] 83%|████████▎ | 2286/2757 [5:12:38<1:02:25, 7.95s/it] 83%|████████▎ | 2287/2757 [5:12:46<1:01:28, 7.85s/it] 83%|████████▎ | 2288/2757 [5:12:54<1:01:03, 7.81s/it] 83%|████████▎ | 2289/2757 [5:13:02<1:01:49, 7.93s/it] 83%|████████▎ | 2290/2757 [5:13:10<1:01:44, 7.93s/it] 83%|████████▎ | 2290/2757 [5:13:10<1:01:44, 7.93s/it] 83%|████████▎ | 2291/2757 [5:13:18<1:02:49, 8.09s/it] 83%|████████▎ | 2292/2757 [5:13:26<1:02:01, 8.00s/it] 83%|████████▎ | 2293/2757 [5:13:34<1:02:34, 8.09s/it]
{'loss': '0.2519', 'grad_norm': '0.5483', 'learning_rate': '3.725e-05', 'rewards/chosen': '-3.012', 'rewards/rejected': '-5.305', 'rewards/accuracies': '0.857', 'rewards/margins': '2.293', 'logps/chosen': '-112.8', 'logps/rejected': '-140.4', 'logits/chosen': '-4.164', 'logits/rejected': '-4.179', 'epoch': '2.459'}
{'loss': '0.2518', 'grad_norm': '0.4968', 'learning_rate': '3.65e-05', 'rewards/chosen': '-3.082', 'rewards/rejected': '-5.411', 'rewards/accuracies': '0.8484', 'rewards/margins': '2.329', 'logps/chosen': '-110.1', 'logps/rejected': '-137.6', 'logits/chosen': '-4.339', 'logits/rejected': '-4.249', 'epoch': '2.47'}
{'loss': '0.2498', 'grad_norm': '0.5726', 'learning_rate': '3.575e-05', 'rewards/chosen': '-3.102', 'rewards/rejected': '-5.512', 'rewards/accuracies': '0.8492', 'rewards/margins': '2.41', 'logps/chosen': '-113.6', 'logps/rejected': '-142.6', 'logits/chosen': '-4.268', 'logits/rejected': '-4.18', 'epoch': '2.481'}
{'loss': '0.26', 'grad_norm': '0.5567', 'learning_rate': '3.5e-05', 'rewards/chosen': '-2.999', 'rewards/rejected': '-5.299', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.3', 'logps/chosen': '-107.5', 'logps/rejected': '-135.8', 'logits/chosen': '-4.303', 'logits/rejected': '-4.221', 'epoch': '2.492'}
{'loss': '0.2627', 'grad_norm': '0.4611', 'learning_rate': '3.426e-05', 'rewards/chosen': '-3.027', 'rewards/rejected': '-5.354', 'rewards/accuracies': '0.8414', 'rewards/margins': '2.327', 'logps/chosen': '-109.5', 'logps/rejected': '-138.5', 'logits/chosen': '-4.257', 'logits/rejected': '-4.285', 'epoch': '2.503'}
{'loss': '0.2664', 'grad_norm': '0.5548', 'learning_rate': '3.351e-05', 'rewards/chosen': '-3.159', 'rewards/rejected': '-5.442', 'rewards/accuracies': '0.8406', 'rewards/margins': '2.282', 'logps/chosen': '-112.9', 'logps/rejected': '-140.6', 'logits/chosen': '-4.131', 'logits/rejected': '-4.09', 'epoch': '2.514'}
{'loss': '0.2697', 'grad_norm': '0.6786', 'learning_rate': '3.276e-05', 'rewards/chosen': '-3.175', 'rewards/rejected': '-5.44', 'rewards/accuracies': '0.8383', 'rewards/margins': '2.265', 'logps/chosen': '-113.3', 'logps/rejected': '-140.3', 'logits/chosen': '-4.26', 'logits/rejected': '-4.25', 'epoch': '2.525'}
{'loss': '0.2606', 'grad_norm': '0.5422', 'learning_rate': '3.201e-05', 'rewards/chosen': '-3.227', 'rewards/rejected': '-5.541', 'rewards/accuracies': '0.843', 'rewards/margins': '2.314', 'logps/chosen': '-112.1', 'logps/rejected': '-142.1', 'logits/chosen': '-4.233', 'logits/rejected': '-4.16', 'epoch': '2.536'}
{'loss': '0.2648', 'grad_norm': '0.6207', 'learning_rate': '3.126e-05', 'rewards/chosen': '-3.262', 'rewards/rejected': '-5.532', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.27', 'logps/chosen': '-110.7', 'logps/rejected': '-138.4', 'logits/chosen': '-4.249', 'logits/rejected': '-4.177', 'epoch': '2.546'}
{'loss': '0.2524', 'grad_norm': '0.6249', 'learning_rate': '3.052e-05', 'rewards/chosen': '-3.352', 'rewards/rejected': '-5.683', 'rewards/accuracies': '0.8562', 'rewards/margins': '2.332', 'logps/chosen': '-115.2', 'logps/rejected': '-144.4', 'logits/chosen': '-4.308', 'logits/rejected': '-4.223', 'epoch': '2.557'}
{'loss': '0.2689', 'grad_norm': '0.6504', 'learning_rate': '2.977e-05', 'rewards/chosen': '-3.239', 'rewards/rejected': '-5.564', 'rewards/accuracies': '0.8398', 'rewards/margins': '2.325', 'logps/chosen': '-111.2', 'logps/rejected': '-141.9', 'logits/chosen': '-4.297', 'logits/rejected': '-4.28', 'epoch': '2.568'}
{'loss': '0.2643', 'grad_norm': '0.6845', 'learning_rate': '2.902e-05', 'rewards/chosen': '-3.097', 'rewards/rejected': '-5.37', 'rewards/accuracies': '0.8336', 'rewards/margins': '2.273', 'logps/chosen': '-111', 'logps/rejected': '-138.2', 'logits/chosen': '-4.262', 'logits/rejected': '-4.167', 'epoch': '2.579'}
{'loss': '0.2551', 'grad_norm': '0.4879', 'learning_rate': '2.827e-05', 'rewards/chosen': '-2.942', 'rewards/rejected': '-5.293', 'rewards/accuracies': '0.8484', 'rewards/margins': '2.351', 'logps/chosen': '-106.5', 'logps/rejected': '-137.2', 'logits/chosen': '-4.409', 'logits/rejected': '-4.352', 'epoch': '2.59'}
{'loss': '0.2424', 'grad_norm': '0.5652', 'learning_rate': '2.752e-05', 'rewards/chosen': '-3.074', 'rewards/rejected': '-5.492', 'rewards/accuracies': '0.8578', 'rewards/margins': '2.419', 'logps/chosen': '-108.9', 'logps/rejected': '-137.6', 'logits/chosen': '-4.418', 'logits/rejected': '-4.38', 'epoch': '2.601'}
{'loss': '0.2535', 'grad_norm': '0.5238', 'learning_rate': '2.678e-05', 'rewards/chosen': '-3.111', 'rewards/rejected': '-5.516', 'rewards/accuracies': '0.8562', 'rewards/margins': '2.405', 'logps/chosen': '-112.7', 'logps/rejected': '-142.6', 'logits/chosen': '-4.35', 'logits/rejected': '-4.247', 'epoch': '2.612'}
{'loss': '0.26', 'grad_norm': '0.519', 'learning_rate': '2.603e-05', 'rewards/chosen': '-3.063', 'rewards/rejected': '-5.384', 'rewards/accuracies': '0.8383', 'rewards/margins': '2.322', 'logps/chosen': '-109.9', 'logps/rejected': '-137.5', 'logits/chosen': '-4.415', 'logits/rejected': '-4.263', 'epoch': '2.623'}
{'loss': '0.2537', 'grad_norm': '0.5785', 'learning_rate': '2.528e-05', 'rewards/chosen': '-3.048', 'rewards/rejected': '-5.36', 'rewards/accuracies': '0.8523', 'rewards/margins': '2.312', 'logps/chosen': '-111.9', 'logps/rejected': '-139.8', 'logits/chosen': '-4.251', 'logits/rejected': '-4.197', 'epoch': '2.633'}
{'loss': '0.2483', 'grad_norm': '0.5177', 'learning_rate': '2.453e-05', 'rewards/chosen': '-3.117', 'rewards/rejected': '-5.505', 'rewards/accuracies': '0.8422', 'rewards/margins': '2.388', 'logps/chosen': '-114.2', 'logps/rejected': '-142.7', 'logits/chosen': '-4.407', 'logits/rejected': '-4.32', 'epoch': '2.644'}
{'loss': '0.2676', 'grad_norm': '0.6798', 'learning_rate': '2.378e-05', 'rewards/chosen': '-2.953', 'rewards/rejected': '-5.227', 'rewards/accuracies': '0.8391', 'rewards/margins': '2.275', 'logps/chosen': '-109.7', 'logps/rejected': '-138.4', 'logits/chosen': '-4.313', 'logits/rejected': '-4.273', 'epoch': '2.655'}
{'loss': '0.2508', 'grad_norm': '0.6013', 'learning_rate': '2.304e-05', 'rewards/chosen': '-2.888', 'rewards/rejected': '-5.234', 'rewards/accuracies': '0.8633', 'rewards/margins': '2.347', 'logps/chosen': '-110.6', 'logps/rejected': '-139.8', 'logits/chosen': '-4.369', 'logits/rejected': '-4.297', 'epoch': '2.666'}
{'loss': '0.2731', 'grad_norm': '0.5708', 'learning_rate': '2.229e-05', 'rewards/chosen': '-2.885', 'rewards/rejected': '-5.138', 'rewards/accuracies': '0.8281', 'rewards/margins': '2.253', 'logps/chosen': '-107.8', 'logps/rejected': '-134.8', 'logits/chosen': '-4.391', 'logits/rejected': '-4.356', 'epoch': '2.677'}
{'loss': '0.2608', 'grad_norm': '0.4749', 'learning_rate': '2.154e-05', 'rewards/chosen': '-2.873', 'rewards/rejected': '-5.198', 'rewards/accuracies': '0.8562', 'rewards/margins': '2.325', 'logps/chosen': '-108.3', 'logps/rejected': '-135.5', 'logits/chosen': '-4.335', 'logits/rejected': '-4.284', 'epoch': '2.688'}
{'loss': '0.2541', 'grad_norm': '0.5339', 'learning_rate': '2.079e-05', 'rewards/chosen': '-2.983', 'rewards/rejected': '-5.276', 'rewards/accuracies': '0.8625', 'rewards/margins': '2.292', 'logps/chosen': '-109.3', 'logps/rejected': '-137.2', 'logits/chosen': '-4.364', 'logits/rejected': '-4.272', 'epoch': '2.699'}
{'loss': '0.258', 'grad_norm': '0.533', 'learning_rate': '2.004e-05', 'rewards/chosen': '-3.023', 'rewards/rejected': '-5.346', 'rewards/accuracies': '0.8586', 'rewards/margins': '2.323', 'logps/chosen': '-109.9', 'logps/rejected': '-138.8', 'logits/chosen': '-4.273', 'logits/rejected': '-4.253', 'epoch': '2.71'}
{'loss': '0.2491', 'grad_norm': '0.4829', 'learning_rate': '1.93e-05', 'rewards/chosen': '-3.075', 'rewards/rejected': '-5.44', 'rewards/accuracies': '0.8445', 'rewards/margins': '2.365', 'logps/chosen': '-114.9', 'logps/rejected': '-143.9', 'logits/chosen': '-4.346', 'logits/rejected': '-4.261', 'epoch': '2.721'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
0%| | 0/300 [00:00<?, ?it/s]
1%| | 2/300 [00:00<00:52, 5.72it/s]
1%| | 3/300 [00:00<01:46, 2.79it/s]
1%|▏ | 4/300 [00:01<01:34, 3.12it/s]
2%|▏ | 5/300 [00:01<01:44, 2.83it/s]
2%|▏ | 6/300 [00:01<01:40, 2.93it/s]
2%|▏ | 7/300 [00:02<01:55, 2.55it/s]
3%|▎ | 8/300 [00:02<01:50, 2.64it/s]
3%|▎ | 9/300 [00:03<01:52, 2.60it/s]
3%|▎ | 10/300 [00:03<01:48, 2.67it/s]
4%|▎ | 11/300 [00:03<01:51, 2.59it/s]
4%|▍ | 12/300 [00:04<01:38, 2.93it/s]
4%|▍ | 13/300 [00:04<01:43, 2.78it/s]
5%|▍ | 14/300 [00:04<01:35, 2.98it/s]
5%|▌ | 15/300 [00:05<01:45, 2.69it/s]
5%|▌ | 16/300 [00:05<01:34, 3.01it/s]
6%|▌ | 17/300 [00:05<01:38, 2.87it/s]
6%|▌ | 18/300 [00:06<01:29, 3.17it/s]
6%|▋ | 19/300 [00:06<01:39, 2.82it/s]
7%|▋ | 20/300 [00:06<01:31, 3.05it/s]
7%|▋ | 21/300 [00:07<01:43, 2.70it/s]
7%|▋ | 22/300 [00:07<01:31, 3.02it/s]
8%|▊ | 23/300 [00:07<01:32, 2.99it/s]
8%|▊ | 24/300 [00:08<01:23, 3.30it/s]
8%|▊ | 25/300 [00:08<01:35, 2.87it/s]
9%|▊ | 26/300 [00:08<01:29, 3.06it/s]
9%|▉ | 27/300 [00:09<01:38, 2.78it/s]
9%|▉ | 28/300 [00:09<01:27, 3.12it/s]
10%|▉ | 29/300 [00:09<01:30, 3.00it/s]
10%|█ | 30/300 [00:10<01:21, 3.30it/s]
10%|█ | 31/300 [00:10<01:33, 2.86it/s]
11%|█ | 32/300 [00:10<01:24, 3.17it/s]
11%|█ | 33/300 [00:11<01:25, 3.12it/s]
11%|█▏ | 34/300 [00:11<01:19, 3.35it/s]
12%|█▏ | 35/300 [00:11<01:30, 2.93it/s]
12%|█▏ | 36/300 [00:12<01:23, 3.14it/s]
12%|█▏ | 37/300 [00:12<01:32, 2.84it/s]
13%|█▎ | 38/300 [00:12<01:21, 3.20it/s]
13%|█▎ | 39/300 [00:13<01:28, 2.95it/s]
13%|█▎ | 40/300 [00:13<01:20, 3.22it/s]
14%|█▎ | 41/300 [00:13<01:27, 2.97it/s]
14%|█▍ | 42/300 [00:14<01:19, 3.26it/s]
14%|█▍ | 43/300 [00:14<01:23, 3.09it/s]
15%|█▍ | 44/300 [00:14<01:16, 3.36it/s]
15%|█▌ | 45/300 [00:15<01:23, 3.07it/s]
15%|█▌ | 46/300 [00:15<01:24, 3.02it/s]
16%|█▌ | 47/300 [00:15<01:29, 2.82it/s]
16%|█▌ | 48/300 [00:16<01:22, 3.05it/s]
16%|█▋ | 49/300 [00:16<01:23, 3.01it/s]
17%|█▋ | 50/300 [00:16<01:14, 3.35it/s]
17%|█▋ | 51/300 [00:17<01:18, 3.19it/s]
17%|█▋ | 52/300 [00:17<01:15, 3.30it/s]
18%|█▊ | 53/300 [00:17<01:28, 2.80it/s]
18%|█▊ | 54/300 [00:18<01:19, 3.10it/s]
18%|█▊ | 55/300 [00:18<01:26, 2.82it/s]
19%|█▊ | 56/300 [00:18<01:15, 3.23it/s]
19%|█▉ | 57/300 [00:19<01:19, 3.04it/s]
19%|█▉ | 58/300 [00:19<01:21, 2.98it/s]
20%|█▉ | 59/300 [00:19<01:26, 2.79it/s]
20%|██ | 60/300 [00:20<01:17, 3.10it/s]
20%|██ | 61/300 [00:20<01:21, 2.95it/s]
21%|██ | 62/300 [00:20<01:20, 2.94it/s]
21%|██ | 63/300 [00:21<01:25, 2.76it/s]
21%|██▏ | 64/300 [00:21<01:23, 2.82it/s]
22%|██▏ | 65/300 [00:21<01:27, 2.70it/s]
22%|██▏ | 66/300 [00:22<01:19, 2.95it/s]
22%|██▏ | 67/300 [00:22<01:26, 2.69it/s]
23%|██▎ | 68/300 [00:22<01:18, 2.97it/s]
23%|██▎ | 69/300 [00:23<01:21, 2.84it/s]
23%|██▎ | 70/300 [00:23<01:13, 3.15it/s]
24%|██▎ | 71/300 [00:23<01:15, 3.03it/s]
24%|██▍ | 72/300 [00:24<01:09, 3.29it/s]
24%|██▍ | 73/300 [00:24<01:12, 3.13it/s]
25%|██▍ | 74/300 [00:24<01:08, 3.29it/s]
25%|██▌ | 75/300 [00:25<01:13, 3.06it/s]
25%|██▌ | 76/300 [00:25<01:14, 3.00it/s]
26%|██▌ | 77/300 [00:25<01:23, 2.67it/s]
26%|██▌ | 78/300 [00:26<01:15, 2.93it/s]
26%|██▋ | 79/300 [00:26<01:16, 2.87it/s]
27%|██▋ | 80/300 [00:26<01:12, 3.05it/s]
27%|██▋ | 81/300 [00:27<01:19, 2.74it/s]
27%|██▋ | 82/300 [00:27<01:11, 3.05it/s]
28%|██▊ | 83/300 [00:27<01:14, 2.92it/s]
28%|██▊ | 84/300 [00:28<01:14, 2.90it/s]
28%|██▊ | 85/300 [00:28<01:18, 2.72it/s]
29%|██▊ | 86/300 [00:28<01:10, 3.03it/s]
29%|██▉ | 87/300 [00:29<01:12, 2.93it/s]
29%|██▉ | 88/300 [00:29<01:06, 3.18it/s]
30%|██▉ | 89/300 [00:29<01:10, 2.97it/s]
30%|███ | 90/300 [00:30<01:05, 3.21it/s]
30%|███ | 91/300 [00:30<01:09, 3.02it/s]
31%|███ | 92/300 [00:30<01:03, 3.28it/s]
31%|███ | 93/300 [00:31<01:09, 3.00it/s]
31%|███▏ | 94/300 [00:31<01:02, 3.30it/s]
32%|███▏ | 95/300 [00:31<01:07, 3.02it/s]
32%|███▏ | 96/300 [00:32<01:03, 3.21it/s]
32%|███▏ | 97/300 [00:32<01:05, 3.08it/s]
33%|███▎ | 98/300 [00:32<01:02, 3.25it/s]
33%|███▎ | 99/300 [00:33<01:07, 2.99it/s]
33%|███▎ | 100/300 [00:33<01:03, 3.14it/s]
34%|███▎ | 101/300 [00:33<01:07, 2.95it/s]
34%|███▍ | 102/300 [00:34<01:05, 3.01it/s]
34%|███▍ | 103/300 [00:34<01:09, 2.82it/s]
35%|███▍ | 104/300 [00:34<01:02, 3.14it/s]
35%|███▌ | 105/300 [00:35<01:05, 2.98it/s]
35%|███▌ | 106/300 [00:35<01:00, 3.18it/s]
36%|███▌ | 107/300 [00:35<01:05, 2.96it/s]
36%|███▌ | 108/300 [00:36<01:02, 3.06it/s]
36%|███▋ | 109/300 [00:36<01:06, 2.88it/s]
37%|███▋ | 110/300 [00:36<01:02, 3.03it/s]
37%|███▋ | 111/300 [00:37<01:11, 2.65it/s]
37%|███▋ | 112/300 [00:37<01:05, 2.85it/s]
38%|███▊ | 113/300 [00:37<01:07, 2.77it/s]
38%|███▊ | 114/300 [00:38<01:06, 2.79it/s]
38%|███▊ | 115/300 [00:38<01:09, 2.65it/s]
39%|███▊ | 116/300 [00:38<01:01, 2.98it/s]
39%|███▉ | 117/300 [00:39<01:03, 2.90it/s]
39%|███▉ | 118/300 [00:39<00:58, 3.13it/s]
40%|███▉ | 119/300 [00:39<01:00, 3.00it/s]
40%|████ | 120/300 [00:40<00:59, 3.03it/s]
40%|████ | 121/300 [00:40<01:01, 2.89it/s]
41%|████ | 122/300 [00:40<00:57, 3.08it/s]
41%|████ | 123/300 [00:41<01:01, 2.87it/s]
41%|████▏ | 124/300 [00:41<00:57, 3.04it/s]
42%|████▏ | 125/300 [00:41<00:59, 2.94it/s]
42%|████▏ | 126/300 [00:42<00:59, 2.91it/s]
42%|████▏ | 127/300 [00:42<01:01, 2.79it/s]
43%|████▎ | 128/300 [00:42<00:57, 2.99it/s]
43%|████▎ | 129/300 [00:43<00:59, 2.87it/s]
43%|████▎ | 130/300 [00:43<00:53, 3.15it/s]
44%|████▎ | 131/300 [00:44<01:00, 2.78it/s]
44%|████▍ | 132/300 [00:44<00:55, 3.03it/s]
44%|████▍ | 133/300 [00:44<01:05, 2.54it/s]
45%|████▍ | 134/300 [00:45<00:58, 2.81it/s]
45%|████▌ | 135/300 [00:45<01:00, 2.74it/s]
45%|████▌ | 136/300 [00:45<00:53, 3.04it/s]
46%|████▌ | 137/300 [00:46<00:54, 3.01it/s]
46%|████▌ | 138/300 [00:46<00:50, 3.23it/s]
46%|████▋ | 139/300 [00:46<00:50, 3.17it/s]
47%|████▋ | 140/300 [00:46<00:46, 3.43it/s]
47%|████▋ | 141/300 [00:47<00:48, 3.26it/s]
47%|████▋ | 142/300 [00:47<00:50, 3.12it/s]
48%|████▊ | 143/300 [00:48<00:53, 2.93it/s]
48%|████▊ | 144/300 [00:48<00:48, 3.23it/s]
48%|████▊ | 145/300 [00:48<00:48, 3.17it/s]
49%|████▊ | 146/300 [00:48<00:46, 3.32it/s]
49%|████▉ | 147/300 [00:49<00:53, 2.87it/s]
49%|████▉ | 148/300 [00:49<00:50, 3.04it/s]
50%|████▉ | 149/300 [00:50<00:53, 2.81it/s]
50%|█████ | 150/300 [00:50<00:49, 3.01it/s]
50%|█████ | 151/300 [00:50<00:51, 2.89it/s]
51%|█████ | 152/300 [00:50<00:46, 3.20it/s]
51%|█████ | 153/300 [00:51<00:47, 3.08it/s]
51%|█████▏ | 154/300 [00:51<00:43, 3.35it/s]
52%|█████▏ | 155/300 [00:51<00:46, 3.14it/s]
52%|█████▏ | 156/300 [00:52<00:41, 3.47it/s]
52%|█████▏ | 157/300 [00:52<00:46, 3.09it/s]
53%|█████▎ | 158/300 [00:52<00:43, 3.24it/s]
53%|█████▎ | 159/300 [00:53<00:50, 2.78it/s]
53%|█████▎ | 160/300 [00:53<00:46, 3.01it/s]
54%|█████▎ | 161/300 [00:54<00:54, 2.54it/s]
54%|█████▍ | 162/300 [00:54<00:52, 2.63it/s]
54%|█████▍ | 163/300 [00:54<00:57, 2.38it/s]
55%|█████▍ | 164/300 [00:55<00:54, 2.50it/s]
55%|█████▌ | 165/300 [00:55<00:53, 2.51it/s]
55%|█████▌ | 166/300 [00:55<00:51, 2.63it/s]
56%|█████▌ | 167/300 [00:56<00:54, 2.42it/s]
56%|█████▌ | 168/300 [00:56<00:48, 2.74it/s]
56%|█████▋ | 169/300 [00:57<00:53, 2.47it/s]
57%|█████▋ | 170/300 [00:57<00:50, 2.57it/s]
57%|█████▋ | 171/300 [00:57<00:50, 2.53it/s]
57%|█████▋ | 172/300 [00:58<00:44, 2.87it/s]
58%|█████▊ | 173/300 [00:58<00:46, 2.71it/s]
58%|█████▊ | 174/300 [00:58<00:41, 3.04it/s]
58%|█████▊ | 175/300 [00:59<00:44, 2.78it/s]
59%|█████▊ | 176/300 [00:59<00:39, 3.11it/s]
59%|█████▉ | 177/300 [00:59<00:44, 2.77it/s]
59%|█████▉ | 178/300 [01:00<00:38, 3.20it/s]
60%|█████▉ | 179/300 [01:00<00:41, 2.94it/s]
60%|██████ | 180/300 [01:00<00:37, 3.21it/s]
60%|██████ | 181/300 [01:01<00:41, 2.88it/s]
61%|██████ | 182/300 [01:01<00:37, 3.19it/s]
61%|██████ | 183/300 [01:01<00:40, 2.86it/s]
61%|██████▏ | 184/300 [01:02<00:40, 2.86it/s]
62%|██████▏ | 185/300 [01:02<00:42, 2.68it/s]
62%|██████▏ | 186/300 [01:03<00:39, 2.88it/s]
62%|██████▏ | 187/300 [01:03<00:39, 2.86it/s]
63%|██████▎ | 188/300 [01:03<00:35, 3.14it/s]
63%|██████▎ | 189/300 [01:04<00:38, 2.85it/s]
63%|██████▎ | 190/300 [01:04<00:35, 3.09it/s]
64%|██████▎ | 191/300 [01:04<00:37, 2.94it/s]
64%|██████▍ | 192/300 [01:05<00:36, 2.97it/s]
64%|██████▍ | 193/300 [01:05<00:41, 2.57it/s]
65%|██████▍ | 194/300 [01:05<00:40, 2.65it/s]
65%|██████▌ | 195/300 [01:06<00:41, 2.55it/s]
65%|██████▌ | 196/300 [01:06<00:37, 2.77it/s]
66%|██████▌ | 197/300 [01:07<00:39, 2.62it/s]
66%|██████▌ | 198/300 [01:07<00:35, 2.87it/s]
66%|██████▋ | 199/300 [01:07<00:36, 2.78it/s]
67%|██████▋ | 200/300 [01:07<00:33, 2.98it/s]
67%|██████▋ | 201/300 [01:08<00:35, 2.78it/s]
67%|██████▋ | 202/300 [01:08<00:31, 3.07it/s]
68%|██████▊ | 203/300 [01:08<00:33, 2.93it/s]
68%|██████▊ | 204/300 [01:09<00:29, 3.22it/s]
68%|██████▊ | 205/300 [01:09<00:31, 2.99it/s]
69%|██████▊ | 206/300 [01:09<00:27, 3.39it/s]
69%|██████▉ | 207/300 [01:10<00:32, 2.88it/s]
69%|██████▉ | 208/300 [01:10<00:31, 2.91it/s]
70%|██████▉ | 209/300 [01:11<00:32, 2.77it/s]
70%|███████ | 210/300 [01:11<00:30, 2.94it/s]
70%|███████ | 211/300 [01:11<00:33, 2.62it/s]
71%|███████ | 212/300 [01:12<00:29, 2.94it/s]
71%|███████ | 213/300 [01:12<00:34, 2.53it/s]
71%|███████▏ | 214/300 [01:12<00:30, 2.82it/s]
72%|███████▏ | 215/300 [01:13<00:31, 2.68it/s]
72%|███████▏ | 216/300 [01:13<00:28, 2.99it/s]
72%|███████▏ | 217/300 [01:13<00:32, 2.58it/s]
73%|███████▎ | 218/300 [01:14<00:27, 3.02it/s]
73%|███████▎ | 219/300 [01:14<00:30, 2.63it/s]
73%|███████▎ | 220/300 [01:14<00:27, 2.95it/s]
74%|███████▎ | 221/300 [01:15<00:29, 2.69it/s]
74%|███████▍ | 222/300 [01:15<00:25, 3.00it/s]
74%|███████▍ | 223/300 [01:16<00:28, 2.70it/s]
75%|███████▍ | 224/300 [01:16<00:26, 2.92it/s]
75%|███████▌ | 225/300 [01:16<00:26, 2.79it/s]
75%|███████▌ | 226/300 [01:17<00:24, 3.03it/s]
76%|███████▌ | 227/300 [01:17<00:26, 2.78it/s]
76%|███████▌ | 228/300 [01:17<00:23, 3.11it/s]
76%|███████▋ | 229/300 [01:18<00:25, 2.75it/s]
77%|███████▋ | 230/300 [01:18<00:24, 2.85it/s]
77%|███████▋ | 231/300 [01:18<00:25, 2.70it/s]
77%|███████▋ | 232/300 [01:19<00:24, 2.77it/s]
78%|███████▊ | 233/300 [01:19<00:27, 2.46it/s]
78%|███████▊ | 234/300 [01:19<00:23, 2.82it/s]
78%|███████▊ | 235/300 [01:20<00:24, 2.64it/s]
79%|███████▊ | 236/300 [01:20<00:23, 2.70it/s]
79%|███████▉ | 237/300 [01:21<00:24, 2.60it/s]
79%|███████▉ | 238/300 [01:21<00:21, 2.91it/s]
80%|███████▉ | 239/300 [01:21<00:21, 2.88it/s]
80%|████████ | 240/300 [01:22<00:18, 3.20it/s]
80%|████████ | 241/300 [01:22<00:19, 3.02it/s]
81%|████████ | 242/300 [01:22<00:18, 3.06it/s]
81%|████████ | 243/300 [01:23<00:20, 2.72it/s]
81%|████████▏ | 244/300 [01:23<00:19, 2.82it/s]
82%|████████▏ | 245/300 [01:23<00:20, 2.67it/s]
82%|████████▏ | 246/300 [01:24<00:19, 2.76it/s]
82%|████████▏ | 247/300 [01:24<00:20, 2.59it/s]
83%|████████▎ | 248/300 [01:24<00:17, 2.93it/s]
83%|████████▎ | 249/300 [01:25<00:18, 2.80it/s]
83%|████████▎ | 250/300 [01:25<00:17, 2.92it/s]
84%|████████▎ | 251/300 [01:26<00:18, 2.59it/s]
84%|████████▍ | 252/300 [01:26<00:16, 2.83it/s]
84%|████████▍ | 253/300 [01:26<00:17, 2.74it/s]
85%|████████▍ | 254/300 [01:26<00:14, 3.12it/s]
85%|████████▌ | 255/300 [01:27<00:16, 2.75it/s]
85%|████████▌ | 256/300 [01:27<00:14, 3.10it/s]
86%|████████▌ | 257/300 [01:28<00:14, 2.93it/s]
86%|████████▌ | 258/300 [01:28<00:13, 3.20it/s]
86%|████████▋ | 259/300 [01:28<00:13, 2.97it/s]
87%|████████▋ | 260/300 [01:28<00:12, 3.16it/s]
87%|████████▋ | 261/300 [01:29<00:13, 2.99it/s]
87%|████████▋ | 262/300 [01:29<00:11, 3.26it/s]
88%|████████▊ | 263/300 [01:29<00:12, 3.00it/s]
88%|████████▊ | 264/300 [01:30<00:11, 3.00it/s]
88%|████████▊ | 265/300 [01:30<00:13, 2.59it/s]
89%|████████▊ | 266/300 [01:31<00:12, 2.69it/s]
89%|████████▉ | 267/300 [01:31<00:13, 2.43it/s]
89%|████████▉ | 268/300 [01:31<00:11, 2.79it/s]
90%|████████▉ | 269/300 [01:32<00:12, 2.47it/s]
90%|█████████ | 270/300 [01:32<00:11, 2.65it/s]
90%|█████████ | 271/300 [01:33<00:12, 2.39it/s]
91%|█████████ | 272/300 [01:33<00:10, 2.65it/s]
91%|█████████ | 273/300 [01:34<00:11, 2.39it/s]
91%|█████████▏| 274/300 [01:34<00:09, 2.73it/s]
92%|█████████▏| 275/300 [01:34<00:10, 2.45it/s]
92%|█████████▏| 276/300 [01:35<00:09, 2.63it/s]
92%|█████████▏| 277/300 [01:35<00:09, 2.53it/s]
93%|█████████▎| 278/300 [01:35<00:07, 2.85it/s]
93%|█████████▎| 279/300 [01:36<00:07, 2.81it/s]
93%|█████████▎| 280/300 [01:36<00:06, 2.90it/s]
94%|█████████▎| 281/300 [01:36<00:06, 2.72it/s]
94%|█████████▍| 282/300 [01:37<00:05, 3.03it/s]
94%|█████████▍| 283/300 [01:37<00:06, 2.80it/s]
95%|█████████▍| 284/300 [01:37<00:05, 3.06it/s]
95%|█████████▌| 285/300 [01:38<00:05, 2.65it/s]
95%|█████████▌| 286/300 [01:38<00:05, 2.71it/s]
96%|█████████▌| 287/300 [01:39<00:04, 2.62it/s]
96%|█████████▌| 288/300 [01:39<00:03, 3.04it/s]
96%|█████████▋| 289/300 [01:39<00:03, 2.79it/s]
97%|█████████▋| 290/300 [01:40<00:03, 2.81it/s]
97%|█████████▋| 291/300 [01:40<00:03, 2.61it/s]
97%|█████████▋| 292/300 [01:40<00:02, 2.93it/s]
98%|█████████▊| 293/300 [01:41<00:02, 2.67it/s]
98%|█████████▊| 294/300 [01:41<00:02, 2.91it/s]
98%|█████████▊| 295/300 [01:41<00:01, 2.56it/s]
99%|█████████▊| 296/300 [01:42<00:01, 2.83it/s]
99%|█████████▉| 297/300 [01:42<00:01, 2.72it/s]
99%|█████████▉| 298/300 [01:42<00:00, 3.11it/s]
100%|█████████▉| 299/300 [01:43<00:00, 2.77it/s]
100%|██████████| 300/300 [01:43<00:00, 2.83it/s]
 91%|█████████ | 2500/2757 [5:42:56<33:59, 7.94s/it]
100%|██████████| 300/300 [01:43<00:00, 2.83it/s]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-2500/tokenizer_config.json.
91%|█████████ | 2501/2757 [5:43:08<2:53:55, 40.76s/it] 91%|█████████ | 2502/2757 [5:43:17<2:12:09, 31.09s/it] 91%|█████████ | 2503/2757 [5:43:25<1:41:53, 24.07s/it] 91%|█████████ | 2504/2757 [5:43:32<1:20:55, 19.19s/it] 91%|█████████ | 2505/2757 [5:43:41<1:06:48, 15.91s/it] 91%|█████████ | 2506/2757 [5:43:48<56:19, 13.46s/it] 91%|█████████ | 2507/2757 [5:43:56<49:16, 11.82s/it] 91%|█████████ | 2508/2757 [5:44:05<44:48, 10.80s/it] 91%|█████████ | 2509/2757 [5:44:13<41:18, 9.99s/it] 91%|█████████ | 2510/2757 [5:44:21<39:05, 9.50s/it] 91%|█████████ | 2510/2757 [5:44:21<39:05, 9.50s/it] 91%|█████████ | 2511/2757 [5:44:29<36:46, 8.97s/it] 91%|█████████ | 2512/2757 [5:44:37<35:59, 8.81s/it] 91%|█████████ | 2513/2757 [5:44:46<35:07, 8.64s/it] 91%|█████████ | 2514/2757 [5:44:54<34:23, 8.49s/it] 91%|█████████ | 2515/2757 [5:45:02<34:02, 8.44s/it] 91%|█████████▏| 2516/2757 [5:45:10<33:27, 8.33s/it] 91%|█████████▏| 2517/2757 [5:45:18<32:51, 8.21s/it] 91%|█████████▏| 2518/2757 [5:45:26<32:04, 8.05s/it] 91%|█████████▏| 2519/2757 [5:45:34<31:53, 8.04s/it] 91%|█████████▏| 2520/2757 [5:45:42<31:43, 8.03s/it] 91%|█████████▏| 2520/2757 [5:45:42<31:43, 8.03s/it] 91%|█████████▏| 2521/2757 [5:45:50<32:01, 8.14s/it] 91%|█████████▏| 2522/2757 [5:45:59<31:59, 8.17s/it] 92%|█████████▏| 2523/2757 [5:46:06<31:19, 8.03s/it] 92%|█████████▏| 2524/2757 [5:46:14<31:05, 8.01s/it] 92%|█████████▏| 2525/2757 [5:46:22<30:43, 7.94s/it] 92%|█████████▏| 2526/2757 [5:46:30<31:00, 8.06s/it] 92%|█████████▏| 2527/2757 [5:46:38<30:57, 8.08s/it] 92%|█████████▏| 2528/2757 [5:46:46<30:30, 7.99s/it] 92%|█████████▏| 2529/2757 [5:46:54<30:18, 7.98s/it] 92%|█████████▏| 2530/2757 [5:47:01<29:27, 7.79s/it] 92%|█████████▏| 2530/2757 [5:47:01<29:27, 7.79s/it] 92%|█████████▏| 2531/2757 [5:47:09<29:30, 7.83s/it] 92%|█████████▏| 2532/2757 [5:47:17<29:37, 7.90s/it] 92%|█████████▏| 2533/2757 [5:47:26<29:47, 7.98s/it] 92%|█████████▏| 2534/2757 [5:47:33<29:15, 7.87s/it] 92%|█████████▏| 2535/2757 [5:47:41<29:29, 7.97s/it] 92%|█████████▏| 2536/2757 [5:47:49<29:14, 7.94s/it] 92%|█████████▏| 2537/2757 [5:47:57<29:10, 7.96s/it] 92%|█████████▏| 2538/2757 [5:48:05<28:47, 7.89s/it] 92%|█████████▏| 2539/2757 [5:48:13<29:01, 7.99s/it] 92%|█████████▏| 2540/2757 [5:48:21<29:03, 8.03s/it] 92%|█████████▏| 2540/2757 [5:48:21<29:03, 8.03s/it] 92%|█████████▏| 2541/2757 [5:48:30<29:35, 8.22s/it] 92%|█████████▏| 2542/2757 [5:48:37<28:30, 7.96s/it] 92%|█████████▏| 2543/2757 [5:48:45<27:56, 7.83s/it] 92%|█████████▏| 2544/2757 [5:48:53<28:07, 7.92s/it] 92%|█████████▏| 2545/2757 [5:49:01<28:05, 7.95s/it] 92%|█████████▏| 2546/2757 [5:49:09<28:16, 8.04s/it] 92%|█████████▏| 2547/2757 [5:49:17<28:01, 8.01s/it] 92%|█████████▏| 2548/2757 [5:49:26<28:16, 8.12s/it] 92%|█████████▏| 2549/2757
{'loss': '0.2396', 'grad_norm': '0.5165', 'learning_rate': '1.855e-05', 'rewards/chosen': '-3.146', 'rewards/rejected': '-5.601', 'rewards/accuracies': '0.8609', 'rewards/margins': '2.456', 'logps/chosen': '-113.9', 'logps/rejected': '-145.7', 'logits/chosen': '-4.321', 'logits/rejected': '-4.34', 'epoch': '2.731'}
{'loss': '0.251', 'grad_norm': '0.5313', 'learning_rate': '1.78e-05', 'rewards/chosen': '-3.095', 'rewards/rejected': '-5.495', 'rewards/accuracies': '0.8547', 'rewards/margins': '2.4', 'logps/chosen': '-112', 'logps/rejected': '-144.2', 'logits/chosen': '-4.337', 'logits/rejected': '-4.252', 'epoch': '2.742'}
{'loss': '0.2669', 'grad_norm': '0.6009', 'learning_rate': '1.705e-05', 'rewards/chosen': '-3.144', 'rewards/rejected': '-5.451', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.306', 'logps/chosen': '-112.2', 'logps/rejected': '-141.5', 'logits/chosen': '-4.345', 'logits/rejected': '-4.265', 'epoch': '2.753'}
{'loss': '0.2554', 'grad_norm': '0.5517', 'learning_rate': '1.631e-05', 'rewards/chosen': '-3.108', 'rewards/rejected': '-5.491', 'rewards/accuracies': '0.8422', 'rewards/margins': '2.383', 'logps/chosen': '-113.3', 'logps/rejected': '-140.6', 'logits/chosen': '-4.365', 'logits/rejected': '-4.302', 'epoch': '2.764'}
{'loss': '0.2582', 'grad_norm': '0.6179', 'learning_rate': '1.556e-05', 'rewards/chosen': '-3.127', 'rewards/rejected': '-5.444', 'rewards/accuracies': '0.8375', 'rewards/margins': '2.317', 'logps/chosen': '-113.7', 'logps/rejected': '-143.8', 'logits/chosen': '-4.218', 'logits/rejected': '-4.297', 'epoch': '2.775'}
{'loss': '0.2668', 'grad_norm': '0.5837', 'learning_rate': '1.481e-05', 'rewards/chosen': '-3.097', 'rewards/rejected': '-5.393', 'rewards/accuracies': '0.8422', 'rewards/margins': '2.297', 'logps/chosen': '-107.7', 'logps/rejected': '-136.1', 'logits/chosen': '-4.289', 'logits/rejected': '-4.224', 'epoch': '2.786'}
{'loss': '0.2648', 'grad_norm': '0.6681', 'learning_rate': '1.406e-05', 'rewards/chosen': '-3.148', 'rewards/rejected': '-5.445', 'rewards/accuracies': '0.8406', 'rewards/margins': '2.297', 'logps/chosen': '-112.1', 'logps/rejected': '-140.3', 'logits/chosen': '-4.223', 'logits/rejected': '-4.183', 'epoch': '2.797'}
{'loss': '0.2467', 'grad_norm': '0.5711', 'learning_rate': '1.331e-05', 'rewards/chosen': '-3.118', 'rewards/rejected': '-5.44', 'rewards/accuracies': '0.8547', 'rewards/margins': '2.322', 'logps/chosen': '-113.7', 'logps/rejected': '-142.5', 'logits/chosen': '-4.21', 'logits/rejected': '-4.185', 'epoch': '2.808'}
{'loss': '0.262', 'grad_norm': '0.5574', 'learning_rate': '1.257e-05', 'rewards/chosen': '-3.077', 'rewards/rejected': '-5.374', 'rewards/accuracies': '0.8438', 'rewards/margins': '2.297', 'logps/chosen': '-109', 'logps/rejected': '-136.3', 'logits/chosen': '-4.284', 'logits/rejected': '-4.183', 'epoch': '2.819'}
{'loss': '0.2469', 'grad_norm': '0.549', 'learning_rate': '1.182e-05', 'rewards/chosen': '-3.052', 'rewards/rejected': '-5.457', 'rewards/accuracies': '0.8555', 'rewards/margins': '2.405', 'logps/chosen': '-111.3', 'logps/rejected': '-140.8', 'logits/chosen': '-4.32', 'logits/rejected': '-4.266', 'epoch': '2.829'}
{'loss': '0.2493', 'grad_norm': '0.4994', 'learning_rate': '1.107e-05', 'rewards/chosen': '-3.06', 'rewards/rejected': '-5.434', 'rewards/accuracies': '0.8508', 'rewards/margins': '2.374', 'logps/chosen': '-108.5', 'logps/rejected': '-137.1', 'logits/chosen': '-4.284', 'logits/rejected': '-4.174', 'epoch': '2.84'}
{'loss': '0.2597', 'grad_norm': '0.596', 'learning_rate': '1.032e-05', 'rewards/chosen': '-3.036', 'rewards/rejected': '-5.337', 'rewards/accuracies': '0.8578', 'rewards/margins': '2.3', 'logps/chosen': '-107.8', 'logps/rejected': '-135.4', 'logits/chosen': '-4.267', 'logits/rejected': '-4.113', 'epoch': '2.851'}
{'loss': '0.2591', 'grad_norm': '0.5377', 'learning_rate': '9.574e-06', 'rewards/chosen': '-3.168', 'rewards/rejected': '-5.573', 'rewards/accuracies': '0.8469', 'rewards/margins': '2.405', 'logps/chosen': '-111.8', 'logps/rejected': '-141.1', 'logits/chosen': '-4.322', 'logits/rejected': '-4.283', 'epoch': '2.862'}
{'loss': '0.2559', 'grad_norm': '0.6059', 'learning_rate': '8.826e-06', 'rewards/chosen': '-3.091', 'rewards/rejected': '-5.442', 'rewards/accuracies': '0.8352', 'rewards/margins': '2.351', 'logps/chosen': '-111.7', 'logps/rejected': '-141', 'logits/chosen': '-4.247', 'logits/rejected': '-4.217', 'epoch': '2.873'}
{'loss': '0.2493', 'grad_norm': '0.6502', 'learning_rate': '8.078e-06', 'rewards/chosen': '-3.102', 'rewards/rejected': '-5.487', 'rewards/accuracies': '0.8523', 'rewards/margins': '2.385', 'logps/chosen': '-112.2', 'logps/rejected': '-142.6', 'logits/chosen': '-4.256', 'logits/rejected': '-4.207', 'epoch': '2.884'}
{'loss': '0.2628', 'grad_norm': '0.524', 'learning_rate': '7.33e-06', 'rewards/chosen': '-3.17', 'rewards/rejected': '-5.479', 'rewards/accuracies': '0.8336', 'rewards/margins': '2.309', 'logps/chosen': '-113', 'logps/rejected': '-140.3', 'logits/chosen': '-4.29', 'logits/rejected': '-4.22', 'epoch': '2.895'}
{'loss': '0.2616', 'grad_norm': '0.6188', 'learning_rate': '6.582e-06', 'rewards/chosen': '-3.144', 'rewards/rejected': '-5.507', 'rewards/accuracies': '0.8484', 'rewards/margins': '2.363', 'logps/chosen': '-111.8', 'logps/rejected': '-140.9', 'logits/chosen': '-4.309', 'logits/rejected': '-4.274', 'epoch': '2.906'}
{'loss': '0.2506', 'grad_norm': '0.66', 'learning_rate': '5.834e-06', 'rewards/chosen': '-3.092', 'rewards/rejected': '-5.493', 'rewards/accuracies': '0.8562', 'rewards/margins': '2.402', 'logps/chosen': '-112.2', 'logps/rejected': '-143.6', 'logits/chosen': '-4.265', 'logits/rejected': '-4.203', 'epoch': '2.916'}
{'loss': '0.2568', 'grad_norm': '0.4364', 'learning_rate': '5.086e-06', 'rewards/chosen': '-3.115', 'rewards/rejected': '-5.444', 'rewards/accuracies': '0.8406', 'rewards/margins': '2.328', 'logps/chosen': '-110', 'logps/rejected': '-139.2', 'logits/chosen': '-4.176', 'logits/rejected': '-4.107', 'epoch': '2.927'}
{'loss': '0.2593', 'grad_norm': '0.5918', 'learning_rate': '4.338e-06', 'rewards/chosen': '-3.071', 'rewards/rejected': '-5.375', 'rewards/accuracies': '0.8438', 'rewards/margins': '2.304', 'logps/chosen': '-112.2', 'logps/rejected': '-139.6', 'logits/chosen': '-4.195', 'logits/rejected': '-4.221', 'epoch': '2.938'}
{'loss': '0.2563', 'grad_norm': '0.4884', 'learning_rate': '3.59e-06', 'rewards/chosen': '-3.157', 'rewards/rejected': '-5.468', 'rewards/accuracies': '0.8375', 'rewards/margins': '2.311', 'logps/chosen': '-113.7', 'logps/rejected': '-140.7', 'logits/chosen': '-4.237', 'logits/rejected': '-4.152', 'epoch': '2.949'}
{'loss': '0.2622', 'grad_norm': '0.6017', 'learning_rate': '2.842e-06', 'rewards/chosen': '-3.124', 'rewards/rejected': '-5.479', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.355', 'logps/chosen': '-112.5', 'logps/rejected': '-141.4', 'logits/chosen': '-4.274', 'logits/rejected': '-4.203', 'epoch': '2.96'}
{'loss': '0.2499', 'grad_norm': '0.6066', 'learning_rate': '2.094e-06', 'rewards/chosen': '-3.07', 'rewards/rejected': '-5.482', 'rewards/accuracies': '0.8547', 'rewards/margins': '2.412', 'logps/chosen': '-112.7', 'logps/rejected': '-142.3', 'logits/chosen': '-4.194', 'logits/rejected': '-4.145', 'epoch': '2.971'}
{'loss': '0.2538', 'grad_norm': '0.6238', 'learning_rate': '1.346e-06', 'rewards/chosen': '-3.098', 'rewards/rejected': '-5.456', 'rewards/accuracies': '0.8492', 'rewards/margins': '2.358', 'logps/chosen': '-111.6', 'logps/rejected': '-141.5', 'logits/chosen': '-4.227', 'logits/rejected': '-4.174', 'epoch': '2.982'}
{'loss': '0.2607', 'grad_norm': '0.6461', 'learning_rate': '5.984e-07', 'rewards/chosen': '-3.128', 'rewards/rejected': '-5.369', 'rewards/accuracies': '0.8492', 'rewards/margins': '2.241', 'logps/chosen': '-109.2', 'logps/rejected': '-137.4', 'logits/chosen': '-4.289', 'logits/rejected': '-4.216', 'epoch': '2.993'} 100%|█████████▉| 2750/2757 [6:16:20<00:56, 8.05s/it] 100%|█████████▉| 2751/2757 [6:16:31<00:54, 9.02s/it] 100%|█████████▉| 2752/2757 [6:16:39<00:43, 8.63s/it] 100%|█████████▉| 2753/2757 [6:16:47<00:33, 8.38s/it] 100%|█████████▉| 2754/2757 [6:16:54<00:24, 8.20s/it] 100%|█████████▉| 2755/2757 [6:17:02<00:16, 8.16s/it] 100%|█████████▉| 2756/2757 [6:17:10<00:08, 8.06s/it] 100%|██████████| 2757/2757 [6:17:16<00:00, 7.44s/it]
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
0%| | 0/300 [00:00<?, ?it/s]
1%| | 2/300 [00:00<00:52, 5.70it/s]
1%| | 3/300 [00:00<01:46, 2.80it/s]
1%|▏ | 4/300 [00:01<01:34, 3.12it/s]
2%|▏ | 5/300 [00:01<01:44, 2.81it/s]
2%|▏ | 6/300 [00:01<01:40, 2.92it/s]
2%|▏ | 7/300 [00:02<01:55, 2.54it/s]
3%|▎ | 8/300 [00:02<01:50, 2.64it/s]
3%|▎ | 9/300 [00:03<01:52, 2.59it/s]
3%|▎ | 10/300 [00:03<01:49, 2.66it/s]
4%|▎ | 11/300 [00:03<01:51, 2.59it/s]
4%|▍ | 12/300 [00:04<01:39, 2.91it/s]
4%|▍ | 13/300 [00:04<01:49, 2.63it/s]
5%|▍ | 14/300 [00:04<01:40, 2.86it/s]
5%|▌ | 15/300 [00:05<01:49, 2.61it/s]
5%|▌ | 16/300 [00:05<01:36, 2.94it/s]
6%|▌ | 17/300 [00:06<01:39, 2.83it/s]
6%|▌ | 18/300 [00:06<01:30, 3.13it/s]
6%|▋ | 19/300 [00:06<01:40, 2.80it/s]
7%|▋ | 20/300 [00:07<01:32, 3.03it/s]
7%|▋ | 21/300 [00:07<01:50, 2.53it/s]
7%|▋ | 22/300 [00:07<01:36, 2.87it/s]
8%|▊ | 23/300 [00:08<01:35, 2.89it/s]
8%|▊ | 24/300 [00:08<01:26, 3.21it/s]
8%|▊ | 25/300 [00:08<01:37, 2.81it/s]
9%|▊ | 26/300 [00:09<01:30, 3.01it/s]
9%|▉ | 27/300 [00:09<01:41, 2.70it/s]
9%|▉ | 28/300 [00:09<01:29, 3.05it/s]
10%|▉ | 29/300 [00:10<01:31, 2.95it/s]
10%|█ | 30/300 [00:10<01:22, 3.26it/s]
10%|█ | 31/300 [00:10<01:25, 3.14it/s]
11%|█ | 32/300 [00:10<01:18, 3.41it/s]
11%|█ | 33/300 [00:11<01:21, 3.27it/s]
11%|█▏ | 34/300 [00:11<01:17, 3.45it/s]
12%|█▏ | 35/300 [00:11<01:23, 3.19it/s]
12%|█▏ | 36/300 [00:12<01:19, 3.34it/s]
12%|█▏ | 37/300 [00:12<01:27, 3.01it/s]
13%|█▎ | 38/300 [00:12<01:18, 3.35it/s]
13%|█▎ | 39/300 [00:13<01:27, 2.97it/s]
13%|█▎ | 40/300 [00:13<01:19, 3.27it/s]
14%|█▎ | 41/300 [00:13<01:26, 3.01it/s]
14%|█▍ | 42/300 [00:14<01:18, 3.30it/s]
14%|█▍ | 43/300 [00:14<01:21, 3.14it/s]
15%|█▍ | 44/300 [00:14<01:15, 3.41it/s]
15%|█▌ | 45/300 [00:15<01:22, 3.10it/s]
15%|█▌ | 46/300 [00:15<01:23, 3.04it/s]
16%|█▌ | 47/300 [00:15<01:29, 2.83it/s]
16%|█▌ | 48/300 [00:16<01:22, 3.06it/s]
16%|█▋ | 49/300 [00:16<01:23, 3.02it/s]
17%|█▋ | 50/300 [00:16<01:14, 3.35it/s]
17%|█▋ | 51/300 [00:17<01:17, 3.21it/s]
17%|█▋ | 52/300 [00:17<01:14, 3.31it/s]
18%|█▊ | 53/300 [00:17<01:27, 2.81it/s]
18%|█▊ | 54/300 [00:18<01:18, 3.11it/s]
18%|█▊ | 55/300 [00:18<01:30, 2.70it/s]
19%|█▊ | 56/300 [00:18<01:18, 3.12it/s]
19%|█▉ | 57/300 [00:19<01:21, 3.00it/s]
19%|█▉ | 58/300 [00:19<01:22, 2.95it/s]
20%|█▉ | 59/300 [00:19<01:27, 2.75it/s]
20%|██ | 60/300 [00:20<01:18, 3.07it/s]
20%|██ | 61/300 [00:20<01:21, 2.93it/s]
21%|██ | 62/300 [00:20<01:26, 2.74it/s]
21%|██ | 63/300 [00:21<01:29, 2.64it/s]
21%|██▏ | 64/300 [00:21<01:26, 2.72it/s]
22%|██▏ | 65/300 [00:22<01:29, 2.62it/s]
22%|██▏ | 66/300 [00:22<01:20, 2.89it/s]
22%|██▏ | 67/300 [00:22<01:27, 2.65it/s]
23%|██▎ | 68/300 [00:23<01:19, 2.94it/s]
23%|██▎ | 69/300 [00:23<01:21, 2.82it/s]
23%|██▎ | 70/300 [00:23<01:13, 3.12it/s]
24%|██▎ | 71/300 [00:23<01:15, 3.02it/s]
24%|██▍ | 72/300 [00:24<01:09, 3.28it/s]
24%|██▍ | 73/300 [00:24<01:12, 3.13it/s]
25%|██▍ | 74/300 [00:24<01:08, 3.31it/s]
25%|██▌ | 75/300 [00:25<01:16, 2.94it/s]
25%|██▌ | 76/300 [00:25<01:16, 2.92it/s]
26%|██▌ | 77/300 [00:26<01:21, 2.74it/s]
26%|██▌ | 78/300 [00:26<01:14, 2.99it/s]
26%|██▋ | 79/300 [00:26<01:16, 2.90it/s]
27%|██▋ | 80/300 [00:26<01:11, 3.06it/s]
27%|██▋ | 81/300 [00:27<01:19, 2.75it/s]
27%|██▋ | 82/300 [00:27<01:11, 3.05it/s]
28%|██▊ | 83/300 [00:28<01:13, 2.94it/s]
28%|██▊ | 84/300 [00:28<01:14, 2.90it/s]
28%|██▊ | 85/300 [00:28<01:19, 2.72it/s]
29%|██▊ | 86/300 [00:29<01:10, 3.01it/s]
29%|██▉ | 87/300 [00:29<01:13, 2.90it/s]
29%|██▉ | 88/300 [00:29<01:07, 3.14it/s]
30%|██▉ | 89/300 [00:30<01:11, 2.95it/s]
30%|███ | 90/300 [00:30<01:06, 3.17it/s]
30%|███ | 91/300 [00:30<01:09, 2.99it/s]
31%|███ | 92/300 [00:30<01:03, 3.26it/s]
31%|███ | 93/300 [00:31<01:09, 2.98it/s]
31%|███▏ | 94/300 [00:31<01:02, 3.27it/s]
32%|███▏ | 95/300 [00:32<01:10, 2.91it/s]
32%|███▏ | 96/300 [00:32<01:05, 3.13it/s]
32%|███▏ | 97/300 [00:32<01:06, 3.04it/s]
33%|███▎ | 98/300 [00:32<01:02, 3.22it/s]
33%|███▎ | 99/300 [00:33<01:07, 2.98it/s]
33%|███▎ | 100/300 [00:33<01:03, 3.15it/s]
34%|███▎ | 101/300 [00:33<01:07, 2.95it/s]
34%|███▍ | 102/300 [00:34<01:05, 3.01it/s]
34%|███▍ | 103/300 [00:34<01:08, 2.88it/s]
35%|███▍ | 104/300 [00:34<01:01, 3.20it/s]
35%|███▌ | 105/300 [00:35<01:04, 3.02it/s]
35%|███▌ | 106/300 [00:35<01:00, 3.22it/s]
36%|███▌ | 107/300 [00:35<01:04, 3.00it/s]
36%|███▌ | 108/300 [00:36<01:02, 3.08it/s]
36%|███▋ | 109/300 [00:36<01:06, 2.89it/s]
37%|███▋ | 110/300 [00:36<01:02, 3.05it/s]
37%|███▋ | 111/300 [00:37<01:11, 2.66it/s]
37%|███▋ | 112/300 [00:37<01:05, 2.86it/s]
38%|███▊ | 113/300 [00:38<01:17, 2.42it/s]
38%|███▊ | 114/300 [00:38<01:13, 2.53it/s]
38%|███▊ | 115/300 [00:39<01:16, 2.43it/s]
39%|███▊ | 116/300 [00:39<01:06, 2.77it/s]
39%|███▉ | 117/300 [00:39<01:10, 2.59it/s]
39%|███▉ | 118/300 [00:39<01:03, 2.87it/s]
40%|███▉ | 119/300 [00:40<01:04, 2.82it/s]
40%|████ | 120/300 [00:40<01:02, 2.89it/s]
40%|████ | 121/300 [00:41<01:03, 2.82it/s]
41%|████ | 122/300 [00:41<00:58, 3.05it/s]
41%|████ | 123/300 [00:41<01:02, 2.85it/s]
41%|████▏ | 124/300 [00:42<00:58, 3.03it/s]
42%|████▏ | 125/300 [00:42<00:59, 2.93it/s]
42%|████▏ | 126/300 [00:42<00:59, 2.91it/s]
42%|████▏ | 127/300 [00:43<01:02, 2.77it/s]
43%|████▎ | 128/300 [00:43<00:57, 2.98it/s]
43%|████▎ | 129/300 [00:43<00:59, 2.86it/s]
43%|████▎ | 130/300 [00:44<00:54, 3.14it/s]
44%|████▎ | 131/300 [00:44<01:00, 2.78it/s]
44%|████▍ | 132/300 [00:44<00:55, 3.03it/s]
44%|████▍ | 133/300 [00:45<01:02, 2.68it/s]
45%|████▍ | 134/300 [00:45<00:56, 2.93it/s]
45%|████▌ | 135/300 [00:45<00:58, 2.80it/s]
45%|████▌ | 136/300 [00:46<00:52, 3.09it/s]
46%|████▌ | 137/300 [00:46<00:53, 3.03it/s]
46%|████▌ | 138/300 [00:46<00:49, 3.25it/s]
46%|████▋ | 139/300 [00:47<00:50, 3.19it/s]
47%|████▋ | 140/300 [00:47<00:46, 3.44it/s]
47%|████▋ | 141/300 [00:47<00:48, 3.27it/s]
47%|████▋ | 142/300 [00:47<00:50, 3.13it/s]
48%|████▊ | 143/300 [00:48<00:53, 2.94it/s]
48%|████▊ | 144/300 [00:48<00:48, 3.24it/s]
48%|████▊ | 145/300 [00:48<00:48, 3.18it/s]
49%|████▊ | 146/300 [00:49<00:46, 3.33it/s]
49%|████▉ | 147/300 [00:49<00:53, 2.88it/s]
49%|████▉ | 148/300 [00:49<00:49, 3.04it/s]
50%|████▉ | 149/300 [00:50<00:53, 2.80it/s]
50%|█████ | 150/300 [00:50<00:50, 3.00it/s]
50%|█████ | 151/300 [00:51<00:51, 2.88it/s]
51%|█████ | 152/300 [00:51<00:46, 3.19it/s]
51%|█████ | 153/300 [00:51<00:47, 3.09it/s]
51%|█████▏ | 154/300 [00:51<00:43, 3.36it/s]
52%|█████▏ | 155/300 [00:52<00:45, 3.17it/s]
52%|█████▏ | 156/300 [00:52<00:41, 3.49it/s]
52%|█████▏ | 157/300 [00:52<00:45, 3.12it/s]
53%|█████▎ | 158/300 [00:53<00:43, 3.26it/s]
53%|█████▎ | 159/300 [00:53<00:50, 2.79it/s]
53%|█████▎ | 160/300 [00:53<00:46, 3.02it/s]
54%|█████▎ | 161/300 [00:54<00:50, 2.76it/s]
54%|█████▍ | 162/300 [00:54<00:49, 2.79it/s]
54%|█████▍ | 163/300 [00:55<00:55, 2.47it/s]
55%|█████▍ | 164/300 [00:55<00:52, 2.58it/s]
55%|█████▌ | 165/300 [00:55<00:52, 2.56it/s]
55%|█████▌ | 166/300 [00:56<00:50, 2.67it/s]
56%|█████▌ | 167/300 [00:56<00:54, 2.44it/s]
56%|█████▌ | 168/300 [00:56<00:48, 2.75it/s]
56%|█████▋ | 169/300 [00:57<00:52, 2.48it/s]
57%|█████▋ | 170/300 [00:57<00:50, 2.58it/s]
57%|█████▋ | 171/300 [00:58<00:50, 2.54it/s]
57%|█████▋ | 172/300 [00:58<00:44, 2.87it/s]
58%|█████▊ | 173/300 [00:58<00:46, 2.71it/s]
58%|█████▊ | 174/300 [00:59<00:41, 3.05it/s]
58%|█████▊ | 175/300 [00:59<00:46, 2.69it/s]
59%|█████▊ | 176/300 [00:59<00:40, 3.03it/s]
59%|█████▉ | 177/300 [01:00<00:44, 2.73it/s]
59%|█████▉ | 178/300 [01:00<00:38, 3.16it/s]
60%|█████▉ | 179/300 [01:00<00:41, 2.92it/s]
60%|██████ | 180/300 [01:01<00:37, 3.19it/s]
60%|██████ | 181/300 [01:01<00:41, 2.88it/s]
61%|██████ | 182/300 [01:01<00:37, 3.18it/s]
61%|██████ | 183/300 [01:02<00:40, 2.91it/s]
61%|██████▏ | 184/300 [01:02<00:40, 2.90it/s]
62%|██████▏ | 185/300 [01:02<00:42, 2.71it/s]
62%|██████▏ | 186/300 [01:03<00:39, 2.90it/s]
62%|██████▏ | 187/300 [01:03<00:39, 2.87it/s]
63%|██████▎ | 188/300 [01:03<00:35, 3.15it/s]
63%|██████▎ | 189/300 [01:04<00:38, 2.86it/s]
63%|██████▎ | 190/300 [01:04<00:35, 3.10it/s]
64%|██████▎ | 191/300 [01:04<00:37, 2.89it/s]
64%|██████▍ | 192/300 [01:05<00:36, 2.94it/s]
64%|██████▍ | 193/300 [01:05<00:41, 2.56it/s]
65%|██████▍ | 194/300 [01:06<00:40, 2.64it/s]
65%|██████▌ | 195/300 [01:06<00:43, 2.42it/s]
65%|██████▌ | 196/300 [01:06<00:39, 2.66it/s]
66%|██████▌ | 197/300 [01:07<00:39, 2.64it/s]
66%|██████▌ | 198/300 [01:07<00:35, 2.87it/s]
66%|██████▋ | 199/300 [01:07<00:37, 2.73it/s]
67%|██████▋ | 200/300 [01:08<00:34, 2.93it/s]
67%|██████▋ | 201/300 [01:08<00:37, 2.63it/s]
67%|██████▋ | 202/300 [01:08<00:33, 2.93it/s]
68%|██████▊ | 203/300 [01:09<00:34, 2.85it/s]
68%|██████▊ | 204/300 [01:09<00:30, 3.15it/s]
68%|██████▊ | 205/300 [01:09<00:32, 2.97it/s]
69%|██████▊ | 206/300 [01:10<00:28, 3.35it/s]
69%|██████▉ | 207/300 [01:10<00:31, 2.91it/s]
69%|██████▉ | 208/300 [01:10<00:31, 2.93it/s]
70%|██████▉ | 209/300 [01:11<00:32, 2.78it/s]
70%|███████ | 210/300 [01:11<00:29, 3.01it/s]
70%|███████ | 211/300 [01:12<00:33, 2.66it/s]
71%|███████ | 212/300 [01:12<00:29, 2.97it/s]
71%|███████ | 213/300 [01:12<00:31, 2.72it/s]
71%|███████▏ | 214/300 [01:13<00:28, 2.97it/s]
72%|███████▏ | 215/300 [01:13<00:30, 2.77it/s]
72%|███████▏ | 216/300 [01:13<00:27, 3.07it/s]
72%|███████▏ | 217/300 [01:14<00:30, 2.75it/s]
73%|███████▎ | 218/300 [01:14<00:26, 3.15it/s]
73%|███████▎ | 219/300 [01:14<00:28, 2.86it/s]
73%|███████▎ | 220/300 [01:15<00:25, 3.14it/s]
74%|███████▎ | 221/300 [01:15<00:28, 2.80it/s]
74%|███████▍ | 222/300 [01:15<00:25, 3.10it/s]
74%|███████▍ | 223/300 [01:16<00:27, 2.75it/s]
75%|███████▍ | 224/300 [01:16<00:25, 2.97it/s]
75%|███████▌ | 225/300 [01:16<00:26, 2.82it/s]
75%|███████▌ | 226/300 [01:17<00:24, 3.05it/s]
76%|███████▌ | 227/300 [01:17<00:26, 2.80it/s]
76%|███████▌ | 228/300 [01:17<00:23, 3.13it/s]
76%|███████▋ | 229/300 [01:18<00:25, 2.76it/s]
77%|███████▋ | 230/300 [01:18<00:24, 2.86it/s]
77%|███████▋ | 231/300 [01:19<00:25, 2.71it/s]
77%|███████▋ | 232/300 [01:19<00:24, 2.77it/s]
78%|███████▊ | 233/300 [01:19<00:27, 2.47it/s]
78%|███████▊ | 234/300 [01:20<00:23, 2.82it/s]
78%|███████▊ | 235/300 [01:20<00:24, 2.63it/s]
79%|███████▊ | 236/300 [01:20<00:23, 2.69it/s]
79%|███████▉ | 237/300 [01:21<00:24, 2.58it/s]
79%|███████▉ | 238/300 [01:21<00:21, 2.91it/s]
80%|███████▉ | 239/300 [01:21<00:21, 2.83it/s]
80%|████████ | 240/300 [01:22<00:18, 3.16it/s]
80%|████████ | 241/300 [01:22<00:19, 3.05it/s]
81%|████████ | 242/300 [01:22<00:18, 3.09it/s]
81%|████████ | 243/300 [01:23<00:20, 2.73it/s]
81%|████████▏ | 244/300 [01:23<00:19, 2.82it/s]
82%|████████▏ | 245/300 [01:24<00:20, 2.67it/s]
82%|████████▏ | 246/300 [01:24<00:19, 2.77it/s]
82%|████████▏ | 247/300 [01:24<00:20, 2.59it/s]
83%|████████▎ | 248/300 [01:25<00:17, 2.92it/s]
83%|████████▎ | 249/300 [01:25<00:18, 2.78it/s]
83%|████████▎ | 250/300 [01:25<00:17, 2.89it/s]
84%|████████▎ | 251/300 [01:26<00:17, 2.78it/s]
84%|████████▍ | 252/300 [01:26<00:16, 2.99it/s]
84%|████████▍ | 253/300 [01:26<00:17, 2.73it/s]
85%|████████▍ | 254/300 [01:27<00:14, 3.11it/s]
85%|████████▌ | 255/300 [01:27<00:16, 2.80it/s]
85%|████████▌ | 256/300 [01:27<00:14, 3.14it/s]
86%|████████▌ | 257/300 [01:28<00:14, 2.95it/s]
86%|████████▌ | 258/300 [01:28<00:13, 3.21it/s]
86%|████████▋ | 259/300 [01:28<00:14, 2.93it/s]
87%|████████▋ | 260/300 [01:29<00:12, 3.13it/s]
87%|████████▋ | 261/300 [01:29<00:13, 2.96it/s]
87%|████████▋ | 262/300 [01:29<00:11, 3.26it/s]
88%|████████▊ | 263/300 [01:30<00:12, 3.00it/s]
88%|████████▊ | 264/300 [01:30<00:12, 2.99it/s]
88%|████████▊ | 265/300 [01:30<00:13, 2.59it/s]
89%|████████▊ | 266/300 [01:31<00:12, 2.68it/s]
89%|████████▉ | 267/300 [01:31<00:13, 2.42it/s]
89%|████████▉ | 268/300 [01:32<00:11, 2.78it/s]
90%|████████▉ | 269/300 [01:32<00:12, 2.46it/s]
90%|█████████ | 270/300 [01:32<00:11, 2.62it/s]
90%|█████████ | 271/300 [01:33<00:12, 2.37it/s]
91%|█████████ | 272/300 [01:33<00:10, 2.63it/s]
91%|█████████ | 273/300 [01:34<00:11, 2.38it/s]
91%|█████████▏| 274/300 [01:34<00:09, 2.72it/s]
92%|█████████▏| 275/300 [01:34<00:10, 2.44it/s]
92%|█████████▏| 276/300 [01:35<00:09, 2.62it/s]
92%|█████████▏| 277/300 [01:35<00:09, 2.53it/s]
93%|█████████▎| 278/300 [01:35<00:07, 2.85it/s]
93%|█████████▎| 279/300 [01:36<00:07, 2.80it/s]
93%|█████████▎| 280/300 [01:36<00:06, 2.89it/s]
94%|█████████▎| 281/300 [01:37<00:06, 2.72it/s]
94%|█████████▍| 282/300 [01:37<00:05, 3.02it/s]
94%|█████████▍| 283/300 [01:37<00:06, 2.79it/s]
95%|█████████▍| 284/300 [01:37<00:05, 3.06it/s]
95%|█████████▌| 285/300 [01:38<00:05, 2.66it/s]
95%|█████████▌| 286/300 [01:38<00:05, 2.71it/s]
96%|█████████▌| 287/300 [01:39<00:05, 2.59it/s]
96%|█████████▌| 288/300 [01:39<00:03, 3.01it/s]
96%|█████████▋| 289/300 [01:39<00:04, 2.68it/s]
97%|█████████▋| 290/300 [01:40<00:03, 2.73it/s]
97%|█████████▋| 291/300 [01:40<00:03, 2.47it/s]
97%|█████████▋| 292/300 [01:40<00:02, 2.80it/s]
98%|█████████▊| 293/300 [01:41<00:02, 2.70it/s]
98%|█████████▊| 294/300 [01:41<00:02, 2.95it/s]
98%|█████████▊| 295/300 [01:42<00:01, 2.89it/s]
99%|█████████▊| 296/300 [01:42<00:01, 3.11it/s]
99%|█████████▉| 297/300 [01:42<00:01, 2.88it/s]
99%|█████████▉| 298/300 [01:42<00:00, 3.25it/s]
100%|█████████▉| 299/300 [01:43<00:00, 2.53it/s]
100%|██████████| 300/300 [01:43<00:00, 2.57it/s]
 100%|██████████| 2757/2757 [6:19:01<00:00, 7.44s/it]
100%|██████████| 300/300 [01:44<00:00, 2.57it/s]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-2757/tokenizer_config.json.
100%|██████████| 2757/2757 [6:19:31<00:00, 7.44s/it] 100%|██████████| 2757/2757 [6:19:32<00:00, 8.26s/it]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/final/tokenizer_config.json.
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/merged/tokenizer_config.json.
{'eval_loss': '0.4576', 'eval_runtime': '104.9', 'eval_samples_per_second': '22.88', 'eval_steps_per_second': '2.86', 'eval_rewards/chosen': '-3.332', 'eval_rewards/rejected': '-4.978', 'eval_rewards/accuracies': '0.7233', 'eval_rewards/margins': '1.646', 'eval_logps/chosen': '-112.9', 'eval_logps/rejected': '-135.6', 'eval_logits/chosen': '-4.33', 'eval_logits/rejected': '-4.281', 'epoch': '3'}
{'train_runtime': '2.277e+04', 'train_samples_per_second': '15.49', 'train_steps_per_second': '0.121', 'train_loss': '0.4124', 'epoch': '3'}
Saved adapter to models/dpo_default/final
Found HuggingFace hub cache directory: /workspace/.cache/huggingface/hub
Checking cache directory for required files...
Unsloth: Copying 1 files from cache to `models/dpo_default/merged`: 0%| | 0/1 [00:00<?, ?it/s] Unsloth: Copying 1 files from cache to `models/dpo_default/merged`: 100%|██████████| 1/1 [00:02<00:00, 2.20s/it] Unsloth: Copying 1 files from cache to `models/dpo_default/merged`: 100%|██████████| 1/1 [00:02<00:00, 2.20s/it]
Successfully copied all 1 files from cache to `models/dpo_default/merged`
Checking cache directory for required files...
Cache check failed: tokenizer.model not found in local cache.
Not all required files found in cache. Will proceed with downloading.
Unsloth: Preparing safetensor model files: 0%| | 0/1 [00:00<?, ?it/s] Unsloth: Preparing safetensor model files: 100%|██████████| 1/1 [00:00<00:00, 37117.73it/s]
Unsloth: Merging weights into 16bit: 0%| | 0/1 [00:00<?, ?it/s] Unsloth: Merging weights into 16bit: 100%|██████████| 1/1 [00:02<00:00, 2.43s/it] Unsloth: Merging weights into 16bit: 100%|██████████| 1/1 [00:02<00:00, 2.43s/it]
Unsloth: Merge process complete. Saved to `/workspace/post-training-experiments/dpo/DPO_SmolLM135M/models/dpo_default/merged`
Saved merged model to models/dpo_default/merged
== Step 2: post-dpo diversity (compare to baseline above) ==
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
`torch_dtype` is deprecated! Use `dtype` instead!
Skipping import of cpp extensions due to incompatible torch version. Please upgrade to torch >= 2.11.0 (found 2.10.0+cu128).
Loading weights: 0%| | 0/272 [00:00<?, ?it/s] Loading weights: 100%|██████████| 272/272 [00:00<00:00, 3064.64it/s]
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
[1/100] 16 responses avg_len=815 chars
t=0.3 ### Q1 **Question:** What metrics are used to control page layout in the dataset...
t=0.3 ### Q1 **Question:** What metrics are used to control page layout in the dataset...
t=0.3 ### Q1 **Question:** What metrics are used to control page layout in the dataset...
t=0.3 ### Q1 **Question:** What does Figure 4 and 5 illustrate? **Answer:** Figure 4 ...
t=0.5 ### Q1 **Question:** What metrics are used to control page layout in the dataset...
t=0.5 ### Q1 **Question:** What does Figure 4 and Table 4 illustrate? **Answer:** Fig...
t=0.5 ### Q1 **Question:** What does Figure 4 and 5 illustrate? **Answer:** Figure 4 ...
t=0.5 ### Q1 **Question:** What does Figure 4 and Figure 5 illustrate? **Answer:** Fi...
t=0.7 **Question:** What metrics are used to control page layout in the dataset? **An...
t=0.7 *'default' is a value that is empty if none. *Figure 4 and Figure 5 provide an o...
t=0.7 ### Q1 **Question:** What metrics are used to control the page layout of the dat...
t=0.7 *['How is the 'default' value defined in the context of the API format?'] *What ...
t=1.0 ### Q1 **Question:** What metrics are used to control page layout in the dataset...
t=1.0 ### Q1 **Question:** What metrics are used to control page layout in the dataset...
t=1.0 *'default' represents the default value, empty if none *Figure 5 illustrates the...
t=1.0 ### Q1 **Question:** What does Figure 4 and Table 4 illustrate? **Answer:** Fig...
[2/100] 16 responses avg_len=240 chars
t=0.3 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
t=0.3 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {" timedelt...
t=0.3 [{"subject": "AoA", "relation": "is a", "object": "second entry of path estimate...
t=0.3 [{"subject": "AoA", "relation": "is a", "object": "second entry of path estimate...
t=0.5 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
t=0.5 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {" timedelt...
t=0.5 [{"subject": "AoA", "relation": "is a", "object": "second entry of path estimate...
t=0.5 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {" timedelt...
t=0.7 [{"subject": "Option", "relation": "is a declaration in RLang", "object": "presy...
t=0.7 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
t=0.7 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"subject":...
t=0.7 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
t=1.0 [{"subject": "Option", "relation": "describes", "object": "sequence of actions s...
t=1.0 [{"subject": "AoA", "relation": "is a specification of", " object": "sequences o...
t=1.0 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
t=1.0 [{"subject": "Option", "relation": "is a declaration in RLang", "object": "pecul...
[3/100] 16 responses avg_len=218 chars
[4/100] 16 responses avg_len=823 chars
[5/100] 16 responses avg_len=880 chars
[6/100] 16 responses avg_len=276 chars
[7/100] 16 responses avg_len=152 chars
[8/100] 16 responses avg_len=295 chars
[9/100] 16 responses avg_len=230 chars
[10/100] 16 responses avg_len=311 chars
[11/100] 16 responses avg_len=741 chars
[12/100] 16 responses avg_len=865 chars
[13/100] 16 responses avg_len=133 chars
[14/100] 16 responses avg_len=736 chars
[15/100] 16 responses avg_len=327 chars
[16/100] 16 responses avg_len=1030 chars
[17/100] 16 responses avg_len=702 chars
[18/100] 16 responses avg_len=1030 chars
[19/100] 16 responses avg_len=297 chars
[20/100] 16 responses avg_len=319 chars
[21/100] 16 responses avg_len=420 chars
[22/100] 16 responses avg_len=70 chars
[23/100] 16 responses avg_len=320 chars
[24/100] 16 responses avg_len=25 chars
[25/100] 16 responses avg_len=68 chars
[26/100] 16 responses avg_len=299 chars
[27/100] 16 responses avg_len=720 chars
[28/100] 16 responses avg_len=1254 chars
[29/100] 16 responses avg_len=137 chars
[30/100] 16 responses avg_len=353 chars
[31/100] 16 responses avg_len=1093 chars
[32/100] 16 responses avg_len=603 chars
[33/100] 16 responses avg_len=75 chars
[34/100] 16 responses avg_len=109 chars
[35/100] 16 responses avg_len=1307 chars
[36/100] 16 responses avg_len=750 chars
[37/100] 16 responses avg_len=714 chars
[38/100] 16 responses avg_len=179 chars
[39/100] 16 responses avg_len=284 chars
[40/100] 16 responses avg_len=120 chars
[41/100] 16 responses avg_len=765 chars
[42/100] 16 responses avg_len=259 chars
[43/100] 16 responses avg_len=481 chars
[44/100] 16 responses avg_len=241 chars
[45/100] 16 responses avg_len=95 chars
[46/100] 16 responses avg_len=320 chars
[47/100] 16 responses avg_len=391 chars
[48/100] 16 responses avg_len=1030 chars
[49/100] 16 responses avg_len=283 chars
[50/100] 16 responses avg_len=822 chars
[51/100] 16 responses avg_len=788 chars
[52/100] 16 responses avg_len=741 chars
[53/100] 16 responses avg_len=294 chars
[54/100] 16 responses avg_len=120 chars
[55/100] 16 responses avg_len=550 chars
[56/100] 16 responses avg_len=1174 chars
[57/100] 16 responses avg_len=205 chars
[58/100] 16 responses avg_len=190 chars
[59/100] 16 responses avg_len=373 chars
[60/100] 16 responses avg_len=674 chars
[61/100] 16 responses avg_len=1170 chars
[62/100] 16 responses avg_len=693 chars
[63/100] 16 responses avg_len=245 chars
[64/100] 16 responses avg_len=705 chars
[65/100] 16 responses avg_len=161 chars
[66/100] 16 responses avg_len=247 chars
[67/100] 16 responses avg_len=846 chars
[68/100] 16 responses avg_len=816 chars
[69/100] 16 responses avg_len=276 chars
[70/100] 16 responses avg_len=527 chars
[71/100] 16 responses avg_len=253 chars
[72/100] 16 responses avg_len=1074 chars
[73/100] 16 responses avg_len=324 chars
[74/100] 16 responses avg_len=110 chars
[75/100] 16 responses avg_len=337 chars
[76/100] 16 responses avg_len=703 chars
[77/100] 16 responses avg_len=1396 chars
[78/100] 16 responses avg_len=449 chars
[79/100] 16 responses avg_len=592 chars
[80/100] 16 responses avg_len=156 chars
[81/100] 16 responses avg_len=243 chars
[82/100] 16 responses avg_len=581 chars
[83/100] 16 responses avg_len=272 chars
[84/100] 16 responses avg_len=152 chars
[85/100] 16 responses avg_len=75 chars
[86/100] 16 responses avg_len=199 chars
[87/100] 16 responses avg_len=306 chars
[88/100] 16 responses avg_len=306 chars
[89/100] 16 responses avg_len=1197 chars
[90/100] 16 responses avg_len=689 chars
[91/100] 16 responses avg_len=597 chars
[92/100] 16 responses avg_len=677 chars
[93/100] 16 responses avg_len=165 chars
[94/100] 16 responses avg_len=274 chars
[95/100] 16 responses avg_len=416 chars
[96/100] 16 responses avg_len=287 chars
[97/100] 16 responses avg_len=308 chars
[98/100] 16 responses avg_len=62 chars
[99/100] 16 responses avg_len=69 chars
[100/100] 16 responses avg_len=129 chars
Saved 100 records to evals/dpo_default_n100_r4.jsonl
Skipping import of cpp extensions due to incompatible torch version. Please upgrade to torch >= 2.11.0 (found 2.10.0+cu128).
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Loading SBERT model: sentence-transformers/all-MiniLM-L6-v2 ...
Loading weights: 0%| | 0/103 [00:00<?, ?it/s] Loading weights: 46%|████▌ | 47/103 [00:00<00:00, 409.92it/s] Loading weights: 91%|█████████▏| 94/103 [00:00<00:00, 414.36it/s] Loading weights: 100%|██████████| 103/103 [00:00<00:00, 450.62it/s]
BertModel LOAD REPORT from: sentence-transformers/all-MiniLM-L6-v2
Key | Status | |
------------------------+------------+--+-
embeddings.position_ids | UNEXPECTED | |
Notes:
- UNEXPECTED: can be ignored when loading from different task/architecture; not ok if you expect identical arch.
Scoring 100 records ...
[1/100] ead=0.2215 sbert=0.4202 vendi=4.7004
[2/100] ead=0.1539 sbert=0.3031 vendi=3.0089
[3/100] ead=0.1791 sbert=0.3479 vendi=3.7807
[4/100] ead=0.2852 sbert=0.5217 vendi=6.8363
[5/100] ead=0.2914 sbert=0.5311 vendi=6.9277
[6/100] ead=0.0645 sbert=0.1331 vendi=1.4444
[7/100] ead=0.1304 sbert=0.2600 vendi=2.8699
[8/100] ead=0.0776 sbert=0.1590 vendi=1.9956
[9/100] ead=0.0983 sbert=0.1994 vendi=2.3667
[10/100] ead=0.0731 sbert=0.1503 vendi=1.9737
[11/100] ead=0.0827 sbert=0.1691 vendi=2.0416
[12/100] ead=0.0821 sbert=0.1679 vendi=2.0866
[13/100] ead=0.0655 sbert=0.1352 vendi=1.7483
[14/100] ead=0.0807 sbert=0.1652 vendi=2.1535
[15/100] ead=0.1073 sbert=0.2166 vendi=2.4434
[16/100] ead=0.3385 sbert=0.5999 vendi=8.1126
[17/100] ead=0.0539 sbert=0.1119 vendi=1.7672
[18/100] ead=0.3339 sbert=0.5934 vendi=8.0090
[19/100] ead=0.2089 sbert=0.3991 vendi=3.5623
[20/100] ead=0.1009 sbert=0.2043 vendi=2.3543
[21/100] ead=0.0806 sbert=0.1650 vendi=1.9491
[22/100] ead=0.1957 sbert=0.3766 vendi=2.7339
[23/100] ead=0.3114 sbert=0.5609 vendi=5.9426
[24/100] ead=0.1103 sbert=0.2223 vendi=1.7700
[25/100] ead=0.0882 sbert=0.1798 vendi=1.6982
[26/100] ead=0.1494 sbert=0.2950 vendi=3.0703
[27/100] ead=0.1351 sbert=0.2688 vendi=3.2119
[28/100] ead=0.2144 sbert=0.4084 vendi=5.0096
[29/100] ead=0.0538 sbert=0.1117 vendi=1.5520
[30/100] ead=0.0033 sbert=0.0070 vendi=1.0495
[31/100] ead=0.2240 sbert=0.4243 vendi=5.3786
[32/100] ead=0.2644 sbert=0.4895 vendi=5.5140
[33/100] ead=0.1268 sbert=0.2533 vendi=2.8508
[34/100] ead=0.0572 sbert=0.1185 vendi=1.5461
[35/100] ead=0.2483 sbert=0.4640 vendi=4.7925
[36/100] ead=0.0421 sbert=0.0880 vendi=1.5689
[37/100] ead=0.0832 sbert=0.1701 vendi=2.1962
[38/100] ead=0.0302 sbert=0.0634 vendi=1.3600
[39/100] ead=0.0455 sbert=0.0948 vendi=1.6311
[40/100] ead=0.0178 sbert=0.0376 vendi=1.1938
[41/100] ead=0.0478 sbert=0.0996 vendi=1.6916
[42/100] ead=0.0623 sbert=0.1289 vendi=1.8174
[43/100] ead=0.0325 sbert=0.0681 vendi=1.4654
[44/100] ead=0.1723 sbert=0.3359 vendi=3.1290
[45/100] ead=0.1127 sbert=0.2269 vendi=2.2373
[46/100] ead=0.0968 sbert=0.1966 vendi=2.1951
[47/100] ead=0.0870 sbert=0.1775 vendi=2.2215
[48/100] ead=0.1293 sbert=0.2581 vendi=3.1035
[49/100] ead=0.1105 sbert=0.2227 vendi=2.5606
[50/100] ead=0.0722 sbert=0.1485 vendi=2.0503
[51/100] ead=0.0434 sbert=0.0905 vendi=1.6095
[52/100] ead=0.1231 sbert=0.2464 vendi=2.9946
[53/100] ead=0.0189 sbert=0.0399 vendi=1.2115
[54/100] ead=0.2173 sbert=0.4132 vendi=3.7740
[55/100] ead=0.0714 sbert=0.1468 vendi=1.9421
[56/100] ead=0.0795 sbert=0.1629 vendi=2.1548
[57/100] ead=0.0646 sbert=0.1334 vendi=1.8689
[58/100] ead=0.2422 sbert=0.4542 vendi=2.8661
[59/100] ead=0.1103 sbert=0.2224 vendi=2.3809
[60/100] ead=0.0845 sbert=0.1727 vendi=2.2505
[61/100] ead=0.2839 sbert=0.5197 vendi=6.7955
[62/100] ead=0.0455 sbert=0.0948 vendi=1.6370
[63/100] ead=0.0812 sbert=0.1662 vendi=1.9496
[64/100] ead=0.0175 sbert=0.0369 vendi=1.2523
[65/100] ead=0.2150 sbert=0.4094 vendi=2.5259
[66/100] ead=0.1168 sbert=0.2345 vendi=2.2798
[67/100] ead=0.1980 sbert=0.3807 vendi=4.4107
[68/100] ead=0.0632 sbert=0.1306 vendi=1.9128
[69/100] ead=0.0313 sbert=0.0657 vendi=1.3866
[70/100] ead=0.0799 sbert=0.1637 vendi=2.2017
[71/100] ead=0.0780 sbert=0.1600 vendi=2.0474
[72/100] ead=0.2311 sbert=0.4360 vendi=5.4815
[73/100] ead=0.1055 sbert=0.2132 vendi=2.1208
[74/100] ead=0.0918 sbert=0.1869 vendi=1.9718
[75/100] ead=0.0958 sbert=0.1945 vendi=2.1223
[76/100] ead=0.0597 sbert=0.1235 vendi=1.8748
[77/100] ead=0.0671 sbert=0.1384 vendi=1.9527
[78/100] ead=0.1057 sbert=0.2136 vendi=2.4193
[79/100] ead=0.0185 sbert=0.0391 vendi=1.2546
[80/100] ead=0.0971 sbert=0.1972 vendi=2.0882
[81/100] ead=0.1035 sbert=0.2094 vendi=2.3673
[82/100] ead=0.1347 sbert=0.2680 vendi=3.1583
[83/100] ead=0.0241 sbert=0.0508 vendi=1.3213
[84/100] ead=0.0043 sbert=0.0092 vendi=1.0509
[85/100] ead=0.2164 sbert=0.4118 vendi=2.9182
[86/100] ead=0.1350 sbert=0.2685 vendi=2.6217
[87/100] ead=0.0750 sbert=0.1541 vendi=2.0077
[88/100] ead=0.0538 sbert=0.1116 vendi=1.5855
[89/100] ead=0.2717 sbert=0.5008 vendi=6.5359
[90/100] ead=0.0692 sbert=0.1426 vendi=2.0181
[91/100] ead=0.2908 sbert=0.5302 vendi=6.5217
[92/100] ead=0.1777 sbert=0.3454 vendi=3.8662
[93/100] ead=0.2116 sbert=0.4037 vendi=3.1923
[94/100] ead=0.0679 sbert=0.1400 vendi=1.8048
[95/100] ead=0.0586 sbert=0.1213 vendi=1.6027
[96/100] ead=0.0668 sbert=0.1378 vendi=1.7351
[97/100] ead=0.1519 sbert=0.2994 vendi=3.0876
[98/100] ead=0.2075 sbert=0.3967 vendi=2.8306
[99/100] ead=0.0000 sbert=0.0000 vendi=1.0000
[100/100] ead=0.0407 sbert=0.0850 vendi=1.4648
--- summary ---
ead mean=0.1193 min=0.0000 max=0.3385
sbert mean=0.2322 min=0.0000 max=0.5999
vendi mean=2.7410 min=1.0000 max=8.1126
Saved to evals/dpo_default_n100_r4_diversity.jsonl
==========================================
Done. Compare the two diversity summaries:
baseline: evals/baseline_smollm_135M_neuraltxt_v1_n100_r4.jsonl
dpo_default: evals/dpo_default_n100_r4.jsonl
A large drop in SBERT/Vendi => mode collapse.
Merged model: models/dpo_default/merged
==========================================