Files
SmolLM-135M-neuraltxt-dpo-v1/logs/dpo_20260601_0917.log
ModelHub XC 62b47dc323 初始化项目,由ModelHub XC社区提供模型
Model: JaydeepR/SmolLM-135M-neuraltxt-dpo-v1
Source: Original Platform
2026-08-02 05:49:18 +08:00

2885 lines
553 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

==========================================
method=dpo run_id=dpo_default
base=paperbd/smollm_135M_neuraltxt_v1
dataset=paperbd/paper_preference_150K-v1
batch=8 grad_accum=16 (eff ~128)
==========================================
== uv sync ==
Resolved 148 packages in 5ms
Checked 130 packages in 418ms
== Step 0: skipped (RUN_BASELINE=0) ==
== Step 1: dpo training ==
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
Unsloth: Your Flash Attention 2 installation seems to be broken. Using Xformers instead. No performance changes will be seen.
🦥 Unsloth Zoo will now patch everything to make training faster!
Method: DPO
Base model: paperbd/smollm_135M_neuraltxt_v1
Dataset: paperbd/paper_preference_150K-v1
==((====))== Unsloth 2026.5.9: Fast Llama patching. Transformers: 5.5.0.
\\ /| NVIDIA GeForce RTX 3090. Num GPUs = 1. Max memory: 23.559 GB. Platform: Linux.
O^O/ \_/ \ Torch: 2.10.0+cu128. CUDA: 8.6. CUDA Toolkit: 12.8. Triton: 3.6.0
\ / Bfloat16 = TRUE. FA [Xformers = 0.0.35. FA2 = False]
"-____-" Free license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
Loading weights: 0%| | 0/272 [00:00<?, ?it/s]
Loading weights: 0%| | 1/272 [00:00<01:06, 4.07it/s]
Loading weights: 1%| | 3/272 [00:00<00:46, 5.83it/s]
Loading weights: 8%|▊ | 23/272 [00:00<00:04, 51.42it/s]
Loading weights: 16%|█▌ | 43/272 [00:00<00:02, 88.67it/s]
Loading weights: 23%|██▎ | 63/272 [00:00<00:01, 117.62it/s]
Loading weights: 31%|███ | 84/272 [00:00<00:01, 141.77it/s]
Loading weights: 39%|███▉ | 107/272 [00:01<00:00, 165.90it/s]
Loading weights: 48%|████▊ | 131/272 [00:01<00:00, 186.38it/s]
Loading weights: 58%|█████▊ | 157/272 [00:01<00:00, 206.87it/s]
Loading weights: 67%|██████▋ | 183/272 [00:01<00:00, 216.32it/s]
Loading weights: 76%|███████▌ | 206/272 [00:01<00:00, 199.31it/s]
Loading weights: 83%|████████▎ | 227/272 [00:01<00:00, 195.80it/s]
Loading weights: 91%|█████████ | 248/272 [00:01<00:00, 179.27it/s]
Loading weights: 100%|██████████| 272/272 [00:01<00:00, 147.70it/s]
Unsloth: Will map <|im_end|> to EOS = <|im_end|>.
Unsloth 2026.5.9 patched 30 layers with 30 QKV layers, 30 O layers and 30 MLP layers.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
Tokenizing train dataset (num_proc=64): 0%| | 0/117600 [00:00<?, ? examples/s]
Tokenizing train dataset (num_proc=64): 0%| | 22/117600 [00:03<4:47:59, 6.80 examples/s]
Tokenizing train dataset (num_proc=64): 0%| | 148/117600 [00:03<32:59, 59.33 examples/s]
Tokenizing train dataset (num_proc=64): 0%| | 347/117600 [00:03<11:42, 166.96 examples/s]
Tokenizing train dataset (num_proc=64): 1%| | 780/117600 [00:03<04:08, 470.67 examples/s]
Tokenizing train dataset (num_proc=64): 1%| | 1462/117600 [00:03<01:48, 1069.04 examples/s]
Tokenizing train dataset (num_proc=64): 2%|▏ | 2474/117600 [00:03<00:54, 2111.82 examples/s]
Tokenizing train dataset (num_proc=64): 3%|▎ | 3639/117600 [00:03<00:33, 3432.29 examples/s]
Tokenizing train dataset (num_proc=64): 4%|▍ | 5143/117600 [00:04<00:21, 5282.21 examples/s]
Tokenizing train dataset (num_proc=64): 6%|▌ | 6825/117600 [00:04<00:15, 7306.57 examples/s]
Tokenizing train dataset (num_proc=64): 7%|▋ | 8586/117600 [00:04<00:11, 9211.19 examples/s]
Tokenizing train dataset (num_proc=64): 9%|▉ | 10716/117600 [00:04<00:09, 11377.87 examples/s]
Tokenizing train dataset (num_proc=64): 10%|█ | 12137/117600 [00:04<00:08, 12036.98 examples/s]
Tokenizing train dataset (num_proc=64): 12%|█▏ | 13754/117600 [00:04<00:08, 12292.25 examples/s]
Tokenizing train dataset (num_proc=64): 13%|█▎ | 15227/117600 [00:04<00:08, 11921.56 examples/s]
Tokenizing train dataset (num_proc=64): 14%|█▍ | 16688/117600 [00:04<00:08, 12451.34 examples/s]
Tokenizing train dataset (num_proc=64): 15%|█▌ | 18043/117600 [00:04<00:07, 12645.89 examples/s]
Tokenizing train dataset (num_proc=64): 17%|█▋ | 19423/117600 [00:05<00:07, 12668.27 examples/s]
Tokenizing train dataset (num_proc=64): 18%|█▊ | 20782/117600 [00:05<00:07, 12260.18 examples/s]
Tokenizing train dataset (num_proc=64): 19%|█▊ | 22040/117600 [00:05<00:07, 12330.93 examples/s]
Tokenizing train dataset (num_proc=64): 20%|█▉ | 23302/117600 [00:05<00:09, 9851.69 examples/s]
Tokenizing train dataset (num_proc=64): 21%|██ | 24699/117600 [00:05<00:10, 9103.06 examples/s]
Tokenizing train dataset (num_proc=64): 22%|██▏ | 25699/117600 [00:05<00:10, 8665.93 examples/s]
Tokenizing train dataset (num_proc=64): 23%|██▎ | 27321/117600 [00:05<00:10, 8799.02 examples/s]
Tokenizing train dataset (num_proc=64): 25%|██▍ | 29095/117600 [00:06<00:09, 9167.80 examples/s]
Tokenizing train dataset (num_proc=64): 26%|██▌ | 30038/117600 [00:06<00:09, 8817.06 examples/s]
Tokenizing train dataset (num_proc=64): 30%|██▉ | 35077/117600 [00:06<00:04, 18054.95 examples/s]
Tokenizing train dataset (num_proc=64): 34%|███▍ | 40286/117600 [00:06<00:02, 26251.45 examples/s]
Tokenizing train dataset (num_proc=64): 37%|███▋ | 43355/117600 [00:06<00:03, 21808.34 examples/s]
Tokenizing train dataset (num_proc=64): 39%|███▉ | 45958/117600 [00:06<00:03, 18784.22 examples/s]
Tokenizing train dataset (num_proc=64): 41%|████ | 48188/117600 [00:06<00:03, 19334.06 examples/s]
Tokenizing train dataset (num_proc=64): 43%|████▎ | 50396/117600 [00:07<00:03, 19834.56 examples/s]
Tokenizing train dataset (num_proc=64): 45%|████▍ | 52599/117600 [00:07<00:03, 16618.19 examples/s]
Tokenizing train dataset (num_proc=64): 46%|████▋ | 54538/117600 [00:07<00:03, 17228.98 examples/s]
Tokenizing train dataset (num_proc=64): 48%|████▊ | 56434/117600 [00:07<00:03, 17513.71 examples/s]
Tokenizing train dataset (num_proc=64): 50%|████▉ | 58386/117600 [00:07<00:03, 18020.73 examples/s]
Tokenizing train dataset (num_proc=64): 51%|█████▏ | 60290/117600 [00:07<00:03, 17977.29 examples/s]
Tokenizing train dataset (num_proc=64): 53%|█████▎ | 62177/117600 [00:07<00:03, 18194.95 examples/s]
Tokenizing train dataset (num_proc=64): 54%|█████▍ | 64075/117600 [00:07<00:03, 14800.11 examples/s]
Tokenizing train dataset (num_proc=64): 56%|█████▌ | 65825/117600 [00:08<00:03, 15407.31 examples/s]
Tokenizing train dataset (num_proc=64): 57%|█████▋ | 67479/117600 [00:08<00:03, 15598.71 examples/s]
Tokenizing train dataset (num_proc=64): 59%|█████▉ | 69675/117600 [00:08<00:02, 17298.65 examples/s]
Tokenizing train dataset (num_proc=64): 61%|██████ | 71493/117600 [00:08<00:02, 16907.54 examples/s]
Tokenizing train dataset (num_proc=64): 63%|██████▎ | 73550/117600 [00:08<00:02, 17285.60 examples/s]
Tokenizing train dataset (num_proc=64): 64%|██████▍ | 75319/117600 [00:08<00:02, 14467.41 examples/s]
Tokenizing train dataset (num_proc=64): 65%|██████▌ | 76880/117600 [00:08<00:02, 14465.11 examples/s]
Tokenizing train dataset (num_proc=64): 67%|██████▋ | 79303/117600 [00:08<00:02, 16971.11 examples/s]
Tokenizing train dataset (num_proc=64): 69%|██████▉ | 81103/117600 [00:09<00:02, 13995.97 examples/s]
Tokenizing train dataset (num_proc=64): 71%|███████ | 83301/117600 [00:09<00:02, 15885.74 examples/s]
Tokenizing train dataset (num_proc=64): 72%|███████▏ | 85054/117600 [00:09<00:02, 16077.23 examples/s]
Tokenizing train dataset (num_proc=64): 74%|███████▍ | 86801/117600 [00:09<00:01, 16341.09 examples/s]
Tokenizing train dataset (num_proc=64): 75%|███████▌ | 88523/117600 [00:09<00:01, 16559.18 examples/s]
Tokenizing train dataset (num_proc=64): 77%|███████▋ | 90599/117600 [00:09<00:01, 16250.03 examples/s]
Tokenizing train dataset (num_proc=64): 79%|███████▊ | 92460/117600 [00:09<00:01, 16877.10 examples/s]
Tokenizing train dataset (num_proc=64): 80%|████████ | 94191/117600 [00:09<00:01, 16783.68 examples/s]
Tokenizing train dataset (num_proc=64): 82%|████████▏ | 96019/117600 [00:09<00:01, 17200.72 examples/s]
Tokenizing train dataset (num_proc=64): 83%|████████▎ | 97766/117600 [00:09<00:01, 16754.61 examples/s]
Tokenizing train dataset (num_proc=64): 85%|████████▌ | 100029/117600 [00:10<00:00, 18416.41 examples/s]
Tokenizing train dataset (num_proc=64): 87%|████████▋ | 101924/117600 [00:10<00:00, 16956.50 examples/s]
Tokenizing train dataset (num_proc=64): 88%|████████▊ | 103663/117600 [00:10<00:00, 16863.61 examples/s]
Tokenizing train dataset (num_proc=64): 90%|████████▉ | 105377/117600 [00:10<00:00, 15894.52 examples/s]
Tokenizing train dataset (num_proc=64): 91%|█████████ | 107162/117600 [00:10<00:00, 16425.19 examples/s]
Tokenizing train dataset (num_proc=64): 93%|█████████▎| 108855/117600 [00:10<00:00, 16506.49 examples/s]
Tokenizing train dataset (num_proc=64): 94%|█████████▍| 110730/117600 [00:10<00:00, 17106.29 examples/s]
Tokenizing train dataset (num_proc=64): 96%|█████████▌| 112464/117600 [00:10<00:00, 15181.05 examples/s]
Tokenizing train dataset (num_proc=64): 97%|█████████▋| 114056/117600 [00:11<00:00, 15324.58 examples/s]
Tokenizing train dataset (num_proc=64): 98%|█████████▊| 115637/117600 [00:11<00:00, 13236.85 examples/s]
Tokenizing train dataset (num_proc=64): 100%|█████████▉| 117035/117600 [00:11<00:00, 11759.07 examples/s]
Tokenizing train dataset (num_proc=64): 100%|█████████▉| 117553/117600 [00:23<00:00, 11759.07 examples/s]
Tokenizing train dataset (num_proc=64): 100%|█████████▉| 117581/117600 [00:31<00:00, 212.82 examples/s]
Tokenizing train dataset (num_proc=64): 100%|█████████▉| 117599/117600 [00:33<00:00, 177.67 examples/s]
Tokenizing train dataset (num_proc=64): 100%|██████████| 117600/117600 [00:37<00:00, 3171.84 examples/s]
Tokenizing eval dataset (num_proc=64): 0%| | 0/2400 [00:00<?, ? examples/s]
Tokenizing eval dataset (num_proc=64): 1%| | 23/2400 [00:03<06:04, 6.53 examples/s]
Tokenizing eval dataset (num_proc=64): 4%|▍ | 103/2400 [00:03<01:01, 37.19 examples/s]
Tokenizing eval dataset (num_proc=64): 8%|▊ | 203/2400 [00:03<00:25, 87.05 examples/s]
Tokenizing eval dataset (num_proc=64): 13%|█▎ | 304/2400 [00:03<00:14, 149.51 examples/s]
Tokenizing eval dataset (num_proc=64): 18%|█▊ | 428/2400 [00:03<00:08, 244.40 examples/s]
Tokenizing eval dataset (num_proc=64): 23%|██▎ | 557/2400 [00:04<00:05, 356.08 examples/s]
Tokenizing eval dataset (num_proc=64): 28%|██▊ | 672/2400 [00:04<00:03, 460.08 examples/s]
Tokenizing eval dataset (num_proc=64): 32%|███▏ | 773/2400 [00:04<00:03, 541.31 examples/s]
Tokenizing eval dataset (num_proc=64): 36%|███▋ | 874/2400 [00:04<00:02, 613.89 examples/s]
Tokenizing eval dataset (num_proc=64): 42%|████▏ | 1000/2400 [00:04<00:01, 725.76 examples/s]
Tokenizing eval dataset (num_proc=64): 47%|████▋ | 1119/2400 [00:04<00:01, 813.13 examples/s]
Tokenizing eval dataset (num_proc=64): 52%|█████▏ | 1242/2400 [00:04<00:01, 901.11 examples/s]
Tokenizing eval dataset (num_proc=64): 56%|█████▋ | 1352/2400 [00:04<00:01, 858.55 examples/s]
Tokenizing eval dataset (num_proc=64): 61%|██████ | 1463/2400 [00:04<00:01, 916.66 examples/s]
Tokenizing eval dataset (num_proc=64): 67%|██████▋ | 1599/2400 [00:05<00:00, 950.68 examples/s]
Tokenizing eval dataset (num_proc=64): 72%|███████▏ | 1724/2400 [00:05<00:00, 1025.60 examples/s]
Tokenizing eval dataset (num_proc=64): 76%|███████▋ | 1833/2400 [00:05<00:00, 1038.31 examples/s]
Tokenizing eval dataset (num_proc=64): 82%|████████▏ | 1970/2400 [00:05<00:00, 1037.88 examples/s]
Tokenizing eval dataset (num_proc=64): 88%|████████▊ | 2104/2400 [00:05<00:00, 1107.38 examples/s]
Tokenizing eval dataset (num_proc=64): 93%|█████████▎| 2225/2400 [00:05<00:00, 1102.42 examples/s]
Tokenizing eval dataset (num_proc=64): 98%|█████████▊| 2349/2400 [00:05<00:00, 1102.17 examples/s]
Tokenizing eval dataset (num_proc=64): 100%|██████████| 2400/2400 [00:06<00:00, 363.13 examples/s]
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': 2}.
==((====))== Unsloth - 2x faster free finetuning | Num GPUs used = 1
\\ /| Num examples = 117,600 | Num Epochs = 3 | Total steps = 2,757
O^O/ \_/ \ Batch size per device = 8 | Gradient accumulation steps = 16
\ / Data Parallel GPUs = 1 | Total batch size (8 x 16 x 1) = 128
"-____-" Trainable parameters = 9,768,960 of 144,283,968 (6.77% trained)
0%| | 0/2757 [00:00<?, ?it/s]`use_return_dict` is deprecated! Use `return_dict` instead!
0%| | 1/2757 [00:18<14:04:38, 18.39s/it]
0%| | 2/2757 [00:26<9:35:41, 12.54s/it]
0%| | 3/2757 [00:35<8:04:08, 10.55s/it]
0%| | 4/2757 [00:42<7:14:13, 9.46s/it]
0%| | 5/2757 [00:50<6:49:58, 8.94s/it]
0%| | 6/2757 [00:58<6:35:17, 8.62s/it]
0%| | 7/2757 [01:06<6:25:30, 8.41s/it]
0%| | 8/2757 [01:15<6:24:42, 8.40s/it]
0%| | 9/2757 [01:23<6:25:05, 8.41s/it]
0%| | 10/2757 [01:31<6:14:47, 8.19s/it]
0%| | 10/2757 [01:31<6:14:47, 8.19s/it]
0%| | 11/2757 [01:39<6:16:32, 8.23s/it]
0%| | 12/2757 [01:47<6:14:45, 8.19s/it]
0%| | 13/2757 [01:55<6:13:45, 8.17s/it]
1%| | 14/2757 [02:04<6:13:33, 8.17s/it]
1%| | 15/2757 [02:11<6:08:16, 8.06s/it]
1%| | 16/2757 [02:19<6:06:42, 8.03s/it]
1%| | 17/2757 [02:27<6:07:00, 8.04s/it]
1%| | 18/2757 [02:35<6:06:41, 8.03s/it]
1%| | 19/2757 [02:43<6:04:08, 7.98s/it]
1%| | 20/2757 [02:51<6:03:17, 7.96s/it]
1%| | 20/2757 [02:51<6:03:17, 7.96s/it]
1%| | 21/2757 [02:59<6:03:00, 7.96s/it]
1%| | 22/2757 [03:07<6:07:19, 8.06s/it]
1%| | 23/2757 [03:16<6:08:49, 8.09s/it]
1%| | 24/2757 [03:23<6:04:23, 8.00s/it]
1%| | 25/2757 [03:31<5:58:22, 7.87s/it]
1%| | 26/2757 [03:40<6:08:21, 8.09s/it]
1%| | 27/2757 [03:47<6:05:53, 8.04s/it]
1%| | 28/2757 [03:56<6:07:42, 8.08s/it]
1%| | 29/2757 [04:04<6:10:32, 8.15s/it]
1%| | 30/2757 [04:11<6:01:58, 7.96s/it]
1%| | 30/2757 [04:11<6:01:58, 7.96s/it]
1%| | 31/2757 [04:19<6:01:21, 7.95s/it]
1%| | 32/2757 [04:27<6:01:29, 7.96s/it]
1%| | 33/2757 [04:36<6:03:54, 8.02s/it]
1%| | 34/2757 [04:43<6:03:26, 8.01s/it]
1%|▏ | 35/2757 [04:52<6:07:57, 8.11s/it]
1%|▏ | 36/2757 [05:00<6:03:40, 8.02s/it]
1%|▏ | 37/2757 [05:08<6:01:54, 7.98s/it]
1%|▏ | 38/2757 [05:16<6:08:12, 8.13s/it]
1%|▏ | 39/2757 [05:24<6:07:04, 8.10s/it]
1%|▏ | 40/2757 [05:32<6:00:19, 7.96s/it]
1%|▏ | 40/2757 [05:32<6:00:19, 7.96s/it]
1%|▏ | 41/2757 [05:39<5:54:11, 7.82s/it]
2%|▏ | 42/2757 [05:47<5:58:43, 7.93s/it]
2%|▏ | 43/2757 [05:55<5:55:50, 7.87s/it]
2%|▏ | 44/2757 [06:03<5:55:02, 7.85s/it]
2%|▏ | 45/2757 [06:11<5:58:09, 7.92s/it]
2%|▏ | 46/2757 [06:19<6:05:15, 8.08s/it]
2%|▏ | 47/2757 [06:27<6:01:18, 8.00s/it]
2%|▏ | 48/2757 [06:35<6:04:18, 8.07s/it]
2%|▏ | 49/2757 [06:43<6:01:21, 8.01s/it]
2%|▏ | 50/2757 [06:51<5:57:05, 7.91s/it]
2%|▏ | 50/2757 [06:51<5:57:05, 7.91s/it]
2%|▏ | 51/2757 [06:59<5:55:45, 7.89s/it]
2%|▏ | 52/2757 [07:07<6:04:10, 8.08s/it]
2%|▏ | 53/2757 [07:15<5:55:19, 7.88s/it]
2%|▏ | 54/2757 [07:23<6:03:22, 8.07s/it]
2%|▏ | 55/2757 [07:32<6:07:54, 8.17s/it]
2%|▏ | 56/2757 [07:39<5:59:55, 8.00s/it]
2%|▏ | 57/2757 [07:48<6:06:46, 8.15s/it]
2%|▏ | 58/2757 [07:56<6:05:33, 8.13s/it]
2%|▏ | 59/2757 [08:04<6:08:12, 8.19s/it]
2%|▏ | 60/2757 [08:12<6:04:13, 8.10s/it]
2%|▏ | 60/2757 [08:12<6:04:13, 8.10s/it]
2%|▏ | 61/2757 [08:20<6:02:48, 8.07s/it]
2%|▏ | 62/2757 [08:28<5:59:11, 8.00s/it]
2%|▏ | 63/2757 [08:36<6:02:13, 8.07s/it]
2%|▏ | 64/2757 [08:44<6:00:34, 8.03s/it]
2%|▏ | 65/2757 [08:52<6:00:39, 8.04s/it]
2%|▏ | 66/2757 [09:00<5:52:05, 7.85s/it]
2%|▏ | 67/2757 [09:08<5:53:32, 7.89s/it]
2%|▏ | 68/2757 [09:15<5:52:41, 7.87s/it]
3%|▎ | 69/2757 [09:24<5:57:26, 7.98s/it]
3%|▎ | 70/2757 [09:32<5:56:28, 7.96s/it]
3%|▎ | 70/2757 [09:32<5:56:28, 7.96s/it]
3%|▎ | 71/2757 [09:39<5:55:12, 7.93s/it]
3%|▎ | 72/2757 [09:47<5:54:07, 7.91s/it]
3%|▎ | 73/2757 [09:55<5:56:56, 7.98s/it]
3%|▎ | 74/2757 [10:03<5:55:35, 7.95s/it]
3%|▎ | 75/2757 [10:11<5:54:36, 7.93s/it]
3%|▎ | 76/2757 [10:19<5:52:56, 7.90s/it]
3%|▎ | 77/2757 [10:27<5:48:11, 7.80s/it]
3%|▎ | 78/2757 [10:34<5:47:47, 7.79s/it]
3%|▎ | 79/2757 [10:42<5:51:32, 7.88s/it]
3%|▎ | 80/2757 [10:50<5:49:30, 7.83s/it]
3%|▎ | 80/2757 [10:50<5:49:30, 7.83s/it]
3%|▎ | 81/2757 [10:58<5:47:39, 7.79s/it]
3%|▎ | 82/2757 [11:06<5:52:02, 7.90s/it]
3%|▎ | 83/2757 [11:14<5:52:07, 7.90s/it]
3%|▎ | 84/2757 [11:22<5:52:55, 7.92s/it]
3%|▎ | 85/2757 [11:30<5:52:10, 7.91s/it]
3%|▎ | 86/2757 [11:38<5:50:13, 7.87s/it]
3%|▎ | 87/2757 [11:45<5:45:48, 7.77s/it]
3%|▎ | 88/2757 [11:53<5:52:14, 7.92s/it]
3%|▎ | 89/2757 [12:02<5:59:16, 8.08s/it]
3%|▎ | 90/2757 [12:10<6:03:47, 8.18s/it]
3%|▎ | 90/2757 [12:10<6:03:47, 8.18s/it]
3%|▎ | 91/2757 [12:18<5:58:38, 8.07s/it]
3%|▎ | 92/2757 [12:26<5:56:40, 8.03s/it]
3%|▎ | 93/2757 [12:34<5:57:56, 8.06s/it]
3%|▎ | 94/2757 [12:42<6:02:15, 8.16s/it]
3%|▎ | 95/2757 [12:50<5:57:54, 8.07s/it]
3%|▎ | 96/2757 [12:58<5:56:07, 8.03s/it]
4%|▎ | 97/2757 [13:06<5:57:03, 8.05s/it]
4%|▎ | 98/2757 [13:14<5:57:09, 8.06s/it]
4%|▎ | 99/2757 [13:22<5:54:14, 8.00s/it]
4%|▎ | 100/2757 [13:30<5:53:32, 7.98s/it]
4%|▎ | 100/2757 [13:30<5:53:32, 7.98s/it]
4%|▎ | 101/2757 [13:38<5:50:59, 7.93s/it]
4%|▎ | 102/2757 [13:46<5:48:13, 7.87s/it]
4%|▎ | 103/2757 [13:54<5:54:47, 8.02s/it]
4%|▍ | 104/2757 [14:02<5:57:02, 8.07s/it]
4%|▍ | 105/2757 [14:10<5:53:03, 7.99s/it]
4%|▍ | 106/2757 [14:18<5:54:04, 8.01s/it]
4%|▍ | 107/2757 [14:26<5:55:47, 8.06s/it]
4%|▍ | 108/2757 [14:35<5:58:09, 8.11s/it]
4%|▍ | 109/2757 [14:43<5:58:13, 8.12s/it]
4%|▍ | 110/2757 [14:51<5:56:32, 8.08s/it]
4%|▍ | 110/2757 [14:51<5:56:32, 8.08s/it]
4%|▍ | 111/2757 [14:58<5:46:26, 7.86s/it]
4%|▍ | 112/2757 [15:06<5:48:07, 7.90s/it]
4%|▍ | 113/2757 [15:14<5:50:11, 7.95s/it]
4%|▍ | 114/2757 [15:22<5:43:15, 7.79s/it]
4%|▍ | 115/2757 [15:30<5:47:38, 7.90s/it]
4%|▍ | 116/2757 [15:37<5:44:38, 7.83s/it]
4%|▍ | 117/2757 [15:45<5:46:51, 7.88s/it]
4%|▍ | 118/2757 [15:53<5:45:31, 7.86s/it]
4%|▍ | 119/2757 [16:01<5:43:45, 7.82s/it]
4%|▍ | 120/2757 [16:09<5:43:17, 7.81s/it]
4%|▍ | 120/2757 [16:09<5:43:17, 7.81s/it]
4%|▍ | 121/2757 [16:17<5:43:40, 7.82s/it]
4%|▍ | 122/2757 [16:25<5:48:11, 7.93s/it]
4%|▍ | 123/2757 [16:32<5:44:34, 7.85s/it]
4%|▍ | 124/2757 [16:40<5:45:00, 7.86s/it]
5%|▍ | 125/2757 [16:48<5:39:24, 7.74s/it]
5%|▍ | 126/2757 [16:56<5:41:44, 7.79s/it]
5%|▍ | 127/2757 [17:04<5:43:58, 7.85s/it]
5%|▍ | 128/2757 [17:11<5:43:42, 7.84s/it]
5%|▍ | 129/2757 [17:20<5:50:34, 8.00s/it]
5%|▍ | 130/2757 [17:28<5:49:17, 7.98s/it]
5%|▍ | 130/2757 [17:28<5:49:17, 7.98s/it]
5%|▍ | 131/2757 [17:36<5:52:59, 8.07s/it]
5%|▍ | 132/2757 [17:44<5:47:31, 7.94s/it]
5%|▍ | 133/2757 [17:52<5:50:43, 8.02s/it]
5%|▍ | 134/2757 [18:00<5:56:00, 8.14s/it]
5%|▍ | 135/2757 [18:08<5:56:06, 8.15s/it]
5%|▍ | 136/2757 [18:16<5:53:39, 8.10s/it]
5%|▍ | 137/2757 [18:25<5:58:56, 8.22s/it]
5%|▌ | 138/2757 [18:33<5:53:32, 8.10s/it]
5%|▌ | 139/2757 [18:41<5:51:06, 8.05s/it]
5%|▌ | 140/2757 [18:49<5:48:37, 7.99s/it]
5%|▌ | 140/2757 [18:49<5:48:37, 7.99s/it]
5%|▌ | 141/2757 [18:56<5:46:16, 7.94s/it]
5%|▌ | 142/2757 [19:04<5:47:20, 7.97s/it]
5%|▌ | 143/2757 [19:13<5:48:47, 8.01s/it]
5%|▌ | 144/2757 [19:21<5:51:13, 8.06s/it]
5%|▌ | 145/2757 [19:29<5:54:24, 8.14s/it]
5%|▌ | 146/2757 [19:37<5:57:37, 8.22s/it]
5%|▌ | 147/2757 [19:45<5:54:26, 8.15s/it]
5%|▌ | 148/2757 [19:53<5:52:32, 8.11s/it]
5%|▌ | 149/2757 [20:01<5:50:33, 8.07s/it]
5%|▌ | 150/2757 [20:09<5:48:17, 8.02s/it]
5%|▌ | 150/2757 [20:09<5:48:17, 8.02s/it]
5%|▌ | 151/2757 [20:18<5:53:52, 8.15s/it]
6%|▌ | 152/2757 [20:26<5:54:13, 8.16s/it]
6%|▌ | 153/2757 [20:34<5:46:55, 7.99s/it]
6%|▌ | 154/2757 [20:41<5:45:24, 7.96s/it]
6%|▌ | 155/2757 [20:50<5:48:40, 8.04s/it]
6%|▌ | 156/2757 [20:58<5:52:52, 8.14s/it]
6%|▌ | 157/2757 [21:06<5:45:25, 7.97s/it]
6%|▌ | 158/2757 [21:14<5:47:00, 8.01s/it]
6%|▌ | 159/2757 [21:21<5:42:50, 7.92s/it]
6%|▌ | 160/2757 [21:30<5:50:19, 8.09s/it]
6%|▌ | 160/2757 [21:30<5:50:19, 8.09s/it]
6%|▌ | 161/2757 [21:38<5:43:46, 7.95s/it]
6%|▌ | 162/2757 [21:46<5:49:53, 8.09s/it]
6%|▌ | 163/2757 [21:54<5:53:17, 8.17s/it]
6%|▌ | 164/2757 [22:03<5:53:42, 8.18s/it]
6%|▌ | 165/2757 [22:11<5:51:56, 8.15s/it]
6%|▌ | 166/2757 [22:18<5:44:45, 7.98s/it]
6%|▌ | 167/2757 [22:26<5:41:45, 7.92s/it]
6%|▌ | 168/2757 [22:34<5:42:09, 7.93s/it]
6%|▌ | 169/2757 [22:42<5:49:06, 8.09s/it]
6%|▌ | 170/2757 [22:50<5:48:26, 8.08s/it]
6%|▌ | 170/2757 [22:50<5:48:26, 8.08s/it]
6%|▌ | 171/2757 [22:59<5:48:44, 8.09s/it]
6%|▌ | 172/2757 [23:07<5:49:53, 8.12s/it]
6%|▋ | 173/2757 [23:15<5:45:13, 8.02s/it]
6%|▋ | 174/2757 [23:22<5:39:45, 7.89s/it]
6%|▋ | 175/2757 [23:30<5:41:57, 7.95s/it]
6%|▋ | 176/2757 [23:38<5:45:40, 8.04s/it]
6%|▋ | 177/2757 [23:46<5:44:38, 8.01s/it]
6%|▋ | 178/2757 [23:55<5:46:14, 8.06s/it]
6%|▋ | 179/2757 [24:02<5:42:59, 7.98s/it]
7%|▋ | 180/2757 [24:11<5:47:18, 8.09s/it]
7%|▋ | 180/2757 [24:11<5:47:18, 8.09s/it]
7%|▋ | 181/2757 [24:19<5:45:26, 8.05s/it]
7%|▋ | 182/2757 [24:27<5:44:17, 8.02s/it]
7%|▋ | 183/2757 [24:35<5:42:33, 7.99s/it]
7%|▋ | 184/2757 [24:42<5:41:50, 7.97s/it]
7%|▋ | 185/2757 [24:51<5:43:10, 8.01s/it]
7%|▋ | 186/2757 [24:58<5:42:01, 7.98s/it]
7%|▋ | 187/2757 [25:07<5:44:24, 8.04s/it]
7%|▋ | 188/2757 [25:15<5:47:33, 8.12s/it]
7%|▋ | 189/2757 [25:23<5:50:12, 8.18s/it]
7%|▋ | 190/2757 [25:31<5:46:29, 8.10s/it]
7%|▋ | 190/2757 [25:31<5:46:29, 8.10s/it]
7%|▋ | 191/2757 [25:39<5:41:36, 7.99s/it]
7%|▋ | 192/2757 [25:47<5:37:46, 7.90s/it]
7%|▋ | 193/2757 [25:55<5:38:33, 7.92s/it]
7%|▋ | 194/2757 [26:03<5:38:24, 7.92s/it]
7%|▋ | 195/2757 [26:11<5:43:54, 8.05s/it]
7%|▋ | 196/2757 [26:19<5:40:39, 7.98s/it]
7%|▋ | 197/2757 [26:27<5:43:10, 8.04s/it]
7%|▋ | 198/2757 [26:35<5:43:13, 8.05s/it]
7%|▋ | 199/2757 [26:43<5:46:33, 8.13s/it]
7%|▋ | 200/2757 [26:51<5:47:05, 8.14s/it]
7%|▋ | 200/2757 [26:51<5:47:05, 8.14s/it]
7%|▋ | 201/2757 [27:00<5:52:34, 8.28s/it]
7%|▋ | 202/2757 [27:08<5:46:34, 8.14s/it]
7%|▋ | 203/2757 [27:16<5:41:01, 8.01s/it]
7%|▋ | 204/2757 [27:24<5:46:03, 8.13s/it]
7%|▋ | 205/2757 [27:32<5:43:59, 8.09s/it]
7%|▋ | 206/2757 [27:40<5:38:35, 7.96s/it]
8%|▊ | 207/2757 [27:48<5:44:00, 8.09s/it]
8%|▊ | 208/2757 [27:56<5:43:58, 8.10s/it]
8%|▊ | 209/2757 [28:04<5:38:30, 7.97s/it]
8%|▊ | 210/2757 [28:12<5:38:50, 7.98s/it]
8%|▊ | 210/2757 [28:12<5:38:50, 7.98s/it]
8%|▊ | 211/2757 [28:20<5:37:48, 7.96s/it]
8%|▊ | 212/2757 [28:28<5:46:09, 8.16s/it]
8%|▊ | 213/2757 [28:36<5:41:16, 8.05s/it]
8%|▊ | 214/2757 [28:45<5:50:48, 8.28s/it]
8%|▊ | 215/2757 [28:53<5:51:39, 8.30s/it]
8%|▊ | 216/2757 [29:01<5:46:50, 8.19s/it]
8%|▊ | 217/2757 [29:09<5:47:46, 8.21s/it]
8%|▊ | 218/2757 [29:18<5:52:22, 8.33s/it]
8%|▊ | 219/2757 [29:26<5:47:59, 8.23s/it]
8%|▊ | 220/2757 [29:34<5:44:29, 8.15s/it]
8%|▊ | 220/2757 [29:34<5:44:29, 8.15s/it]
8%|▊ | 221/2757 [29:42<5:44:06, 8.14s/it]
8%|▊ | 222/2757 [29:50<5:44:42, 8.16s/it]
8%|▊ | 223/2757 [29:58<5:43:59, 8.15s/it]
8%|▊ | 224/2757 [30:06<5:38:55, 8.03s/it]
8%|▊ | 225/2757 [30:14<5:31:42, 7.86s/it]
8%|▊ | 226/2757 [30:21<5:29:38, 7.81s/it]
8%|▊ | 227/2757 [30:30<5:34:35, 7.93s/it]
8%|▊ | 228/2757 [30:38<5:35:18, 7.96s/it]
8%|▊ | 229/2757 [30:46<5:35:13, 7.96s/it]
8%|▊ | 230/2757 [30:53<5:32:12, 7.89s/it]
8%|▊ | 230/2757 [30:53<5:32:12, 7.89s/it]
8%|▊ | 231/2757 [31:01<5:34:40, 7.95s/it]
8%|▊ | 232/2757 [31:09<5:36:14, 7.99s/it]
8%|▊ | 233/2757 [31:18<5:41:15, 8.11s/it]
8%|▊ | 234/2757 [31:27<5:49:52, 8.32s/it]
9%|▊ | 235/2757 [31:35<5:47:49, 8.27s/it]
9%|▊ | 236/2757 [31:43<5:42:59, 8.16s/it]
9%|▊ | 237/2757 [31:51<5:42:50, 8.16s/it]
9%|▊ | 238/2757 [31:59<5:38:35, 8.06s/it]
9%|▊ | 239/2757 [32:07<5:40:33, 8.12s/it]
9%|▊ | 240/2757 [32:15<5:42:59, 8.18s/it]
9%|▊ | 240/2757 [32:15<5:42:59, 8.18s/it]
9%|▊ | 241/2757 [32:24<5:46:28, 8.26s/it]
9%|▉ | 242/2757 [32:32<5:50:12, 8.35s/it]
9%|▉ | 243/2757 [32:41<5:49:12, 8.33s/it]
9%|▉ | 244/2757 [32:49<5:47:50, 8.31s/it]
9%|▉ | 245/2757 [32:57<5:46:34, 8.28s/it]
9%|▉ | 246/2757 [33:05<5:42:16, 8.18s/it]
9%|▉ | 247/2757 [33:13<5:43:12, 8.20s/it]
9%|▉ | 248/2757 [33:22<5:44:39, 8.24s/it]
9%|▉ | 249/2757 [33:30<5:48:51, 8.35s/it]
9%|▉ | 250/2757 [33:38<5:40:57, 8.16s/it]
9%|▉ | 250/2757 [33:38<5:40:57, 8.16s/it]
9%|▉ | 251/2757 [33:50<6:24:38, 9.21s/it]
9%|▉ | 252/2757 [33:57<6:01:28, 8.66s/it]
9%|▉ | 253/2757 [34:04<5:45:47, 8.29s/it]
9%|▉ | 254/2757 [34:12<5:40:16, 8.16s/it]
9%|▉ | 255/2757 [34:20<5:34:10, 8.01s/it]
9%|▉ | 256/2757 [34:28<5:30:07, 7.92s/it]
9%|▉ | 257/2757 [34:36<5:30:47, 7.94s/it]
9%|▉ | 258/2757 [34:44<5:34:20, 8.03s/it]
9%|▉ | 259/2757 [34:52<5:36:06, 8.07s/it]
9%|▉ | 260/2757 [35:00<5:38:48, 8.14s/it]
Unsloth: Will smartly offload gradients to save VRAM!
Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
{'loss': '0.693', 'grad_norm': '0.7077', 'learning_rate': '2.169e-05', 'rewards/chosen': '0.001227', 'rewards/rejected': '-0.00209', 'rewards/accuracies': '0.3492', 'rewards/margins': '0.003317', 'logps/chosen': '-76.98', 'logps/rejected': '-83.68', 'logits/chosen': '-2.212', 'logits/rejected': '-2.102', 'epoch': '0.01088'}
{'loss': '0.6921', 'grad_norm': '0.9575', 'learning_rate': '4.578e-05', 'rewards/chosen': '0.0005895', 'rewards/rejected': '-0.005486', 'rewards/accuracies': '0.4672', 'rewards/margins': '0.006076', 'logps/chosen': '-79.9', 'logps/rejected': '-86.09', 'logits/chosen': '-2.361', 'logits/rejected': '-2.318', 'epoch': '0.02177'}
{'loss': '0.6918', 'grad_norm': '0.6923', 'learning_rate': '6.988e-05', 'rewards/chosen': '-0.01929', 'rewards/rejected': '-0.02695', 'rewards/accuracies': '0.4766', 'rewards/margins': '0.007659', 'logps/chosen': '-80.27', 'logps/rejected': '-87.22', 'logits/chosen': '-2.296', 'logits/rejected': '-2.237', 'epoch': '0.03265'}
{'loss': '0.6888', 'grad_norm': '0.6989', 'learning_rate': '9.398e-05', 'rewards/chosen': '-0.08199', 'rewards/rejected': '-0.0995', 'rewards/accuracies': '0.4875', 'rewards/margins': '0.01751', 'logps/chosen': '-84.66', 'logps/rejected': '-91.65', 'logits/chosen': '-2.305', 'logits/rejected': '-2.264', 'epoch': '0.04354'}
{'loss': '0.6795', 'grad_norm': '0.7515', 'learning_rate': '0.0001181', 'rewards/chosen': '-0.1621', 'rewards/rejected': '-0.2124', 'rewards/accuracies': '0.5016', 'rewards/margins': '0.05032', 'logps/chosen': '-86.99', 'logps/rejected': '-93.27', 'logits/chosen': '-2.576', 'logits/rejected': '-2.516', 'epoch': '0.05442'}
{'loss': '0.6819', 'grad_norm': '0.864', 'learning_rate': '0.0001422', 'rewards/chosen': '-0.202', 'rewards/rejected': '-0.2602', 'rewards/accuracies': '0.5117', 'rewards/margins': '0.05819', 'logps/chosen': '-83.06', 'logps/rejected': '-88.82', 'logits/chosen': '-2.496', 'logits/rejected': '-2.35', 'epoch': '0.06531'}
{'loss': '0.6768', 'grad_norm': '0.8207', 'learning_rate': '0.0001663', 'rewards/chosen': '-0.1923', 'rewards/rejected': '-0.2661', 'rewards/accuracies': '0.5203', 'rewards/margins': '0.07378', 'logps/chosen': '-81.74', 'logps/rejected': '-87.19', 'logits/chosen': '-2.699', 'logits/rejected': '-2.621', 'epoch': '0.07619'}
{'loss': '0.6668', 'grad_norm': '0.8357', 'learning_rate': '0.0001904', 'rewards/chosen': '-0.2063', 'rewards/rejected': '-0.32', 'rewards/accuracies': '0.5289', 'rewards/margins': '0.1137', 'logps/chosen': '-82.09', 'logps/rejected': '-89.19', 'logits/chosen': '-2.595', 'logits/rejected': '-2.484', 'epoch': '0.08707'}
{'loss': '0.6667', 'grad_norm': '0.7759', 'learning_rate': '0.0001996', 'rewards/chosen': '-0.1989', 'rewards/rejected': '-0.3181', 'rewards/accuracies': '0.5469', 'rewards/margins': '0.1192', 'logps/chosen': '-80.27', 'logps/rejected': '-88.54', 'logits/chosen': '-2.37', 'logits/rejected': '-2.255', 'epoch': '0.09796'}
{'loss': '0.6667', 'grad_norm': '0.7275', 'learning_rate': '0.0001988', 'rewards/chosen': '-0.2421', 'rewards/rejected': '-0.3706', 'rewards/accuracies': '0.5492', 'rewards/margins': '0.1285', 'logps/chosen': '-85.48', 'logps/rejected': '-92.23', 'logits/chosen': '-2.458', 'logits/rejected': '-2.306', 'epoch': '0.1088'}
{'loss': '0.6598', 'grad_norm': '0.8685', 'learning_rate': '0.0001981', 'rewards/chosen': '-0.1943', 'rewards/rejected': '-0.3449', 'rewards/accuracies': '0.5523', 'rewards/margins': '0.1507', 'logps/chosen': '-81.67', 'logps/rejected': '-89.74', 'logits/chosen': '-2.194', 'logits/rejected': '-2.128', 'epoch': '0.1197'}
{'loss': '0.6632', 'grad_norm': '0.7957', 'learning_rate': '0.0001973', 'rewards/chosen': '-0.1917', 'rewards/rejected': '-0.3336', 'rewards/accuracies': '0.5422', 'rewards/margins': '0.1418', 'logps/chosen': '-84.79', 'logps/rejected': '-90.5', 'logits/chosen': '-2.447', 'logits/rejected': '-2.352', 'epoch': '0.1306'}
{'loss': '0.6509', 'grad_norm': '0.6242', 'learning_rate': '0.0001966', 'rewards/chosen': '-0.1273', 'rewards/rejected': '-0.2984', 'rewards/accuracies': '0.5508', 'rewards/margins': '0.1711', 'logps/chosen': '-79.19', 'logps/rejected': '-85.88', 'logits/chosen': '-2.458', 'logits/rejected': '-2.388', 'epoch': '0.1415'}
{'loss': '0.6572', 'grad_norm': '0.6959', 'learning_rate': '0.0001958', 'rewards/chosen': '-0.1161', 'rewards/rejected': '-0.282', 'rewards/accuracies': '0.5641', 'rewards/margins': '0.166', 'logps/chosen': '-82.53', 'logps/rejected': '-89.38', 'logits/chosen': '-2.645', 'logits/rejected': '-2.575', 'epoch': '0.1524'}
{'loss': '0.6591', 'grad_norm': '0.704', 'learning_rate': '0.0001951', 'rewards/chosen': '-0.0406', 'rewards/rejected': '-0.2036', 'rewards/accuracies': '0.5641', 'rewards/margins': '0.163', 'logps/chosen': '-84.36', 'logps/rejected': '-91.33', 'logits/chosen': '-2.601', 'logits/rejected': '-2.53', 'epoch': '0.1633'}
{'loss': '0.6441', 'grad_norm': '0.6618', 'learning_rate': '0.0001943', 'rewards/chosen': '-0.01285', 'rewards/rejected': '-0.201', 'rewards/accuracies': '0.5883', 'rewards/margins': '0.1881', 'logps/chosen': '-83.15', 'logps/rejected': '-89.39', 'logits/chosen': '-2.715', 'logits/rejected': '-2.597', 'epoch': '0.1741'}
{'loss': '0.653', 'grad_norm': '0.6484', 'learning_rate': '0.0001936', 'rewards/chosen': '-0.05956', 'rewards/rejected': '-0.2501', 'rewards/accuracies': '0.5641', 'rewards/margins': '0.1905', 'logps/chosen': '-79.66', 'logps/rejected': '-85.64', 'logits/chosen': '-2.808', 'logits/rejected': '-2.71', 'epoch': '0.185'}
{'loss': '0.671', 'grad_norm': '0.6358', 'learning_rate': '0.0001928', 'rewards/chosen': '-0.1839', 'rewards/rejected': '-0.3243', 'rewards/accuracies': '0.557', 'rewards/margins': '0.1405', 'logps/chosen': '-86.25', 'logps/rejected': '-91.76', 'logits/chosen': '-2.726', 'logits/rejected': '-2.632', 'epoch': '0.1959'}
{'loss': '0.6496', 'grad_norm': '0.7087', 'learning_rate': '0.0001921', 'rewards/chosen': '-0.259', 'rewards/rejected': '-0.4433', 'rewards/accuracies': '0.5711', 'rewards/margins': '0.1842', 'logps/chosen': '-83.1', 'logps/rejected': '-90.81', 'logits/chosen': '-2.575', 'logits/rejected': '-2.476', 'epoch': '0.2068'}
{'loss': '0.6528', 'grad_norm': '0.7277', 'learning_rate': '0.0001913', 'rewards/chosen': '-0.2878', 'rewards/rejected': '-0.4894', 'rewards/accuracies': '0.5523', 'rewards/margins': '0.2016', 'logps/chosen': '-84.37', 'logps/rejected': '-92.43', 'logits/chosen': '-2.665', 'logits/rejected': '-2.642', 'epoch': '0.2177'}
{'loss': '0.6556', 'grad_norm': '0.7041', 'learning_rate': '0.0001906', 'rewards/chosen': '-0.175', 'rewards/rejected': '-0.3603', 'rewards/accuracies': '0.5703', 'rewards/margins': '0.1853', 'logps/chosen': '-83.11', 'logps/rejected': '-90.62', 'logits/chosen': '-2.796', 'logits/rejected': '-2.704', 'epoch': '0.2286'}
{'loss': '0.6463', 'grad_norm': '0.6894', 'learning_rate': '0.0001898', 'rewards/chosen': '0.06025', 'rewards/rejected': '-0.1298', 'rewards/accuracies': '0.5805', 'rewards/margins': '0.19', 'logps/chosen': '-78.92', 'logps/rejected': '-84.79', 'logits/chosen': '-2.694', 'logits/rejected': '-2.68', 'epoch': '0.2395'}
{'loss': '0.6312', 'grad_norm': '0.6098', 'learning_rate': '0.0001891', 'rewards/chosen': '0.03963', 'rewards/rejected': '-0.1992', 'rewards/accuracies': '0.5922', 'rewards/margins': '0.2389', 'logps/chosen': '-80.18', 'logps/rejected': '-86.77', 'logits/chosen': '-2.949', 'logits/rejected': '-2.881', 'epoch': '0.2503'}
{'loss': '0.6405', 'grad_norm': '0.6649', 'learning_rate': '0.0001883', 'rewards/chosen': '-0.1528', 'rewards/rejected': '-0.3926', 'rewards/accuracies': '0.593', 'rewards/margins': '0.2397', 'logps/chosen': '-82.76', 'logps/rejected': '-92.11', 'logits/chosen': '-3.144', 'logits/rejected': '-3.156', 'epoch': '0.2612'}
{'loss': '0.629', 'grad_norm': '0.7529', 'learning_rate': '0.0001876', 'rewards/chosen': '-0.3856', 'rewards/rejected': '-0.6723', 'rewards/accuracies': '0.5914', 'rewards/margins': '0.2867', 'logps/chosen': '-88.3', 'logps/rejected': '-97.05', 'logits/chosen': '-3.041', 'logits/rejected': '-3.004', 'epoch': '0.2721'}
{'loss': '0.62', 'grad_norm': '0.6279', 'learning_rate': '0.0001868', 'rewards/chosen': '-0.438', 'rewards/rejected': '-0.7219', 'rewards/accuracies': '0.5859', 'rewards/margins': '0.284', 'logps/chosen': '-87.58', 'logps/rejected': '-93.7', 'logits/chosen': '-3.091', 'logits/rejected': '-2.95', 'epoch': '0.283'}
9%|▉ | 260/2757 [35:00<5:38:48, 8.14s/it]
9%|▉ | 261/2757 [35:08<5:36:28, 8.09s/it]
10%|▉ | 262/2757 [35:16<5:35:03, 8.06s/it]
10%|▉ | 263/2757 [35:24<5:34:47, 8.05s/it]
10%|▉ | 264/2757 [35:32<5:35:24, 8.07s/it]
10%|▉ | 265/2757 [35:41<5:40:45, 8.20s/it]
10%|▉ | 266/2757 [35:49<5:38:07, 8.14s/it]
10%|▉ | 267/2757 [35:57<5:41:59, 8.24s/it]
10%|▉ | 268/2757 [36:05<5:31:09, 7.98s/it]
10%|▉ | 269/2757 [36:13<5:32:00, 8.01s/it]
10%|▉ | 270/2757 [36:20<5:25:24, 7.85s/it]
10%|▉ | 270/2757 [36:20<5:25:24, 7.85s/it]
10%|▉ | 271/2757 [36:28<5:27:27, 7.90s/it]
10%|▉ | 272/2757 [36:36<5:29:00, 7.94s/it]
10%|▉ | 273/2757 [36:45<5:34:02, 8.07s/it]
10%|▉ | 274/2757 [36:53<5:33:43, 8.06s/it]
10%|▉ | 275/2757 [37:01<5:31:46, 8.02s/it]
10%|█ | 276/2757 [37:08<5:25:47, 7.88s/it]
10%|█ | 277/2757 [37:17<5:36:08, 8.13s/it]
10%|█ | 278/2757 [37:25<5:34:53, 8.11s/it]
10%|█ | 279/2757 [37:34<5:39:02, 8.21s/it]
10%|█ | 280/2757 [37:42<5:36:54, 8.16s/it]
10%|█ | 280/2757 [37:42<5:36:54, 8.16s/it]
10%|█ | 281/2757 [37:49<5:30:11, 8.00s/it]
10%|█ | 282/2757 [37:57<5:32:12, 8.05s/it]
10%|█ | 283/2757 [38:06<5:35:45, 8.14s/it]
10%|█ | 284/2757 [38:14<5:34:14, 8.11s/it]
10%|█ | 285/2757 [38:21<5:29:31, 8.00s/it]
10%|█ | 286/2757 [38:30<5:35:09, 8.14s/it]
10%|█ | 287/2757 [38:38<5:28:24, 7.98s/it]
10%|█ | 288/2757 [38:46<5:36:57, 8.19s/it]
10%|█ | 289/2757 [38:54<5:32:11, 8.08s/it]
11%|█ | 290/2757 [39:02<5:33:59, 8.12s/it]
11%|█ | 290/2757 [39:02<5:33:59, 8.12s/it]
11%|█ | 291/2757 [39:10<5:27:12, 7.96s/it]
11%|█ | 292/2757 [39:18<5:25:56, 7.93s/it]
11%|█ | 293/2757 [39:25<5:20:08, 7.80s/it]
11%|█ | 294/2757 [39:33<5:21:08, 7.82s/it]
11%|█ | 295/2757 [39:42<5:30:09, 8.05s/it]
11%|█ | 296/2757 [39:49<5:26:05, 7.95s/it]
11%|█ | 297/2757 [39:57<5:25:10, 7.93s/it]
11%|█ | 298/2757 [40:06<5:28:38, 8.02s/it]
11%|█ | 299/2757 [40:13<5:27:05, 7.98s/it]
11%|█ | 300/2757 [40:22<5:29:03, 8.04s/it]
11%|█ | 300/2757 [40:22<5:29:03, 8.04s/it]
11%|█ | 301/2757 [40:30<5:36:05, 8.21s/it]
11%|█ | 302/2757 [40:38<5:29:15, 8.05s/it]
11%|█ | 303/2757 [40:46<5:28:38, 8.04s/it]
11%|█ | 304/2757 [40:54<5:25:46, 7.97s/it]
11%|█ | 305/2757 [41:02<5:25:20, 7.96s/it]
11%|█ | 306/2757 [41:10<5:25:19, 7.96s/it]
11%|█ | 307/2757 [41:18<5:28:42, 8.05s/it]
11%|█ | 308/2757 [41:26<5:32:15, 8.14s/it]
11%|█ | 309/2757 [41:34<5:29:04, 8.07s/it]
11%|█ | 310/2757 [41:43<5:34:50, 8.21s/it]
11%|█ | 310/2757 [41:43<5:34:50, 8.21s/it]
11%|█▏ | 311/2757 [41:51<5:34:07, 8.20s/it]
11%|█▏ | 312/2757 [41:59<5:29:01, 8.07s/it]
11%|█▏ | 313/2757 [42:07<5:28:13, 8.06s/it]
11%|█▏ | 314/2757 [42:15<5:30:44, 8.12s/it]
11%|█▏ | 315/2757 [42:23<5:27:30, 8.05s/it]
11%|█▏ | 316/2757 [42:31<5:24:03, 7.97s/it]
11%|█▏ | 317/2757 [42:39<5:28:08, 8.07s/it]
12%|█▏ | 318/2757 [42:47<5:24:56, 7.99s/it]
12%|█▏ | 319/2757 [42:55<5:28:55, 8.09s/it]
12%|█▏ | 320/2757 [43:03<5:25:58, 8.03s/it]
12%|█▏ | 320/2757 [43:03<5:25:58, 8.03s/it]
12%|█▏ | 321/2757 [43:10<5:20:37, 7.90s/it]
12%|█▏ | 322/2757 [43:18<5:19:10, 7.86s/it]
12%|█▏ | 323/2757 [43:27<5:25:30, 8.02s/it]
12%|█▏ | 324/2757 [43:35<5:24:09, 7.99s/it]
12%|█▏ | 325/2757 [43:42<5:20:58, 7.92s/it]
12%|█▏ | 326/2757 [43:50<5:18:49, 7.87s/it]
12%|█▏ | 327/2757 [43:58<5:19:37, 7.89s/it]
12%|█▏ | 328/2757 [44:06<5:22:48, 7.97s/it]
12%|█▏ | 329/2757 [44:14<5:26:50, 8.08s/it]
12%|█▏ | 330/2757 [44:23<5:26:45, 8.08s/it]
12%|█▏ | 330/2757 [44:23<5:26:45, 8.08s/it]
12%|█▏ | 331/2757 [44:31<5:25:44, 8.06s/it]
12%|█▏ | 332/2757 [44:39<5:28:03, 8.12s/it]
12%|█▏ | 333/2757 [44:46<5:19:24, 7.91s/it]
12%|█▏ | 334/2757 [44:54<5:20:36, 7.94s/it]
12%|█▏ | 335/2757 [45:02<5:24:13, 8.03s/it]
12%|█▏ | 336/2757 [45:11<5:24:53, 8.05s/it]
12%|█▏ | 337/2757 [45:19<5:24:41, 8.05s/it]
12%|█▏ | 338/2757 [45:26<5:21:47, 7.98s/it]
12%|█▏ | 339/2757 [45:34<5:20:56, 7.96s/it]
12%|█▏ | 340/2757 [45:42<5:18:59, 7.92s/it]
12%|█▏ | 340/2757 [45:42<5:18:59, 7.92s/it]
12%|█▏ | 341/2757 [45:50<5:17:05, 7.87s/it]
12%|█▏ | 342/2757 [45:58<5:19:30, 7.94s/it]
12%|█▏ | 343/2757 [46:06<5:18:26, 7.91s/it]
12%|█▏ | 344/2757 [46:14<5:16:16, 7.86s/it]
13%|█▎ | 345/2757 [46:22<5:19:59, 7.96s/it]
13%|█▎ | 346/2757 [46:30<5:16:48, 7.88s/it]
13%|█▎ | 347/2757 [46:38<5:24:02, 8.07s/it]
13%|█▎ | 348/2757 [46:46<5:24:11, 8.07s/it]
13%|█▎ | 349/2757 [46:54<5:27:11, 8.15s/it]
13%|█▎ | 350/2757 [47:02<5:24:27, 8.09s/it]
13%|█▎ | 350/2757 [47:02<5:24:27, 8.09s/it]
13%|█▎ | 351/2757 [47:10<5:18:21, 7.94s/it]
13%|█▎ | 352/2757 [47:17<5:12:46, 7.80s/it]
13%|█▎ | 353/2757 [47:25<5:13:10, 7.82s/it]
13%|█▎ | 354/2757 [47:33<5:14:14, 7.85s/it]
13%|█▎ | 355/2757 [47:42<5:19:11, 7.97s/it]
13%|█▎ | 356/2757 [47:49<5:16:20, 7.91s/it]
13%|█▎ | 357/2757 [47:57<5:18:02, 7.95s/it]
13%|█▎ | 358/2757 [48:05<5:19:20, 7.99s/it]
13%|█▎ | 359/2757 [48:13<5:19:04, 7.98s/it]
13%|█▎ | 360/2757 [48:22<5:21:16, 8.04s/it]
13%|█▎ | 360/2757 [48:22<5:21:16, 8.04s/it]
13%|█▎ | 361/2757 [48:30<5:25:30, 8.15s/it]
13%|█▎ | 362/2757 [48:38<5:22:37, 8.08s/it]
13%|█▎ | 363/2757 [48:46<5:23:13, 8.10s/it]
13%|█▎ | 364/2757 [48:54<5:24:42, 8.14s/it]
13%|█▎ | 365/2757 [49:02<5:23:44, 8.12s/it]
13%|█▎ | 366/2757 [49:10<5:22:04, 8.08s/it]
13%|█▎ | 367/2757 [49:18<5:22:52, 8.11s/it]
13%|█▎ | 368/2757 [49:26<5:17:57, 7.99s/it]
13%|█▎ | 369/2757 [49:34<5:21:07, 8.07s/it]
13%|█▎ | 370/2757 [49:42<5:18:35, 8.01s/it]
13%|█▎ | 370/2757 [49:42<5:18:35, 8.01s/it]
13%|█▎ | 371/2757 [49:50<5:18:15, 8.00s/it]
13%|█▎ | 372/2757 [49:59<5:20:53, 8.07s/it]
14%|█▎ | 373/2757 [50:07<5:23:28, 8.14s/it]
14%|█▎ | 374/2757 [50:15<5:21:53, 8.10s/it]
14%|█▎ | 375/2757 [50:23<5:21:22, 8.10s/it]
14%|█▎ | 376/2757 [50:31<5:17:58, 8.01s/it]
14%|█▎ | 377/2757 [50:39<5:22:25, 8.13s/it]
14%|█▎ | 378/2757 [50:47<5:20:49, 8.09s/it]
14%|█▎ | 379/2757 [50:55<5:21:13, 8.11s/it]
14%|█▍ | 380/2757 [51:03<5:12:13, 7.88s/it]
14%|█▍ | 380/2757 [51:03<5:12:13, 7.88s/it]
14%|█▍ | 381/2757 [51:11<5:15:14, 7.96s/it]
14%|█▍ | 382/2757 [51:18<5:11:11, 7.86s/it]
14%|█▍ | 383/2757 [51:27<5:17:57, 8.04s/it]
14%|█▍ | 384/2757 [51:35<5:18:56, 8.06s/it]
14%|█▍ | 385/2757 [51:42<5:11:05, 7.87s/it]
14%|█▍ | 386/2757 [51:50<5:10:02, 7.85s/it]
14%|█▍ | 387/2757 [51:58<5:14:48, 7.97s/it]
14%|█▍ | 388/2757 [52:06<5:08:57, 7.82s/it]
14%|█▍ | 389/2757 [52:14<5:11:59, 7.91s/it]
14%|█▍ | 390/2757 [52:22<5:18:05, 8.06s/it]
14%|█▍ | 390/2757 [52:22<5:18:05, 8.06s/it]
14%|█▍ | 391/2757 [52:30<5:14:31, 7.98s/it]
14%|█▍ | 392/2757 [52:39<5:18:29, 8.08s/it]
14%|█▍ | 393/2757 [52:46<5:12:04, 7.92s/it]
14%|█▍ | 394/2757 [52:55<5:19:01, 8.10s/it]
14%|█▍ | 395/2757 [53:02<5:13:11, 7.96s/it]
14%|█▍ | 396/2757 [53:10<5:09:18, 7.86s/it]
14%|█▍ | 397/2757 [53:18<5:09:50, 7.88s/it]
14%|█▍ | 398/2757 [53:26<5:14:45, 8.01s/it]
14%|█▍ | 399/2757 [53:34<5:15:25, 8.03s/it]
15%|█▍ | 400/2757 [53:42<5:15:01, 8.02s/it]
15%|█▍ | 400/2757 [53:42<5:15:01, 8.02s/it]
15%|█▍ | 401/2757 [53:50<5:16:02, 8.05s/it]
15%|█▍ | 402/2757 [53:58<5:14:40, 8.02s/it]
15%|█▍ | 403/2757 [54:06<5:11:09, 7.93s/it]
15%|█▍ | 404/2757 [54:15<5:20:32, 8.17s/it]
15%|█▍ | 405/2757 [54:23<5:18:59, 8.14s/it]
15%|█▍ | 406/2757 [54:31<5:19:48, 8.16s/it]
15%|█▍ | 407/2757 [54:39<5:20:58, 8.19s/it]
15%|█▍ | 408/2757 [54:48<5:22:04, 8.23s/it]
15%|█▍ | 409/2757 [54:56<5:22:25, 8.24s/it]
15%|█▍ | 410/2757 [55:04<5:15:49, 8.07s/it]
15%|█▍ | 410/2757 [55:04<5:15:49, 8.07s/it]
15%|█▍ | 411/2757 [55:12<5:17:32, 8.12s/it]
15%|█▍ | 412/2757 [55:20<5:14:54, 8.06s/it]
15%|█▍ | 413/2757 [55:28<5:15:28, 8.08s/it]
15%|█▌ | 414/2757 [55:36<5:13:26, 8.03s/it]
15%|█▌ | 415/2757 [55:44<5:13:15, 8.03s/it]
15%|█▌ | 416/2757 [55:52<5:11:22, 7.98s/it]
15%|█▌ | 417/2757 [55:59<5:07:04, 7.87s/it]
15%|█▌ | 418/2757 [56:07<5:06:17, 7.86s/it]
15%|█▌ | 419/2757 [56:15<5:07:04, 7.88s/it]
15%|█▌ | 420/2757 [56:23<5:12:47, 8.03s/it]
15%|█▌ | 420/2757 [56:23<5:12:47, 8.03s/it]
15%|█▌ | 421/2757 [56:31<5:11:26, 8.00s/it]
15%|█▌ | 422/2757 [56:39<5:05:53, 7.86s/it]
15%|█▌ | 423/2757 [56:47<5:05:54, 7.86s/it]
15%|█▌ | 424/2757 [56:55<5:07:56, 7.92s/it]
15%|█▌ | 425/2757 [57:03<5:10:41, 7.99s/it]
15%|█▌ | 426/2757 [57:11<5:13:28, 8.07s/it]
15%|█▌ | 427/2757 [57:19<5:06:31, 7.89s/it]
16%|█▌ | 428/2757 [57:26<5:02:23, 7.79s/it]
16%|█▌ | 429/2757 [57:34<5:02:58, 7.81s/it]
16%|█▌ | 430/2757 [57:42<5:01:14, 7.77s/it]
16%|█▌ | 430/2757 [57:42<5:01:14, 7.77s/it]
16%|█▌ | 431/2757 [57:49<4:57:04, 7.66s/it]
16%|█▌ | 432/2757 [57:57<5:04:20, 7.85s/it]
16%|█▌ | 433/2757 [58:05<5:02:16, 7.80s/it]
16%|█▌ | 434/2757 [58:13<5:04:47, 7.87s/it]
16%|█▌ | 435/2757 [58:21<5:08:18, 7.97s/it]
16%|█▌ | 436/2757 [58:29<5:07:47, 7.96s/it]
16%|█▌ | 437/2757 [58:37<5:08:24, 7.98s/it]
16%|█▌ | 438/2757 [58:45<5:05:49, 7.91s/it]
16%|█▌ | 439/2757 [58:53<5:05:28, 7.91s/it]
16%|█▌ | 440/2757 [59:01<5:11:12, 8.06s/it]
16%|█▌ | 440/2757 [59:01<5:11:12, 8.06s/it]
16%|█▌ | 441/2757 [59:09<5:11:28, 8.07s/it]
16%|█▌ | 442/2757 [59:18<5:12:07, 8.09s/it]
16%|█▌ | 443/2757 [59:25<5:09:21, 8.02s/it]
16%|█▌ | 444/2757 [59:34<5:13:29, 8.13s/it]
16%|█▌ | 445/2757 [59:42<5:13:48, 8.14s/it]
16%|█▌ | 446/2757 [59:50<5:15:04, 8.18s/it]
16%|█▌ | 447/2757 [59:58<5:11:52, 8.10s/it]
16%|█▌ | 448/2757 [1:00:06<5:12:55, 8.13s/it]
16%|█▋ | 449/2757 [1:00:14<5:07:45, 8.00s/it]
16%|█▋ | 450/2757 [1:00:22<5:07:50, 8.01s/it]
16%|█▋ | 450/2757 [1:00:22<5:07:50, 8.01s/it]
16%|█▋ | 451/2757 [1:00:30<5:04:26, 7.92s/it]
16%|█▋ | 452/2757 [1:00:38<5:03:37, 7.90s/it]
16%|█▋ | 453/2757 [1:00:46<5:05:07, 7.95s/it]
16%|█▋ | 454/2757 [1:00:54<5:06:05, 7.97s/it]
17%|█▋ | 455/2757 [1:01:02<5:10:57, 8.10s/it]
17%|█▋ | 456/2757 [1:01:10<5:05:31, 7.97s/it]
17%|█▋ | 457/2757 [1:01:18<5:06:38, 8.00s/it]
17%|█▋ | 458/2757 [1:01:26<5:02:25, 7.89s/it]
17%|█▋ | 459/2757 [1:01:33<4:59:22, 7.82s/it]
17%|█▋ | 460/2757 [1:01:41<4:58:28, 7.80s/it]
17%|█▋ | 460/2757 [1:01:41<4:58:28, 7.80s/it]
17%|█▋ | 461/2757 [1:01:49<4:59:33, 7.83s/it]
17%|█▋ | 462/2757 [1:01:56<4:55:28, 7.72s/it]
17%|█▋ | 463/2757 [1:02:04<4:59:32, 7.83s/it]
17%|█▋ | 464/2757 [1:02:12<5:01:20, 7.89s/it]
17%|█▋ | 465/2757 [1:02:20<5:00:44, 7.87s/it]
17%|█▋ | 466/2757 [1:02:28<5:03:07, 7.94s/it]
17%|█▋ | 467/2757 [1:02:37<5:07:49, 8.07s/it]
17%|█▋ | 468/2757 [1:02:45<5:08:24, 8.08s/it]
17%|█▋ | 469/2757 [1:02:52<5:01:37, 7.91s/it]
17%|█▋ | 470/2757 [1:03:00<4:59:47, 7.87s/it]
17%|█▋ | 470/2757 [1:03:00<4:59:47, 7.87s/it]
17%|█▋ | 471/2757 [1:03:08<5:03:42, 7.97s/it]
17%|█▋ | 472/2757 [1:03:17<5:06:44, 8.05s/it]
17%|█▋ | 473/2757 [1:03:25<5:06:40, 8.06s/it]
17%|█▋ | 474/2757 [1:03:32<5:04:02, 7.99s/it]
17%|█▋ | 475/2757 [1:03:41<5:10:18, 8.16s/it]
17%|█▋ | 476/2757 [1:03:49<5:10:22, 8.16s/it]
17%|█▋ | 477/2757 [1:03:57<5:05:12, 8.03s/it]
17%|█▋ | 478/2757 [1:04:05<5:04:56, 8.03s/it]
17%|█▋ | 479/2757 [1:04:13<5:09:02, 8.14s/it]
17%|█▋ | 480/2757 [1:04:21<5:09:00, 8.14s/it]
17%|█▋ | 480/2757 [1:04:21<5:09:00, 8.14s/it]
17%|█▋ | 481/2757 [1:04:29<5:04:30, 8.03s/it]
17%|█▋ | 482/2757 [1:04:38<5:07:14, 8.10s/it]
18%|█▊ | 483/2757 [1:04:45<5:03:11, 8.00s/it]
18%|█▊ | 484/2757 [1:04:53<5:04:33, 8.04s/it]
18%|█▊ | 485/2757 [1:05:02<5:06:42, 8.10s/it]
18%|█▊ | 486/2757 [1:05:10<5:07:29, 8.12s/it]
18%|█▊ | 487/2757 [1:05:18<5:07:07, 8.12s/it]
18%|█▊ | 488/2757 [1:05:26<5:02:58, 8.01s/it]
18%|█▊ | 489/2757 [1:05:34<5:01:55, 7.99s/it]
18%|█▊ | 490/2757 [1:05:42<5:00:30, 7.95s/it]
18%|█▊ | 490/2757 [1:05:42<5:00:30, 7.95s/it]
18%|█▊ | 491/2757 [1:05:50<5:00:43, 7.96s/it]
18%|█▊ | 492/2757 [1:05:57<4:59:23, 7.93s/it]
18%|█▊ | 493/2757 [1:06:05<4:59:31, 7.94s/it]
18%|█▊ | 494/2757 [1:06:14<5:02:51, 8.03s/it]
18%|█▊ | 495/2757 [1:06:22<5:04:03, 8.07s/it]
18%|█▊ | 496/2757 [1:06:30<5:01:46, 8.01s/it]
18%|█▊ | 497/2757 [1:06:38<5:06:35, 8.14s/it]
18%|█▊ | 498/2757 [1:06:46<5:07:37, 8.17s/it]
18%|█▊ | 499/2757 [1:06:54<5:05:40, 8.12s/it]
18%|█▊ | 500/2757 [1:07:02<5:03:35, 8.07s/it]
18%|█▊ | 500/2757 [1:07:02<5:03:35, 8.07s/it]
{'loss': '0.6296', 'grad_norm': '0.5629', 'learning_rate': '0.0001861', 'rewards/chosen': '-0.3549', 'rewards/rejected': '-0.6289', 'rewards/accuracies': '0.5961', 'rewards/margins': '0.274', 'logps/chosen': '-80.02', 'logps/rejected': '-88.44', 'logits/chosen': '-3.056', 'logits/rejected': '-2.951', 'epoch': '0.2939'}
{'loss': '0.626', 'grad_norm': '0.6267', 'learning_rate': '0.0001853', 'rewards/chosen': '-0.3159', 'rewards/rejected': '-0.5841', 'rewards/accuracies': '0.5914', 'rewards/margins': '0.2681', 'logps/chosen': '-79.61', 'logps/rejected': '-87.73', 'logits/chosen': '-3.073', 'logits/rejected': '-2.979', 'epoch': '0.3048'}
{'loss': '0.6292', 'grad_norm': '0.6236', 'learning_rate': '0.0001846', 'rewards/chosen': '-0.253', 'rewards/rejected': '-0.538', 'rewards/accuracies': '0.5852', 'rewards/margins': '0.285', 'logps/chosen': '-82', 'logps/rejected': '-90.57', 'logits/chosen': '-3.13', 'logits/rejected': '-3.074', 'epoch': '0.3156'}
{'loss': '0.6195', 'grad_norm': '0.6637', 'learning_rate': '0.0001838', 'rewards/chosen': '-0.08129', 'rewards/rejected': '-0.3927', 'rewards/accuracies': '0.6008', 'rewards/margins': '0.3114', 'logps/chosen': '-80.13', 'logps/rejected': '-88.26', 'logits/chosen': '-3.198', 'logits/rejected': '-3.114', 'epoch': '0.3265'}
{'loss': '0.6184', 'grad_norm': '0.6673', 'learning_rate': '0.0001831', 'rewards/chosen': '-0.3788', 'rewards/rejected': '-0.6948', 'rewards/accuracies': '0.5984', 'rewards/margins': '0.316', 'logps/chosen': '-86.59', 'logps/rejected': '-95.23', 'logits/chosen': '-2.986', 'logits/rejected': '-2.906', 'epoch': '0.3374'}
{'loss': '0.6248', 'grad_norm': '0.6171', 'learning_rate': '0.0001823', 'rewards/chosen': '-0.5991', 'rewards/rejected': '-0.8997', 'rewards/accuracies': '0.5844', 'rewards/margins': '0.3006', 'logps/chosen': '-88.97', 'logps/rejected': '-97.9', 'logits/chosen': '-3.004', 'logits/rejected': '-2.952', 'epoch': '0.3483'}
{'loss': '0.6249', 'grad_norm': '0.7893', 'learning_rate': '0.0001816', 'rewards/chosen': '-0.639', 'rewards/rejected': '-0.9381', 'rewards/accuracies': '0.5859', 'rewards/margins': '0.2991', 'logps/chosen': '-84.33', 'logps/rejected': '-94.08', 'logits/chosen': '-3.131', 'logits/rejected': '-3.113', 'epoch': '0.3592'}
{'loss': '0.6234', 'grad_norm': '0.5814', 'learning_rate': '0.0001809', 'rewards/chosen': '-0.6253', 'rewards/rejected': '-0.9446', 'rewards/accuracies': '0.5922', 'rewards/margins': '0.3192', 'logps/chosen': '-86.04', 'logps/rejected': '-93.89', 'logits/chosen': '-3.178', 'logits/rejected': '-3.083', 'epoch': '0.3701'}
{'loss': '0.6068', 'grad_norm': '0.6354', 'learning_rate': '0.0001801', 'rewards/chosen': '-0.5151', 'rewards/rejected': '-0.8645', 'rewards/accuracies': '0.6133', 'rewards/margins': '0.3494', 'logps/chosen': '-89.97', 'logps/rejected': '-98.78', 'logits/chosen': '-2.931', 'logits/rejected': '-2.912', 'epoch': '0.381'}
{'loss': '0.6245', 'grad_norm': '0.6956', 'learning_rate': '0.0001794', 'rewards/chosen': '-0.5855', 'rewards/rejected': '-0.8816', 'rewards/accuracies': '0.6016', 'rewards/margins': '0.2961', 'logps/chosen': '-84.48', 'logps/rejected': '-91.43', 'logits/chosen': '-2.768', 'logits/rejected': '-2.695', 'epoch': '0.3918'}
{'loss': '0.623', 'grad_norm': '0.5492', 'learning_rate': '0.0001786', 'rewards/chosen': '-0.6255', 'rewards/rejected': '-0.9144', 'rewards/accuracies': '0.6023', 'rewards/margins': '0.2889', 'logps/chosen': '-88.09', 'logps/rejected': '-94.99', 'logits/chosen': '-2.877', 'logits/rejected': '-2.792', 'epoch': '0.4027'}
{'loss': '0.6103', 'grad_norm': '0.6273', 'learning_rate': '0.0001779', 'rewards/chosen': '-0.5627', 'rewards/rejected': '-0.8772', 'rewards/accuracies': '0.6078', 'rewards/margins': '0.3145', 'logps/chosen': '-89.07', 'logps/rejected': '-97.89', 'logits/chosen': '-2.693', 'logits/rejected': '-2.596', 'epoch': '0.4136'}
{'loss': '0.6128', 'grad_norm': '0.6782', 'learning_rate': '0.0001771', 'rewards/chosen': '-0.5093', 'rewards/rejected': '-0.8476', 'rewards/accuracies': '0.6219', 'rewards/margins': '0.3384', 'logps/chosen': '-85.96', 'logps/rejected': '-94.58', 'logits/chosen': '-2.902', 'logits/rejected': '-2.792', 'epoch': '0.4245'}
{'loss': '0.6213', 'grad_norm': '0.6319', 'learning_rate': '0.0001764', 'rewards/chosen': '-0.3547', 'rewards/rejected': '-0.6653', 'rewards/accuracies': '0.6102', 'rewards/margins': '0.3106', 'logps/chosen': '-82.64', 'logps/rejected': '-93.14', 'logits/chosen': '-2.978', 'logits/rejected': '-2.966', 'epoch': '0.4354'}
{'loss': '0.6141', 'grad_norm': '0.6016', 'learning_rate': '0.0001756', 'rewards/chosen': '-0.3532', 'rewards/rejected': '-0.698', 'rewards/accuracies': '0.5945', 'rewards/margins': '0.3448', 'logps/chosen': '-84.97', 'logps/rejected': '-93.02', 'logits/chosen': '-3.077', 'logits/rejected': '-3.008', 'epoch': '0.4463'}
{'loss': '0.6004', 'grad_norm': '0.6052', 'learning_rate': '0.0001749', 'rewards/chosen': '-0.2189', 'rewards/rejected': '-0.587', 'rewards/accuracies': '0.6258', 'rewards/margins': '0.3681', 'logps/chosen': '-79.92', 'logps/rejected': '-88.4', 'logits/chosen': '-3.087', 'logits/rejected': '-2.981', 'epoch': '0.4571'}
{'loss': '0.62', 'grad_norm': '0.73', 'learning_rate': '0.0001741', 'rewards/chosen': '-0.2221', 'rewards/rejected': '-0.5518', 'rewards/accuracies': '0.6047', 'rewards/margins': '0.3297', 'logps/chosen': '-84.05', 'logps/rejected': '-91.39', 'logits/chosen': '-3.061', 'logits/rejected': '-3.027', 'epoch': '0.468'}
{'loss': '0.6032', 'grad_norm': '0.5996', 'learning_rate': '0.0001734', 'rewards/chosen': '-0.5063', 'rewards/rejected': '-0.86', 'rewards/accuracies': '0.6305', 'rewards/margins': '0.3537', 'logps/chosen': '-88.16', 'logps/rejected': '-94.57', 'logits/chosen': '-2.93', 'logits/rejected': '-2.835', 'epoch': '0.4789'}
{'loss': '0.5989', 'grad_norm': '0.6094', 'learning_rate': '0.0001726', 'rewards/chosen': '-0.5802', 'rewards/rejected': '-0.9776', 'rewards/accuracies': '0.6117', 'rewards/margins': '0.3973', 'logps/chosen': '-85.01', 'logps/rejected': '-94.11', 'logits/chosen': '-3.055', 'logits/rejected': '-2.933', 'epoch': '0.4898'}
{'loss': '0.6101', 'grad_norm': '0.6395', 'learning_rate': '0.0001719', 'rewards/chosen': '-0.5156', 'rewards/rejected': '-0.8643', 'rewards/accuracies': '0.6172', 'rewards/margins': '0.3487', 'logps/chosen': '-86.3', 'logps/rejected': '-94.5', 'logits/chosen': '-2.982', 'logits/rejected': '-2.935', 'epoch': '0.5007'}
{'loss': '0.6019', 'grad_norm': '0.5491', 'learning_rate': '0.0001711', 'rewards/chosen': '-0.3541', 'rewards/rejected': '-0.7315', 'rewards/accuracies': '0.6062', 'rewards/margins': '0.3774', 'logps/chosen': '-81.23', 'logps/rejected': '-88.91', 'logits/chosen': '-2.927', 'logits/rejected': '-2.875', 'epoch': '0.5116'}
{'loss': '0.5966', 'grad_norm': '0.6346', 'learning_rate': '0.0001704', 'rewards/chosen': '-0.242', 'rewards/rejected': '-0.6034', 'rewards/accuracies': '0.6055', 'rewards/margins': '0.3614', 'logps/chosen': '-82.53', 'logps/rejected': '-91.44', 'logits/chosen': '-2.846', 'logits/rejected': '-2.819', 'epoch': '0.5224'}
{'loss': '0.5878', 'grad_norm': '0.6204', 'learning_rate': '0.0001696', 'rewards/chosen': '-0.3545', 'rewards/rejected': '-0.7606', 'rewards/accuracies': '0.643', 'rewards/margins': '0.4061', 'logps/chosen': '-83.22', 'logps/rejected': '-91.02', 'logits/chosen': '-3.038', 'logits/rejected': '-2.952', 'epoch': '0.5333'}
{'loss': '0.5975', 'grad_norm': '0.6997', 'learning_rate': '0.0001689', 'rewards/chosen': '-0.6454', 'rewards/rejected': '-1.051', 'rewards/accuracies': '0.6109', 'rewards/margins': '0.4053', 'logps/chosen': '-88.25', 'logps/rejected': '-98.36', 'logits/chosen': '-3.002', 'logits/rejected': '-2.958', 'epoch': '0.5442'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
0%| | 0/300 [00:00<?, ?it/s]
1%| | 2/300 [00:00<00:52, 5.64it/s]
1%| | 3/300 [00:00<01:46, 2.79it/s]
1%|▏ | 4/300 [00:01<01:34, 3.14it/s]
2%|▏ | 5/300 [00:01<01:44, 2.82it/s]
2%|▏ | 6/300 [00:01<01:40, 2.93it/s]
2%|▏ | 7/300 [00:02<01:54, 2.56it/s]
3%|▎ | 8/300 [00:02<01:50, 2.64it/s]
3%|▎ | 9/300 [00:03<01:53, 2.57it/s]
3%|▎ | 10/300 [00:03<01:49, 2.66it/s]
4%|▎ | 11/300 [00:03<01:51, 2.59it/s]
4%|▍ | 12/300 [00:04<01:38, 2.92it/s]
4%|▍ | 13/300 [00:04<01:45, 2.72it/s]
5%|▍ | 14/300 [00:04<01:37, 2.93it/s]
5%|▌ | 15/300 [00:05<01:47, 2.66it/s]
5%|▌ | 16/300 [00:05<01:34, 2.99it/s]
6%|▌ | 17/300 [00:05<01:38, 2.87it/s]
6%|▌ | 18/300 [00:06<01:28, 3.18it/s]
6%|▋ | 19/300 [00:06<01:39, 2.83it/s]
7%|▋ | 20/300 [00:06<01:31, 3.05it/s]
7%|▋ | 21/300 [00:07<01:41, 2.74it/s]
7%|▋ | 22/300 [00:07<01:30, 3.06it/s]
8%|▊ | 23/300 [00:07<01:32, 3.00it/s]
8%|▊ | 24/300 [00:08<01:23, 3.31it/s]
8%|▊ | 25/300 [00:08<01:35, 2.87it/s]
9%|▊ | 26/300 [00:08<01:29, 3.08it/s]
9%|▉ | 27/300 [00:09<01:37, 2.80it/s]
9%|▉ | 28/300 [00:09<01:26, 3.15it/s]
10%|▉ | 29/300 [00:09<01:29, 3.03it/s]
10%|█ | 30/300 [00:10<01:21, 3.33it/s]
10%|█ | 31/300 [00:10<01:23, 3.22it/s]
11%|█ | 32/300 [00:10<01:17, 3.48it/s]
11%|█ | 33/300 [00:11<01:19, 3.34it/s]
11%|█▏ | 34/300 [00:11<01:15, 3.52it/s]
12%|█▏ | 35/300 [00:11<01:22, 3.23it/s]
12%|█▏ | 36/300 [00:11<01:17, 3.39it/s]
12%|█▏ | 37/300 [00:12<01:26, 3.04it/s]
13%|█▎ | 38/300 [00:12<01:17, 3.36it/s]
13%|█▎ | 39/300 [00:13<01:40, 2.60it/s]
13%|█▎ | 40/300 [00:13<01:28, 2.94it/s]
14%|█▎ | 41/300 [00:13<01:32, 2.81it/s]
14%|█▍ | 42/300 [00:14<01:22, 3.14it/s]
14%|█▍ | 43/300 [00:14<01:25, 3.02it/s]
15%|█▍ | 44/300 [00:14<01:17, 3.30it/s]
15%|█▌ | 45/300 [00:15<01:23, 3.04it/s]
15%|█▌ | 46/300 [00:15<01:24, 3.00it/s]
16%|█▌ | 47/300 [00:15<01:30, 2.80it/s]
16%|█▌ | 48/300 [00:16<01:22, 3.04it/s]
16%|█▋ | 49/300 [00:16<01:23, 3.02it/s]
17%|█▋ | 50/300 [00:16<01:14, 3.36it/s]
17%|█▋ | 51/300 [00:16<01:18, 3.19it/s]
17%|█▋ | 52/300 [00:17<01:15, 3.29it/s]
18%|█▊ | 53/300 [00:17<01:28, 2.80it/s]
18%|█▊ | 54/300 [00:17<01:19, 3.09it/s]
18%|█▊ | 55/300 [00:18<01:29, 2.73it/s]
19%|█▊ | 56/300 [00:18<01:17, 3.15it/s]
19%|█▉ | 57/300 [00:18<01:20, 3.01it/s]
19%|█▉ | 58/300 [00:19<01:21, 2.96it/s]
20%|█▉ | 59/300 [00:19<01:26, 2.79it/s]
20%|██ | 60/300 [00:20<01:18, 3.08it/s]
20%|██ | 61/300 [00:20<01:27, 2.72it/s]
21%|██ | 62/300 [00:20<01:25, 2.78it/s]
21%|██ | 63/300 [00:21<01:32, 2.56it/s]
21%|██▏ | 64/300 [00:21<01:28, 2.66it/s]
22%|██▏ | 65/300 [00:22<01:30, 2.59it/s]
22%|██▏ | 66/300 [00:22<01:21, 2.87it/s]
22%|██▏ | 67/300 [00:22<01:29, 2.62it/s]
23%|██▎ | 68/300 [00:22<01:19, 2.91it/s]
23%|██▎ | 69/300 [00:23<01:22, 2.81it/s]
23%|██▎ | 70/300 [00:23<01:13, 3.12it/s]
24%|██▎ | 71/300 [00:23<01:15, 3.03it/s]
24%|██▍ | 72/300 [00:24<01:09, 3.29it/s]
24%|██▍ | 73/300 [00:24<01:12, 3.14it/s]
25%|██▍ | 74/300 [00:24<01:08, 3.32it/s]
25%|██▌ | 75/300 [00:25<01:15, 2.98it/s]
25%|██▌ | 76/300 [00:25<01:15, 2.95it/s]
26%|██▌ | 77/300 [00:26<01:20, 2.76it/s]
26%|██▌ | 78/300 [00:26<01:13, 3.01it/s]
26%|██▋ | 79/300 [00:26<01:15, 2.91it/s]
27%|██▋ | 80/300 [00:26<01:11, 3.08it/s]
27%|██▋ | 81/300 [00:27<01:19, 2.76it/s]
27%|██▋ | 82/300 [00:27<01:11, 3.07it/s]
28%|██▊ | 83/300 [00:27<01:13, 2.97it/s]
28%|██▊ | 84/300 [00:28<01:13, 2.93it/s]
28%|██▊ | 85/300 [00:28<01:18, 2.74it/s]
29%|██▊ | 86/300 [00:28<01:10, 3.04it/s]
29%|██▉ | 87/300 [00:29<01:12, 2.94it/s]
29%|██▉ | 88/300 [00:29<01:06, 3.19it/s]
30%|██▉ | 89/300 [00:29<01:10, 2.98it/s]
30%|███ | 90/300 [00:30<01:05, 3.22it/s]
30%|███ | 91/300 [00:30<01:09, 3.03it/s]
31%|███ | 92/300 [00:30<01:03, 3.28it/s]
31%|███ | 93/300 [00:31<01:09, 2.99it/s]
31%|███▏ | 94/300 [00:31<01:02, 3.29it/s]
32%|███▏ | 95/300 [00:31<01:07, 3.02it/s]
32%|███▏ | 96/300 [00:32<01:03, 3.21it/s]
32%|███▏ | 97/300 [00:32<01:05, 3.09it/s]
33%|███▎ | 98/300 [00:32<01:02, 3.25it/s]
33%|███▎ | 99/300 [00:33<01:07, 3.00it/s]
33%|███▎ | 100/300 [00:33<01:03, 3.16it/s]
34%|███▎ | 101/300 [00:33<01:07, 2.93it/s]
34%|███▍ | 102/300 [00:34<01:06, 3.00it/s]
34%|███▍ | 103/300 [00:34<01:09, 2.85it/s]
35%|███▍ | 104/300 [00:34<01:01, 3.18it/s]
35%|███▌ | 105/300 [00:35<01:04, 3.01it/s]
35%|███▌ | 106/300 [00:35<01:00, 3.21it/s]
36%|███▌ | 107/300 [00:35<01:05, 2.96it/s]
36%|███▌ | 108/300 [00:36<01:02, 3.06it/s]
36%|███▋ | 109/300 [00:36<01:06, 2.87it/s]
37%|███▋ | 110/300 [00:36<01:02, 3.03it/s]
37%|███▋ | 111/300 [00:37<01:11, 2.65it/s]
37%|███▋ | 112/300 [00:37<01:05, 2.85it/s]
38%|███▊ | 113/300 [00:37<01:07, 2.75it/s]
38%|███▊ | 114/300 [00:38<01:06, 2.79it/s]
38%|███▊ | 115/300 [00:38<01:09, 2.65it/s]
39%|███▊ | 116/300 [00:38<01:01, 2.98it/s]
39%|███▉ | 117/300 [00:39<01:03, 2.90it/s]
39%|███▉ | 118/300 [00:39<00:58, 3.12it/s]
40%|███▉ | 119/300 [00:39<01:00, 2.99it/s]
40%|████ | 120/300 [00:40<00:59, 3.01it/s]
40%|████ | 121/300 [00:40<01:03, 2.83it/s]
41%|████ | 122/300 [00:40<00:57, 3.07it/s]
41%|████ | 123/300 [00:41<01:01, 2.87it/s]
41%|████▏ | 124/300 [00:41<00:57, 3.04it/s]
42%|████▏ | 125/300 [00:42<00:59, 2.94it/s]
42%|████▏ | 126/300 [00:42<00:59, 2.91it/s]
42%|████▏ | 127/300 [00:42<01:02, 2.76it/s]
43%|████▎ | 128/300 [00:43<00:58, 2.97it/s]
43%|████▎ | 129/300 [00:43<01:09, 2.46it/s]
43%|████▎ | 130/300 [00:43<01:00, 2.79it/s]
44%|████▎ | 131/300 [00:44<01:05, 2.58it/s]
44%|████▍ | 132/300 [00:44<00:58, 2.86it/s]
44%|████▍ | 133/300 [00:45<01:05, 2.56it/s]
45%|████▍ | 134/300 [00:45<00:58, 2.84it/s]
45%|████▌ | 135/300 [00:45<01:02, 2.65it/s]
45%|████▌ | 136/300 [00:46<00:55, 2.96it/s]
46%|████▌ | 137/300 [00:46<00:55, 2.93it/s]
46%|████▌ | 138/300 [00:46<00:51, 3.17it/s]
46%|████▋ | 139/300 [00:46<00:51, 3.12it/s]
47%|████▋ | 140/300 [00:47<00:47, 3.37it/s]
47%|████▋ | 141/300 [00:47<00:49, 3.20it/s]
47%|████▋ | 142/300 [00:47<00:51, 3.09it/s]
48%|████▊ | 143/300 [00:48<00:54, 2.89it/s]
48%|████▊ | 144/300 [00:48<00:48, 3.20it/s]
48%|████▊ | 145/300 [00:48<00:50, 3.09it/s]
49%|████▊ | 146/300 [00:49<00:47, 3.26it/s]
49%|████▉ | 147/300 [00:49<00:53, 2.84it/s]
49%|████▉ | 148/300 [00:49<00:50, 3.01it/s]
50%|████▉ | 149/300 [00:50<00:54, 2.79it/s]
50%|█████ | 150/300 [00:50<00:50, 2.99it/s]
50%|█████ | 151/300 [00:50<00:51, 2.88it/s]
51%|█████ | 152/300 [00:51<00:46, 3.20it/s]
51%|█████ | 153/300 [00:51<00:47, 3.09it/s]
51%|█████▏ | 154/300 [00:51<00:43, 3.37it/s]
52%|█████▏ | 155/300 [00:52<00:45, 3.17it/s]
52%|█████▏ | 156/300 [00:52<00:41, 3.50it/s]
52%|█████▏ | 157/300 [00:52<00:48, 2.97it/s]
53%|█████▎ | 158/300 [00:53<00:45, 3.14it/s]
53%|█████▎ | 159/300 [00:53<00:51, 2.72it/s]
53%|█████▎ | 160/300 [00:53<00:47, 2.96it/s]
54%|█████▎ | 161/300 [00:54<00:51, 2.72it/s]
54%|█████▍ | 162/300 [00:54<00:50, 2.76it/s]
54%|█████▍ | 163/300 [00:55<00:55, 2.45it/s]
55%|█████▍ | 164/300 [00:55<00:53, 2.56it/s]
55%|█████▌ | 165/300 [00:55<00:52, 2.56it/s]
55%|█████▌ | 166/300 [00:56<00:50, 2.67it/s]
56%|█████▌ | 167/300 [00:56<00:54, 2.43it/s]
56%|█████▌ | 168/300 [00:56<00:47, 2.75it/s]
56%|█████▋ | 169/300 [00:57<00:52, 2.48it/s]
57%|█████▋ | 170/300 [00:57<00:50, 2.58it/s]
57%|█████▋ | 171/300 [00:58<00:50, 2.55it/s]
57%|█████▋ | 172/300 [00:58<00:44, 2.88it/s]
58%|█████▊ | 173/300 [00:58<00:46, 2.72it/s]
58%|█████▊ | 174/300 [00:59<00:41, 3.04it/s]
58%|█████▊ | 175/300 [00:59<00:45, 2.73it/s]
59%|█████▊ | 176/300 [00:59<00:40, 3.06it/s]
59%|█████▉ | 177/300 [01:00<00:44, 2.74it/s]
59%|█████▉ | 178/300 [01:00<00:38, 3.15it/s]
60%|█████▉ | 179/300 [01:00<00:41, 2.93it/s]
60%|██████ | 180/300 [01:01<00:37, 3.21it/s]
60%|██████ | 181/300 [01:01<00:41, 2.88it/s]
61%|██████ | 182/300 [01:01<00:36, 3.19it/s]
61%|██████ | 183/300 [01:02<00:39, 2.93it/s]
61%|██████▏ | 184/300 [01:02<00:39, 2.90it/s]
62%|██████▏ | 185/300 [01:02<00:42, 2.72it/s]
62%|██████▏ | 186/300 [01:03<00:39, 2.91it/s]
62%|██████▏ | 187/300 [01:03<00:39, 2.87it/s]
63%|██████▎ | 188/300 [01:03<00:35, 3.15it/s]
63%|██████▎ | 189/300 [01:04<00:38, 2.87it/s]
63%|██████▎ | 190/300 [01:04<00:35, 3.10it/s]
64%|██████▎ | 191/300 [01:04<00:36, 2.96it/s]
64%|██████▍ | 192/300 [01:05<00:36, 2.98it/s]
64%|██████▍ | 193/300 [01:05<00:41, 2.58it/s]
65%|██████▍ | 194/300 [01:06<00:39, 2.66it/s]
65%|██████▌ | 195/300 [01:06<00:42, 2.50it/s]
65%|██████▌ | 196/300 [01:06<00:38, 2.73it/s]
66%|██████▌ | 197/300 [01:07<00:37, 2.71it/s]
66%|██████▌ | 198/300 [01:07<00:34, 2.94it/s]
66%|██████▋ | 199/300 [01:07<00:35, 2.83it/s]
67%|██████▋ | 200/300 [01:08<00:33, 3.01it/s]
67%|██████▋ | 201/300 [01:08<00:35, 2.82it/s]
67%|██████▋ | 202/300 [01:08<00:31, 3.11it/s]
68%|██████▊ | 203/300 [01:09<00:32, 2.96it/s]
68%|██████▊ | 204/300 [01:09<00:29, 3.24it/s]
68%|██████▊ | 205/300 [01:09<00:31, 3.03it/s]
69%|██████▊ | 206/300 [01:10<00:27, 3.42it/s]
69%|██████▉ | 207/300 [01:10<00:31, 2.94it/s]
69%|██████▉ | 208/300 [01:10<00:31, 2.96it/s]
70%|██████▉ | 209/300 [01:11<00:32, 2.81it/s]
70%|███████ | 210/300 [01:11<00:29, 3.03it/s]
70%|███████ | 211/300 [01:11<00:33, 2.67it/s]
71%|███████ | 212/300 [01:12<00:29, 2.99it/s]
71%|███████ | 213/300 [01:12<00:31, 2.75it/s]
71%|███████▏ | 214/300 [01:12<00:28, 3.00it/s]
72%|███████▏ | 215/300 [01:13<00:31, 2.70it/s]
72%|███████▏ | 216/300 [01:13<00:27, 3.01it/s]
72%|███████▏ | 217/300 [01:14<00:34, 2.41it/s]
73%|███████▎ | 218/300 [01:14<00:28, 2.85it/s]
73%|███████▎ | 219/300 [01:14<00:30, 2.69it/s]
73%|███████▎ | 220/300 [01:15<00:26, 3.00it/s]
74%|███████▎ | 221/300 [01:15<00:28, 2.73it/s]
74%|███████▍ | 222/300 [01:15<00:25, 3.04it/s]
74%|███████▍ | 223/300 [01:16<00:28, 2.73it/s]
75%|███████▍ | 224/300 [01:16<00:25, 2.94it/s]
75%|███████▌ | 225/300 [01:16<00:26, 2.81it/s]
75%|███████▌ | 226/300 [01:17<00:24, 3.05it/s]
76%|███████▌ | 227/300 [01:17<00:26, 2.78it/s]
76%|███████▌ | 228/300 [01:17<00:23, 3.11it/s]
76%|███████▋ | 229/300 [01:18<00:25, 2.75it/s]
77%|███████▋ | 230/300 [01:18<00:24, 2.86it/s]
77%|███████▋ | 231/300 [01:18<00:25, 2.70it/s]
77%|███████▋ | 232/300 [01:19<00:24, 2.77it/s]
78%|███████▊ | 233/300 [01:19<00:27, 2.45it/s]
78%|███████▊ | 234/300 [01:20<00:23, 2.81it/s]
78%|███████▊ | 235/300 [01:20<00:24, 2.63it/s]
79%|███████▊ | 236/300 [01:20<00:23, 2.70it/s]
79%|███████▉ | 237/300 [01:21<00:25, 2.49it/s]
79%|███████▉ | 238/300 [01:21<00:21, 2.82it/s]
80%|███████▉ | 239/300 [01:21<00:21, 2.83it/s]
80%|████████ | 240/300 [01:22<00:18, 3.16it/s]
80%|████████ | 241/300 [01:22<00:19, 3.04it/s]
81%|████████ | 242/300 [01:22<00:18, 3.08it/s]
81%|████████ | 243/300 [01:23<00:20, 2.73it/s]
81%|████████▏ | 244/300 [01:23<00:19, 2.83it/s]
82%|████████▏ | 245/300 [01:24<00:20, 2.67it/s]
82%|████████▏ | 246/300 [01:24<00:19, 2.77it/s]
82%|████████▏ | 247/300 [01:24<00:20, 2.60it/s]
83%|████████▎ | 248/300 [01:25<00:17, 2.94it/s]
83%|████████▎ | 249/300 [01:25<00:18, 2.80it/s]
83%|████████▎ | 250/300 [01:25<00:17, 2.92it/s]
84%|████████▎ | 251/300 [01:26<00:17, 2.81it/s]
84%|████████▍ | 252/300 [01:26<00:15, 3.01it/s]
84%|████████▍ | 253/300 [01:26<00:17, 2.74it/s]
85%|████████▍ | 254/300 [01:27<00:14, 3.12it/s]
85%|████████▌ | 255/300 [01:27<00:16, 2.81it/s]
85%|████████▌ | 256/300 [01:27<00:13, 3.15it/s]
86%|████████▌ | 257/300 [01:28<00:14, 2.96it/s]
86%|████████▌ | 258/300 [01:28<00:13, 3.22it/s]
86%|████████▋ | 259/300 [01:28<00:13, 2.94it/s]
87%|████████▋ | 260/300 [01:29<00:12, 3.14it/s]
87%|████████▋ | 261/300 [01:29<00:13, 2.99it/s]
87%|████████▋ | 262/300 [01:29<00:11, 3.28it/s]
88%|████████▊ | 263/300 [01:30<00:12, 3.02it/s]
88%|████████▊ | 264/300 [01:30<00:11, 3.01it/s]
88%|████████▊ | 265/300 [01:30<00:13, 2.60it/s]
89%|████████▊ | 266/300 [01:31<00:12, 2.70it/s]
89%|████████▉ | 267/300 [01:31<00:13, 2.44it/s]
89%|████████▉ | 268/300 [01:31<00:11, 2.80it/s]
90%|████████▉ | 269/300 [01:32<00:12, 2.47it/s]
90%|█████████ | 270/300 [01:32<00:11, 2.64it/s]
90%|█████████ | 271/300 [01:33<00:12, 2.39it/s]
91%|█████████ | 272/300 [01:33<00:10, 2.65it/s]
91%|█████████ | 273/300 [01:34<00:11, 2.39it/s]
91%|█████████▏| 274/300 [01:34<00:09, 2.74it/s]
92%|█████████▏| 275/300 [01:34<00:10, 2.28it/s]
92%|█████████▏| 276/300 [01:35<00:09, 2.49it/s]
92%|█████████▏| 277/300 [01:35<00:09, 2.45it/s]
93%|█████████▎| 278/300 [01:35<00:07, 2.78it/s]
93%|█████████▎| 279/300 [01:36<00:07, 2.77it/s]
93%|█████████▎| 280/300 [01:36<00:06, 2.87it/s]
94%|█████████▎| 281/300 [01:37<00:07, 2.71it/s]
94%|█████████▍| 282/300 [01:37<00:05, 3.02it/s]
94%|█████████▍| 283/300 [01:37<00:06, 2.77it/s]
95%|█████████▍| 284/300 [01:37<00:05, 3.03it/s]
95%|█████████▌| 285/300 [01:38<00:05, 2.62it/s]
95%|█████████▌| 286/300 [01:38<00:05, 2.69it/s]
96%|█████████▌| 287/300 [01:39<00:04, 2.62it/s]
96%|█████████▌| 288/300 [01:39<00:03, 3.05it/s]
96%|█████████▋| 289/300 [01:39<00:03, 2.80it/s]
97%|█████████▋| 290/300 [01:40<00:03, 2.82it/s]
97%|█████████▋| 291/300 [01:40<00:03, 2.47it/s]
97%|█████████▋| 292/300 [01:40<00:02, 2.81it/s]
98%|█████████▊| 293/300 [01:41<00:02, 2.70it/s]
98%|█████████▊| 294/300 [01:41<00:02, 2.96it/s]
98%|█████████▊| 295/300 [01:41<00:01, 2.92it/s]
99%|█████████▊| 296/300 [01:42<00:01, 3.13it/s]
99%|█████████▉| 297/300 [01:42<00:01, 2.90it/s]
99%|█████████▉| 298/300 [01:42<00:00, 3.27it/s]
100%|█████████▉| 299/300 [01:43<00:00, 2.87it/s]
100%|██████████| 300/300 [01:43<00:00, 2.76it/s]

18%|█▊ | 500/2757 [1:08:47<5:03:35, 8.07s/it]
100%|██████████| 300/300 [01:43<00:00, 2.76it/s]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-500/tokenizer_config.json.
18%|█▊ | 501/2757 [1:09:01<25:49:43, 41.22s/it]
18%|█▊ | 502/2757 [1:09:09<19:33:58, 31.24s/it]
18%|█▊ | 503/2757 [1:09:17<15:09:23, 24.21s/it]
18%|█▊ | 504/2757 [1:09:24<11:57:50, 19.12s/it]
18%|█▊ | 505/2757 [1:09:32<9:49:08, 15.70s/it]
18%|█▊ | 506/2757 [1:09:39<8:19:28, 13.31s/it]
18%|█▊ | 507/2757 [1:09:47<7:20:56, 11.76s/it]
18%|█▊ | 508/2757 [1:09:56<6:40:22, 10.68s/it]
18%|█▊ | 509/2757 [1:10:03<6:07:33, 9.81s/it]
18%|█▊ | 510/2757 [1:10:11<5:41:59, 9.13s/it]
18%|█▊ | 510/2757 [1:10:11<5:41:59, 9.13s/it]
19%|█▊ | 511/2757 [1:10:19<5:25:41, 8.70s/it]
19%|█▊ | 512/2757 [1:10:27<5:17:03, 8.47s/it]
19%|█▊ | 513/2757 [1:10:35<5:12:13, 8.35s/it]
19%|█▊ | 514/2757 [1:10:43<5:07:24, 8.22s/it]
19%|█▊ | 515/2757 [1:10:50<4:59:38, 8.02s/it]
19%|█▊ | 516/2757 [1:10:58<5:01:44, 8.08s/it]
19%|█▉ | 517/2757 [1:11:07<5:03:39, 8.13s/it]
19%|█▉ | 518/2757 [1:11:14<5:00:38, 8.06s/it]
19%|█▉ | 519/2757 [1:11:23<5:04:44, 8.17s/it]
19%|█▉ | 520/2757 [1:11:31<4:59:29, 8.03s/it]
19%|█▉ | 520/2757 [1:11:31<4:59:29, 8.03s/it]
19%|█▉ | 521/2757 [1:11:39<5:02:53, 8.13s/it]
19%|█▉ | 522/2757 [1:11:47<5:02:38, 8.12s/it]
19%|█▉ | 523/2757 [1:11:55<4:59:37, 8.05s/it]
19%|█▉ | 524/2757 [1:12:03<4:56:07, 7.96s/it]
19%|█▉ | 525/2757 [1:12:10<4:51:43, 7.84s/it]
19%|█▉ | 526/2757 [1:12:18<4:54:56, 7.93s/it]
19%|█▉ | 527/2757 [1:12:26<4:53:28, 7.90s/it]
19%|█▉ | 528/2757 [1:12:34<4:55:12, 7.95s/it]
19%|█▉ | 529/2757 [1:12:42<4:54:24, 7.93s/it]
19%|█▉ | 530/2757 [1:12:50<4:54:31, 7.94s/it]
19%|█▉ | 530/2757 [1:12:50<4:54:31, 7.94s/it]
19%|█▉ | 531/2757 [1:12:58<4:55:44, 7.97s/it]
19%|█▉ | 532/2757 [1:13:06<4:55:46, 7.98s/it]
19%|█▉ | 533/2757 [1:13:14<4:52:19, 7.89s/it]
19%|█▉ | 534/2757 [1:13:22<4:53:03, 7.91s/it]
19%|█▉ | 535/2757 [1:13:30<4:51:32, 7.87s/it]
19%|█▉ | 536/2757 [1:13:38<4:54:57, 7.97s/it]
19%|█▉ | 537/2757 [1:13:45<4:51:27, 7.88s/it]
20%|█▉ | 538/2757 [1:13:53<4:53:09, 7.93s/it]
20%|█▉ | 539/2757 [1:14:01<4:50:54, 7.87s/it]
20%|█▉ | 540/2757 [1:14:09<4:53:03, 7.93s/it]
20%|█▉ | 540/2757 [1:14:09<4:53:03, 7.93s/it]
20%|█▉ | 541/2757 [1:14:17<4:54:32, 7.98s/it]
20%|█▉ | 542/2757 [1:14:25<4:53:00, 7.94s/it]
20%|█▉ | 543/2757 [1:14:34<4:57:26, 8.06s/it]
20%|█▉ | 544/2757 [1:14:41<4:51:29, 7.90s/it]
20%|█▉ | 545/2757 [1:14:49<4:54:20, 7.98s/it]
20%|█▉ | 546/2757 [1:14:58<4:58:13, 8.09s/it]
20%|█▉ | 547/2757 [1:15:05<4:54:39, 8.00s/it]
20%|█▉ | 548/2757 [1:15:14<4:56:26, 8.05s/it]
20%|█▉ | 549/2757 [1:15:22<5:00:59, 8.18s/it]
20%|█▉ | 550/2757 [1:15:30<5:02:26, 8.22s/it]
20%|█▉ | 550/2757 [1:15:30<5:02:26, 8.22s/it]
20%|█▉ | 551/2757 [1:15:39<5:03:46, 8.26s/it]
20%|██ | 552/2757 [1:15:46<4:57:16, 8.09s/it]
20%|██ | 553/2757 [1:15:54<4:56:03, 8.06s/it]
20%|██ | 554/2757 [1:16:02<4:55:32, 8.05s/it]
20%|██ | 555/2757 [1:16:11<4:59:00, 8.15s/it]
20%|██ | 556/2757 [1:16:19<5:00:26, 8.19s/it]
20%|██ | 557/2757 [1:16:27<4:54:49, 8.04s/it]
20%|██ | 558/2757 [1:16:35<4:53:26, 8.01s/it]
20%|██ | 559/2757 [1:16:43<4:54:02, 8.03s/it]
20%|██ | 560/2757 [1:16:52<5:03:55, 8.30s/it]
20%|██ | 560/2757 [1:16:52<5:03:55, 8.30s/it]
20%|██ | 561/2757 [1:17:00<5:02:05, 8.25s/it]
20%|██ | 562/2757 [1:17:08<5:00:18, 8.21s/it]
20%|██ | 563/2757 [1:17:16<4:59:14, 8.18s/it]
20%|██ | 564/2757 [1:17:24<4:56:18, 8.11s/it]
20%|██ | 565/2757 [1:17:33<5:01:51, 8.26s/it]
21%|██ | 566/2757 [1:17:41<5:00:49, 8.24s/it]
21%|██ | 567/2757 [1:17:49<4:55:55, 8.11s/it]
21%|██ | 568/2757 [1:17:57<4:55:24, 8.10s/it]
21%|██ | 569/2757 [1:18:05<4:52:35, 8.02s/it]
21%|██ | 570/2757 [1:18:13<4:56:29, 8.13s/it]
21%|██ | 570/2757 [1:18:13<4:56:29, 8.13s/it]
21%|██ | 571/2757 [1:18:21<4:59:14, 8.21s/it]
21%|██ | 572/2757 [1:18:29<4:58:35, 8.20s/it]
21%|██ | 573/2757 [1:18:38<5:00:35, 8.26s/it]
21%|██ | 574/2757 [1:18:46<4:57:10, 8.17s/it]
21%|██ | 575/2757 [1:18:54<4:52:03, 8.03s/it]
21%|██ | 576/2757 [1:19:02<4:54:31, 8.10s/it]
21%|██ | 577/2757 [1:19:10<4:52:34, 8.05s/it]
21%|██ | 578/2757 [1:19:17<4:49:02, 7.96s/it]
21%|██ | 579/2757 [1:19:25<4:43:39, 7.81s/it]
21%|██ | 580/2757 [1:19:33<4:43:06, 7.80s/it]
21%|██ | 580/2757 [1:19:33<4:43:06, 7.80s/it]
21%|██ | 581/2757 [1:19:41<4:46:11, 7.89s/it]
21%|██ | 582/2757 [1:19:49<4:47:43, 7.94s/it]
21%|██ | 583/2757 [1:19:57<4:46:39, 7.91s/it]
21%|██ | 584/2757 [1:20:05<4:45:10, 7.87s/it]
21%|██ | 585/2757 [1:20:12<4:45:50, 7.90s/it]
21%|██▏ | 586/2757 [1:20:21<4:47:26, 7.94s/it]
21%|██▏ | 587/2757 [1:20:28<4:46:32, 7.92s/it]
21%|██▏ | 588/2757 [1:20:36<4:48:21, 7.98s/it]
21%|██▏ | 589/2757 [1:20:44<4:44:12, 7.87s/it]
21%|██▏ | 590/2757 [1:20:52<4:40:45, 7.77s/it]
21%|██▏ | 590/2757 [1:20:52<4:40:45, 7.77s/it]
21%|██▏ | 591/2757 [1:21:00<4:43:43, 7.86s/it]
21%|██▏ | 592/2757 [1:21:08<4:43:31, 7.86s/it]
22%|██▏ | 593/2757 [1:21:16<4:47:21, 7.97s/it]
22%|██▏ | 594/2757 [1:21:24<4:45:34, 7.92s/it]
22%|██▏ | 595/2757 [1:21:32<4:47:49, 7.99s/it]
22%|██▏ | 596/2757 [1:21:40<4:47:23, 7.98s/it]
22%|██▏ | 597/2757 [1:21:48<4:48:15, 8.01s/it]
22%|██▏ | 598/2757 [1:21:56<4:49:02, 8.03s/it]
22%|██▏ | 599/2757 [1:22:04<4:47:48, 8.00s/it]
22%|██▏ | 600/2757 [1:22:12<4:44:38, 7.92s/it]
22%|██▏ | 600/2757 [1:22:12<4:44:38, 7.92s/it]
22%|██▏ | 601/2757 [1:22:20<4:46:12, 7.97s/it]
22%|██▏ | 602/2757 [1:22:28<4:49:08, 8.05s/it]
22%|██▏ | 603/2757 [1:22:36<4:44:57, 7.94s/it]
22%|██▏ | 604/2757 [1:22:44<4:45:59, 7.97s/it]
22%|██▏ | 605/2757 [1:22:52<4:50:52, 8.11s/it]
22%|██▏ | 606/2757 [1:23:00<4:53:05, 8.18s/it]
22%|██▏ | 607/2757 [1:23:08<4:51:14, 8.13s/it]
22%|██▏ | 608/2757 [1:23:16<4:48:07, 8.04s/it]
22%|██▏ | 609/2757 [1:23:24<4:48:23, 8.06s/it]
22%|██▏ | 610/2757 [1:23:32<4:49:19, 8.09s/it]
22%|██▏ | 610/2757 [1:23:32<4:49:19, 8.09s/it]
22%|██▏ | 611/2757 [1:23:41<4:50:24, 8.12s/it]
22%|██▏ | 612/2757 [1:23:49<4:54:34, 8.24s/it]
22%|██▏ | 613/2757 [1:23:57<4:51:55, 8.17s/it]
22%|██▏ | 614/2757 [1:24:05<4:49:26, 8.10s/it]
22%|██▏ | 615/2757 [1:24:13<4:50:31, 8.14s/it]
22%|██▏ | 616/2757 [1:24:22<4:52:21, 8.19s/it]
22%|██▏ | 617/2757 [1:24:30<4:51:10, 8.16s/it]
22%|██▏ | 618/2757 [1:24:38<4:47:57, 8.08s/it]
22%|██▏ | 619/2757 [1:24:46<4:48:07, 8.09s/it]
22%|██▏ | 620/2757 [1:24:53<4:42:34, 7.93s/it]
22%|██▏ | 620/2757 [1:24:53<4:42:34, 7.93s/it]
23%|██▎ | 621/2757 [1:25:01<4:40:57, 7.89s/it]
23%|██▎ | 622/2757 [1:25:09<4:44:11, 7.99s/it]
23%|██▎ | 623/2757 [1:25:17<4:36:22, 7.77s/it]
23%|██▎ | 624/2757 [1:25:25<4:41:35, 7.92s/it]
23%|██▎ | 625/2757 [1:25:33<4:43:06, 7.97s/it]
23%|██▎ | 626/2757 [1:25:41<4:43:43, 7.99s/it]
23%|██▎ | 627/2757 [1:25:49<4:44:41, 8.02s/it]
23%|██▎ | 628/2757 [1:25:57<4:44:18, 8.01s/it]
23%|██▎ | 629/2757 [1:26:05<4:43:39, 8.00s/it]
23%|██▎ | 630/2757 [1:26:13<4:47:36, 8.11s/it]
23%|██▎ | 630/2757 [1:26:13<4:47:36, 8.11s/it]
23%|██▎ | 631/2757 [1:26:22<4:47:46, 8.12s/it]
23%|██▎ | 632/2757 [1:26:30<4:54:37, 8.32s/it]
23%|██▎ | 633/2757 [1:26:38<4:48:41, 8.16s/it]
23%|██▎ | 634/2757 [1:26:47<4:52:48, 8.28s/it]
23%|██▎ | 635/2757 [1:26:55<4:51:05, 8.23s/it]
23%|██▎ | 636/2757 [1:27:03<4:48:05, 8.15s/it]
23%|██▎ | 637/2757 [1:27:11<4:52:19, 8.27s/it]
23%|██▎ | 638/2757 [1:27:20<4:51:59, 8.27s/it]
23%|██▎ | 639/2757 [1:27:28<4:50:23, 8.23s/it]
23%|██▎ | 640/2757 [1:27:35<4:43:36, 8.04s/it]
23%|██▎ | 640/2757 [1:27:35<4:43:36, 8.04s/it]
23%|██▎ | 641/2757 [1:27:43<4:40:02, 7.94s/it]
23%|██▎ | 642/2757 [1:27:51<4:38:05, 7.89s/it]
23%|██▎ | 643/2757 [1:27:59<4:39:02, 7.92s/it]
23%|██▎ | 644/2757 [1:28:07<4:41:22, 7.99s/it]
23%|██▎ | 645/2757 [1:28:16<4:50:53, 8.26s/it]
23%|██▎ | 646/2757 [1:28:24<4:50:04, 8.24s/it]
23%|██▎ | 647/2757 [1:28:32<4:44:08, 8.08s/it]
24%|██▎ | 648/2757 [1:28:40<4:42:29, 8.04s/it]
24%|██▎ | 649/2757 [1:28:48<4:41:44, 8.02s/it]
24%|██▎ | 650/2757 [1:28:56<4:42:18, 8.04s/it]
24%|██▎ | 650/2757 [1:28:56<4:42:18, 8.04s/it]
24%|██▎ | 651/2757 [1:29:04<4:44:25, 8.10s/it]
24%|██▎ | 652/2757 [1:29:12<4:47:45, 8.20s/it]
24%|██▎ | 653/2757 [1:29:21<4:47:57, 8.21s/it]
24%|██▎ | 654/2757 [1:29:29<4:47:35, 8.21s/it]
24%|██▍ | 655/2757 [1:29:37<4:43:45, 8.10s/it]
24%|██▍ | 656/2757 [1:29:45<4:43:11, 8.09s/it]
24%|██▍ | 657/2757 [1:29:53<4:47:40, 8.22s/it]
24%|██▍ | 658/2757 [1:30:01<4:41:47, 8.06s/it]
24%|██▍ | 659/2757 [1:30:09<4:40:42, 8.03s/it]
24%|██▍ | 660/2757 [1:30:17<4:38:10, 7.96s/it]
24%|██▍ | 660/2757 [1:30:17<4:38:10, 7.96s/it]
24%|██▍ | 661/2757 [1:30:24<4:31:12, 7.76s/it]
24%|██▍ | 662/2757 [1:30:31<4:27:54, 7.67s/it]
24%|██▍ | 663/2757 [1:30:40<4:33:08, 7.83s/it]
24%|██▍ | 664/2757 [1:30:48<4:34:16, 7.86s/it]
24%|██▍ | 665/2757 [1:30:56<4:40:29, 8.04s/it]
24%|██▍ | 666/2757 [1:31:04<4:41:44, 8.08s/it]
24%|██▍ | 667/2757 [1:31:13<4:45:48, 8.21s/it]
24%|██▍ | 668/2757 [1:31:21<4:41:54, 8.10s/it]
24%|██▍ | 669/2757 [1:31:28<4:38:12, 7.99s/it]
24%|██▍ | 670/2757 [1:31:36<4:39:08, 8.03s/it]
24%|██▍ | 670/2757 [1:31:36<4:39:08, 8.03s/it]
24%|██▍ | 671/2757 [1:31:45<4:40:37, 8.07s/it]
24%|██▍ | 672/2757 [1:31:52<4:35:39, 7.93s/it]
24%|██▍ | 673/2757 [1:32:00<4:32:02, 7.83s/it]
24%|██▍ | 674/2757 [1:32:08<4:34:47, 7.92s/it]
24%|██▍ | 675/2757 [1:32:15<4:31:05, 7.81s/it]
25%|██▍ | 676/2757 [1:32:23<4:30:35, 7.80s/it]
25%|██▍ | 677/2757 [1:32:32<4:36:49, 7.99s/it]
25%|██▍ | 678/2757 [1:32:40<4:41:07, 8.11s/it]
25%|██▍ | 679/2757 [1:32:48<4:41:14, 8.12s/it]
25%|██▍ | 680/2757 [1:32:56<4:38:54, 8.06s/it]
25%|██▍ | 680/2757 [1:32:56<4:38:54, 8.06s/it]
25%|██▍ | 681/2757 [1:33:04<4:31:58, 7.86s/it]
25%|██▍ | 682/2757 [1:33:12<4:33:14, 7.90s/it]
25%|██▍ | 683/2757 [1:33:19<4:31:25, 7.85s/it]
25%|██▍ | 684/2757 [1:33:27<4:34:15, 7.94s/it]
25%|██▍ | 685/2757 [1:33:35<4:33:51, 7.93s/it]
25%|██▍ | 686/2757 [1:33:43<4:33:15, 7.92s/it]
25%|██▍ | 687/2757 [1:33:51<4:33:59, 7.94s/it]
25%|██▍ | 688/2757 [1:33:59<4:27:13, 7.75s/it]
25%|██▍ | 689/2757 [1:34:06<4:28:54, 7.80s/it]
25%|██▌ | 690/2757 [1:34:15<4:33:13, 7.93s/it]
25%|██▌ | 690/2757 [1:34:15<4:33:13, 7.93s/it]
25%|██▌ | 691/2757 [1:34:22<4:29:59, 7.84s/it]
25%|██▌ | 692/2757 [1:34:30<4:29:43, 7.84s/it]
25%|██▌ | 693/2757 [1:34:38<4:33:57, 7.96s/it]
25%|██▌ | 694/2757 [1:34:47<4:36:22, 8.04s/it]
25%|██▌ | 695/2757 [1:34:55<4:40:55, 8.17s/it]
25%|██▌ | 696/2757 [1:35:03<4:36:55, 8.06s/it]
25%|██▌ | 697/2757 [1:35:11<4:35:05, 8.01s/it]
25%|██▌ | 698/2757 [1:35:19<4:36:50, 8.07s/it]
25%|██▌ | 699/2757 [1:35:27<4:36:34, 8.06s/it]
25%|██▌ | 700/2757 [1:35:35<4:34:19, 8.00s/it]
25%|██▌ | 700/2757 [1:35:35<4:34:19, 8.00s/it]
25%|██▌ | 701/2757 [1:35:43<4:34:35, 8.01s/it]
25%|██▌ | 702/2757 [1:35:51<4:33:25, 7.98s/it]
25%|██▌ | 703/2757 [1:35:59<4:30:29, 7.90s/it]
26%|██▌ | 704/2757 [1:36:06<4:28:10, 7.84s/it]
26%|██▌ | 705/2757 [1:36:14<4:27:55, 7.83s/it]
26%|██▌ | 706/2757 [1:36:23<4:33:58, 8.01s/it]
26%|██▌ | 707/2757 [1:36:30<4:32:56, 7.99s/it]
26%|██▌ | 708/2757 [1:36:38<4:32:11, 7.97s/it]
26%|██▌ | 709/2757 [1:36:46<4:27:26, 7.84s/it]
26%|██▌ | 710/2757 [1:36:54<4:29:01, 7.89s/it]
26%|██▌ | 710/2757 [1:36:54<4:29:01, 7.89s/it]
26%|██▌ | 711/2757 [1:37:02<4:31:54, 7.97s/it]
26%|██▌ | 712/2757 [1:37:10<4:30:56, 7.95s/it]
26%|██▌ | 713/2757 [1:37:18<4:29:57, 7.92s/it]
26%|██▌ | 714/2757 [1:37:26<4:30:56, 7.96s/it]
26%|██▌ | 715/2757 [1:37:34<4:28:21, 7.89s/it]
26%|██▌ | 716/2757 [1:37:42<4:28:57, 7.91s/it]
26%|██▌ | 717/2757 [1:37:50<4:33:00, 8.03s/it]
26%|██▌ | 718/2757 [1:37:57<4:28:44, 7.91s/it]
26%|██▌ | 719/2757 [1:38:05<4:25:26, 7.81s/it]
26%|██▌ | 720/2757 [1:38:13<4:25:41, 7.83s/it]
26%|██▌ | 720/2757 [1:38:13<4:25:41, 7.83s/it]
26%|██▌ | 721/2757 [1:38:20<4:22:46, 7.74s/it]
26%|██▌ | 722/2757 [1:38:28<4:23:05, 7.76s/it]
26%|██▌ | 723/2757 [1:38:37<4:30:15, 7.97s/it]
26%|██▋ | 724/2757 [1:38:44<4:27:46, 7.90s/it]
26%|██▋ | 725/2757 [1:38:52<4:24:20, 7.81s/it]
26%|██▋ | 726/2757 [1:39:01<4:31:17, 8.01s/it]
26%|██▋ | 727/2757 [1:39:09<4:37:03, 8.19s/it]
26%|██▋ | 728/2757 [1:39:17<4:34:19, 8.11s/it]
26%|██▋ | 729/2757 [1:39:25<4:32:14, 8.05s/it]
26%|██▋ | 730/2757 [1:39:33<4:29:33, 7.98s/it]
26%|██▋ | 730/2757 [1:39:33<4:29:33, 7.98s/it]
27%|██▋ | 731/2757 [1:39:41<4:27:45, 7.93s/it]
27%|██▋ | 732/2757 [1:39:49<4:28:19, 7.95s/it]
27%|██▋ | 733/2757 [1:39:57<4:28:42, 7.97s/it]
27%|██▋ | 734/2757 [1:40:05<4:28:14, 7.96s/it]
27%|██▋ | 735/2757 [1:40:13<4:33:13, 8.11s/it]
27%|██▋ | 736/2757 [1:40:21<4:34:56, 8.16s/it]
27%|██▋ | 737/2757 [1:40:29<4:32:04, 8.08s/it]
27%|██▋ | 738/2757 [1:40:37<4:29:44, 8.02s/it]
27%|██▋ | 739/2757 [1:40:45<4:24:21, 7.86s/it]
27%|██▋ | 740/2757 [1:40:53<4:33:32, 8.14s/it]
27%|██▋ | 740/2757 [1:40:53<4:33:32, 8.14s/it]
27%|██▋ | 741/2757 [1:41:01<4:32:02, 8.10s/it]
27%|██▋ | 742/2757 [1:41:09<4:32:17, 8.11s/it]
27%|██▋ | 743/2757 [1:41:18<4:37:26, 8.27s/it]
27%|██▋ | 744/2757 [1:41:26<4:32:39, 8.13s/it]
27%|██▋ | 745/2757 [1:41:34<4:32:33, 8.13s/it]
27%|██▋ | 746/2757 [1:41:42<4:27:01, 7.97s/it]
27%|██▋ | 747/2757 [1:41:49<4:22:32, 7.84s/it]
27%|██▋ | 748/2757 [1:41:57<4:25:00, 7.91s/it]
27%|██▋ | 749/2757 [1:42:05<4:25:26, 7.93s/it]
27%|██▋ | 750/2757 [1:42:13<4:25:53, 7.95s/it]
{'eval_loss': '0.5771', 'eval_runtime': '104.9', 'eval_samples_per_second': '22.88', 'eval_steps_per_second': '2.86', 'eval_rewards/chosen': '-0.6041', 'eval_rewards/rejected': '-1.069', 'eval_rewards/accuracies': '0.625', 'eval_rewards/margins': '0.4646', 'eval_logps/chosen': '-85.65', 'eval_logps/rejected': '-96.54', 'eval_logits/chosen': '-3.086', 'eval_logits/rejected': '-3.039', 'epoch': '0.5442'}
{'loss': '0.5776', 'grad_norm': '0.5719', 'learning_rate': '0.0001681', 'rewards/chosen': '-0.6292', 'rewards/rejected': '-1.103', 'rewards/accuracies': '0.6281', 'rewards/margins': '0.4741', 'logps/chosen': '-84.31', 'logps/rejected': '-96.12', 'logits/chosen': '-2.981', 'logits/rejected': '-3.005', 'epoch': '0.5551'}
{'loss': '0.5797', 'grad_norm': '0.6719', 'learning_rate': '0.0001674', 'rewards/chosen': '-0.6672', 'rewards/rejected': '-1.152', 'rewards/accuracies': '0.6539', 'rewards/margins': '0.4843', 'logps/chosen': '-84.52', 'logps/rejected': '-95.7', 'logits/chosen': '-3.107', 'logits/rejected': '-3.058', 'epoch': '0.566'}
{'loss': '0.5927', 'grad_norm': '0.6522', 'learning_rate': '0.0001666', 'rewards/chosen': '-0.6613', 'rewards/rejected': '-1.109', 'rewards/accuracies': '0.625', 'rewards/margins': '0.4473', 'logps/chosen': '-85.24', 'logps/rejected': '-94.5', 'logits/chosen': '-3.016', 'logits/rejected': '-3.001', 'epoch': '0.5769'}
{'loss': '0.6019', 'grad_norm': '0.9451', 'learning_rate': '0.0001659', 'rewards/chosen': '-0.6853', 'rewards/rejected': '-1.141', 'rewards/accuracies': '0.6266', 'rewards/margins': '0.4554', 'logps/chosen': '-87.74', 'logps/rejected': '-98.06', 'logits/chosen': '-2.828', 'logits/rejected': '-2.82', 'epoch': '0.5878'}
{'loss': '0.5943', 'grad_norm': '0.6104', 'learning_rate': '0.0001651', 'rewards/chosen': '-0.7626', 'rewards/rejected': '-1.175', 'rewards/accuracies': '0.6313', 'rewards/margins': '0.4125', 'logps/chosen': '-86.3', 'logps/rejected': '-96.72', 'logits/chosen': '-2.712', 'logits/rejected': '-2.636', 'epoch': '0.5986'}
{'loss': '0.5997', 'grad_norm': '0.5953', 'learning_rate': '0.0001644', 'rewards/chosen': '-0.7641', 'rewards/rejected': '-1.156', 'rewards/accuracies': '0.632', 'rewards/margins': '0.3917', 'logps/chosen': '-85.59', 'logps/rejected': '-94.78', 'logits/chosen': '-2.773', 'logits/rejected': '-2.675', 'epoch': '0.6095'}
{'loss': '0.5831', 'grad_norm': '0.6583', 'learning_rate': '0.0001636', 'rewards/chosen': '-0.7877', 'rewards/rejected': '-1.254', 'rewards/accuracies': '0.6258', 'rewards/margins': '0.4662', 'logps/chosen': '-90.34', 'logps/rejected': '-99.95', 'logits/chosen': '-2.784', 'logits/rejected': '-2.651', 'epoch': '0.6204'}
{'loss': '0.5975', 'grad_norm': '0.5589', 'learning_rate': '0.0001629', 'rewards/chosen': '-0.5206', 'rewards/rejected': '-0.9202', 'rewards/accuracies': '0.625', 'rewards/margins': '0.3997', 'logps/chosen': '-83.24', 'logps/rejected': '-91.77', 'logits/chosen': '-2.907', 'logits/rejected': '-2.885', 'epoch': '0.6313'}
{'loss': '0.5781', 'grad_norm': '0.6037', 'learning_rate': '0.0001622', 'rewards/chosen': '-0.4098', 'rewards/rejected': '-0.8762', 'rewards/accuracies': '0.6414', 'rewards/margins': '0.4664', 'logps/chosen': '-83.76', 'logps/rejected': '-94.21', 'logits/chosen': '-2.92', 'logits/rejected': '-2.842', 'epoch': '0.6422'}
{'loss': '0.5839', 'grad_norm': '0.6424', 'learning_rate': '0.0001614', 'rewards/chosen': '-0.5493', 'rewards/rejected': '-0.9931', 'rewards/accuracies': '0.6586', 'rewards/margins': '0.4438', 'logps/chosen': '-86.28', 'logps/rejected': '-96.4', 'logits/chosen': '-2.929', 'logits/rejected': '-2.954', 'epoch': '0.6531'}
{'loss': '0.5724', 'grad_norm': '0.5479', 'learning_rate': '0.0001607', 'rewards/chosen': '-0.5622', 'rewards/rejected': '-1.034', 'rewards/accuracies': '0.6438', 'rewards/margins': '0.472', 'logps/chosen': '-87.9', 'logps/rejected': '-96.35', 'logits/chosen': '-2.836', 'logits/rejected': '-2.8', 'epoch': '0.6639'}
{'loss': '0.5745', 'grad_norm': '0.6719', 'learning_rate': '0.0001599', 'rewards/chosen': '-0.7492', 'rewards/rejected': '-1.227', 'rewards/accuracies': '0.6531', 'rewards/margins': '0.478', 'logps/chosen': '-88.12', 'logps/rejected': '-98.25', 'logits/chosen': '-2.963', 'logits/rejected': '-2.848', 'epoch': '0.6748'}
{'loss': '0.5667', 'grad_norm': '0.6403', 'learning_rate': '0.0001592', 'rewards/chosen': '-0.9368', 'rewards/rejected': '-1.455', 'rewards/accuracies': '0.6602', 'rewards/margins': '0.5185', 'logps/chosen': '-90.78', 'logps/rejected': '-103.6', 'logits/chosen': '-2.943', 'logits/rejected': '-2.97', 'epoch': '0.6857'}
{'loss': '0.5546', 'grad_norm': '0.6475', 'learning_rate': '0.0001584', 'rewards/chosen': '-0.9921', 'rewards/rejected': '-1.549', 'rewards/accuracies': '0.6461', 'rewards/margins': '0.557', 'logps/chosen': '-89.43', 'logps/rejected': '-100.7', 'logits/chosen': '-2.979', 'logits/rejected': '-2.938', 'epoch': '0.6966'}
{'loss': '0.574', 'grad_norm': '0.7136', 'learning_rate': '0.0001577', 'rewards/chosen': '-1.028', 'rewards/rejected': '-1.534', 'rewards/accuracies': '0.6375', 'rewards/margins': '0.5058', 'logps/chosen': '-91.74', 'logps/rejected': '-102.1', 'logits/chosen': '-3.021', 'logits/rejected': '-2.974', 'epoch': '0.7075'}
{'loss': '0.56', 'grad_norm': '0.6012', 'learning_rate': '0.0001569', 'rewards/chosen': '-1.082', 'rewards/rejected': '-1.642', 'rewards/accuracies': '0.6555', 'rewards/margins': '0.5598', 'logps/chosen': '-90.38', 'logps/rejected': '-101.6', 'logits/chosen': '-3.157', 'logits/rejected': '-3.125', 'epoch': '0.7184'}
{'loss': '0.5732', 'grad_norm': '0.6408', 'learning_rate': '0.0001562', 'rewards/chosen': '-1.105', 'rewards/rejected': '-1.613', 'rewards/accuracies': '0.6477', 'rewards/margins': '0.5072', 'logps/chosen': '-89.5', 'logps/rejected': '-100.5', 'logits/chosen': '-3.229', 'logits/rejected': '-3.236', 'epoch': '0.7293'}
{'loss': '0.5617', 'grad_norm': '0.6599', 'learning_rate': '0.0001554', 'rewards/chosen': '-0.8735', 'rewards/rejected': '-1.432', 'rewards/accuracies': '0.6641', 'rewards/margins': '0.5582', 'logps/chosen': '-91.57', 'logps/rejected': '-102.2', 'logits/chosen': '-3.211', 'logits/rejected': '-3.173', 'epoch': '0.7401'}
{'loss': '0.5498', 'grad_norm': '0.6686', 'learning_rate': '0.0001547', 'rewards/chosen': '-0.6966', 'rewards/rejected': '-1.305', 'rewards/accuracies': '0.6695', 'rewards/margins': '0.6086', 'logps/chosen': '-90.9', 'logps/rejected': '-102.5', 'logits/chosen': '-3.235', 'logits/rejected': '-3.097', 'epoch': '0.751'}
{'loss': '0.5865', 'grad_norm': '0.7763', 'learning_rate': '0.0001539', 'rewards/chosen': '-0.608', 'rewards/rejected': '-1.119', 'rewards/accuracies': '0.6539', 'rewards/margins': '0.5112', 'logps/chosen': '-85.77', 'logps/rejected': '-96.22', 'logits/chosen': '-3.261', 'logits/rejected': '-3.15', 'epoch': '0.7619'}
{'loss': '0.5637', 'grad_norm': '0.6534', 'learning_rate': '0.0001532', 'rewards/chosen': '-0.7708', 'rewards/rejected': '-1.309', 'rewards/accuracies': '0.6508', 'rewards/margins': '0.5377', 'logps/chosen': '-88.75', 'logps/rejected': '-99.82', 'logits/chosen': '-2.995', 'logits/rejected': '-2.962', 'epoch': '0.7728'}
{'loss': '0.5601', 'grad_norm': '0.5064', 'learning_rate': '0.0001524', 'rewards/chosen': '-0.8121', 'rewards/rejected': '-1.338', 'rewards/accuracies': '0.6375', 'rewards/margins': '0.5259', 'logps/chosen': '-84.73', 'logps/rejected': '-96.15', 'logits/chosen': '-3.161', 'logits/rejected': '-3.103', 'epoch': '0.7837'}
{'loss': '0.5589', 'grad_norm': '0.603', 'learning_rate': '0.0001517', 'rewards/chosen': '-0.7111', 'rewards/rejected': '-1.252', 'rewards/accuracies': '0.65', 'rewards/margins': '0.5404', 'logps/chosen': '-85.85', 'logps/rejected': '-96.86', 'logits/chosen': '-3.101', 'logits/rejected': '-3.052', 'epoch': '0.7946'}
{'loss': '0.5412', 'grad_norm': '0.608', 'learning_rate': '0.0001509', 'rewards/chosen': '-0.7364', 'rewards/rejected': '-1.331', 'rewards/accuracies': '0.6672', 'rewards/margins': '0.5946', 'logps/chosen': '-91.16', 'logps/rejected': '-101.2', 'logits/chosen': '-3.177', 'logits/rejected': '-3.198', 'epoch': '0.8054'}
{'loss': '0.5476', 'grad_norm': '0.6249', 'learning_rate': '0.0001502', 'rewards/chosen': '-0.6863', 'rewards/rejected': '-1.317', 'rewards/accuracies': '0.668', 'rewards/margins': '0.6311', 'logps/chosen': '-85.68', 'logps/rejected': '-97.83', 'logits/chosen': '-3.299', 'logits/rejected': '-3.16', 'epoch': '0.8163'}
27%|██▋ | 750/2757 [1:42:13<4:25:53, 7.95s/it]
27%|██▋ | 751/2757 [1:42:25<5:02:01, 9.03s/it]
27%|██▋ | 752/2757 [1:42:33<4:50:08, 8.68s/it]
27%|██▋ | 753/2757 [1:42:40<4:37:16, 8.30s/it]
27%|██▋ | 754/2757 [1:42:48<4:35:19, 8.25s/it]
27%|██▋ | 755/2757 [1:42:57<4:36:26, 8.29s/it]
27%|██▋ | 756/2757 [1:43:05<4:34:48, 8.24s/it]
27%|██▋ | 757/2757 [1:43:13<4:35:00, 8.25s/it]
27%|██▋ | 758/2757 [1:43:21<4:33:42, 8.22s/it]
28%|██▊ | 759/2757 [1:43:30<4:35:52, 8.28s/it]
28%|██▊ | 760/2757 [1:43:37<4:31:39, 8.16s/it]
28%|██▊ | 760/2757 [1:43:37<4:31:39, 8.16s/it]
28%|██▊ | 761/2757 [1:43:45<4:28:49, 8.08s/it]
28%|██▊ | 762/2757 [1:43:54<4:30:20, 8.13s/it]
28%|██▊ | 763/2757 [1:44:02<4:31:47, 8.18s/it]
28%|██▊ | 764/2757 [1:44:09<4:25:25, 7.99s/it]
28%|██▊ | 765/2757 [1:44:17<4:21:32, 7.88s/it]
28%|██▊ | 766/2757 [1:44:25<4:22:31, 7.91s/it]
28%|██▊ | 767/2757 [1:44:33<4:26:36, 8.04s/it]
28%|██▊ | 768/2757 [1:44:42<4:29:57, 8.14s/it]
28%|██▊ | 769/2757 [1:44:50<4:26:13, 8.03s/it]
28%|██▊ | 770/2757 [1:44:57<4:23:28, 7.96s/it]
28%|██▊ | 770/2757 [1:44:57<4:23:28, 7.96s/it]
28%|██▊ | 771/2757 [1:45:05<4:25:31, 8.02s/it]
28%|██▊ | 772/2757 [1:45:14<4:27:16, 8.08s/it]
28%|██▊ | 773/2757 [1:45:22<4:27:16, 8.08s/it]
28%|██▊ | 774/2757 [1:45:30<4:25:37, 8.04s/it]
28%|██▊ | 775/2757 [1:45:38<4:25:14, 8.03s/it]
28%|██▊ | 776/2757 [1:45:46<4:26:07, 8.06s/it]
28%|██▊ | 777/2757 [1:45:54<4:22:47, 7.96s/it]
28%|██▊ | 778/2757 [1:46:01<4:20:55, 7.91s/it]
28%|██▊ | 779/2757 [1:46:09<4:22:31, 7.96s/it]
28%|██▊ | 780/2757 [1:46:17<4:18:58, 7.86s/it]
28%|██▊ | 780/2757 [1:46:17<4:18:58, 7.86s/it]
28%|██▊ | 781/2757 [1:46:26<4:25:22, 8.06s/it]
28%|██▊ | 782/2757 [1:46:34<4:24:28, 8.03s/it]
28%|██▊ | 783/2757 [1:46:41<4:20:20, 7.91s/it]
28%|██▊ | 784/2757 [1:46:49<4:20:02, 7.91s/it]
28%|██▊ | 785/2757 [1:46:57<4:23:30, 8.02s/it]
29%|██▊ | 786/2757 [1:47:05<4:19:49, 7.91s/it]
29%|██▊ | 787/2757 [1:47:13<4:19:16, 7.90s/it]
29%|██▊ | 788/2757 [1:47:21<4:20:24, 7.94s/it]
29%|██▊ | 789/2757 [1:47:29<4:22:53, 8.02s/it]
29%|██▊ | 790/2757 [1:47:37<4:18:18, 7.88s/it]
29%|██▊ | 790/2757 [1:47:37<4:18:18, 7.88s/it]
29%|██▊ | 791/2757 [1:47:44<4:16:32, 7.83s/it]
29%|██▊ | 792/2757 [1:47:52<4:14:41, 7.78s/it]
29%|██▉ | 793/2757 [1:48:00<4:13:26, 7.74s/it]
29%|██▉ | 794/2757 [1:48:07<4:11:02, 7.67s/it]
29%|██▉ | 795/2757 [1:48:15<4:12:41, 7.73s/it]
29%|██▉ | 796/2757 [1:48:23<4:15:42, 7.82s/it]
29%|██▉ | 797/2757 [1:48:31<4:16:44, 7.86s/it]
29%|██▉ | 798/2757 [1:48:39<4:17:43, 7.89s/it]
29%|██▉ | 799/2757 [1:48:47<4:17:56, 7.90s/it]
29%|██▉ | 800/2757 [1:48:55<4:18:34, 7.93s/it]
29%|██▉ | 800/2757 [1:48:55<4:18:34, 7.93s/it]
29%|██▉ | 801/2757 [1:49:03<4:20:00, 7.98s/it]
29%|██▉ | 802/2757 [1:49:11<4:20:17, 7.99s/it]
29%|██▉ | 803/2757 [1:49:19<4:19:35, 7.97s/it]
29%|██▉ | 804/2757 [1:49:27<4:20:49, 8.01s/it]
29%|██▉ | 805/2757 [1:49:35<4:21:01, 8.02s/it]
29%|██▉ | 806/2757 [1:49:43<4:19:24, 7.98s/it]
29%|██▉ | 807/2757 [1:49:51<4:16:11, 7.88s/it]
29%|██▉ | 808/2757 [1:49:58<4:12:47, 7.78s/it]
29%|██▉ | 809/2757 [1:50:06<4:14:54, 7.85s/it]
29%|██▉ | 810/2757 [1:50:14<4:14:54, 7.86s/it]
29%|██▉ | 810/2757 [1:50:14<4:14:54, 7.86s/it]
29%|██▉ | 811/2757 [1:50:22<4:17:39, 7.94s/it]
29%|██▉ | 812/2757 [1:50:30<4:18:44, 7.98s/it]
29%|██▉ | 813/2757 [1:50:39<4:21:04, 8.06s/it]
30%|██▉ | 814/2757 [1:50:47<4:20:41, 8.05s/it]
30%|██▉ | 815/2757 [1:50:55<4:21:15, 8.07s/it]
30%|██▉ | 816/2757 [1:51:03<4:21:10, 8.07s/it]
30%|██▉ | 817/2757 [1:51:11<4:21:42, 8.09s/it]
30%|██▉ | 818/2757 [1:51:19<4:18:45, 8.01s/it]
30%|██▉ | 819/2757 [1:51:27<4:18:13, 7.99s/it]
30%|██▉ | 820/2757 [1:51:35<4:16:44, 7.95s/it]
30%|██▉ | 820/2757 [1:51:35<4:16:44, 7.95s/it]
30%|██▉ | 821/2757 [1:51:43<4:16:28, 7.95s/it]
30%|██▉ | 822/2757 [1:51:51<4:17:36, 7.99s/it]
30%|██▉ | 823/2757 [1:51:58<4:14:10, 7.89s/it]
30%|██▉ | 824/2757 [1:52:06<4:11:44, 7.81s/it]
30%|██▉ | 825/2757 [1:52:13<4:07:02, 7.67s/it]
30%|██▉ | 826/2757 [1:52:21<4:10:58, 7.80s/it]
30%|██▉ | 827/2757 [1:52:29<4:10:36, 7.79s/it]
30%|███ | 828/2757 [1:52:37<4:11:05, 7.81s/it]
30%|███ | 829/2757 [1:52:45<4:09:20, 7.76s/it]
30%|███ | 830/2757 [1:52:52<4:10:41, 7.81s/it]
30%|███ | 830/2757 [1:52:52<4:10:41, 7.81s/it]
30%|███ | 831/2757 [1:53:01<4:15:33, 7.96s/it]
30%|███ | 832/2757 [1:53:09<4:12:58, 7.89s/it]
30%|███ | 833/2757 [1:53:17<4:14:32, 7.94s/it]
30%|███ | 834/2757 [1:53:24<4:13:02, 7.89s/it]
30%|███ | 835/2757 [1:53:32<4:12:32, 7.88s/it]
30%|███ | 836/2757 [1:53:40<4:12:39, 7.89s/it]
30%|███ | 837/2757 [1:53:48<4:16:51, 8.03s/it]
30%|███ | 838/2757 [1:53:56<4:13:19, 7.92s/it]
30%|███ | 839/2757 [1:54:05<4:17:09, 8.04s/it]
30%|███ | 840/2757 [1:54:13<4:16:53, 8.04s/it]
30%|███ | 840/2757 [1:54:13<4:16:53, 8.04s/it]
31%|███ | 841/2757 [1:54:21<4:19:30, 8.13s/it]
31%|███ | 842/2757 [1:54:29<4:20:05, 8.15s/it]
31%|███ | 843/2757 [1:54:38<4:23:11, 8.25s/it]
31%|███ | 844/2757 [1:54:45<4:18:12, 8.10s/it]
31%|███ | 845/2757 [1:54:53<4:16:49, 8.06s/it]
31%|███ | 846/2757 [1:55:01<4:13:59, 7.97s/it]
31%|███ | 847/2757 [1:55:09<4:09:02, 7.82s/it]
31%|███ | 848/2757 [1:55:16<4:08:12, 7.80s/it]
31%|███ | 849/2757 [1:55:24<4:10:15, 7.87s/it]
31%|███ | 850/2757 [1:55:32<4:12:21, 7.94s/it]
31%|███ | 850/2757 [1:55:32<4:12:21, 7.94s/it]
31%|███ | 851/2757 [1:55:40<4:11:58, 7.93s/it]
31%|███ | 852/2757 [1:55:48<4:12:03, 7.94s/it]
31%|███ | 853/2757 [1:55:56<4:13:45, 8.00s/it]
31%|███ | 854/2757 [1:56:04<4:13:21, 7.99s/it]
31%|███ | 855/2757 [1:56:12<4:14:35, 8.03s/it]
31%|███ | 856/2757 [1:56:20<4:09:33, 7.88s/it]
31%|███ | 857/2757 [1:56:28<4:12:43, 7.98s/it]
31%|███ | 858/2757 [1:56:36<4:08:40, 7.86s/it]
31%|███ | 859/2757 [1:56:44<4:14:31, 8.05s/it]
31%|███ | 860/2757 [1:56:52<4:15:06, 8.07s/it]
31%|███ | 860/2757 [1:56:52<4:15:06, 8.07s/it]
31%|███ | 861/2757 [1:57:01<4:17:50, 8.16s/it]
31%|███▏ | 862/2757 [1:57:09<4:17:01, 8.14s/it]
31%|███▏ | 863/2757 [1:57:17<4:17:28, 8.16s/it]
31%|███▏ | 864/2757 [1:57:25<4:16:48, 8.14s/it]
31%|███▏ | 865/2757 [1:57:33<4:16:26, 8.13s/it]
31%|███▏ | 866/2757 [1:57:41<4:16:47, 8.15s/it]
31%|███▏ | 867/2757 [1:57:49<4:15:30, 8.11s/it]
31%|███▏ | 868/2757 [1:57:57<4:12:44, 8.03s/it]
32%|███▏ | 869/2757 [1:58:06<4:14:20, 8.08s/it]
32%|███▏ | 870/2757 [1:58:13<4:12:29, 8.03s/it]
32%|███▏ | 870/2757 [1:58:13<4:12:29, 8.03s/it]
32%|███▏ | 871/2757 [1:58:22<4:15:38, 8.13s/it]
32%|███▏ | 872/2757 [1:58:30<4:11:17, 8.00s/it]
32%|███▏ | 873/2757 [1:58:37<4:07:54, 7.90s/it]
32%|███▏ | 874/2757 [1:58:45<4:07:19, 7.88s/it]
32%|███▏ | 875/2757 [1:58:53<4:06:46, 7.87s/it]
32%|███▏ | 876/2757 [1:59:01<4:07:48, 7.90s/it]
32%|███▏ | 877/2757 [1:59:10<4:18:00, 8.23s/it]
32%|███▏ | 878/2757 [1:59:18<4:14:29, 8.13s/it]
32%|███▏ | 879/2757 [1:59:26<4:14:30, 8.13s/it]
32%|███▏ | 880/2757 [1:59:34<4:11:47, 8.05s/it]
32%|███▏ | 880/2757 [1:59:34<4:11:47, 8.05s/it]
32%|███▏ | 881/2757 [1:59:42<4:12:35, 8.08s/it]
32%|███▏ | 882/2757 [1:59:50<4:12:27, 8.08s/it]
32%|███▏ | 883/2757 [1:59:58<4:12:35, 8.09s/it]
32%|███▏ | 884/2757 [2:00:06<4:11:39, 8.06s/it]
32%|███▏ | 885/2757 [2:00:14<4:11:25, 8.06s/it]
32%|███▏ | 886/2757 [2:00:23<4:16:52, 8.24s/it]
32%|███▏ | 887/2757 [2:00:31<4:15:02, 8.18s/it]
32%|███▏ | 888/2757 [2:00:39<4:11:31, 8.07s/it]
32%|███▏ | 889/2757 [2:00:46<4:08:38, 7.99s/it]
32%|███▏ | 890/2757 [2:00:54<4:05:52, 7.90s/it]
32%|███▏ | 890/2757 [2:00:54<4:05:52, 7.90s/it]
32%|███▏ | 891/2757 [2:01:02<4:07:10, 7.95s/it]
32%|███▏ | 892/2757 [2:01:10<4:03:06, 7.82s/it]
32%|███▏ | 893/2757 [2:01:17<4:02:31, 7.81s/it]
32%|███▏ | 894/2757 [2:01:25<4:03:47, 7.85s/it]
32%|███▏ | 895/2757 [2:01:33<4:00:13, 7.74s/it]
32%|███▏ | 896/2757 [2:01:41<4:05:54, 7.93s/it]
33%|███▎ | 897/2757 [2:01:49<4:05:23, 7.92s/it]
33%|███▎ | 898/2757 [2:01:57<4:05:21, 7.92s/it]
33%|███▎ | 899/2757 [2:02:05<4:06:15, 7.95s/it]
33%|███▎ | 900/2757 [2:02:14<4:10:53, 8.11s/it]
33%|███▎ | 900/2757 [2:02:14<4:10:53, 8.11s/it]
33%|███▎ | 901/2757 [2:02:21<4:08:48, 8.04s/it]
33%|███▎ | 902/2757 [2:02:29<4:05:04, 7.93s/it]
33%|███▎ | 903/2757 [2:02:37<4:01:10, 7.80s/it]
33%|███▎ | 904/2757 [2:02:44<3:58:52, 7.74s/it]
33%|███▎ | 905/2757 [2:02:52<4:03:40, 7.89s/it]
33%|███▎ | 906/2757 [2:03:00<4:03:28, 7.89s/it]
33%|███▎ | 907/2757 [2:03:09<4:06:13, 7.99s/it]
33%|███▎ | 908/2757 [2:03:17<4:08:47, 8.07s/it]
33%|███▎ | 909/2757 [2:03:25<4:07:38, 8.04s/it]
33%|███▎ | 910/2757 [2:03:33<4:06:17, 8.00s/it]
33%|███▎ | 910/2757 [2:03:33<4:06:17, 8.00s/it]
33%|███▎ | 911/2757 [2:03:41<4:08:09, 8.07s/it]
33%|███▎ | 912/2757 [2:03:49<4:05:18, 7.98s/it]
33%|███▎ | 913/2757 [2:03:56<4:01:59, 7.87s/it]
33%|███▎ | 914/2757 [2:04:05<4:08:51, 8.10s/it]
33%|███▎ | 915/2757 [2:04:13<4:11:25, 8.19s/it]
33%|███▎ | 916/2757 [2:04:21<4:09:42, 8.14s/it]
33%|███▎ | 917/2757 [2:04:29<4:04:35, 7.98s/it]
33%|███▎ | 918/2757 [2:04:37<4:04:56, 7.99s/it]
33%|███▎ | 919/2757 [2:04:43<3:45:56, 7.38s/it]
{'loss': '0.5396', 'grad_norm': '0.7027', 'learning_rate': '0.0001494', 'rewards/chosen': '-0.8498', 'rewards/rejected': '-1.501', 'rewards/accuracies': '0.6695', 'rewards/margins': '0.6513', 'logps/chosen': '-87.9', 'logps/rejected': '-101.8', 'logits/chosen': '-3.371', 'logits/rejected': '-3.293', 'epoch': '0.8272'}
{'loss': '0.5349', 'grad_norm': '0.6295', 'learning_rate': '0.0001487', 'rewards/chosen': '-1.063', 'rewards/rejected': '-1.742', 'rewards/accuracies': '0.6594', 'rewards/margins': '0.6796', 'logps/chosen': '-89.56', 'logps/rejected': '-102.7', 'logits/chosen': '-3.39', 'logits/rejected': '-3.347', 'epoch': '0.8381'}
{'loss': '0.5414', 'grad_norm': '0.6378', 'learning_rate': '0.0001479', 'rewards/chosen': '-1.214', 'rewards/rejected': '-1.858', 'rewards/accuracies': '0.6672', 'rewards/margins': '0.6445', 'logps/chosen': '-92.15', 'logps/rejected': '-104.5', 'logits/chosen': '-3.227', 'logits/rejected': '-3.154', 'epoch': '0.849'}
{'loss': '0.5625', 'grad_norm': '0.7089', 'learning_rate': '0.0001472', 'rewards/chosen': '-1.144', 'rewards/rejected': '-1.71', 'rewards/accuracies': '0.6352', 'rewards/margins': '0.5667', 'logps/chosen': '-92.22', 'logps/rejected': '-102.2', 'logits/chosen': '-3.079', 'logits/rejected': '-3.059', 'epoch': '0.8599'}
{'loss': '0.5564', 'grad_norm': '0.672', 'learning_rate': '0.0001464', 'rewards/chosen': '-1.028', 'rewards/rejected': '-1.614', 'rewards/accuracies': '0.6586', 'rewards/margins': '0.5857', 'logps/chosen': '-87.96', 'logps/rejected': '-99.78', 'logits/chosen': '-3.219', 'logits/rejected': '-3.18', 'epoch': '0.8707'}
{'loss': '0.5628', 'grad_norm': '0.7438', 'learning_rate': '0.0001457', 'rewards/chosen': '-1.04', 'rewards/rejected': '-1.64', 'rewards/accuracies': '0.6594', 'rewards/margins': '0.5995', 'logps/chosen': '-90.28', 'logps/rejected': '-101.8', 'logits/chosen': '-3.175', 'logits/rejected': '-3.129', 'epoch': '0.8816'}
{'loss': '0.5592', 'grad_norm': '0.7402', 'learning_rate': '0.000145', 'rewards/chosen': '-1.04', 'rewards/rejected': '-1.605', 'rewards/accuracies': '0.6562', 'rewards/margins': '0.5652', 'logps/chosen': '-92.35', 'logps/rejected': '-103.3', 'logits/chosen': '-3.144', 'logits/rejected': '-3.037', 'epoch': '0.8925'}
{'loss': '0.5435', 'grad_norm': '0.6177', 'learning_rate': '0.0001442', 'rewards/chosen': '-0.964', 'rewards/rejected': '-1.562', 'rewards/accuracies': '0.675', 'rewards/margins': '0.5981', 'logps/chosen': '-92.94', 'logps/rejected': '-102.4', 'logits/chosen': '-3.209', 'logits/rejected': '-3.142', 'epoch': '0.9034'}
{'loss': '0.5488', 'grad_norm': '0.6203', 'learning_rate': '0.0001435', 'rewards/chosen': '-0.9898', 'rewards/rejected': '-1.58', 'rewards/accuracies': '0.6789', 'rewards/margins': '0.5904', 'logps/chosen': '-91.24', 'logps/rejected': '-101.4', 'logits/chosen': '-3.352', 'logits/rejected': '-3.235', 'epoch': '0.9143'}
{'loss': '0.5207', 'grad_norm': '0.669', 'learning_rate': '0.0001427', 'rewards/chosen': '-0.9211', 'rewards/rejected': '-1.579', 'rewards/accuracies': '0.6836', 'rewards/margins': '0.6583', 'logps/chosen': '-89.67', 'logps/rejected': '-102.7', 'logits/chosen': '-3.454', 'logits/rejected': '-3.354', 'epoch': '0.9252'}
{'loss': '0.523', 'grad_norm': '0.7382', 'learning_rate': '0.000142', 'rewards/chosen': '-0.9207', 'rewards/rejected': '-1.614', 'rewards/accuracies': '0.6977', 'rewards/margins': '0.6931', 'logps/chosen': '-92.43', 'logps/rejected': '-103.6', 'logits/chosen': '-3.535', 'logits/rejected': '-3.453', 'epoch': '0.9361'}
{'loss': '0.532', 'grad_norm': '0.7022', 'learning_rate': '0.0001412', 'rewards/chosen': '-1.145', 'rewards/rejected': '-1.819', 'rewards/accuracies': '0.6953', 'rewards/margins': '0.6743', 'logps/chosen': '-91.38', 'logps/rejected': '-104.2', 'logits/chosen': '-3.44', 'logits/rejected': '-3.353', 'epoch': '0.9469'}
{'loss': '0.5318', 'grad_norm': '0.6528', 'learning_rate': '0.0001405', 'rewards/chosen': '-1.164', 'rewards/rejected': '-1.821', 'rewards/accuracies': '0.6773', 'rewards/margins': '0.6567', 'logps/chosen': '-90.99', 'logps/rejected': '-103.5', 'logits/chosen': '-3.513', 'logits/rejected': '-3.527', 'epoch': '0.9578'}
{'loss': '0.532', 'grad_norm': '0.6701', 'learning_rate': '0.0001397', 'rewards/chosen': '-1.009', 'rewards/rejected': '-1.716', 'rewards/accuracies': '0.6789', 'rewards/margins': '0.7072', 'logps/chosen': '-89.25', 'logps/rejected': '-101.4', 'logits/chosen': '-3.612', 'logits/rejected': '-3.547', 'epoch': '0.9687'}
{'loss': '0.5377', 'grad_norm': '0.6882', 'learning_rate': '0.000139', 'rewards/chosen': '-0.9364', 'rewards/rejected': '-1.605', 'rewards/accuracies': '0.6805', 'rewards/margins': '0.6687', 'logps/chosen': '-88.02', 'logps/rejected': '-99.57', 'logits/chosen': '-3.528', 'logits/rejected': '-3.411', 'epoch': '0.9796'}
{'loss': '0.5362', 'grad_norm': '0.6982', 'learning_rate': '0.0001382', 'rewards/chosen': '-0.8833', 'rewards/rejected': '-1.537', 'rewards/accuracies': '0.6523', 'rewards/margins': '0.654', 'logps/chosen': '-87.5', 'logps/rejected': '-98.83', 'logits/chosen': '-3.544', 'logits/rejected': '-3.5', 'epoch': '0.9905'}
33%|███▎ | 920/2757 [2:04:52<3:57:58, 7.77s/it]
33%|███▎ | 920/2757 [2:04:52<3:57:58, 7.77s/it]
33%|███▎ | 921/2757 [2:05:00<4:03:27, 7.96s/it]
33%|███▎ | 922/2757 [2:05:09<4:10:14, 8.18s/it]
33%|███▎ | 923/2757 [2:05:17<4:09:05, 8.15s/it]
34%|███▎ | 924/2757 [2:05:25<4:09:27, 8.17s/it]
34%|███▎ | 925/2757 [2:05:33<4:10:49, 8.21s/it]
34%|███▎ | 926/2757 [2:05:41<4:07:28, 8.11s/it]
34%|███▎ | 927/2757 [2:05:49<4:05:37, 8.05s/it]
34%|███▎ | 928/2757 [2:05:57<4:03:45, 8.00s/it]
34%|███▎ | 929/2757 [2:06:05<4:04:49, 8.04s/it]
34%|███▎ | 930/2757 [2:06:13<4:04:38, 8.03s/it]
34%|███▎ | 930/2757 [2:06:13<4:04:38, 8.03s/it]
34%|███▍ | 931/2757 [2:06:21<4:06:45, 8.11s/it]
34%|███▍ | 932/2757 [2:06:30<4:06:35, 8.11s/it]
34%|███▍ | 933/2757 [2:06:38<4:06:31, 8.11s/it]
34%|███▍ | 934/2757 [2:06:46<4:05:03, 8.07s/it]
34%|███▍ | 935/2757 [2:06:53<4:01:31, 7.95s/it]
34%|███▍ | 936/2757 [2:07:01<3:58:12, 7.85s/it]
34%|███▍ | 937/2757 [2:07:09<4:01:53, 7.97s/it]
34%|███▍ | 938/2757 [2:07:18<4:08:38, 8.20s/it]
34%|███▍ | 939/2757 [2:07:26<4:09:15, 8.23s/it]
34%|███▍ | 940/2757 [2:07:34<4:06:23, 8.14s/it]
34%|███▍ | 940/2757 [2:07:34<4:06:23, 8.14s/it]
34%|███▍ | 941/2757 [2:07:42<4:05:38, 8.12s/it]
34%|███▍ | 942/2757 [2:07:50<4:01:14, 7.97s/it]
34%|███▍ | 943/2757 [2:07:58<3:58:23, 7.89s/it]
34%|███▍ | 944/2757 [2:08:06<4:02:31, 8.03s/it]
34%|███▍ | 945/2757 [2:08:14<4:02:37, 8.03s/it]
34%|███▍ | 946/2757 [2:08:22<4:04:50, 8.11s/it]
34%|███▍ | 947/2757 [2:08:30<4:00:51, 7.98s/it]
34%|███▍ | 948/2757 [2:08:38<4:02:03, 8.03s/it]
34%|███▍ | 949/2757 [2:08:46<4:05:32, 8.15s/it]
34%|███▍ | 950/2757 [2:08:55<4:07:08, 8.21s/it]
34%|███▍ | 950/2757 [2:08:55<4:07:08, 8.21s/it]
34%|███▍ | 951/2757 [2:09:03<4:07:01, 8.21s/it]
35%|███▍ | 952/2757 [2:09:11<4:06:29, 8.19s/it]
35%|███▍ | 953/2757 [2:09:19<4:04:44, 8.14s/it]
35%|███▍ | 954/2757 [2:09:27<4:00:41, 8.01s/it]
35%|███▍ | 955/2757 [2:09:35<4:04:11, 8.13s/it]
35%|███▍ | 956/2757 [2:09:44<4:07:09, 8.23s/it]
35%|███▍ | 957/2757 [2:09:52<4:03:52, 8.13s/it]
35%|███▍ | 958/2757 [2:10:00<4:08:07, 8.28s/it]
35%|███▍ | 959/2757 [2:10:08<4:05:23, 8.19s/it]
35%|███▍ | 960/2757 [2:10:16<4:02:57, 8.11s/it]
35%|███▍ | 960/2757 [2:10:16<4:02:57, 8.11s/it]
35%|███▍ | 961/2757 [2:10:24<4:01:14, 8.06s/it]
35%|███▍ | 962/2757 [2:10:32<4:01:22, 8.07s/it]
35%|███▍ | 963/2757 [2:10:41<4:02:59, 8.13s/it]
35%|███▍ | 964/2757 [2:10:48<3:58:51, 7.99s/it]
35%|███▌ | 965/2757 [2:10:56<3:59:00, 8.00s/it]
35%|███▌ | 966/2757 [2:11:05<4:05:08, 8.21s/it]
35%|███▌ | 967/2757 [2:11:13<4:02:15, 8.12s/it]
35%|███▌ | 968/2757 [2:11:21<4:05:00, 8.22s/it]
35%|███▌ | 969/2757 [2:11:29<4:04:20, 8.20s/it]
35%|███▌ | 970/2757 [2:11:37<4:00:06, 8.06s/it]
35%|███▌ | 970/2757 [2:11:37<4:00:06, 8.06s/it]
35%|███▌ | 971/2757 [2:11:46<4:03:06, 8.17s/it]
35%|███▌ | 972/2757 [2:11:54<4:00:59, 8.10s/it]
35%|███▌ | 973/2757 [2:12:01<3:57:25, 7.99s/it]
35%|███▌ | 974/2757 [2:12:09<3:58:25, 8.02s/it]
35%|███▌ | 975/2757 [2:12:17<3:58:08, 8.02s/it]
35%|███▌ | 976/2757 [2:12:25<3:57:40, 8.01s/it]
35%|███▌ | 977/2757 [2:12:33<3:58:22, 8.04s/it]
35%|███▌ | 978/2757 [2:12:41<3:55:08, 7.93s/it]
36%|███▌ | 979/2757 [2:12:49<3:57:19, 8.01s/it]
36%|███▌ | 980/2757 [2:12:57<3:55:22, 7.95s/it]
36%|███▌ | 980/2757 [2:12:57<3:55:22, 7.95s/it]
36%|███▌ | 981/2757 [2:13:05<3:55:53, 7.97s/it]
36%|███▌ | 982/2757 [2:13:13<3:55:58, 7.98s/it]
36%|███▌ | 983/2757 [2:13:22<3:59:21, 8.10s/it]
36%|███▌ | 984/2757 [2:13:30<3:59:10, 8.09s/it]
36%|███▌ | 985/2757 [2:13:38<4:03:46, 8.25s/it]
36%|███▌ | 986/2757 [2:13:47<4:03:57, 8.26s/it]
36%|███▌ | 987/2757 [2:13:54<3:59:02, 8.10s/it]
36%|███▌ | 988/2757 [2:14:02<3:59:24, 8.12s/it]
36%|███▌ | 989/2757 [2:14:10<3:52:49, 7.90s/it]
36%|███▌ | 990/2757 [2:14:18<3:52:25, 7.89s/it]
36%|███▌ | 990/2757 [2:14:18<3:52:25, 7.89s/it]
36%|███▌ | 991/2757 [2:14:26<3:54:55, 7.98s/it]
36%|███▌ | 992/2757 [2:14:34<3:58:39, 8.11s/it]
36%|███▌ | 993/2757 [2:14:42<3:57:48, 8.09s/it]
36%|███▌ | 994/2757 [2:14:50<3:57:20, 8.08s/it]
36%|███▌ | 995/2757 [2:14:58<3:56:39, 8.06s/it]
36%|███▌ | 996/2757 [2:15:07<3:58:26, 8.12s/it]
36%|███▌ | 997/2757 [2:15:15<3:56:13, 8.05s/it]
36%|███▌ | 998/2757 [2:15:23<3:57:57, 8.12s/it]
36%|███▌ | 999/2757 [2:15:30<3:52:36, 7.94s/it]
36%|███▋ | 1000/2757 [2:15:39<3:55:40, 8.05s/it]
36%|███▋ | 1000/2757 [2:15:39<3:55:40, 8.05s/it]{'loss': '0.5034', 'grad_norm': '0.4905', 'learning_rate': '0.0001375', 'rewards/chosen': '-0.9122', 'rewards/rejected': '-1.683', 'rewards/accuracies': '0.7075', 'rewards/margins': '0.7706', 'logps/chosen': '-86.75', 'logps/rejected': '-98.81', 'logits/chosen': '-3.483', 'logits/rejected': '-3.456', 'epoch': '1.001'}
{'loss': '0.4122', 'grad_norm': '0.5277', 'learning_rate': '0.0001367', 'rewards/chosen': '-1.053', 'rewards/rejected': '-2.115', 'rewards/accuracies': '0.7789', 'rewards/margins': '1.062', 'logps/chosen': '-92.64', 'logps/rejected': '-107.8', 'logits/chosen': '-3.593', 'logits/rejected': '-3.548', 'epoch': '1.012'}
{'loss': '0.4067', 'grad_norm': '0.5503', 'learning_rate': '0.000136', 'rewards/chosen': '-1.235', 'rewards/rejected': '-2.371', 'rewards/accuracies': '0.7844', 'rewards/margins': '1.136', 'logps/chosen': '-94.07', 'logps/rejected': '-112.1', 'logits/chosen': '-3.677', 'logits/rejected': '-3.588', 'epoch': '1.023'}
{'loss': '0.4037', 'grad_norm': '0.5421', 'learning_rate': '0.0001352', 'rewards/chosen': '-1.13', 'rewards/rejected': '-2.287', 'rewards/accuracies': '0.7844', 'rewards/margins': '1.158', 'logps/chosen': '-90.41', 'logps/rejected': '-106.1', 'logits/chosen': '-3.779', 'logits/rejected': '-3.727', 'epoch': '1.034'}
{'loss': '0.4048', 'grad_norm': '0.5324', 'learning_rate': '0.0001345', 'rewards/chosen': '-0.9678', 'rewards/rejected': '-2.156', 'rewards/accuracies': '0.7937', 'rewards/margins': '1.188', 'logps/chosen': '-92.28', 'logps/rejected': '-108.5', 'logits/chosen': '-3.955', 'logits/rejected': '-3.926', 'epoch': '1.045'}
{'loss': '0.3986', 'grad_norm': '0.4539', 'learning_rate': '0.0001337', 'rewards/chosen': '-0.9384', 'rewards/rejected': '-2.162', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.224', 'logps/chosen': '-90.46', 'logps/rejected': '-109.6', 'logits/chosen': '-3.724', 'logits/rejected': '-3.732', 'epoch': '1.056'}
{'loss': '0.4087', 'grad_norm': '0.5155', 'learning_rate': '0.000133', 'rewards/chosen': '-1.096', 'rewards/rejected': '-2.233', 'rewards/accuracies': '0.7719', 'rewards/margins': '1.137', 'logps/chosen': '-88.56', 'logps/rejected': '-105.9', 'logits/chosen': '-3.808', 'logits/rejected': '-3.751', 'epoch': '1.066'}
{'loss': '0.3877', 'grad_norm': '0.5626', 'learning_rate': '0.0001322', 'rewards/chosen': '-1.119', 'rewards/rejected': '-2.363', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.244', 'logps/chosen': '-93.86', 'logps/rejected': '-112.1', 'logits/chosen': '-3.833', 'logits/rejected': '-3.733', 'epoch': '1.077'}
{'loss': '0.3857', 'grad_norm': '0.4981', 'learning_rate': '0.0001315', 'rewards/chosen': '-1.254', 'rewards/rejected': '-2.53', 'rewards/accuracies': '0.7844', 'rewards/margins': '1.277', 'logps/chosen': '-90.55', 'logps/rejected': '-110.4', 'logits/chosen': '-3.934', 'logits/rejected': '-3.891', 'epoch': '1.088'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
0%| | 0/300 [00:00<?, ?it/s]
1%| | 2/300 [00:00<00:51, 5.74it/s]
1%| | 3/300 [00:00<01:46, 2.80it/s]
1%|▏ | 4/300 [00:01<01:34, 3.13it/s]
2%|▏ | 5/300 [00:01<01:44, 2.83it/s]
2%|▏ | 6/300 [00:01<01:40, 2.94it/s]
2%|▏ | 7/300 [00:02<01:54, 2.56it/s]
3%|▎ | 8/300 [00:02<01:50, 2.65it/s]
3%|▎ | 9/300 [00:03<01:51, 2.61it/s]
3%|▎ | 10/300 [00:03<01:48, 2.67it/s]
4%|▎ | 11/300 [00:03<01:50, 2.61it/s]
4%|▍ | 12/300 [00:04<01:37, 2.94it/s]
4%|▍ | 13/300 [00:04<01:42, 2.80it/s]
5%|▍ | 14/300 [00:04<01:35, 3.00it/s]
5%|▌ | 15/300 [00:05<01:45, 2.70it/s]
5%|▌ | 16/300 [00:05<01:33, 3.04it/s]
6%|▌ | 17/300 [00:05<01:38, 2.89it/s]
6%|▌ | 18/300 [00:06<01:28, 3.19it/s]
6%|▋ | 19/300 [00:06<01:38, 2.85it/s]
7%|▋ | 20/300 [00:06<01:31, 3.06it/s]
7%|▋ | 21/300 [00:07<01:41, 2.76it/s]
7%|▋ | 22/300 [00:07<01:30, 3.08it/s]
8%|▊ | 23/300 [00:07<01:31, 3.04it/s]
8%|▊ | 24/300 [00:08<01:22, 3.35it/s]
8%|▊ | 25/300 [00:08<01:35, 2.89it/s]
9%|▊ | 26/300 [00:08<01:28, 3.10it/s]
9%|▉ | 27/300 [00:09<01:36, 2.82it/s]
9%|▉ | 28/300 [00:09<01:26, 3.16it/s]
10%|▉ | 29/300 [00:09<01:34, 2.85it/s]
10%|█ | 30/300 [00:10<01:24, 3.18it/s]
10%|█ | 31/300 [00:10<01:26, 3.12it/s]
11%|█ | 32/300 [00:10<01:18, 3.40it/s]
11%|█ | 33/300 [00:11<01:21, 3.29it/s]
11%|█▏ | 34/300 [00:11<01:16, 3.47it/s]
12%|█▏ | 35/300 [00:11<01:22, 3.22it/s]
12%|█▏ | 36/300 [00:11<01:18, 3.38it/s]
12%|█▏ | 37/300 [00:12<01:26, 3.03it/s]
13%|█▎ | 38/300 [00:12<01:17, 3.37it/s]
13%|█▎ | 39/300 [00:12<01:25, 3.05it/s]
13%|█▎ | 40/300 [00:13<01:17, 3.34it/s]
14%|█▎ | 41/300 [00:13<01:25, 3.03it/s]
14%|█▍ | 42/300 [00:13<01:17, 3.32it/s]
14%|█▍ | 43/300 [00:14<01:21, 3.15it/s]
15%|█▍ | 44/300 [00:14<01:14, 3.42it/s]
15%|█▌ | 45/300 [00:14<01:22, 3.11it/s]
15%|█▌ | 46/300 [00:15<01:23, 3.05it/s]
16%|█▌ | 47/300 [00:15<01:35, 2.66it/s]
16%|█▌ | 48/300 [00:15<01:26, 2.92it/s]
16%|█▋ | 49/300 [00:16<01:40, 2.49it/s]
17%|█▋ | 50/300 [00:16<01:26, 2.88it/s]
17%|█▋ | 51/300 [00:17<01:26, 2.88it/s]
17%|█▋ | 52/300 [00:17<01:21, 3.05it/s]
18%|█▊ | 53/300 [00:17<01:32, 2.68it/s]
18%|█▊ | 54/300 [00:18<01:22, 2.99it/s]
18%|█▊ | 55/300 [00:18<01:29, 2.74it/s]
19%|█▊ | 56/300 [00:18<01:17, 3.15it/s]
19%|█▉ | 57/300 [00:19<01:21, 2.98it/s]
19%|█▉ | 58/300 [00:19<01:22, 2.94it/s]
20%|█▉ | 59/300 [00:19<01:26, 2.78it/s]
20%|██ | 60/300 [00:20<01:18, 3.07it/s]
20%|██ | 61/300 [00:20<01:21, 2.95it/s]
21%|██ | 62/300 [00:20<01:20, 2.95it/s]
21%|██ | 63/300 [00:21<01:25, 2.76it/s]
21%|██▏ | 64/300 [00:21<01:23, 2.81it/s]
22%|██▏ | 65/300 [00:21<01:27, 2.70it/s]
22%|██▏ | 66/300 [00:22<01:19, 2.95it/s]
22%|██▏ | 67/300 [00:22<01:26, 2.69it/s]
23%|██▎ | 68/300 [00:22<01:17, 2.97it/s]
23%|██▎ | 69/300 [00:23<01:21, 2.85it/s]
23%|██▎ | 70/300 [00:23<01:12, 3.15it/s]
24%|██▎ | 71/300 [00:23<01:15, 3.05it/s]
24%|██▍ | 72/300 [00:24<01:09, 3.30it/s]
24%|██▍ | 73/300 [00:24<01:12, 3.14it/s]
25%|██▍ | 74/300 [00:24<01:08, 3.32it/s]
25%|██▌ | 75/300 [00:25<01:12, 3.09it/s]
25%|██▌ | 76/300 [00:25<01:14, 3.02it/s]
26%|██▌ | 77/300 [00:25<01:20, 2.79it/s]
26%|██▌ | 78/300 [00:26<01:13, 3.03it/s]
26%|██▋ | 79/300 [00:26<01:15, 2.93it/s]
27%|██▋ | 80/300 [00:26<01:11, 3.09it/s]
27%|██▋ | 81/300 [00:27<01:19, 2.77it/s]
27%|██▋ | 82/300 [00:27<01:10, 3.07it/s]
28%|██▊ | 83/300 [00:27<01:13, 2.97it/s]
28%|██▊ | 84/300 [00:28<01:13, 2.94it/s]
28%|██▊ | 85/300 [00:28<01:18, 2.75it/s]
29%|██▊ | 86/300 [00:28<01:10, 3.05it/s]
29%|██▉ | 87/300 [00:29<01:12, 2.94it/s]
29%|██▉ | 88/300 [00:29<01:06, 3.19it/s]
30%|██▉ | 89/300 [00:29<01:10, 2.98it/s]
30%|███ | 90/300 [00:30<01:05, 3.21it/s]
30%|███ | 91/300 [00:30<01:09, 3.01it/s]
31%|███ | 92/300 [00:30<01:03, 3.28it/s]
31%|███ | 93/300 [00:31<01:09, 3.00it/s]
31%|███▏ | 94/300 [00:31<01:02, 3.29it/s]
32%|███▏ | 95/300 [00:31<01:07, 3.02it/s]
32%|███▏ | 96/300 [00:32<01:03, 3.21it/s]
32%|███▏ | 97/300 [00:32<01:05, 3.09it/s]
33%|███▎ | 98/300 [00:32<01:01, 3.26it/s]
33%|███▎ | 99/300 [00:33<01:06, 3.00it/s]
33%|███▎ | 100/300 [00:33<01:03, 3.16it/s]
34%|███▎ | 101/300 [00:33<01:07, 2.97it/s]
34%|███▍ | 102/300 [00:34<01:05, 3.02it/s]
34%|███▍ | 103/300 [00:34<01:09, 2.84it/s]
35%|███▍ | 104/300 [00:34<01:02, 3.16it/s]
35%|███▌ | 105/300 [00:35<01:05, 2.99it/s]
35%|███▌ | 106/300 [00:35<01:00, 3.20it/s]
36%|███▌ | 107/300 [00:35<01:04, 2.98it/s]
36%|███▌ | 108/300 [00:35<01:02, 3.07it/s]
36%|███▋ | 109/300 [00:36<01:06, 2.89it/s]
37%|███▋ | 110/300 [00:36<01:02, 3.04it/s]
37%|███▋ | 111/300 [00:37<01:11, 2.66it/s]
37%|███▋ | 112/300 [00:37<01:05, 2.85it/s]
38%|███▊ | 113/300 [00:37<01:07, 2.75it/s]
38%|███▊ | 114/300 [00:38<01:06, 2.78it/s]
38%|███▊ | 115/300 [00:38<01:09, 2.65it/s]
39%|███▊ | 116/300 [00:38<01:01, 2.97it/s]
39%|███▉ | 117/300 [00:39<01:03, 2.90it/s]
39%|███▉ | 118/300 [00:39<00:58, 3.12it/s]
40%|███▉ | 119/300 [00:39<01:00, 2.97it/s]
40%|████ | 120/300 [00:40<01:05, 2.76it/s]
40%|████ | 121/300 [00:40<01:05, 2.72it/s]
41%|████ | 122/300 [00:40<00:59, 2.97it/s]
41%|████ | 123/300 [00:41<01:03, 2.77it/s]
41%|████▏ | 124/300 [00:41<00:59, 2.96it/s]
42%|████▏ | 125/300 [00:42<01:00, 2.87it/s]
42%|████▏ | 126/300 [00:42<01:00, 2.87it/s]
42%|████▏ | 127/300 [00:42<01:03, 2.73it/s]
43%|████▎ | 128/300 [00:43<00:58, 2.94it/s]
43%|████▎ | 129/300 [00:43<01:00, 2.83it/s]
43%|████▎ | 130/300 [00:43<00:54, 3.12it/s]
44%|████▎ | 131/300 [00:44<01:01, 2.76it/s]
44%|████▍ | 132/300 [00:44<00:55, 3.01it/s]
44%|████▍ | 133/300 [00:44<01:05, 2.57it/s]
45%|████▍ | 134/300 [00:45<00:58, 2.84it/s]
45%|████▌ | 135/300 [00:45<01:04, 2.57it/s]
45%|████▌ | 136/300 [00:45<00:56, 2.89it/s]
46%|████▌ | 137/300 [00:46<00:58, 2.79it/s]
46%|████▌ | 138/300 [00:46<00:53, 3.05it/s]
46%|████▋ | 139/300 [00:46<00:53, 3.03it/s]
47%|████▋ | 140/300 [00:47<00:48, 3.32it/s]
47%|████▋ | 141/300 [00:47<00:49, 3.18it/s]
47%|████▋ | 142/300 [00:47<00:51, 3.08it/s]
48%|████▊ | 143/300 [00:48<00:55, 2.85it/s]
48%|████▊ | 144/300 [00:48<00:49, 3.16it/s]
48%|████▊ | 145/300 [00:48<00:49, 3.13it/s]
49%|████▊ | 146/300 [00:49<00:46, 3.29it/s]
49%|████▉ | 147/300 [00:49<00:53, 2.86it/s]
49%|████▉ | 148/300 [00:49<00:50, 3.02it/s]
50%|████▉ | 149/300 [00:50<00:53, 2.81it/s]
50%|█████ | 150/300 [00:50<00:49, 3.01it/s]
50%|█████ | 151/300 [00:50<00:51, 2.89it/s]
51%|█████ | 152/300 [00:51<00:46, 3.20it/s]
51%|█████ | 153/300 [00:51<00:47, 3.09it/s]
51%|█████▏ | 154/300 [00:51<00:43, 3.37it/s]
52%|█████▏ | 155/300 [00:52<00:45, 3.17it/s]
52%|█████▏ | 156/300 [00:52<00:41, 3.49it/s]
52%|█████▏ | 157/300 [00:52<00:45, 3.13it/s]
53%|█████▎ | 158/300 [00:52<00:43, 3.27it/s]
53%|█████▎ | 159/300 [00:53<00:52, 2.67it/s]
53%|█████▎ | 160/300 [00:53<00:48, 2.90it/s]
54%|█████▎ | 161/300 [00:54<00:51, 2.70it/s]
54%|█████▍ | 162/300 [00:54<00:50, 2.74it/s]
54%|█████▍ | 163/300 [00:55<00:56, 2.43it/s]
55%|█████▍ | 164/300 [00:55<00:53, 2.55it/s]
55%|█████▌ | 165/300 [00:55<00:53, 2.54it/s]
55%|█████▌ | 166/300 [00:56<00:50, 2.65it/s]
56%|█████▌ | 167/300 [00:56<00:54, 2.43it/s]
56%|█████▌ | 168/300 [00:56<00:47, 2.75it/s]
56%|█████▋ | 169/300 [00:57<00:52, 2.48it/s]
57%|█████▋ | 170/300 [00:57<00:50, 2.59it/s]
57%|█████▋ | 171/300 [00:58<00:50, 2.54it/s]
57%|█████▋ | 172/300 [00:58<00:44, 2.88it/s]
58%|█████▊ | 173/300 [00:58<00:48, 2.63it/s]
58%|█████▊ | 174/300 [00:59<00:42, 2.97it/s]
58%|█████▊ | 175/300 [00:59<00:45, 2.73it/s]
59%|█████▊ | 176/300 [00:59<00:40, 3.07it/s]
59%|█████▉ | 177/300 [01:00<00:46, 2.65it/s]
59%|█████▉ | 178/300 [01:00<00:39, 3.08it/s]
60%|█████▉ | 179/300 [01:00<00:42, 2.88it/s]
60%|██████ | 180/300 [01:01<00:38, 3.16it/s]
60%|██████ | 181/300 [01:01<00:41, 2.85it/s]
61%|██████ | 182/300 [01:01<00:37, 3.16it/s]
61%|██████ | 183/300 [01:02<00:40, 2.90it/s]
61%|██████▏ | 184/300 [01:02<00:40, 2.89it/s]
62%|██████▏ | 185/300 [01:02<00:42, 2.70it/s]
62%|██████▏ | 186/300 [01:03<00:39, 2.89it/s]
62%|██████▏ | 187/300 [01:03<00:39, 2.87it/s]
63%|██████▎ | 188/300 [01:03<00:35, 3.15it/s]
63%|██████▎ | 189/300 [01:04<00:38, 2.86it/s]
63%|██████▎ | 190/300 [01:04<00:35, 3.09it/s]
64%|██████▎ | 191/300 [01:04<00:37, 2.94it/s]
64%|██████▍ | 192/300 [01:05<00:36, 2.97it/s]
64%|██████▍ | 193/300 [01:05<00:41, 2.56it/s]
65%|██████▍ | 194/300 [01:06<00:40, 2.64it/s]
65%|██████▌ | 195/300 [01:06<00:41, 2.54it/s]
65%|██████▌ | 196/300 [01:06<00:37, 2.75it/s]
66%|██████▌ | 197/300 [01:07<00:37, 2.72it/s]
66%|██████▌ | 198/300 [01:07<00:34, 2.94it/s]
66%|██████▋ | 199/300 [01:07<00:35, 2.81it/s]
67%|██████▋ | 200/300 [01:08<00:33, 2.99it/s]
67%|██████▋ | 201/300 [01:08<00:35, 2.79it/s]
67%|██████▋ | 202/300 [01:08<00:31, 3.08it/s]
68%|██████▊ | 203/300 [01:09<00:33, 2.93it/s]
68%|██████▊ | 204/300 [01:09<00:29, 3.22it/s]
68%|██████▊ | 205/300 [01:09<00:31, 3.01it/s]
69%|██████▊ | 206/300 [01:09<00:27, 3.40it/s]
69%|██████▉ | 207/300 [01:10<00:31, 2.92it/s]
69%|██████▉ | 208/300 [01:10<00:31, 2.93it/s]
70%|██████▉ | 209/300 [01:11<00:32, 2.79it/s]
70%|███████ | 210/300 [01:11<00:29, 3.00it/s]
70%|███████ | 211/300 [01:11<00:33, 2.64it/s]
71%|███████ | 212/300 [01:12<00:29, 2.96it/s]
71%|███████ | 213/300 [01:12<00:32, 2.69it/s]
71%|███████▏ | 214/300 [01:12<00:29, 2.94it/s]
72%|███████▏ | 215/300 [01:13<00:30, 2.76it/s]
72%|███████▏ | 216/300 [01:13<00:27, 3.06it/s]
72%|███████▏ | 217/300 [01:14<00:31, 2.64it/s]
73%|███████▎ | 218/300 [01:14<00:26, 3.04it/s]
73%|███████▎ | 219/300 [01:14<00:28, 2.80it/s]
73%|███████▎ | 220/300 [01:14<00:25, 3.09it/s]
74%|███████▎ | 221/300 [01:15<00:28, 2.78it/s]
74%|███████▍ | 222/300 [01:15<00:25, 3.09it/s]
74%|███████▍ | 223/300 [01:16<00:28, 2.74it/s]
75%|███████▍ | 224/300 [01:16<00:25, 2.94it/s]
75%|███████▌ | 225/300 [01:16<00:26, 2.80it/s]
75%|███████▌ | 226/300 [01:17<00:24, 3.03it/s]
76%|███████▌ | 227/300 [01:17<00:26, 2.79it/s]
76%|███████▌ | 228/300 [01:17<00:23, 3.12it/s]
76%|███████▋ | 229/300 [01:18<00:25, 2.75it/s]
77%|███████▋ | 230/300 [01:18<00:24, 2.85it/s]
77%|███████▋ | 231/300 [01:18<00:25, 2.71it/s]
77%|███████▋ | 232/300 [01:19<00:24, 2.78it/s]
78%|███████▊ | 233/300 [01:19<00:27, 2.47it/s]
78%|███████▊ | 234/300 [01:19<00:23, 2.82it/s]
78%|███████▊ | 235/300 [01:20<00:24, 2.64it/s]
79%|███████▊ | 236/300 [01:20<00:23, 2.71it/s]
79%|███████▉ | 237/300 [01:21<00:24, 2.61it/s]
79%|███████▉ | 238/300 [01:21<00:21, 2.93it/s]
80%|███████▉ | 239/300 [01:21<00:21, 2.90it/s]
80%|████████ | 240/300 [01:21<00:18, 3.21it/s]
80%|████████ | 241/300 [01:22<00:19, 3.07it/s]
81%|████████ | 242/300 [01:22<00:18, 3.10it/s]
81%|████████ | 243/300 [01:23<00:20, 2.74it/s]
81%|████████▏ | 244/300 [01:23<00:19, 2.84it/s]
82%|████████▏ | 245/300 [01:23<00:20, 2.69it/s]
82%|████████▏ | 246/300 [01:24<00:19, 2.78it/s]
82%|████████▏ | 247/300 [01:24<00:20, 2.60it/s]
83%|████████▎ | 248/300 [01:24<00:17, 2.94it/s]
83%|████████▎ | 249/300 [01:25<00:18, 2.80it/s]
83%|████████▎ | 250/300 [01:25<00:17, 2.90it/s]
84%|████████▎ | 251/300 [01:25<00:17, 2.80it/s]
84%|████████▍ | 252/300 [01:26<00:16, 2.99it/s]
84%|████████▍ | 253/300 [01:26<00:16, 2.84it/s]
85%|████████▍ | 254/300 [01:26<00:14, 3.21it/s]
85%|████████▌ | 255/300 [01:27<00:15, 2.85it/s]
85%|████████▌ | 256/300 [01:27<00:13, 3.19it/s]
86%|████████▌ | 257/300 [01:27<00:14, 2.98it/s]
86%|████████▌ | 258/300 [01:28<00:12, 3.24it/s]
86%|████████▋ | 259/300 [01:28<00:13, 3.00it/s]
87%|████████▋ | 260/300 [01:28<00:12, 3.18it/s]
87%|████████▋ | 261/300 [01:29<00:12, 3.01it/s]
87%|████████▋ | 262/300 [01:29<00:11, 3.30it/s]
88%|████████▊ | 263/300 [01:29<00:12, 3.01it/s]
88%|████████▊ | 264/300 [01:30<00:12, 3.00it/s]
88%|████████▊ | 265/300 [01:30<00:13, 2.59it/s]
89%|████████▊ | 266/300 [01:31<00:12, 2.68it/s]
89%|████████▉ | 267/300 [01:31<00:13, 2.42it/s]
89%|████████▉ | 268/300 [01:31<00:11, 2.77it/s]
90%|████████▉ | 269/300 [01:32<00:12, 2.45it/s]
90%|█████████ | 270/300 [01:32<00:11, 2.63it/s]
90%|█████████ | 271/300 [01:33<00:12, 2.38it/s]
91%|█████████ | 272/300 [01:33<00:10, 2.64it/s]
91%|█████████ | 273/300 [01:33<00:11, 2.38it/s]
91%|█████████▏| 274/300 [01:34<00:09, 2.72it/s]
92%|█████████▏| 275/300 [01:34<00:10, 2.43it/s]
92%|█████████▏| 276/300 [01:34<00:09, 2.61it/s]
92%|█████████▏| 277/300 [01:35<00:09, 2.53it/s]
93%|█████████▎| 278/300 [01:35<00:07, 2.85it/s]
93%|█████████▎| 279/300 [01:36<00:07, 2.82it/s]
93%|█████████▎| 280/300 [01:36<00:06, 2.91it/s]
94%|█████████▎| 281/300 [01:36<00:06, 2.73it/s]
94%|█████████▍| 282/300 [01:37<00:05, 3.03it/s]
94%|█████████▍| 283/300 [01:37<00:06, 2.81it/s]
95%|█████████▍| 284/300 [01:37<00:05, 3.08it/s]
95%|█████████▌| 285/300 [01:38<00:05, 2.67it/s]
95%|█████████▌| 286/300 [01:38<00:05, 2.73it/s]
96%|█████████▌| 287/300 [01:38<00:04, 2.61it/s]
96%|█████████▌| 288/300 [01:39<00:03, 3.03it/s]
96%|█████████▋| 289/300 [01:39<00:03, 2.79it/s]
97%|█████████▋| 290/300 [01:39<00:03, 2.81it/s]
97%|█████████▋| 291/300 [01:40<00:03, 2.61it/s]
97%|█████████▋| 292/300 [01:40<00:02, 2.93it/s]
98%|█████████▊| 293/300 [01:41<00:02, 2.53it/s]
98%|█████████▊| 294/300 [01:41<00:02, 2.80it/s]
98%|█████████▊| 295/300 [01:41<00:01, 2.79it/s]
99%|█████████▊| 296/300 [01:42<00:01, 3.02it/s]
99%|█████████▉| 297/300 [01:42<00:01, 2.83it/s]
99%|█████████▉| 298/300 [01:42<00:00, 3.20it/s]
100%|█████████▉| 299/300 [01:43<00:00, 2.84it/s]
100%|██████████| 300/300 [01:43<00:00, 2.79it/s]

36%|███▋ | 1000/2757 [2:17:23<3:55:40, 8.05s/it]
100%|██████████| 300/300 [01:43<00:00, 2.79it/s]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-1000/tokenizer_config.json.
36%|███▋ | 1001/2757 [2:17:37<20:00:37, 41.02s/it]
36%|███▋ | 1002/2757 [2:17:44<15:08:38, 31.06s/it]
36%|███▋ | 1003/2757 [2:17:52<11:41:21, 23.99s/it]
36%|███▋ | 1004/2757 [2:18:00<9:22:18, 19.25s/it]
36%|███▋ | 1005/2757 [2:18:08<7:42:17, 15.83s/it]
36%|███▋ | 1006/2757 [2:18:16<6:34:13, 13.51s/it]
37%|███▋ | 1007/2757 [2:18:24<5:44:52, 11.82s/it]
37%|███▋ | 1008/2757 [2:18:32<5:11:16, 10.68s/it]
37%|███▋ | 1009/2757 [2:18:40<4:49:29, 9.94s/it]
37%|███▋ | 1010/2757 [2:18:48<4:33:40, 9.40s/it]
37%|███▋ | 1010/2757 [2:18:48<4:33:40, 9.40s/it]
37%|███▋ | 1011/2757 [2:18:57<4:23:38, 9.06s/it]
37%|███▋ | 1012/2757 [2:19:04<4:11:48, 8.66s/it]
37%|███▋ | 1013/2757 [2:19:12<4:04:05, 8.40s/it]
37%|███▋ | 1014/2757 [2:19:20<4:01:30, 8.31s/it]
37%|███▋ | 1015/2757 [2:19:29<4:02:12, 8.34s/it]
37%|███▋ | 1016/2757 [2:19:37<4:04:17, 8.42s/it]
37%|███▋ | 1017/2757 [2:19:45<3:58:22, 8.22s/it]
37%|███▋ | 1018/2757 [2:19:53<3:58:32, 8.23s/it]
37%|███▋ | 1019/2757 [2:20:01<3:56:37, 8.17s/it]
37%|███▋ | 1020/2757 [2:20:09<3:55:06, 8.12s/it]
37%|███▋ | 1020/2757 [2:20:09<3:55:06, 8.12s/it]
37%|███▋ | 1021/2757 [2:20:18<3:57:07, 8.20s/it]
37%|███▋ | 1022/2757 [2:20:25<3:53:43, 8.08s/it]
37%|███▋ | 1023/2757 [2:20:33<3:52:17, 8.04s/it]
37%|███▋ | 1024/2757 [2:20:41<3:51:47, 8.02s/it]
37%|███▋ | 1025/2757 [2:20:50<3:56:14, 8.18s/it]
37%|███▋ | 1026/2757 [2:20:58<3:57:40, 8.24s/it]
37%|███▋ | 1027/2757 [2:21:07<4:00:06, 8.33s/it]
37%|███▋ | 1028/2757 [2:21:15<3:55:40, 8.18s/it]
37%|███▋ | 1029/2757 [2:21:22<3:50:48, 8.01s/it]
37%|███▋ | 1030/2757 [2:21:30<3:51:07, 8.03s/it]
37%|███▋ | 1030/2757 [2:21:30<3:51:07, 8.03s/it]
37%|███▋ | 1031/2757 [2:21:38<3:49:36, 7.98s/it]
37%|███▋ | 1032/2757 [2:21:46<3:48:41, 7.95s/it]
37%|███▋ | 1033/2757 [2:21:54<3:49:13, 7.98s/it]
38%|███▊ | 1034/2757 [2:22:02<3:47:37, 7.93s/it]
38%|███▊ | 1035/2757 [2:22:10<3:50:18, 8.02s/it]
38%|███▊ | 1036/2757 [2:22:18<3:52:05, 8.09s/it]
38%|███▊ | 1037/2757 [2:22:26<3:47:29, 7.94s/it]
38%|███▊ | 1038/2757 [2:22:34<3:44:24, 7.83s/it]
38%|███▊ | 1039/2757 [2:22:41<3:44:49, 7.85s/it]
38%|███▊ | 1040/2757 [2:22:50<3:47:11, 7.94s/it]
38%|███▊ | 1040/2757 [2:22:50<3:47:11, 7.94s/it]
38%|███▊ | 1041/2757 [2:22:58<3:49:38, 8.03s/it]
38%|███▊ | 1042/2757 [2:23:06<3:46:26, 7.92s/it]
38%|███▊ | 1043/2757 [2:23:14<3:47:21, 7.96s/it]
38%|███▊ | 1044/2757 [2:23:21<3:45:36, 7.90s/it]
38%|███▊ | 1045/2757 [2:23:29<3:45:19, 7.90s/it]
38%|███▊ | 1046/2757 [2:23:37<3:47:21, 7.97s/it]
38%|███▊ | 1047/2757 [2:23:45<3:45:12, 7.90s/it]
38%|███▊ | 1048/2757 [2:23:53<3:44:51, 7.89s/it]
38%|███▊ | 1049/2757 [2:24:01<3:48:04, 8.01s/it]
38%|███▊ | 1050/2757 [2:24:09<3:46:46, 7.97s/it]
38%|███▊ | 1050/2757 [2:24:09<3:46:46, 7.97s/it]
38%|███▊ | 1051/2757 [2:24:17<3:43:57, 7.88s/it]
38%|███▊ | 1052/2757 [2:24:25<3:46:32, 7.97s/it]
38%|███▊ | 1053/2757 [2:24:33<3:46:54, 7.99s/it]
38%|███▊ | 1054/2757 [2:24:41<3:46:38, 7.99s/it]
38%|███▊ | 1055/2757 [2:24:49<3:44:33, 7.92s/it]
38%|███▊ | 1056/2757 [2:24:57<3:47:43, 8.03s/it]
38%|███▊ | 1057/2757 [2:25:06<3:52:17, 8.20s/it]
38%|███▊ | 1058/2757 [2:25:13<3:46:57, 8.02s/it]
38%|███▊ | 1059/2757 [2:25:21<3:47:24, 8.04s/it]
38%|███▊ | 1060/2757 [2:25:29<3:44:58, 7.95s/it]
38%|███▊ | 1060/2757 [2:25:29<3:44:58, 7.95s/it]
38%|███▊ | 1061/2757 [2:25:37<3:45:14, 7.97s/it]
39%|███▊ | 1062/2757 [2:25:45<3:47:24, 8.05s/it]
39%|███▊ | 1063/2757 [2:25:53<3:43:27, 7.91s/it]
39%|███▊ | 1064/2757 [2:26:01<3:43:26, 7.92s/it]
39%|███▊ | 1065/2757 [2:26:09<3:42:52, 7.90s/it]
39%|███▊ | 1066/2757 [2:26:16<3:39:21, 7.78s/it]
39%|███▊ | 1067/2757 [2:26:24<3:42:07, 7.89s/it]
39%|███▊ | 1068/2757 [2:26:32<3:41:29, 7.87s/it]
39%|███▉ | 1069/2757 [2:26:39<3:35:54, 7.67s/it]
39%|███▉ | 1070/2757 [2:26:47<3:36:09, 7.69s/it]
39%|███▉ | 1070/2757 [2:26:47<3:36:09, 7.69s/it]
39%|███▉ | 1071/2757 [2:26:56<3:44:08, 7.98s/it]
39%|███▉ | 1072/2757 [2:27:04<3:44:50, 8.01s/it]
39%|███▉ | 1073/2757 [2:27:12<3:48:06, 8.13s/it]
39%|███▉ | 1074/2757 [2:27:20<3:45:16, 8.03s/it]
39%|███▉ | 1075/2757 [2:27:28<3:42:10, 7.93s/it]
39%|███▉ | 1076/2757 [2:27:36<3:41:03, 7.89s/it]
39%|███▉ | 1077/2757 [2:27:43<3:40:19, 7.87s/it]
39%|███▉ | 1078/2757 [2:27:51<3:37:01, 7.76s/it]
39%|███▉ | 1079/2757 [2:27:59<3:39:38, 7.85s/it]
39%|███▉ | 1080/2757 [2:28:07<3:39:26, 7.85s/it]
39%|███▉ | 1080/2757 [2:28:07<3:39:26, 7.85s/it]
39%|███▉ | 1081/2757 [2:28:15<3:41:14, 7.92s/it]
39%|███▉ | 1082/2757 [2:28:23<3:43:29, 8.01s/it]
39%|███▉ | 1083/2757 [2:28:31<3:44:33, 8.05s/it]
39%|███▉ | 1084/2757 [2:28:39<3:44:05, 8.04s/it]
39%|███▉ | 1085/2757 [2:28:47<3:43:14, 8.01s/it]
39%|███▉ | 1086/2757 [2:28:55<3:41:24, 7.95s/it]
39%|███▉ | 1087/2757 [2:29:03<3:44:07, 8.05s/it]
39%|███▉ | 1088/2757 [2:29:12<3:49:19, 8.24s/it]
39%|███▉ | 1089/2757 [2:29:20<3:45:48, 8.12s/it]
40%|███▉ | 1090/2757 [2:29:28<3:46:57, 8.17s/it]
40%|███▉ | 1090/2757 [2:29:28<3:46:57, 8.17s/it]
40%|███▉ | 1091/2757 [2:29:36<3:45:01, 8.10s/it]
40%|███▉ | 1092/2757 [2:29:44<3:42:01, 8.00s/it]
40%|███▉ | 1093/2757 [2:29:52<3:40:48, 7.96s/it]
40%|███▉ | 1094/2757 [2:30:00<3:42:19, 8.02s/it]
40%|███▉ | 1095/2757 [2:30:08<3:41:06, 7.98s/it]
40%|███▉ | 1096/2757 [2:30:16<3:39:58, 7.95s/it]
40%|███▉ | 1097/2757 [2:30:24<3:40:51, 7.98s/it]
40%|███▉ | 1098/2757 [2:30:32<3:40:32, 7.98s/it]
40%|███▉ | 1099/2757 [2:30:40<3:43:54, 8.10s/it]
40%|███▉ | 1100/2757 [2:30:48<3:46:13, 8.19s/it]
40%|███▉ | 1100/2757 [2:30:48<3:46:13, 8.19s/it]
40%|███▉ | 1101/2757 [2:30:56<3:41:36, 8.03s/it]
40%|███▉ | 1102/2757 [2:31:04<3:41:38, 8.04s/it]
40%|████ | 1103/2757 [2:31:12<3:41:56, 8.05s/it]
40%|████ | 1104/2757 [2:31:20<3:40:57, 8.02s/it]
40%|████ | 1105/2757 [2:31:28<3:40:23, 8.00s/it]
40%|████ | 1106/2757 [2:31:36<3:40:05, 8.00s/it]
40%|████ | 1107/2757 [2:31:44<3:41:23, 8.05s/it]
40%|████ | 1108/2757 [2:31:52<3:40:40, 8.03s/it]
40%|████ | 1109/2757 [2:32:00<3:38:25, 7.95s/it]
40%|████ | 1110/2757 [2:32:09<3:44:33, 8.18s/it]
40%|████ | 1110/2757 [2:32:09<3:44:33, 8.18s/it]
40%|████ | 1111/2757 [2:32:16<3:40:05, 8.02s/it]
40%|████ | 1112/2757 [2:32:24<3:36:24, 7.89s/it]
40%|████ | 1113/2757 [2:32:32<3:35:27, 7.86s/it]
40%|████ | 1114/2757 [2:32:40<3:34:33, 7.84s/it]
40%|████ | 1115/2757 [2:32:47<3:34:06, 7.82s/it]
40%|████ | 1116/2757 [2:32:56<3:37:40, 7.96s/it]
41%|████ | 1117/2757 [2:33:04<3:37:24, 7.95s/it]
41%|████ | 1118/2757 [2:33:12<3:37:08, 7.95s/it]
41%|████ | 1119/2757 [2:33:19<3:34:09, 7.84s/it]
41%|████ | 1120/2757 [2:33:28<3:38:55, 8.02s/it]
41%|████ | 1120/2757 [2:33:28<3:38:55, 8.02s/it]
41%|████ | 1121/2757 [2:33:35<3:37:24, 7.97s/it]
41%|████ | 1122/2757 [2:33:44<3:42:50, 8.18s/it]
41%|████ | 1123/2757 [2:33:52<3:42:00, 8.15s/it]
41%|████ | 1124/2757 [2:34:00<3:39:24, 8.06s/it]
41%|████ | 1125/2757 [2:34:08<3:36:24, 7.96s/it]
41%|████ | 1126/2757 [2:34:15<3:32:23, 7.81s/it]
41%|████ | 1127/2757 [2:34:23<3:32:42, 7.83s/it]
41%|████ | 1128/2757 [2:34:31<3:33:16, 7.86s/it]
41%|████ | 1129/2757 [2:34:39<3:31:42, 7.80s/it]
41%|████ | 1130/2757 [2:34:47<3:34:22, 7.91s/it]
41%|████ | 1130/2757 [2:34:47<3:34:22, 7.91s/it]
41%|████ | 1131/2757 [2:34:55<3:36:04, 7.97s/it]
41%|████ | 1132/2757 [2:35:03<3:38:48, 8.08s/it]
41%|████ | 1133/2757 [2:35:11<3:38:52, 8.09s/it]
41%|████ | 1134/2757 [2:35:19<3:35:26, 7.96s/it]
41%|████ | 1135/2757 [2:35:27<3:33:57, 7.91s/it]
41%|████ | 1136/2757 [2:35:35<3:34:05, 7.92s/it]
41%|████ | 1137/2757 [2:35:43<3:35:11, 7.97s/it]
41%|████▏ | 1138/2757 [2:35:51<3:36:04, 8.01s/it]
41%|████▏ | 1139/2757 [2:35:59<3:36:32, 8.03s/it]
41%|████▏ | 1140/2757 [2:36:07<3:37:38, 8.08s/it]
41%|████▏ | 1140/2757 [2:36:07<3:37:38, 8.08s/it]
41%|████▏ | 1141/2757 [2:36:15<3:38:28, 8.11s/it]
41%|████▏ | 1142/2757 [2:36:23<3:36:06, 8.03s/it]
41%|████▏ | 1143/2757 [2:36:31<3:36:47, 8.06s/it]
41%|████▏ | 1144/2757 [2:36:40<3:37:03, 8.07s/it]
42%|████▏ | 1145/2757 [2:36:47<3:34:09, 7.97s/it]
42%|████▏ | 1146/2757 [2:36:55<3:34:12, 7.98s/it]
42%|████▏ | 1147/2757 [2:37:03<3:31:59, 7.90s/it]
42%|████▏ | 1148/2757 [2:37:11<3:30:10, 7.84s/it]
42%|████▏ | 1149/2757 [2:37:19<3:32:15, 7.92s/it]
42%|████▏ | 1150/2757 [2:37:27<3:34:28, 8.01s/it]
42%|████▏ | 1150/2757 [2:37:27<3:34:28, 8.01s/it]
42%|████▏ | 1151/2757 [2:37:35<3:32:27, 7.94s/it]
42%|████▏ | 1152/2757 [2:37:42<3:30:41, 7.88s/it]
42%|████▏ | 1153/2757 [2:37:51<3:31:55, 7.93s/it]
42%|████▏ | 1154/2757 [2:37:58<3:30:24, 7.88s/it]
42%|████▏ | 1155/2757 [2:38:07<3:33:42, 8.00s/it]
42%|████▏ | 1156/2757 [2:38:14<3:32:42, 7.97s/it]
42%|████▏ | 1157/2757 [2:38:22<3:30:22, 7.89s/it]
42%|████▏ | 1158/2757 [2:38:30<3:33:08, 8.00s/it]
42%|████▏ | 1159/2757 [2:38:39<3:33:56, 8.03s/it]
42%|████▏ | 1160/2757 [2:38:47<3:36:16, 8.13s/it]
42%|████▏ | 1160/2757 [2:38:47<3:36:16, 8.13s/it]
42%|████▏ | 1161/2757 [2:38:55<3:35:30, 8.10s/it]
42%|████▏ | 1162/2757 [2:39:03<3:37:23, 8.18s/it]
42%|████▏ | 1163/2757 [2:39:11<3:34:36, 8.08s/it]
42%|████▏ | 1164/2757 [2:39:19<3:32:32, 8.01s/it]
42%|████▏ | 1165/2757 [2:39:27<3:32:39, 8.01s/it]
42%|████▏ | 1166/2757 [2:39:36<3:36:55, 8.18s/it]
42%|████▏ | 1167/2757 [2:39:44<3:39:19, 8.28s/it]
42%|████▏ | 1168/2757 [2:39:52<3:35:20, 8.13s/it]
42%|████▏ | 1169/2757 [2:40:00<3:36:03, 8.16s/it]
42%|████▏ | 1170/2757 [2:40:08<3:33:27, 8.07s/it]
42%|████▏ | 1170/2757 [2:40:08<3:33:27, 8.07s/it]
42%|████▏ | 1171/2757 [2:40:16<3:31:19, 7.99s/it]
43%|████▎ | 1172/2757 [2:40:23<3:27:05, 7.84s/it]
43%|████▎ | 1173/2757 [2:40:31<3:30:00, 7.95s/it]
43%|████▎ | 1174/2757 [2:40:39<3:30:24, 7.98s/it]
43%|████▎ | 1175/2757 [2:40:47<3:27:18, 7.86s/it]
43%|████▎ | 1176/2757 [2:40:55<3:27:51, 7.89s/it]
43%|████▎ | 1177/2757 [2:41:03<3:30:12, 7.98s/it]
43%|████▎ | 1178/2757 [2:41:11<3:29:20, 7.95s/it]
43%|████▎ | 1179/2757 [2:41:19<3:30:36, 8.01s/it]
43%|████▎ | 1180/2757 [2:41:27<3:31:47, 8.06s/it]
43%|████▎ | 1180/2757 [2:41:27<3:31:47, 8.06s/it]
43%|████▎ | 1181/2757 [2:41:35<3:29:14, 7.97s/it]
43%|████▎ | 1182/2757 [2:41:43<3:30:10, 8.01s/it]
43%|████▎ | 1183/2757 [2:41:51<3:29:26, 7.98s/it]
43%|████▎ | 1184/2757 [2:41:59<3:27:46, 7.93s/it]
43%|████▎ | 1185/2757 [2:42:07<3:25:36, 7.85s/it]
43%|████▎ | 1186/2757 [2:42:15<3:28:24, 7.96s/it]
43%|████▎ | 1187/2757 [2:42:23<3:25:33, 7.86s/it]
43%|████▎ | 1188/2757 [2:42:30<3:21:06, 7.69s/it]
43%|████▎ | 1189/2757 [2:42:38<3:21:58, 7.73s/it]
43%|████▎ | 1190/2757 [2:42:46<3:24:12, 7.82s/it]
43%|████▎ | 1190/2757 [2:42:46<3:24:12, 7.82s/it]
43%|████▎ | 1191/2757 [2:42:54<3:24:38, 7.84s/it]
43%|████▎ | 1192/2757 [2:43:02<3:26:02, 7.90s/it]
43%|████▎ | 1193/2757 [2:43:10<3:31:12, 8.10s/it]
43%|████▎ | 1194/2757 [2:43:18<3:30:15, 8.07s/it]
43%|████▎ | 1195/2757 [2:43:26<3:27:19, 7.96s/it]
43%|████▎ | 1196/2757 [2:43:34<3:26:53, 7.95s/it]
43%|████▎ | 1197/2757 [2:43:42<3:28:13, 8.01s/it]
43%|████▎ | 1198/2757 [2:43:50<3:25:55, 7.93s/it]
43%|████▎ | 1199/2757 [2:43:57<3:24:56, 7.89s/it]
44%|████▎ | 1200/2757 [2:44:05<3:24:46, 7.89s/it]
44%|████▎ | 1200/2757 [2:44:05<3:24:46, 7.89s/it]
44%|████▎ | 1201/2757 [2:44:13<3:19:18, 7.69s/it]
44%|████▎ | 1202/2757 [2:44:20<3:20:08, 7.72s/it]
44%|████▎ | 1203/2757 [2:44:29<3:25:54, 7.95s/it]
44%|████▎ | 1204/2757 [2:44:36<3:21:35, 7.79s/it]
44%|████▎ | 1205/2757 [2:44:44<3:23:11, 7.86s/it]
44%|████▎ | 1206/2757 [2:44:52<3:21:44, 7.80s/it]
44%|████▍ | 1207/2757 [2:45:01<3:27:20, 8.03s/it]
44%|████▍ | 1208/2757 [2:45:09<3:27:17, 8.03s/it]
44%|████▍ | 1209/2757 [2:45:17<3:27:02, 8.03s/it]
44%|████▍ | 1210/2757 [2:45:25<3:27:06, 8.03s/it]
44%|████▍ | 1210/2757 [2:45:25<3:27:06, 8.03s/it]
44%|████▍ | 1211/2757 [2:45:33<3:27:14, 8.04s/it]
44%|████▍ | 1212/2757 [2:45:41<3:26:38, 8.03s/it]
44%|████▍ | 1213/2757 [2:45:49<3:27:54, 8.08s/it]
44%|████▍ | 1214/2757 [2:45:57<3:29:26, 8.14s/it]
44%|████▍ | 1215/2757 [2:46:05<3:25:30, 8.00s/it]
44%|████▍ | 1216/2757 [2:46:13<3:24:54, 7.98s/it]
44%|████▍ | 1217/2757 [2:46:21<3:26:17, 8.04s/it]
44%|████▍ | 1218/2757 [2:46:29<3:26:47, 8.06s/it]
44%|████▍ | 1219/2757 [2:46:37<3:28:13, 8.12s/it]
44%|████▍ | 1220/2757 [2:46:46<3:30:16, 8.21s/it]
44%|████▍ | 1220/2757 [2:46:46<3:30:16, 8.21s/it]
44%|████▍ | 1221/2757 [2:46:53<3:26:35, 8.07s/it]
44%|████▍ | 1222/2757 [2:47:01<3:25:39, 8.04s/it]
44%|████▍ | 1223/2757 [2:47:09<3:21:08, 7.87s/it]
44%|████▍ | 1224/2757 [2:47:17<3:20:22, 7.84s/it]
44%|████▍ | 1225/2757 [2:47:24<3:19:43, 7.82s/it]
44%|████▍ | 1226/2757 [2:47:32<3:20:51, 7.87s/it]
45%|████▍ | 1227/2757 [2:47:40<3:20:02, 7.84s/it]
45%|████▍ | 1228/2757 [2:47:48<3:22:20, 7.94s/it]
45%|████▍ | 1229/2757 [2:47:56<3:22:31, 7.95s/it]
45%|████▍ | 1230/2757 [2:48:05<3:24:41, 8.04s/it]
45%|████▍ | 1230/2757 [2:48:05<3:24:41, 8.04s/it]
45%|████▍ | 1231/2757 [2:48:13<3:27:11, 8.15s/it]
45%|████▍ | 1232/2757 [2:48:21<3:24:10, 8.03s/it]
45%|████▍ | 1233/2757 [2:48:29<3:24:21, 8.05s/it]
45%|████▍ | 1234/2757 [2:48:37<3:24:53, 8.07s/it]
45%|████▍ | 1235/2757 [2:48:45<3:23:15, 8.01s/it]
45%|████▍ | 1236/2757 [2:48:53<3:24:18, 8.06s/it]
45%|████▍ | 1237/2757 [2:49:02<3:27:55, 8.21s/it]
45%|████▍ | 1238/2757 [2:49:09<3:25:16, 8.11s/it]
45%|████▍ | 1239/2757 [2:49:18<3:28:38, 8.25s/it]
45%|████▍ | 1240/2757 [2:49:26<3:23:16, 8.04s/it]
45%|████▍ | 1240/2757 [2:49:26<3:23:16, 8.04s/it]
45%|████▌ | 1241/2757 [2:49:33<3:21:53, 7.99s/it]
45%|████▌ | 1242/2757 [2:49:41<3:20:26, 7.94s/it]
45%|████▌ | 1243/2757 [2:49:49<3:19:45, 7.92s/it]
45%|████▌ | 1244/2757 [2:49:57<3:21:52, 8.01s/it]
45%|████▌ | 1245/2757 [2:50:06<3:23:45, 8.09s/it]
45%|████▌ | 1246/2757 [2:50:14<3:22:41, 8.05s/it]
45%|████▌ | 1247/2757 [2:50:22<3:25:53, 8.18s/it]
45%|████▌ | 1248/2757 [2:50:30<3:24:47, 8.14s/it]
45%|████▌ | 1249/2757 [2:50:38<3:21:29, 8.02s/it]
45%|████▌ | 1250/2757 [2:50:46<3:25:23, 8.18s/it]
{'eval_loss': '0.5218', 'eval_runtime': '104.5', 'eval_samples_per_second': '22.97', 'eval_steps_per_second': '2.871', 'eval_rewards/chosen': '-1.5', 'eval_rewards/rejected': '-2.382', 'eval_rewards/accuracies': '0.6712', 'eval_rewards/margins': '0.8821', 'eval_logps/chosen': '-94.61', 'eval_logps/rejected': '-109.7', 'eval_logits/chosen': '-4.036', 'eval_logits/rejected': '-3.984', 'epoch': '1.088'}
{'loss': '0.3826', 'grad_norm': '0.5595', 'learning_rate': '0.0001307', 'rewards/chosen': '-1.484', 'rewards/rejected': '-2.78', 'rewards/accuracies': '0.7914', 'rewards/margins': '1.296', 'logps/chosen': '-99.29', 'logps/rejected': '-118.4', 'logits/chosen': '-3.986', 'logits/rejected': '-3.91', 'epoch': '1.099'}
{'loss': '0.3769', 'grad_norm': '0.5151', 'learning_rate': '0.00013', 'rewards/chosen': '-1.544', 'rewards/rejected': '-2.896', 'rewards/accuracies': '0.7859', 'rewards/margins': '1.352', 'logps/chosen': '-95.98', 'logps/rejected': '-114.6', 'logits/chosen': '-3.936', 'logits/rejected': '-3.925', 'epoch': '1.11'}
{'loss': '0.3913', 'grad_norm': '0.5914', 'learning_rate': '0.0001292', 'rewards/chosen': '-1.531', 'rewards/rejected': '-2.838', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.307', 'logps/chosen': '-95.18', 'logps/rejected': '-113.3', 'logits/chosen': '-3.944', 'logits/rejected': '-3.977', 'epoch': '1.121'}
{'loss': '0.4064', 'grad_norm': '0.609', 'learning_rate': '0.0001285', 'rewards/chosen': '-1.362', 'rewards/rejected': '-2.581', 'rewards/accuracies': '0.7563', 'rewards/margins': '1.219', 'logps/chosen': '-91.94', 'logps/rejected': '-109.3', 'logits/chosen': '-3.923', 'logits/rejected': '-3.879', 'epoch': '1.132'}
{'loss': '0.3909', 'grad_norm': '0.558', 'learning_rate': '0.0001277', 'rewards/chosen': '-1.129', 'rewards/rejected': '-2.423', 'rewards/accuracies': '0.7781', 'rewards/margins': '1.294', 'logps/chosen': '-93.53', 'logps/rejected': '-110.6', 'logits/chosen': '-4.08', 'logits/rejected': '-4.015', 'epoch': '1.143'}
{'loss': '0.4011', 'grad_norm': '0.6687', 'learning_rate': '0.000127', 'rewards/chosen': '-1.27', 'rewards/rejected': '-2.525', 'rewards/accuracies': '0.7695', 'rewards/margins': '1.256', 'logps/chosen': '-92.7', 'logps/rejected': '-109', 'logits/chosen': '-3.949', 'logits/rejected': '-3.86', 'epoch': '1.153'}
{'loss': '0.3885', 'grad_norm': '0.5773', 'learning_rate': '0.0001263', 'rewards/chosen': '-1.37', 'rewards/rejected': '-2.672', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.302', 'logps/chosen': '-93.09', 'logps/rejected': '-112.1', 'logits/chosen': '-4.051', 'logits/rejected': '-3.921', 'epoch': '1.164'}
{'loss': '0.3903', 'grad_norm': '0.5501', 'learning_rate': '0.0001255', 'rewards/chosen': '-1.401', 'rewards/rejected': '-2.699', 'rewards/accuracies': '0.7797', 'rewards/margins': '1.299', 'logps/chosen': '-93.73', 'logps/rejected': '-112.1', 'logits/chosen': '-3.901', 'logits/rejected': '-3.798', 'epoch': '1.175'}
{'loss': '0.3811', 'grad_norm': '0.602', 'learning_rate': '0.0001248', 'rewards/chosen': '-1.542', 'rewards/rejected': '-2.905', 'rewards/accuracies': '0.7859', 'rewards/margins': '1.362', 'logps/chosen': '-97.4', 'logps/rejected': '-116.5', 'logits/chosen': '-3.874', 'logits/rejected': '-3.843', 'epoch': '1.186'}
{'loss': '0.3859', 'grad_norm': '0.5466', 'learning_rate': '0.000124', 'rewards/chosen': '-1.562', 'rewards/rejected': '-2.871', 'rewards/accuracies': '0.7867', 'rewards/margins': '1.309', 'logps/chosen': '-97.63', 'logps/rejected': '-116.3', 'logits/chosen': '-3.822', 'logits/rejected': '-3.789', 'epoch': '1.197'}
{'loss': '0.3874', 'grad_norm': '0.6201', 'learning_rate': '0.0001233', 'rewards/chosen': '-1.538', 'rewards/rejected': '-2.896', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.358', 'logps/chosen': '-94.8', 'logps/rejected': '-114.1', 'logits/chosen': '-3.869', 'logits/rejected': '-3.814', 'epoch': '1.208'}
{'loss': '0.3829', 'grad_norm': '0.6293', 'learning_rate': '0.0001225', 'rewards/chosen': '-1.702', 'rewards/rejected': '-3.064', 'rewards/accuracies': '0.782', 'rewards/margins': '1.362', 'logps/chosen': '-98.96', 'logps/rejected': '-118.2', 'logits/chosen': '-3.824', 'logits/rejected': '-3.681', 'epoch': '1.219'}
{'loss': '0.3882', 'grad_norm': '0.6542', 'learning_rate': '0.0001218', 'rewards/chosen': '-1.607', 'rewards/rejected': '-2.912', 'rewards/accuracies': '0.7711', 'rewards/margins': '1.305', 'logps/chosen': '-95.04', 'logps/rejected': '-113.7', 'logits/chosen': '-3.806', 'logits/rejected': '-3.804', 'epoch': '1.23'}
{'loss': '0.3934', 'grad_norm': '0.6825', 'learning_rate': '0.000121', 'rewards/chosen': '-1.642', 'rewards/rejected': '-2.956', 'rewards/accuracies': '0.7797', 'rewards/margins': '1.314', 'logps/chosen': '-97.03', 'logps/rejected': '-116', 'logits/chosen': '-3.831', 'logits/rejected': '-3.731', 'epoch': '1.241'}
{'loss': '0.3808', 'grad_norm': '0.5602', 'learning_rate': '0.0001203', 'rewards/chosen': '-1.608', 'rewards/rejected': '-2.998', 'rewards/accuracies': '0.7898', 'rewards/margins': '1.39', 'logps/chosen': '-96.62', 'logps/rejected': '-116.4', 'logits/chosen': '-3.864', 'logits/rejected': '-3.806', 'epoch': '1.251'}
{'loss': '0.3848', 'grad_norm': '0.5861', 'learning_rate': '0.0001195', 'rewards/chosen': '-1.649', 'rewards/rejected': '-3.021', 'rewards/accuracies': '0.775', 'rewards/margins': '1.372', 'logps/chosen': '-95.55', 'logps/rejected': '-114.5', 'logits/chosen': '-3.834', 'logits/rejected': '-3.734', 'epoch': '1.262'}
{'loss': '0.3743', 'grad_norm': '0.5589', 'learning_rate': '0.0001188', 'rewards/chosen': '-1.889', 'rewards/rejected': '-3.314', 'rewards/accuracies': '0.7883', 'rewards/margins': '1.426', 'logps/chosen': '-98.93', 'logps/rejected': '-120.1', 'logits/chosen': '-3.808', 'logits/rejected': '-3.658', 'epoch': '1.273'}
{'loss': '0.3917', 'grad_norm': '0.7734', 'learning_rate': '0.000118', 'rewards/chosen': '-2.133', 'rewards/rejected': '-3.482', 'rewards/accuracies': '0.7727', 'rewards/margins': '1.35', 'logps/chosen': '-100.4', 'logps/rejected': '-119.5', 'logits/chosen': '-3.791', 'logits/rejected': '-3.751', 'epoch': '1.284'}
{'loss': '0.3873', 'grad_norm': '0.6508', 'learning_rate': '0.0001173', 'rewards/chosen': '-1.986', 'rewards/rejected': '-3.376', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.39', 'logps/chosen': '-102.7', 'logps/rejected': '-119.9', 'logits/chosen': '-3.632', 'logits/rejected': '-3.56', 'epoch': '1.295'}
{'loss': '0.3865', 'grad_norm': '0.5762', 'learning_rate': '0.0001165', 'rewards/chosen': '-1.77', 'rewards/rejected': '-3.147', 'rewards/accuracies': '0.7789', 'rewards/margins': '1.377', 'logps/chosen': '-96.95', 'logps/rejected': '-118.6', 'logits/chosen': '-3.736', 'logits/rejected': '-3.719', 'epoch': '1.306'}
{'loss': '0.3787', 'grad_norm': '0.7514', 'learning_rate': '0.0001158', 'rewards/chosen': '-1.872', 'rewards/rejected': '-3.307', 'rewards/accuracies': '0.7898', 'rewards/margins': '1.435', 'logps/chosen': '-99.3', 'logps/rejected': '-118.7', 'logits/chosen': '-3.761', 'logits/rejected': '-3.714', 'epoch': '1.317'}
{'loss': '0.3696', 'grad_norm': '0.6535', 'learning_rate': '0.000115', 'rewards/chosen': '-1.846', 'rewards/rejected': '-3.268', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.422', 'logps/chosen': '-100.5', 'logps/rejected': '-121', 'logits/chosen': '-3.695', 'logits/rejected': '-3.616', 'epoch': '1.328'}
{'loss': '0.3887', 'grad_norm': '0.6603', 'learning_rate': '0.0001143', 'rewards/chosen': '-1.846', 'rewards/rejected': '-3.164', 'rewards/accuracies': '0.7828', 'rewards/margins': '1.318', 'logps/chosen': '-98.53', 'logps/rejected': '-115.1', 'logits/chosen': '-3.787', 'logits/rejected': '-3.732', 'epoch': '1.339'}
{'loss': '0.381', 'grad_norm': '0.7037', 'learning_rate': '0.0001135', 'rewards/chosen': '-1.664', 'rewards/rejected': '-3.028', 'rewards/accuracies': '0.7875', 'rewards/margins': '1.364', 'logps/chosen': '-96.31', 'logps/rejected': '-115.1', 'logits/chosen': '-3.723', 'logits/rejected': '-3.703', 'epoch': '1.349'}
{'loss': '0.3888', 'grad_norm': '0.638', 'learning_rate': '0.0001128', 'rewards/chosen': '-1.572', 'rewards/rejected': '-2.942', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.371', 'logps/chosen': '-95.81', 'logps/rejected': '-114.3', 'logits/chosen': '-3.744', 'logits/rejected': '-3.591', 'epoch': '1.36'}
45%|████▌ | 1250/2757 [2:50:46<3:25:23, 8.18s/it]
45%|████▌ | 1251/2757 [2:50:59<3:56:02, 9.40s/it]
45%|████▌ | 1252/2757 [2:51:07<3:46:44, 9.04s/it]
45%|████▌ | 1253/2757 [2:51:14<3:35:40, 8.60s/it]
45%|████▌ | 1254/2757 [2:51:22<3:30:16, 8.39s/it]
46%|████▌ | 1255/2757 [2:51:31<3:30:03, 8.39s/it]
46%|████▌ | 1256/2757 [2:51:38<3:24:39, 8.18s/it]
46%|████▌ | 1257/2757 [2:51:46<3:18:50, 7.95s/it]
46%|████▌ | 1258/2757 [2:51:54<3:17:15, 7.90s/it]
46%|████▌ | 1259/2757 [2:52:02<3:18:40, 7.96s/it]
46%|████▌ | 1260/2757 [2:52:10<3:20:41, 8.04s/it]
46%|████▌ | 1260/2757 [2:52:10<3:20:41, 8.04s/it]
46%|████▌ | 1261/2757 [2:52:18<3:19:17, 7.99s/it]
46%|████▌ | 1262/2757 [2:52:26<3:16:47, 7.90s/it]
46%|████▌ | 1263/2757 [2:52:34<3:20:51, 8.07s/it]
46%|████▌ | 1264/2757 [2:52:42<3:19:12, 8.01s/it]
46%|████▌ | 1265/2757 [2:52:50<3:18:42, 7.99s/it]
46%|████▌ | 1266/2757 [2:52:58<3:22:26, 8.15s/it]
46%|████▌ | 1267/2757 [2:53:07<3:23:52, 8.21s/it]
46%|████▌ | 1268/2757 [2:53:15<3:22:54, 8.18s/it]
46%|████▌ | 1269/2757 [2:53:23<3:19:56, 8.06s/it]
46%|████▌ | 1270/2757 [2:53:31<3:20:16, 8.08s/it]
46%|████▌ | 1270/2757 [2:53:31<3:20:16, 8.08s/it]
46%|████▌ | 1271/2757 [2:53:39<3:20:24, 8.09s/it]
46%|████▌ | 1272/2757 [2:53:47<3:21:51, 8.16s/it]
46%|████▌ | 1273/2757 [2:53:55<3:18:37, 8.03s/it]
46%|████▌ | 1274/2757 [2:54:03<3:20:09, 8.10s/it]
46%|████▌ | 1275/2757 [2:54:11<3:18:43, 8.05s/it]
46%|████▋ | 1276/2757 [2:54:19<3:18:58, 8.06s/it]
46%|████▋ | 1277/2757 [2:54:27<3:19:54, 8.10s/it]
46%|████▋ | 1278/2757 [2:54:35<3:18:47, 8.06s/it]
46%|████▋ | 1279/2757 [2:54:43<3:18:24, 8.05s/it]
46%|████▋ | 1280/2757 [2:54:51<3:16:33, 7.98s/it]
46%|████▋ | 1280/2757 [2:54:51<3:16:33, 7.98s/it]
46%|████▋ | 1281/2757 [2:54:59<3:14:45, 7.92s/it]
46%|████▋ | 1282/2757 [2:55:07<3:13:50, 7.89s/it]
47%|████▋ | 1283/2757 [2:55:15<3:14:51, 7.93s/it]
47%|████▋ | 1284/2757 [2:55:23<3:15:39, 7.97s/it]
47%|████▋ | 1285/2757 [2:55:31<3:17:58, 8.07s/it]
47%|████▋ | 1286/2757 [2:55:39<3:17:15, 8.05s/it]
47%|████▋ | 1287/2757 [2:55:47<3:18:06, 8.09s/it]
47%|████▋ | 1288/2757 [2:55:55<3:16:46, 8.04s/it]
47%|████▋ | 1289/2757 [2:56:03<3:14:41, 7.96s/it]
47%|████▋ | 1290/2757 [2:56:11<3:17:04, 8.06s/it]
47%|████▋ | 1290/2757 [2:56:11<3:17:04, 8.06s/it]
47%|████▋ | 1291/2757 [2:56:20<3:20:07, 8.19s/it]
47%|████▋ | 1292/2757 [2:56:28<3:20:44, 8.22s/it]
47%|████▋ | 1293/2757 [2:56:36<3:20:21, 8.21s/it]
47%|████▋ | 1294/2757 [2:56:45<3:22:26, 8.30s/it]
47%|████▋ | 1295/2757 [2:56:53<3:18:58, 8.17s/it]
47%|████▋ | 1296/2757 [2:57:01<3:20:48, 8.25s/it]
47%|████▋ | 1297/2757 [2:57:09<3:19:47, 8.21s/it]
47%|████▋ | 1298/2757 [2:57:17<3:17:15, 8.11s/it]
47%|████▋ | 1299/2757 [2:57:25<3:15:30, 8.05s/it]
47%|████▋ | 1300/2757 [2:57:33<3:14:02, 7.99s/it]
47%|████▋ | 1300/2757 [2:57:33<3:14:02, 7.99s/it]
47%|████▋ | 1301/2757 [2:57:41<3:12:54, 7.95s/it]
47%|████▋ | 1302/2757 [2:57:49<3:13:23, 7.98s/it]
47%|████▋ | 1303/2757 [2:57:57<3:15:53, 8.08s/it]
47%|████▋ | 1304/2757 [2:58:05<3:15:15, 8.06s/it]
47%|████▋ | 1305/2757 [2:58:13<3:15:58, 8.10s/it]
47%|████▋ | 1306/2757 [2:58:21<3:12:35, 7.96s/it]
47%|████▋ | 1307/2757 [2:58:29<3:13:35, 8.01s/it]
47%|████▋ | 1308/2757 [2:58:37<3:11:35, 7.93s/it]
47%|████▋ | 1309/2757 [2:58:45<3:13:05, 8.00s/it]
48%|████▊ | 1310/2757 [2:58:53<3:12:11, 7.97s/it]
48%|████▊ | 1310/2757 [2:58:53<3:12:11, 7.97s/it]
48%|████▊ | 1311/2757 [2:59:01<3:12:06, 7.97s/it]
48%|████▊ | 1312/2757 [2:59:09<3:10:03, 7.89s/it]
48%|████▊ | 1313/2757 [2:59:16<3:09:00, 7.85s/it]
48%|████▊ | 1314/2757 [2:59:24<3:08:56, 7.86s/it]
48%|████▊ | 1315/2757 [2:59:32<3:08:55, 7.86s/it]
48%|████▊ | 1316/2757 [2:59:41<3:13:52, 8.07s/it]
48%|████▊ | 1317/2757 [2:59:48<3:11:52, 7.99s/it]
48%|████▊ | 1318/2757 [2:59:57<3:13:06, 8.05s/it]
48%|████▊ | 1319/2757 [3:00:05<3:12:08, 8.02s/it]
48%|████▊ | 1320/2757 [3:00:13<3:13:20, 8.07s/it]
48%|████▊ | 1320/2757 [3:00:13<3:13:20, 8.07s/it]
48%|████▊ | 1321/2757 [3:00:21<3:12:11, 8.03s/it]
48%|████▊ | 1322/2757 [3:00:29<3:11:50, 8.02s/it]
48%|████▊ | 1323/2757 [3:00:37<3:14:13, 8.13s/it]
48%|████▊ | 1324/2757 [3:00:45<3:09:25, 7.93s/it]
48%|████▊ | 1325/2757 [3:00:53<3:11:57, 8.04s/it]
48%|████▊ | 1326/2757 [3:01:00<3:08:20, 7.90s/it]
48%|████▊ | 1327/2757 [3:01:08<3:05:27, 7.78s/it]
48%|████▊ | 1328/2757 [3:01:16<3:06:54, 7.85s/it]
48%|████▊ | 1329/2757 [3:01:24<3:06:53, 7.85s/it]
48%|████▊ | 1330/2757 [3:01:32<3:06:35, 7.85s/it]
48%|████▊ | 1330/2757 [3:01:32<3:06:35, 7.85s/it]
48%|████▊ | 1331/2757 [3:01:40<3:07:09, 7.88s/it]
48%|████▊ | 1332/2757 [3:01:48<3:11:17, 8.05s/it]
48%|████▊ | 1333/2757 [3:01:56<3:11:39, 8.08s/it]
48%|████▊ | 1334/2757 [3:02:04<3:11:44, 8.08s/it]
48%|████▊ | 1335/2757 [3:02:12<3:09:34, 8.00s/it]
48%|████▊ | 1336/2757 [3:02:20<3:11:21, 8.08s/it]
48%|████▊ | 1337/2757 [3:02:28<3:11:00, 8.07s/it]
49%|████▊ | 1338/2757 [3:02:36<3:11:16, 8.09s/it]
49%|████▊ | 1339/2757 [3:02:44<3:08:38, 7.98s/it]
49%|████▊ | 1340/2757 [3:02:52<3:08:37, 7.99s/it]
49%|████▊ | 1340/2757 [3:02:52<3:08:37, 7.99s/it]
49%|████▊ | 1341/2757 [3:03:00<3:07:30, 7.95s/it]
49%|████▊ | 1342/2757 [3:03:08<3:10:27, 8.08s/it]
49%|████▊ | 1343/2757 [3:03:17<3:10:31, 8.08s/it]
49%|████▊ | 1344/2757 [3:03:24<3:07:43, 7.97s/it]
49%|████▉ | 1345/2757 [3:03:32<3:06:41, 7.93s/it]
49%|████▉ | 1346/2757 [3:03:40<3:07:15, 7.96s/it]
49%|████▉ | 1347/2757 [3:03:48<3:08:46, 8.03s/it]
49%|████▉ | 1348/2757 [3:03:57<3:11:35, 8.16s/it]
49%|████▉ | 1349/2757 [3:04:05<3:10:40, 8.13s/it]
49%|████▉ | 1350/2757 [3:04:14<3:16:22, 8.37s/it]
49%|████▉ | 1350/2757 [3:04:14<3:16:22, 8.37s/it]
49%|████▉ | 1351/2757 [3:04:22<3:13:11, 8.24s/it]
49%|████▉ | 1352/2757 [3:04:30<3:13:27, 8.26s/it]
49%|████▉ | 1353/2757 [3:04:38<3:09:31, 8.10s/it]
49%|████▉ | 1354/2757 [3:04:46<3:09:28, 8.10s/it]
49%|████▉ | 1355/2757 [3:04:54<3:06:19, 7.97s/it]
49%|████▉ | 1356/2757 [3:05:02<3:10:52, 8.17s/it]
49%|████▉ | 1357/2757 [3:05:10<3:11:18, 8.20s/it]
49%|████▉ | 1358/2757 [3:05:18<3:07:08, 8.03s/it]
49%|████▉ | 1359/2757 [3:05:26<3:06:58, 8.02s/it]
49%|████▉ | 1360/2757 [3:05:34<3:07:45, 8.06s/it]
49%|████▉ | 1360/2757 [3:05:34<3:07:45, 8.06s/it]
49%|████▉ | 1361/2757 [3:05:43<3:11:06, 8.21s/it]
49%|████▉ | 1362/2757 [3:05:51<3:09:22, 8.15s/it]
49%|████▉ | 1363/2757 [3:05:58<3:06:20, 8.02s/it]
49%|████▉ | 1364/2757 [3:06:07<3:07:24, 8.07s/it]
50%|████▉ | 1365/2757 [3:06:14<3:03:48, 7.92s/it]
50%|████▉ | 1366/2757 [3:06:22<3:02:14, 7.86s/it]
50%|████▉ | 1367/2757 [3:06:30<3:02:49, 7.89s/it]
50%|████▉ | 1368/2757 [3:06:38<3:01:50, 7.86s/it]
50%|████▉ | 1369/2757 [3:06:46<3:03:11, 7.92s/it]
50%|████▉ | 1370/2757 [3:06:54<3:03:10, 7.92s/it]
50%|████▉ | 1370/2757 [3:06:54<3:03:10, 7.92s/it]
50%|████▉ | 1371/2757 [3:07:02<3:07:24, 8.11s/it]
50%|████▉ | 1372/2757 [3:07:11<3:08:26, 8.16s/it]
50%|████▉ | 1373/2757 [3:07:19<3:10:48, 8.27s/it]
50%|████▉ | 1374/2757 [3:07:27<3:09:40, 8.23s/it]
50%|████▉ | 1375/2757 [3:07:35<3:08:18, 8.18s/it]
50%|████▉ | 1376/2757 [3:07:43<3:06:53, 8.12s/it]
50%|████▉ | 1377/2757 [3:07:51<3:06:19, 8.10s/it]
50%|████▉ | 1378/2757 [3:07:59<3:02:09, 7.93s/it]
50%|█████ | 1379/2757 [3:08:07<3:03:40, 8.00s/it]
50%|█████ | 1380/2757 [3:08:15<3:03:55, 8.01s/it]
50%|█████ | 1380/2757 [3:08:15<3:03:55, 8.01s/it]
50%|█████ | 1381/2757 [3:08:23<3:03:27, 8.00s/it]
50%|█████ | 1382/2757 [3:08:31<3:04:35, 8.05s/it]
50%|█████ | 1383/2757 [3:08:40<3:09:10, 8.26s/it]
50%|█████ | 1384/2757 [3:08:48<3:09:37, 8.29s/it]
50%|█████ | 1385/2757 [3:08:56<3:08:25, 8.24s/it]
50%|█████ | 1386/2757 [3:09:04<3:04:38, 8.08s/it]
50%|█████ | 1387/2757 [3:09:12<3:01:43, 7.96s/it]
50%|█████ | 1388/2757 [3:09:20<3:04:36, 8.09s/it]
50%|█████ | 1389/2757 [3:09:28<3:03:47, 8.06s/it]
50%|█████ | 1390/2757 [3:09:36<3:05:02, 8.12s/it]
50%|█████ | 1390/2757 [3:09:36<3:05:02, 8.12s/it]
50%|█████ | 1391/2757 [3:09:44<3:02:06, 8.00s/it]
50%|█████ | 1392/2757 [3:09:52<3:03:50, 8.08s/it]
51%|█████ | 1393/2757 [3:10:01<3:05:17, 8.15s/it]
51%|█████ | 1394/2757 [3:10:08<3:01:30, 7.99s/it]
51%|█████ | 1395/2757 [3:10:16<3:00:49, 7.97s/it]
51%|█████ | 1396/2757 [3:10:24<2:59:38, 7.92s/it]
51%|█████ | 1397/2757 [3:10:32<3:00:00, 7.94s/it]
51%|█████ | 1398/2757 [3:10:40<3:01:48, 8.03s/it]
51%|█████ | 1399/2757 [3:10:48<3:00:37, 7.98s/it]
51%|█████ | 1400/2757 [3:10:56<3:01:22, 8.02s/it]
51%|█████ | 1400/2757 [3:10:56<3:01:22, 8.02s/it]
51%|█████ | 1401/2757 [3:11:05<3:02:56, 8.09s/it]
51%|█████ | 1402/2757 [3:11:12<3:01:03, 8.02s/it]
51%|█████ | 1403/2757 [3:11:20<2:57:20, 7.86s/it]
51%|█████ | 1404/2757 [3:11:28<2:59:22, 7.95s/it]
51%|█████ | 1405/2757 [3:11:36<2:56:15, 7.82s/it]
51%|█████ | 1406/2757 [3:11:44<2:58:16, 7.92s/it]
51%|█████ | 1407/2757 [3:11:52<2:58:30, 7.93s/it]
51%|█████ | 1408/2757 [3:12:00<2:59:55, 8.00s/it]
51%|█████ | 1409/2757 [3:12:08<3:00:00, 8.01s/it]
51%|█████ | 1410/2757 [3:12:16<3:00:09, 8.02s/it]
51%|█████ | 1410/2757 [3:12:16<3:00:09, 8.02s/it]
51%|█████ | 1411/2757 [3:12:25<3:03:58, 8.20s/it]
51%|█████ | 1412/2757 [3:12:32<3:00:56, 8.07s/it]
51%|█████▏ | 1413/2757 [3:12:40<3:00:47, 8.07s/it]
51%|█████▏ | 1414/2757 [3:12:48<2:58:51, 7.99s/it]
51%|█████▏ | 1415/2757 [3:12:56<2:59:31, 8.03s/it]
51%|█████▏ | 1416/2757 [3:13:04<2:58:24, 7.98s/it]
51%|█████▏ | 1417/2757 [3:13:12<2:57:24, 7.94s/it]
51%|█████▏ | 1418/2757 [3:13:20<2:55:21, 7.86s/it]
51%|█████▏ | 1419/2757 [3:13:28<2:57:04, 7.94s/it]
52%|█████▏ | 1420/2757 [3:13:36<2:56:02, 7.90s/it]
52%|█████▏ | 1420/2757 [3:13:36<2:56:02, 7.90s/it]
52%|█████▏ | 1421/2757 [3:13:44<2:57:16, 7.96s/it]
52%|█████▏ | 1422/2757 [3:13:52<2:59:59, 8.09s/it]
52%|█████▏ | 1423/2757 [3:14:00<2:58:51, 8.04s/it]
52%|█████▏ | 1424/2757 [3:14:08<2:57:54, 8.01s/it]
52%|█████▏ | 1425/2757 [3:14:16<2:57:20, 7.99s/it]
52%|█████▏ | 1426/2757 [3:14:24<2:59:31, 8.09s/it]
52%|█████▏ | 1427/2757 [3:14:33<3:01:16, 8.18s/it]
52%|█████▏ | 1428/2757 [3:14:41<2:59:45, 8.12s/it]
52%|█████▏ | 1429/2757 [3:14:49<3:00:33, 8.16s/it]
52%|█████▏ | 1430/2757 [3:14:57<2:57:46, 8.04s/it]
52%|█████▏ | 1430/2757 [3:14:57<2:57:46, 8.04s/it]
52%|█████▏ | 1431/2757 [3:15:05<2:59:10, 8.11s/it]
52%|█████▏ | 1432/2757 [3:15:13<2:56:01, 7.97s/it]
52%|█████▏ | 1433/2757 [3:15:20<2:55:31, 7.95s/it]
52%|█████▏ | 1434/2757 [3:15:28<2:52:28, 7.82s/it]
52%|█████▏ | 1435/2757 [3:15:36<2:54:57, 7.94s/it]
52%|█████▏ | 1436/2757 [3:15:44<2:56:56, 8.04s/it]
52%|█████▏ | 1437/2757 [3:15:53<2:57:01, 8.05s/it]
52%|█████▏ | 1438/2757 [3:16:01<2:56:35, 8.03s/it]
52%|█████▏ | 1439/2757 [3:16:08<2:52:47, 7.87s/it]
52%|█████▏ | 1440/2757 [3:16:16<2:54:25, 7.95s/it]
52%|█████▏ | 1440/2757 [3:16:16<2:54:25, 7.95s/it]
52%|█████▏ | 1441/2757 [3:16:24<2:53:38, 7.92s/it]
52%|█████▏ | 1442/2757 [3:16:32<2:56:11, 8.04s/it]
52%|█████▏ | 1443/2757 [3:16:41<2:59:12, 8.18s/it]
52%|█████▏ | 1444/2757 [3:16:49<2:56:38, 8.07s/it]
52%|█████▏ | 1445/2757 [3:16:57<2:55:19, 8.02s/it]
52%|█████▏ | 1446/2757 [3:17:05<2:56:00, 8.06s/it]
52%|█████▏ | 1447/2757 [3:17:12<2:54:06, 7.97s/it]
53%|█████▎ | 1448/2757 [3:17:20<2:53:12, 7.94s/it]
53%|█████▎ | 1449/2757 [3:17:29<2:55:14, 8.04s/it]
53%|█████▎ | 1450/2757 [3:17:37<2:56:59, 8.13s/it]
53%|█████▎ | 1450/2757 [3:17:37<2:56:59, 8.13s/it]
53%|█████▎ | 1451/2757 [3:17:45<2:59:53, 8.26s/it]
53%|█████▎ | 1452/2757 [3:17:54<3:00:28, 8.30s/it]
53%|█████▎ | 1453/2757 [3:18:02<2:59:46, 8.27s/it]
53%|█████▎ | 1454/2757 [3:18:10<2:58:12, 8.21s/it]
53%|█████▎ | 1455/2757 [3:18:18<2:55:35, 8.09s/it]
53%|█████▎ | 1456/2757 [3:18:26<2:56:48, 8.15s/it]
53%|█████▎ | 1457/2757 [3:18:34<2:56:19, 8.14s/it]
53%|█████▎ | 1458/2757 [3:18:42<2:53:41, 8.02s/it]
53%|█████▎ | 1459/2757 [3:18:50<2:55:00, 8.09s/it]
53%|█████▎ | 1460/2757 [3:18:58<2:52:27, 7.98s/it]
53%|█████▎ | 1460/2757 [3:18:58<2:52:27, 7.98s/it]
53%|█████▎ | 1461/2757 [3:19:06<2:54:08, 8.06s/it]
53%|█████▎ | 1462/2757 [3:19:14<2:52:08, 7.98s/it]
53%|█████▎ | 1463/2757 [3:19:22<2:51:51, 7.97s/it]
53%|█████▎ | 1464/2757 [3:19:30<2:53:01, 8.03s/it]
53%|█████▎ | 1465/2757 [3:19:38<2:54:25, 8.10s/it]
53%|█████▎ | 1466/2757 [3:19:47<2:54:37, 8.12s/it]
53%|█████▎ | 1467/2757 [3:19:55<2:56:10, 8.19s/it]
53%|█████▎ | 1468/2757 [3:20:03<2:57:44, 8.27s/it]
53%|█████▎ | 1469/2757 [3:20:12<2:56:20, 8.21s/it]
53%|█████▎ | 1470/2757 [3:20:20<2:57:17, 8.26s/it]
53%|█████▎ | 1470/2757 [3:20:20<2:57:17, 8.26s/it]
53%|█████▎ | 1471/2757 [3:20:28<2:55:03, 8.17s/it]
53%|█████▎ | 1472/2757 [3:20:36<2:54:08, 8.13s/it]
53%|█████▎ | 1473/2757 [3:20:43<2:49:53, 7.94s/it]
53%|█████▎ | 1474/2757 [3:20:51<2:49:18, 7.92s/it]
54%|█████▎ | 1475/2757 [3:20:59<2:46:52, 7.81s/it]
54%|█████▎ | 1476/2757 [3:21:07<2:48:25, 7.89s/it]
54%|█████▎ | 1477/2757 [3:21:15<2:49:12, 7.93s/it]
54%|█████▎ | 1478/2757 [3:21:23<2:48:49, 7.92s/it]
54%|█████▎ | 1479/2757 [3:21:30<2:45:53, 7.79s/it]
54%|█████▎ | 1480/2757 [3:21:38<2:45:55, 7.80s/it]
54%|█████▎ | 1480/2757 [3:21:38<2:45:55, 7.80s/it]
54%|█████▎ | 1481/2757 [3:21:46<2:45:43, 7.79s/it]
54%|█████▍ | 1482/2757 [3:21:54<2:46:29, 7.83s/it]
54%|█████▍ | 1483/2757 [3:22:02<2:47:57, 7.91s/it]
54%|█████▍ | 1484/2757 [3:22:10<2:48:01, 7.92s/it]
54%|█████▍ | 1485/2757 [3:22:18<2:47:51, 7.92s/it]
54%|█████▍ | 1486/2757 [3:22:26<2:46:56, 7.88s/it]
54%|█████▍ | 1487/2757 [3:22:33<2:45:08, 7.80s/it]
54%|█████▍ | 1488/2757 [3:22:41<2:47:17, 7.91s/it]
54%|█████▍ | 1489/2757 [3:22:49<2:47:41, 7.93s/it]
54%|█████▍ | 1490/2757 [3:22:57<2:46:24, 7.88s/it]
54%|█████▍ | 1490/2757 [3:22:57<2:46:24, 7.88s/it]
54%|█████▍ | 1491/2757 [3:23:05<2:44:54, 7.82s/it]
54%|█████▍ | 1492/2757 [3:23:12<2:43:34, 7.76s/it]
54%|█████▍ | 1493/2757 [3:23:21<2:47:17, 7.94s/it]
54%|█████▍ | 1494/2757 [3:23:29<2:47:01, 7.93s/it]
54%|█████▍ | 1495/2757 [3:23:37<2:47:16, 7.95s/it]
54%|█████▍ | 1496/2757 [3:23:44<2:45:00, 7.85s/it]
54%|█████▍ | 1497/2757 [3:23:53<2:48:12, 8.01s/it]
54%|█████▍ | 1498/2757 [3:24:01<2:47:40, 7.99s/it]
54%|█████▍ | 1499/2757 [3:24:09<2:47:46, 8.00s/it]
54%|█████▍ | 1500/2757 [3:24:17<2:49:38, 8.10s/it]
54%|█████▍ | 1500/2757 [3:24:17<2:49:38, 8.10s/it]
{'loss': '0.3822', 'grad_norm': '0.6954', 'learning_rate': '0.000112', 'rewards/chosen': '-1.646', 'rewards/rejected': '-3.055', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.409', 'logps/chosen': '-94.91', 'logps/rejected': '-116.6', 'logits/chosen': '-3.762', 'logits/rejected': '-3.743', 'epoch': '1.371'}
{'loss': '0.3603', 'grad_norm': '0.5787', 'learning_rate': '0.0001113', 'rewards/chosen': '-1.818', 'rewards/rejected': '-3.225', 'rewards/accuracies': '0.8047', 'rewards/margins': '1.406', 'logps/chosen': '-98.85', 'logps/rejected': '-118.1', 'logits/chosen': '-3.881', 'logits/rejected': '-3.811', 'epoch': '1.382'}
{'loss': '0.3662', 'grad_norm': '0.668', 'learning_rate': '0.0001105', 'rewards/chosen': '-1.788', 'rewards/rejected': '-3.26', 'rewards/accuracies': '0.8039', 'rewards/margins': '1.471', 'logps/chosen': '-100.2', 'logps/rejected': '-120.3', 'logits/chosen': '-3.82', 'logits/rejected': '-3.791', 'epoch': '1.393'}
{'loss': '0.3878', 'grad_norm': '0.6281', 'learning_rate': '0.0001098', 'rewards/chosen': '-1.594', 'rewards/rejected': '-3.012', 'rewards/accuracies': '0.7703', 'rewards/margins': '1.418', 'logps/chosen': '-95.59', 'logps/rejected': '-115.6', 'logits/chosen': '-3.73', 'logits/rejected': '-3.709', 'epoch': '1.404'}
{'loss': '0.3713', 'grad_norm': '0.6394', 'learning_rate': '0.0001091', 'rewards/chosen': '-1.553', 'rewards/rejected': '-2.992', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.439', 'logps/chosen': '-97.09', 'logps/rejected': '-116', 'logits/chosen': '-3.678', 'logits/rejected': '-3.603', 'epoch': '1.415'}
{'loss': '0.3837', 'grad_norm': '0.5887', 'learning_rate': '0.0001083', 'rewards/chosen': '-1.771', 'rewards/rejected': '-3.158', 'rewards/accuracies': '0.7773', 'rewards/margins': '1.388', 'logps/chosen': '-97.09', 'logps/rejected': '-115', 'logits/chosen': '-3.858', 'logits/rejected': '-3.729', 'epoch': '1.426'}
{'loss': '0.3637', 'grad_norm': '0.6475', 'learning_rate': '0.0001076', 'rewards/chosen': '-1.754', 'rewards/rejected': '-3.263', 'rewards/accuracies': '0.7883', 'rewards/margins': '1.509', 'logps/chosen': '-97.05', 'logps/rejected': '-119.6', 'logits/chosen': '-3.94', 'logits/rejected': '-3.782', 'epoch': '1.436'}
{'loss': '0.3758', 'grad_norm': '0.6646', 'learning_rate': '0.0001068', 'rewards/chosen': '-1.683', 'rewards/rejected': '-3.217', 'rewards/accuracies': '0.7719', 'rewards/margins': '1.534', 'logps/chosen': '-96.41', 'logps/rejected': '-117.3', 'logits/chosen': '-3.977', 'logits/rejected': '-3.893', 'epoch': '1.447'}
{'loss': '0.3911', 'grad_norm': '0.7959', 'learning_rate': '0.0001061', 'rewards/chosen': '-1.504', 'rewards/rejected': '-2.96', 'rewards/accuracies': '0.7672', 'rewards/margins': '1.456', 'logps/chosen': '-92.38', 'logps/rejected': '-112.3', 'logits/chosen': '-4.019', 'logits/rejected': '-3.9', 'epoch': '1.458'}
{'loss': '0.3842', 'grad_norm': '0.6806', 'learning_rate': '0.0001053', 'rewards/chosen': '-1.673', 'rewards/rejected': '-3.111', 'rewards/accuracies': '0.7758', 'rewards/margins': '1.438', 'logps/chosen': '-101.7', 'logps/rejected': '-121', 'logits/chosen': '-3.86', 'logits/rejected': '-3.82', 'epoch': '1.469'}
{'loss': '0.3684', 'grad_norm': '0.6531', 'learning_rate': '0.0001046', 'rewards/chosen': '-1.91', 'rewards/rejected': '-3.38', 'rewards/accuracies': '0.7852', 'rewards/margins': '1.47', 'logps/chosen': '-100.1', 'logps/rejected': '-119.4', 'logits/chosen': '-3.826', 'logits/rejected': '-3.773', 'epoch': '1.48'}
{'loss': '0.3801', 'grad_norm': '0.6556', 'learning_rate': '0.0001038', 'rewards/chosen': '-1.97', 'rewards/rejected': '-3.358', 'rewards/accuracies': '0.7648', 'rewards/margins': '1.388', 'logps/chosen': '-101.1', 'logps/rejected': '-120.3', 'logits/chosen': '-3.857', 'logits/rejected': '-3.755', 'epoch': '1.491'}
{'loss': '0.37', 'grad_norm': '0.5829', 'learning_rate': '0.0001031', 'rewards/chosen': '-1.854', 'rewards/rejected': '-3.286', 'rewards/accuracies': '0.8031', 'rewards/margins': '1.432', 'logps/chosen': '-99.89', 'logps/rejected': '-118.7', 'logits/chosen': '-3.933', 'logits/rejected': '-3.822', 'epoch': '1.502'}
{'loss': '0.3734', 'grad_norm': '0.7183', 'learning_rate': '0.0001023', 'rewards/chosen': '-1.893', 'rewards/rejected': '-3.374', 'rewards/accuracies': '0.7828', 'rewards/margins': '1.481', 'logps/chosen': '-97.94', 'logps/rejected': '-117.8', 'logits/chosen': '-4.136', 'logits/rejected': '-3.982', 'epoch': '1.513'}
{'loss': '0.3624', 'grad_norm': '0.6229', 'learning_rate': '0.0001016', 'rewards/chosen': '-1.698', 'rewards/rejected': '-3.211', 'rewards/accuracies': '0.7875', 'rewards/margins': '1.513', 'logps/chosen': '-101.3', 'logps/rejected': '-122.4', 'logits/chosen': '-3.943', 'logits/rejected': '-3.885', 'epoch': '1.524'}
{'loss': '0.3612', 'grad_norm': '0.7118', 'learning_rate': '0.0001008', 'rewards/chosen': '-1.797', 'rewards/rejected': '-3.256', 'rewards/accuracies': '0.7961', 'rewards/margins': '1.459', 'logps/chosen': '-96.92', 'logps/rejected': '-115.9', 'logits/chosen': '-4.146', 'logits/rejected': '-4.064', 'epoch': '1.534'}
{'loss': '0.3773', 'grad_norm': '0.4945', 'learning_rate': '0.0001001', 'rewards/chosen': '-1.945', 'rewards/rejected': '-3.407', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.461', 'logps/chosen': '-100.1', 'logps/rejected': '-120.7', 'logits/chosen': '-4.004', 'logits/rejected': '-3.955', 'epoch': '1.545'}
{'loss': '0.3719', 'grad_norm': '0.5279', 'learning_rate': '9.933e-05', 'rewards/chosen': '-1.995', 'rewards/rejected': '-3.502', 'rewards/accuracies': '0.7797', 'rewards/margins': '1.507', 'logps/chosen': '-102.5', 'logps/rejected': '-122.1', 'logits/chosen': '-3.986', 'logits/rejected': '-3.933', 'epoch': '1.556'}
{'loss': '0.3743', 'grad_norm': '0.5255', 'learning_rate': '9.858e-05', 'rewards/chosen': '-1.921', 'rewards/rejected': '-3.381', 'rewards/accuracies': '0.7891', 'rewards/margins': '1.461', 'logps/chosen': '-100.2', 'logps/rejected': '-119.8', 'logits/chosen': '-3.869', 'logits/rejected': '-3.812', 'epoch': '1.567'}
{'loss': '0.3566', 'grad_norm': '0.6176', 'learning_rate': '9.783e-05', 'rewards/chosen': '-1.973', 'rewards/rejected': '-3.487', 'rewards/accuracies': '0.8', 'rewards/margins': '1.513', 'logps/chosen': '-101.5', 'logps/rejected': '-121.8', 'logits/chosen': '-3.869', 'logits/rejected': '-3.742', 'epoch': '1.578'}
{'loss': '0.3684', 'grad_norm': '0.6701', 'learning_rate': '9.708e-05', 'rewards/chosen': '-2.017', 'rewards/rejected': '-3.525', 'rewards/accuracies': '0.7781', 'rewards/margins': '1.508', 'logps/chosen': '-97.51', 'logps/rejected': '-118.8', 'logits/chosen': '-3.919', 'logits/rejected': '-3.9', 'epoch': '1.589'}
{'loss': '0.369', 'grad_norm': '0.8422', 'learning_rate': '9.634e-05', 'rewards/chosen': '-1.953', 'rewards/rejected': '-3.507', 'rewards/accuracies': '0.7836', 'rewards/margins': '1.554', 'logps/chosen': '-103.5', 'logps/rejected': '-125.3', 'logits/chosen': '-3.815', 'logits/rejected': '-3.763', 'epoch': '1.6'}
{'loss': '0.3787', 'grad_norm': '0.626', 'learning_rate': '9.559e-05', 'rewards/chosen': '-1.782', 'rewards/rejected': '-3.268', 'rewards/accuracies': '0.7883', 'rewards/margins': '1.486', 'logps/chosen': '-98.55', 'logps/rejected': '-118.6', 'logits/chosen': '-3.939', 'logits/rejected': '-3.889', 'epoch': '1.611'}
{'loss': '0.3668', 'grad_norm': '0.6023', 'learning_rate': '9.484e-05', 'rewards/chosen': '-1.582', 'rewards/rejected': '-3.109', 'rewards/accuracies': '0.7812', 'rewards/margins': '1.527', 'logps/chosen': '-96.4', 'logps/rejected': '-115.9', 'logits/chosen': '-4.026', 'logits/rejected': '-3.953', 'epoch': '1.621'}
{'loss': '0.3673', 'grad_norm': '0.5291', 'learning_rate': '9.409e-05', 'rewards/chosen': '-1.715', 'rewards/rejected': '-3.256', 'rewards/accuracies': '0.7898', 'rewards/margins': '1.542', 'logps/chosen': '-96.86', 'logps/rejected': '-118.7', 'logits/chosen': '-3.99', 'logits/rejected': '-3.968', 'epoch': '1.632'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
0%| | 0/300 [00:00<?, ?it/s]
1%| | 2/300 [00:00<00:52, 5.73it/s]
1%| | 3/300 [00:00<01:47, 2.76it/s]
1%|▏ | 4/300 [00:01<01:35, 3.09it/s]
2%|▏ | 5/300 [00:01<01:45, 2.80it/s]
2%|▏ | 6/300 [00:01<01:40, 2.91it/s]
2%|▏ | 7/300 [00:02<01:55, 2.54it/s]
3%|▎ | 8/300 [00:02<01:51, 2.62it/s]
3%|▎ | 9/300 [00:03<01:52, 2.58it/s]
3%|▎ | 10/300 [00:03<01:51, 2.60it/s]
4%|▎ | 11/300 [00:04<01:53, 2.56it/s]
4%|▍ | 12/300 [00:04<01:39, 2.88it/s]
4%|▍ | 13/300 [00:04<01:45, 2.73it/s]
5%|▍ | 14/300 [00:04<01:37, 2.94it/s]
5%|▌ | 15/300 [00:05<01:47, 2.66it/s]
5%|▌ | 16/300 [00:05<01:35, 2.99it/s]
6%|▌ | 17/300 [00:06<01:38, 2.86it/s]
6%|▌ | 18/300 [00:06<01:29, 3.17it/s]
6%|▋ | 19/300 [00:06<01:44, 2.69it/s]
7%|▋ | 20/300 [00:07<01:35, 2.92it/s]
7%|▋ | 21/300 [00:07<01:47, 2.59it/s]
7%|▋ | 22/300 [00:07<01:35, 2.92it/s]
8%|▊ | 23/300 [00:08<01:35, 2.90it/s]
8%|▊ | 24/300 [00:08<01:25, 3.22it/s]
8%|▊ | 25/300 [00:08<01:37, 2.82it/s]
9%|▊ | 26/300 [00:09<01:30, 3.03it/s]
9%|▉ | 27/300 [00:10<02:19, 1.95it/s]
9%|▉ | 28/300 [00:10<01:57, 2.32it/s]
10%|▉ | 29/300 [00:10<02:04, 2.17it/s]
10%|█ | 30/300 [00:11<01:47, 2.52it/s]
10%|█ | 31/300 [00:11<01:55, 2.34it/s]
11%|█ | 32/300 [00:11<01:55, 2.31it/s]
11%|█ | 33/300 [00:12<01:46, 2.50it/s]
11%|█▏ | 34/300 [00:12<01:34, 2.80it/s]
12%|█▏ | 35/300 [00:12<01:37, 2.72it/s]
12%|█▏ | 36/300 [00:13<01:28, 2.97it/s]
12%|█▏ | 37/300 [00:13<01:39, 2.63it/s]
13%|█▎ | 38/300 [00:13<01:27, 3.00it/s]
13%|█▎ | 39/300 [00:14<01:36, 2.70it/s]
13%|█▎ | 40/300 [00:14<01:25, 3.04it/s]
14%|█▎ | 41/300 [00:15<01:35, 2.71it/s]
14%|█▍ | 42/300 [00:15<01:24, 3.05it/s]
14%|█▍ | 43/300 [00:15<01:27, 2.95it/s]
15%|█▍ | 44/300 [00:15<01:19, 3.24it/s]
15%|█▌ | 45/300 [00:16<01:25, 2.99it/s]
15%|█▌ | 46/300 [00:16<01:25, 2.97it/s]
16%|█▌ | 47/300 [00:17<01:30, 2.78it/s]
16%|█▌ | 48/300 [00:17<01:23, 3.02it/s]
16%|█▋ | 49/300 [00:17<01:23, 3.00it/s]
17%|█▋ | 50/300 [00:17<01:15, 3.33it/s]
17%|█▋ | 51/300 [00:18<01:18, 3.18it/s]
17%|█▋ | 52/300 [00:18<01:15, 3.29it/s]
18%|█▊ | 53/300 [00:18<01:28, 2.79it/s]
18%|█▊ | 54/300 [00:19<01:19, 3.09it/s]
18%|█▊ | 55/300 [00:19<01:27, 2.80it/s]
19%|█▊ | 56/300 [00:19<01:15, 3.21it/s]
19%|█▉ | 57/300 [00:20<01:29, 2.72it/s]
19%|█▉ | 58/300 [00:20<01:28, 2.73it/s]
20%|█▉ | 59/300 [00:21<01:31, 2.62it/s]
20%|██ | 60/300 [00:21<01:21, 2.95it/s]
20%|██ | 61/300 [00:21<01:23, 2.85it/s]
21%|██ | 62/300 [00:22<01:22, 2.88it/s]
21%|██ | 63/300 [00:22<01:27, 2.70it/s]
21%|██▏ | 64/300 [00:22<01:25, 2.76it/s]
22%|██▏ | 65/300 [00:23<01:28, 2.66it/s]
22%|██▏ | 66/300 [00:23<01:20, 2.92it/s]
22%|██▏ | 67/300 [00:24<01:27, 2.66it/s]
23%|██▎ | 68/300 [00:24<01:18, 2.95it/s]
23%|██▎ | 69/300 [00:24<01:21, 2.84it/s]
23%|██▎ | 70/300 [00:24<01:13, 3.14it/s]
24%|██▎ | 71/300 [00:25<01:15, 3.02it/s]
24%|██▍ | 72/300 [00:25<01:09, 3.28it/s]
24%|██▍ | 73/300 [00:25<01:12, 3.14it/s]
25%|██▍ | 74/300 [00:26<01:08, 3.32it/s]
25%|██▌ | 75/300 [00:26<01:13, 3.08it/s]
25%|██▌ | 76/300 [00:26<01:14, 3.01it/s]
26%|██▌ | 77/300 [00:27<01:19, 2.81it/s]
26%|██▌ | 78/300 [00:27<01:12, 3.05it/s]
26%|██▋ | 79/300 [00:27<01:15, 2.94it/s]
27%|██▋ | 80/300 [00:28<01:11, 3.06it/s]
27%|██▋ | 81/300 [00:28<01:20, 2.74it/s]
27%|██▋ | 82/300 [00:28<01:11, 3.04it/s]
28%|██▊ | 83/300 [00:29<01:15, 2.88it/s]
28%|██▊ | 84/300 [00:29<01:15, 2.86it/s]
28%|██▊ | 85/300 [00:30<01:19, 2.70it/s]
29%|██▊ | 86/300 [00:30<01:11, 3.00it/s]
29%|██▉ | 87/300 [00:30<01:13, 2.91it/s]
29%|██▉ | 88/300 [00:30<01:07, 3.16it/s]
30%|██▉ | 89/300 [00:31<01:11, 2.96it/s]
30%|███ | 90/300 [00:31<01:05, 3.19it/s]
30%|███ | 91/300 [00:31<01:09, 3.00it/s]
31%|███ | 92/300 [00:32<01:03, 3.27it/s]
31%|███ | 93/300 [00:32<01:09, 2.98it/s]
31%|███▏ | 94/300 [00:32<01:02, 3.28it/s]
32%|███▏ | 95/300 [00:33<01:08, 3.01it/s]
32%|███▏ | 96/300 [00:33<01:03, 3.20it/s]
32%|███▏ | 97/300 [00:33<01:15, 2.67it/s]
33%|███▎ | 98/300 [00:34<01:09, 2.92it/s]
33%|███▎ | 99/300 [00:34<01:12, 2.78it/s]
33%|███▎ | 100/300 [00:34<01:06, 2.99it/s]
34%|███▎ | 101/300 [00:35<01:09, 2.86it/s]
34%|███▍ | 102/300 [00:35<01:07, 2.94it/s]
34%|███▍ | 103/300 [00:36<01:10, 2.79it/s]
35%|███▍ | 104/300 [00:36<01:02, 3.11it/s]
35%|███▌ | 105/300 [00:36<01:05, 2.96it/s]
35%|███▌ | 106/300 [00:36<01:01, 3.17it/s]
36%|███▌ | 107/300 [00:37<01:05, 2.95it/s]
36%|███▌ | 108/300 [00:37<01:03, 3.05it/s]
36%|███▋ | 109/300 [00:38<01:06, 2.86it/s]
37%|███▋ | 110/300 [00:38<01:02, 3.03it/s]
37%|███▋ | 111/300 [00:38<01:11, 2.65it/s]
37%|███▋ | 112/300 [00:39<01:06, 2.84it/s]
38%|███▊ | 113/300 [00:39<01:09, 2.70it/s]
38%|███▊ | 114/300 [00:39<01:07, 2.74it/s]
38%|███▊ | 115/300 [00:40<01:10, 2.62it/s]
39%|███▊ | 116/300 [00:40<01:02, 2.95it/s]
39%|███▉ | 117/300 [00:40<01:03, 2.88it/s]
39%|███▉ | 118/300 [00:41<00:58, 3.10it/s]
40%|███▉ | 119/300 [00:41<01:00, 2.98it/s]
40%|████ | 120/300 [00:41<00:59, 3.01it/s]
40%|████ | 121/300 [00:42<01:02, 2.89it/s]
41%|████ | 122/300 [00:42<00:57, 3.11it/s]
41%|████ | 123/300 [00:42<01:02, 2.84it/s]
41%|████▏ | 124/300 [00:43<00:58, 3.02it/s]
42%|████▏ | 125/300 [00:43<00:59, 2.92it/s]
42%|████▏ | 126/300 [00:43<01:00, 2.90it/s]
42%|████▏ | 127/300 [00:44<01:02, 2.78it/s]
43%|████▎ | 128/300 [00:44<00:57, 2.98it/s]
43%|████▎ | 129/300 [00:44<00:59, 2.86it/s]
43%|████▎ | 130/300 [00:45<00:54, 3.14it/s]
44%|████▎ | 131/300 [00:45<01:00, 2.78it/s]
44%|████▍ | 132/300 [00:45<00:55, 3.02it/s]
44%|████▍ | 133/300 [00:46<01:02, 2.67it/s]
45%|████▍ | 134/300 [00:46<00:56, 2.92it/s]
45%|████▌ | 135/300 [00:47<00:58, 2.80it/s]
45%|████▌ | 136/300 [00:47<00:53, 3.09it/s]
46%|████▌ | 137/300 [00:47<00:53, 3.02it/s]
46%|████▌ | 138/300 [00:47<00:50, 3.23it/s]
46%|████▋ | 139/300 [00:48<00:51, 3.16it/s]
47%|████▋ | 140/300 [00:48<00:46, 3.42it/s]
47%|████▋ | 141/300 [00:48<00:48, 3.26it/s]
47%|████▋ | 142/300 [00:49<00:50, 3.13it/s]
48%|████▊ | 143/300 [00:49<00:54, 2.88it/s]
48%|████▊ | 144/300 [00:49<00:48, 3.19it/s]
48%|████▊ | 145/300 [00:50<00:49, 3.14it/s]
49%|████▊ | 146/300 [00:50<00:46, 3.29it/s]
49%|████▉ | 147/300 [00:50<00:53, 2.86it/s]
49%|████▉ | 148/300 [00:51<00:50, 3.03it/s]
50%|████▉ | 149/300 [00:51<00:53, 2.80it/s]
50%|█████ | 150/300 [00:51<00:49, 3.00it/s]
50%|█████ | 151/300 [00:52<00:51, 2.88it/s]
51%|█████ | 152/300 [00:52<00:46, 3.19it/s]
51%|█████ | 153/300 [00:52<00:47, 3.08it/s]
51%|█████▏ | 154/300 [00:53<00:43, 3.36it/s]
52%|█████▏ | 155/300 [00:53<00:45, 3.17it/s]
52%|█████▏ | 156/300 [00:53<00:41, 3.49it/s]
52%|█████▏ | 157/300 [00:54<00:45, 3.13it/s]
53%|█████▎ | 158/300 [00:54<00:43, 3.27it/s]
53%|█████▎ | 159/300 [00:54<00:53, 2.66it/s]
53%|█████▎ | 160/300 [00:55<00:48, 2.91it/s]
54%|█████▎ | 161/300 [00:55<00:51, 2.71it/s]
54%|█████▍ | 162/300 [00:55<00:50, 2.75it/s]
54%|█████▍ | 163/300 [00:56<00:56, 2.41it/s]
55%|█████▍ | 164/300 [00:56<00:53, 2.53it/s]
55%|█████▌ | 165/300 [00:57<00:53, 2.54it/s]
55%|█████▌ | 166/300 [00:57<00:50, 2.65it/s]
56%|█████▌ | 167/300 [00:57<00:54, 2.43it/s]
56%|█████▌ | 168/300 [00:58<00:48, 2.74it/s]
56%|█████▋ | 169/300 [00:58<00:52, 2.48it/s]
57%|█████▋ | 170/300 [00:59<00:50, 2.58it/s]
57%|█████▋ | 171/300 [00:59<00:50, 2.54it/s]
57%|█████▋ | 172/300 [00:59<00:44, 2.88it/s]
58%|█████▊ | 173/300 [01:00<00:46, 2.72it/s]
58%|█████▊ | 174/300 [01:00<00:41, 3.05it/s]
58%|█████▊ | 175/300 [01:00<00:44, 2.79it/s]
59%|█████▊ | 176/300 [01:01<00:39, 3.11it/s]
59%|█████▉ | 177/300 [01:01<00:46, 2.65it/s]
59%|█████▉ | 178/300 [01:01<00:39, 3.08it/s]
60%|█████▉ | 179/300 [01:02<00:42, 2.88it/s]
60%|██████ | 180/300 [01:02<00:38, 3.16it/s]
60%|██████ | 181/300 [01:02<00:41, 2.85it/s]
61%|██████ | 182/300 [01:03<00:37, 3.15it/s]
61%|██████ | 183/300 [01:03<00:40, 2.89it/s]
61%|██████▏ | 184/300 [01:03<00:40, 2.88it/s]
62%|██████▏ | 185/300 [01:04<00:48, 2.38it/s]
62%|██████▏ | 186/300 [01:04<00:43, 2.63it/s]
62%|██████▏ | 187/300 [01:05<00:42, 2.68it/s]
63%|██████▎ | 188/300 [01:05<00:37, 2.99it/s]
63%|██████▎ | 189/300 [01:05<00:40, 2.76it/s]
63%|██████▎ | 190/300 [01:05<00:36, 3.02it/s]
64%|██████▎ | 191/300 [01:06<00:39, 2.77it/s]
64%|██████▍ | 192/300 [01:06<00:38, 2.84it/s]
64%|██████▍ | 193/300 [01:07<00:42, 2.51it/s]
65%|██████▍ | 194/300 [01:07<00:40, 2.61it/s]
65%|██████▌ | 195/300 [01:08<00:43, 2.42it/s]
65%|██████▌ | 196/300 [01:08<00:38, 2.67it/s]
66%|██████▌ | 197/300 [01:08<00:38, 2.67it/s]
66%|██████▌ | 198/300 [01:09<00:35, 2.91it/s]
66%|██████▋ | 199/300 [01:09<00:36, 2.81it/s]
67%|██████▋ | 200/300 [01:09<00:33, 2.99it/s]
67%|██████▋ | 201/300 [01:10<00:35, 2.79it/s]
67%|██████▋ | 202/300 [01:10<00:31, 3.08it/s]
68%|██████▊ | 203/300 [01:10<00:33, 2.94it/s]
68%|██████▊ | 204/300 [01:10<00:29, 3.23it/s]
68%|██████▊ | 205/300 [01:11<00:31, 3.01it/s]
69%|██████▊ | 206/300 [01:11<00:27, 3.40it/s]
69%|██████▉ | 207/300 [01:12<00:31, 2.93it/s]
69%|██████▉ | 208/300 [01:12<00:31, 2.94it/s]
70%|██████▉ | 209/300 [01:12<00:32, 2.80it/s]
70%|███████ | 210/300 [01:13<00:29, 3.01it/s]
70%|███████ | 211/300 [01:13<00:33, 2.66it/s]
71%|███████ | 212/300 [01:13<00:29, 2.98it/s]
71%|███████ | 213/300 [01:14<00:31, 2.74it/s]
71%|███████▏ | 214/300 [01:14<00:28, 3.00it/s]
72%|███████▏ | 215/300 [01:14<00:30, 2.77it/s]
72%|███████▏ | 216/300 [01:15<00:27, 3.07it/s]
72%|███████▏ | 217/300 [01:15<00:30, 2.75it/s]
73%|███████▎ | 218/300 [01:15<00:25, 3.17it/s]
73%|███████▎ | 219/300 [01:16<00:28, 2.87it/s]
73%|███████▎ | 220/300 [01:16<00:25, 3.15it/s]
74%|███████▎ | 221/300 [01:16<00:28, 2.79it/s]
74%|███████▍ | 222/300 [01:17<00:25, 3.09it/s]
74%|███████▍ | 223/300 [01:17<00:28, 2.75it/s]
75%|███████▍ | 224/300 [01:17<00:25, 2.95it/s]
75%|███████▌ | 225/300 [01:18<00:26, 2.82it/s]
75%|███████▌ | 226/300 [01:18<00:24, 3.05it/s]
76%|███████▌ | 227/300 [01:18<00:26, 2.79it/s]
76%|███████▌ | 228/300 [01:19<00:23, 3.12it/s]
76%|███████▋ | 229/300 [01:19<00:25, 2.76it/s]
77%|███████▋ | 230/300 [01:19<00:24, 2.86it/s]
77%|███████▋ | 231/300 [01:20<00:26, 2.59it/s]
77%|███████▋ | 232/300 [01:20<00:25, 2.69it/s]
78%|███████▊ | 233/300 [01:21<00:27, 2.42it/s]
78%|███████▊ | 234/300 [01:21<00:23, 2.78it/s]
78%|███████▊ | 235/300 [01:21<00:25, 2.55it/s]
79%|███████▊ | 236/300 [01:22<00:24, 2.64it/s]
79%|███████▉ | 237/300 [01:22<00:24, 2.56it/s]
79%|███████▉ | 238/300 [01:22<00:21, 2.88it/s]
80%|███████▉ | 239/300 [01:23<00:21, 2.87it/s]
80%|████████ | 240/300 [01:23<00:18, 3.18it/s]
80%|████████ | 241/300 [01:23<00:19, 3.07it/s]
81%|████████ | 242/300 [01:24<00:18, 3.10it/s]
81%|████████ | 243/300 [01:24<00:20, 2.74it/s]
81%|████████▏ | 244/300 [01:25<00:19, 2.84it/s]
82%|████████▏ | 245/300 [01:25<00:20, 2.67it/s]
82%|████████▏ | 246/300 [01:25<00:19, 2.76it/s]
82%|████████▏ | 247/300 [01:26<00:20, 2.59it/s]
83%|████████▎ | 248/300 [01:26<00:17, 2.92it/s]
83%|████████▎ | 249/300 [01:26<00:18, 2.78it/s]
83%|████████▎ | 250/300 [01:27<00:17, 2.90it/s]
84%|████████▎ | 251/300 [01:27<00:17, 2.80it/s]
84%|████████▍ | 252/300 [01:27<00:16, 2.99it/s]
84%|████████▍ | 253/300 [01:28<00:16, 2.83it/s]
85%|████████▍ | 254/300 [01:28<00:14, 3.20it/s]
85%|████████▌ | 255/300 [01:28<00:15, 2.86it/s]
85%|████████▌ | 256/300 [01:29<00:13, 3.18it/s]
86%|████████▌ | 257/300 [01:29<00:14, 2.97it/s]
86%|████████▌ | 258/300 [01:29<00:12, 3.24it/s]
86%|████████▋ | 259/300 [01:30<00:13, 2.99it/s]
87%|████████▋ | 260/300 [01:30<00:12, 3.17it/s]
87%|████████▋ | 261/300 [01:30<00:13, 3.00it/s]
87%|████████▋ | 262/300 [01:31<00:11, 3.29it/s]
88%|████████▊ | 263/300 [01:31<00:12, 3.02it/s]
88%|████████▊ | 264/300 [01:31<00:12, 2.99it/s]
88%|████████▊ | 265/300 [01:32<00:13, 2.58it/s]
89%|████████▊ | 266/300 [01:32<00:12, 2.68it/s]
89%|████████▉ | 267/300 [01:33<00:13, 2.42it/s]
89%|████████▉ | 268/300 [01:33<00:11, 2.78it/s]
90%|████████▉ | 269/300 [01:33<00:12, 2.46it/s]
90%|█████████ | 270/300 [01:34<00:11, 2.63it/s]
90%|█████████ | 271/300 [01:34<00:12, 2.37it/s]
91%|█████████ | 272/300 [01:35<00:10, 2.64it/s]
91%|█████████ | 273/300 [01:35<00:11, 2.29it/s]
91%|█████████▏| 274/300 [01:35<00:09, 2.63it/s]
92%|█████████▏| 275/300 [01:36<00:10, 2.40it/s]
92%|█████████▏| 276/300 [01:36<00:09, 2.59it/s]
92%|█████████▏| 277/300 [01:37<00:09, 2.48it/s]
93%|█████████▎| 278/300 [01:37<00:07, 2.81it/s]
93%|█████████▎| 279/300 [01:37<00:07, 2.70it/s]
93%|█████████▎| 280/300 [01:38<00:07, 2.82it/s]
94%|█████████▎| 281/300 [01:38<00:07, 2.67it/s]
94%|█████████▍| 282/300 [01:38<00:06, 2.99it/s]
94%|█████████▍| 283/300 [01:39<00:06, 2.78it/s]
95%|█████████▍| 284/300 [01:39<00:05, 3.06it/s]
95%|█████████▌| 285/300 [01:39<00:05, 2.65it/s]
95%|█████████▌| 286/300 [01:40<00:05, 2.71it/s]
96%|█████████▌| 287/300 [01:40<00:04, 2.64it/s]
96%|█████████▌| 288/300 [01:40<00:03, 3.06it/s]
96%|█████████▋| 289/300 [01:41<00:03, 2.80it/s]
97%|█████████▋| 290/300 [01:41<00:03, 2.82it/s]
97%|█████████▋| 291/300 [01:42<00:03, 2.56it/s]
97%|█████████▋| 292/300 [01:42<00:02, 2.89it/s]
98%|█████████▊| 293/300 [01:42<00:02, 2.75it/s]
98%|█████████▊| 294/300 [01:42<00:02, 2.99it/s]
98%|█████████▊| 295/300 [01:43<00:01, 2.94it/s]
99%|█████████▊| 296/300 [01:43<00:01, 3.14it/s]
99%|█████████▉| 297/300 [01:44<00:01, 2.90it/s]
99%|█████████▉| 298/300 [01:44<00:00, 3.24it/s]
100%|█████████▉| 299/300 [01:44<00:00, 2.85it/s]
100%|██████████| 300/300 [01:45<00:00, 2.89it/s]

54%|█████▍ | 1500/2757 [3:26:03<2:49:38, 8.10s/it]
100%|██████████| 300/300 [01:45<00:00, 2.89it/s]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-1500/tokenizer_config.json.
54%|█████▍ | 1501/2757 [3:26:15<14:22:18, 41.19s/it]
54%|█████▍ | 1502/2757 [3:26:23<10:53:54, 31.26s/it]
55%|█████▍ | 1503/2757 [3:26:32<8:28:57, 24.35s/it]
55%|█████▍ | 1504/2757 [3:26:40<6:45:34, 19.42s/it]
55%|█████▍ | 1505/2757 [3:26:48<5:34:40, 16.04s/it]
55%|█████▍ | 1506/2757 [3:26:56<4:45:26, 13.69s/it]
55%|█████▍ | 1507/2757 [3:27:04<4:08:18, 11.92s/it]
55%|█████▍ | 1508/2757 [3:27:12<3:42:04, 10.67s/it]
55%|█████▍ | 1509/2757 [3:27:20<3:29:03, 10.05s/it]
55%|█████▍ | 1510/2757 [3:27:28<3:15:28, 9.41s/it]
55%|█████▍ | 1510/2757 [3:27:28<3:15:28, 9.41s/it]
55%|█████▍ | 1511/2757 [3:27:36<3:06:48, 9.00s/it]
55%|█████▍ | 1512/2757 [3:27:44<3:01:00, 8.72s/it]
55%|█████▍ | 1513/2757 [3:27:52<2:56:43, 8.52s/it]
55%|█████▍ | 1514/2757 [3:28:00<2:53:37, 8.38s/it]
55%|█████▍ | 1515/2757 [3:28:09<2:52:54, 8.35s/it]
55%|█████▍ | 1516/2757 [3:28:16<2:47:34, 8.10s/it]
55%|█████▌ | 1517/2757 [3:28:24<2:45:43, 8.02s/it]
55%|█████▌ | 1518/2757 [3:28:32<2:45:29, 8.01s/it]
55%|█████▌ | 1519/2757 [3:28:40<2:46:13, 8.06s/it]
55%|█████▌ | 1520/2757 [3:28:48<2:45:21, 8.02s/it]
55%|█████▌ | 1520/2757 [3:28:48<2:45:21, 8.02s/it]
55%|█████▌ | 1521/2757 [3:28:56<2:44:21, 7.98s/it]
55%|█████▌ | 1522/2757 [3:29:04<2:42:17, 7.88s/it]
55%|█████▌ | 1523/2757 [3:29:11<2:42:34, 7.90s/it]
55%|█████▌ | 1524/2757 [3:29:20<2:43:54, 7.98s/it]
55%|█████▌ | 1525/2757 [3:29:27<2:42:59, 7.94s/it]
55%|█████▌ | 1526/2757 [3:29:35<2:41:58, 7.89s/it]
55%|█████▌ | 1527/2757 [3:29:43<2:43:24, 7.97s/it]
55%|█████▌ | 1528/2757 [3:29:51<2:42:23, 7.93s/it]
55%|█████▌ | 1529/2757 [3:29:59<2:42:14, 7.93s/it]
55%|█████▌ | 1530/2757 [3:30:07<2:42:41, 7.96s/it]
55%|█████▌ | 1530/2757 [3:30:07<2:42:41, 7.96s/it]
56%|█████▌ | 1531/2757 [3:30:15<2:43:13, 7.99s/it]
56%|█████▌ | 1532/2757 [3:30:23<2:43:22, 8.00s/it]
56%|█████▌ | 1533/2757 [3:30:31<2:44:05, 8.04s/it]
56%|█████▌ | 1534/2757 [3:30:40<2:44:21, 8.06s/it]
56%|█████▌ | 1535/2757 [3:30:47<2:40:01, 7.86s/it]
56%|█████▌ | 1536/2757 [3:30:55<2:40:58, 7.91s/it]
56%|█████▌ | 1537/2757 [3:31:02<2:38:28, 7.79s/it]
56%|█████▌ | 1538/2757 [3:31:11<2:40:21, 7.89s/it]
56%|█████▌ | 1539/2757 [3:31:19<2:41:45, 7.97s/it]
56%|█████▌ | 1540/2757 [3:31:27<2:41:57, 7.98s/it]
56%|█████▌ | 1540/2757 [3:31:27<2:41:57, 7.98s/it]
56%|█████▌ | 1541/2757 [3:31:34<2:40:00, 7.90s/it]
56%|█████▌ | 1542/2757 [3:31:43<2:41:37, 7.98s/it]
56%|█████▌ | 1543/2757 [3:31:51<2:44:50, 8.15s/it]
56%|█████▌ | 1544/2757 [3:31:59<2:45:32, 8.19s/it]
56%|█████▌ | 1545/2757 [3:32:07<2:43:52, 8.11s/it]
56%|█████▌ | 1546/2757 [3:32:15<2:40:52, 7.97s/it]
56%|█████▌ | 1547/2757 [3:32:23<2:40:24, 7.95s/it]
56%|█████▌ | 1548/2757 [3:32:31<2:40:04, 7.94s/it]
56%|█████▌ | 1549/2757 [3:32:38<2:34:09, 7.66s/it]
56%|█████▌ | 1550/2757 [3:32:46<2:39:44, 7.94s/it]
56%|█████▌ | 1550/2757 [3:32:46<2:39:44, 7.94s/it]
56%|█████▋ | 1551/2757 [3:32:55<2:43:11, 8.12s/it]
56%|█████▋ | 1552/2757 [3:33:03<2:44:55, 8.21s/it]
56%|█████▋ | 1553/2757 [3:33:11<2:42:47, 8.11s/it]
56%|█████▋ | 1554/2757 [3:33:20<2:43:39, 8.16s/it]
56%|█████▋ | 1555/2757 [3:33:27<2:40:03, 7.99s/it]
56%|█████▋ | 1556/2757 [3:33:35<2:41:31, 8.07s/it]
56%|█████▋ | 1557/2757 [3:33:44<2:42:18, 8.12s/it]
57%|█████▋ | 1558/2757 [3:33:52<2:43:52, 8.20s/it]
57%|█████▋ | 1559/2757 [3:34:00<2:41:43, 8.10s/it]
57%|█████▋ | 1560/2757 [3:34:08<2:38:52, 7.96s/it]
57%|█████▋ | 1560/2757 [3:34:08<2:38:52, 7.96s/it]
57%|█████▋ | 1561/2757 [3:34:15<2:37:05, 7.88s/it]
57%|█████▋ | 1562/2757 [3:34:23<2:37:48, 7.92s/it]
57%|█████▋ | 1563/2757 [3:34:31<2:34:50, 7.78s/it]
57%|█████▋ | 1564/2757 [3:34:39<2:36:01, 7.85s/it]
57%|█████▋ | 1565/2757 [3:34:47<2:35:44, 7.84s/it]
57%|█████▋ | 1566/2757 [3:34:54<2:35:34, 7.84s/it]
57%|█████▋ | 1567/2757 [3:35:03<2:37:16, 7.93s/it]
57%|█████▋ | 1568/2757 [3:35:10<2:36:32, 7.90s/it]
57%|█████▋ | 1569/2757 [3:35:18<2:35:05, 7.83s/it]
57%|█████▋ | 1570/2757 [3:35:26<2:37:06, 7.94s/it]
57%|█████▋ | 1570/2757 [3:35:26<2:37:06, 7.94s/it]
57%|█████▋ | 1571/2757 [3:35:34<2:36:30, 7.92s/it]
57%|█████▋ | 1572/2757 [3:35:42<2:33:50, 7.79s/it]
57%|█████▋ | 1573/2757 [3:35:50<2:34:37, 7.84s/it]
57%|█████▋ | 1574/2757 [3:35:58<2:37:04, 7.97s/it]
57%|█████▋ | 1575/2757 [3:36:06<2:37:12, 7.98s/it]
57%|█████▋ | 1576/2757 [3:36:14<2:36:36, 7.96s/it]
57%|█████▋ | 1577/2757 [3:36:22<2:36:21, 7.95s/it]
57%|█████▋ | 1578/2757 [3:36:29<2:35:30, 7.91s/it]
57%|█████▋ | 1579/2757 [3:36:37<2:34:24, 7.86s/it]
57%|█████▋ | 1580/2757 [3:36:45<2:33:15, 7.81s/it]
57%|█████▋ | 1580/2757 [3:36:45<2:33:15, 7.81s/it]
57%|█████▋ | 1581/2757 [3:36:53<2:36:48, 8.00s/it]
57%|█████▋ | 1582/2757 [3:37:01<2:35:05, 7.92s/it]
57%|█████▋ | 1583/2757 [3:37:09<2:35:15, 7.93s/it]
57%|█████▋ | 1584/2757 [3:37:17<2:37:33, 8.06s/it]
57%|█████▋ | 1585/2757 [3:37:25<2:36:44, 8.02s/it]
58%|█████▊ | 1586/2757 [3:37:33<2:35:44, 7.98s/it]
58%|█████▊ | 1587/2757 [3:37:41<2:36:08, 8.01s/it]
58%|█████▊ | 1588/2757 [3:37:49<2:35:20, 7.97s/it]
58%|█████▊ | 1589/2757 [3:37:57<2:33:21, 7.88s/it]
58%|█████▊ | 1590/2757 [3:38:04<2:31:41, 7.80s/it]
58%|█████▊ | 1590/2757 [3:38:04<2:31:41, 7.80s/it]
58%|█████▊ | 1591/2757 [3:38:13<2:33:55, 7.92s/it]
58%|█████▊ | 1592/2757 [3:38:21<2:35:36, 8.01s/it]
58%|█████▊ | 1593/2757 [3:38:29<2:34:55, 7.99s/it]
58%|█████▊ | 1594/2757 [3:38:37<2:36:52, 8.09s/it]
58%|█████▊ | 1595/2757 [3:38:45<2:37:45, 8.15s/it]
58%|█████▊ | 1596/2757 [3:38:53<2:35:29, 8.04s/it]
58%|█████▊ | 1597/2757 [3:39:01<2:33:29, 7.94s/it]
58%|█████▊ | 1598/2757 [3:39:09<2:33:27, 7.94s/it]
58%|█████▊ | 1599/2757 [3:39:17<2:33:29, 7.95s/it]
58%|█████▊ | 1600/2757 [3:39:25<2:35:19, 8.05s/it]
58%|█████▊ | 1600/2757 [3:39:25<2:35:19, 8.05s/it]
58%|█████▊ | 1601/2757 [3:39:33<2:33:13, 7.95s/it]
58%|█████▊ | 1602/2757 [3:39:41<2:34:49, 8.04s/it]
58%|█████▊ | 1603/2757 [3:39:49<2:32:06, 7.91s/it]
58%|█████▊ | 1604/2757 [3:39:56<2:28:41, 7.74s/it]
58%|█████▊ | 1605/2757 [3:40:04<2:30:23, 7.83s/it]
58%|█████▊ | 1606/2757 [3:40:12<2:29:21, 7.79s/it]
58%|█████▊ | 1607/2757 [3:40:20<2:30:20, 7.84s/it]
58%|█████▊ | 1608/2757 [3:40:28<2:33:01, 7.99s/it]
58%|█████▊ | 1609/2757 [3:40:36<2:32:22, 7.96s/it]
58%|█████▊ | 1610/2757 [3:40:44<2:31:20, 7.92s/it]
58%|█████▊ | 1610/2757 [3:40:44<2:31:20, 7.92s/it]
58%|█████▊ | 1611/2757 [3:40:52<2:32:06, 7.96s/it]
58%|█████▊ | 1612/2757 [3:41:00<2:30:30, 7.89s/it]
59%|█████▊ | 1613/2757 [3:41:08<2:32:52, 8.02s/it]
59%|█████▊ | 1614/2757 [3:41:16<2:35:14, 8.15s/it]
59%|█████▊ | 1615/2757 [3:41:25<2:36:37, 8.23s/it]
59%|█████▊ | 1616/2757 [3:41:33<2:34:48, 8.14s/it]
59%|█████▊ | 1617/2757 [3:41:41<2:33:45, 8.09s/it]
59%|█████▊ | 1618/2757 [3:41:49<2:32:13, 8.02s/it]
59%|█████▊ | 1619/2757 [3:41:57<2:33:27, 8.09s/it]
59%|█████▉ | 1620/2757 [3:42:04<2:30:09, 7.92s/it]
59%|█████▉ | 1620/2757 [3:42:04<2:30:09, 7.92s/it]
59%|█████▉ | 1621/2757 [3:42:12<2:28:50, 7.86s/it]
59%|█████▉ | 1622/2757 [3:42:20<2:28:14, 7.84s/it]
59%|█████▉ | 1623/2757 [3:42:28<2:29:12, 7.89s/it]
59%|█████▉ | 1624/2757 [3:42:36<2:29:59, 7.94s/it]
59%|█████▉ | 1625/2757 [3:42:44<2:29:35, 7.93s/it]
59%|█████▉ | 1626/2757 [3:42:52<2:32:18, 8.08s/it]
59%|█████▉ | 1627/2757 [3:43:00<2:28:16, 7.87s/it]
59%|█████▉ | 1628/2757 [3:43:07<2:26:55, 7.81s/it]
59%|█████▉ | 1629/2757 [3:43:15<2:27:58, 7.87s/it]
59%|█████▉ | 1630/2757 [3:43:23<2:29:21, 7.95s/it]
59%|█████▉ | 1630/2757 [3:43:23<2:29:21, 7.95s/it]
59%|█████▉ | 1631/2757 [3:43:32<2:32:20, 8.12s/it]
59%|█████▉ | 1632/2757 [3:43:40<2:33:26, 8.18s/it]
59%|█████▉ | 1633/2757 [3:43:48<2:31:39, 8.10s/it]
59%|█████▉ | 1634/2757 [3:43:56<2:29:50, 8.01s/it]
59%|█████▉ | 1635/2757 [3:44:04<2:28:14, 7.93s/it]
59%|█████▉ | 1636/2757 [3:44:12<2:29:27, 8.00s/it]
59%|█████▉ | 1637/2757 [3:44:20<2:27:23, 7.90s/it]
59%|█████▉ | 1638/2757 [3:44:28<2:28:46, 7.98s/it]
59%|█████▉ | 1639/2757 [3:44:36<2:29:47, 8.04s/it]
59%|█████▉ | 1640/2757 [3:44:44<2:29:50, 8.05s/it]
59%|█████▉ | 1640/2757 [3:44:44<2:29:50, 8.05s/it]
60%|█████▉ | 1641/2757 [3:44:52<2:30:20, 8.08s/it]
60%|█████▉ | 1642/2757 [3:45:00<2:30:49, 8.12s/it]
60%|█████▉ | 1643/2757 [3:45:08<2:30:50, 8.12s/it]
60%|█████▉ | 1644/2757 [3:45:16<2:29:15, 8.05s/it]
60%|█████▉ | 1645/2757 [3:45:24<2:29:03, 8.04s/it]
60%|█████▉ | 1646/2757 [3:45:32<2:29:28, 8.07s/it]
60%|█████▉ | 1647/2757 [3:45:40<2:28:48, 8.04s/it]
60%|█████▉ | 1648/2757 [3:45:48<2:26:24, 7.92s/it]
60%|█████▉ | 1649/2757 [3:45:56<2:26:27, 7.93s/it]
60%|█████▉ | 1650/2757 [3:46:04<2:26:23, 7.93s/it]
60%|█████▉ | 1650/2757 [3:46:04<2:26:23, 7.93s/it]
60%|█████▉ | 1651/2757 [3:46:12<2:27:02, 7.98s/it]
60%|█████▉ | 1652/2757 [3:46:20<2:28:14, 8.05s/it]
60%|█████▉ | 1653/2757 [3:46:28<2:27:35, 8.02s/it]
60%|█████▉ | 1654/2757 [3:46:36<2:28:01, 8.05s/it]
60%|██████ | 1655/2757 [3:46:44<2:26:05, 7.95s/it]
60%|██████ | 1656/2757 [3:46:52<2:27:05, 8.02s/it]
60%|██████ | 1657/2757 [3:47:00<2:27:28, 8.04s/it]
60%|██████ | 1658/2757 [3:47:08<2:25:39, 7.95s/it]
60%|██████ | 1659/2757 [3:47:16<2:26:34, 8.01s/it]
60%|██████ | 1660/2757 [3:47:24<2:27:34, 8.07s/it]
60%|██████ | 1660/2757 [3:47:24<2:27:34, 8.07s/it]
60%|██████ | 1661/2757 [3:47:33<2:28:13, 8.11s/it]
60%|██████ | 1662/2757 [3:47:41<2:28:19, 8.13s/it]
60%|██████ | 1663/2757 [3:47:49<2:26:46, 8.05s/it]
60%|██████ | 1664/2757 [3:47:56<2:23:03, 7.85s/it]
60%|██████ | 1665/2757 [3:48:04<2:24:51, 7.96s/it]
60%|██████ | 1666/2757 [3:48:12<2:25:12, 7.99s/it]
60%|██████ | 1667/2757 [3:48:21<2:28:03, 8.15s/it]
61%|██████ | 1668/2757 [3:48:28<2:24:42, 7.97s/it]
61%|██████ | 1669/2757 [3:48:37<2:26:02, 8.05s/it]
61%|██████ | 1670/2757 [3:48:45<2:24:51, 8.00s/it]
61%|██████ | 1670/2757 [3:48:45<2:24:51, 8.00s/it]
61%|██████ | 1671/2757 [3:48:52<2:23:56, 7.95s/it]
61%|██████ | 1672/2757 [3:49:00<2:24:08, 7.97s/it]
61%|██████ | 1673/2757 [3:49:08<2:22:48, 7.90s/it]
61%|██████ | 1674/2757 [3:49:16<2:23:38, 7.96s/it]
61%|██████ | 1675/2757 [3:49:24<2:24:13, 8.00s/it]
61%|██████ | 1676/2757 [3:49:32<2:23:45, 7.98s/it]
61%|██████ | 1677/2757 [3:49:40<2:21:42, 7.87s/it]
61%|██████ | 1678/2757 [3:49:48<2:22:07, 7.90s/it]
61%|██████ | 1679/2757 [3:49:56<2:21:31, 7.88s/it]
61%|██████ | 1680/2757 [3:50:04<2:22:24, 7.93s/it]
61%|██████ | 1680/2757 [3:50:04<2:22:24, 7.93s/it]
61%|██████ | 1681/2757 [3:50:11<2:19:00, 7.75s/it]
61%|██████ | 1682/2757 [3:50:20<2:22:43, 7.97s/it]
61%|██████ | 1683/2757 [3:50:27<2:21:00, 7.88s/it]
61%|██████ | 1684/2757 [3:50:35<2:21:08, 7.89s/it]
61%|██████ | 1685/2757 [3:50:43<2:21:24, 7.91s/it]
61%|██████ | 1686/2757 [3:50:51<2:21:32, 7.93s/it]
61%|██████ | 1687/2757 [3:50:59<2:23:33, 8.05s/it]
61%|██████ | 1688/2757 [3:51:08<2:24:46, 8.13s/it]
61%|██████▏ | 1689/2757 [3:51:16<2:22:52, 8.03s/it]
61%|██████▏ | 1690/2757 [3:51:24<2:23:29, 8.07s/it]
61%|██████▏ | 1690/2757 [3:51:24<2:23:29, 8.07s/it]
61%|██████▏ | 1691/2757 [3:51:32<2:25:51, 8.21s/it]
61%|██████▏ | 1692/2757 [3:51:40<2:21:26, 7.97s/it]
61%|██████▏ | 1693/2757 [3:51:47<2:19:42, 7.88s/it]
61%|██████▏ | 1694/2757 [3:51:55<2:20:31, 7.93s/it]
61%|██████▏ | 1695/2757 [3:52:04<2:22:27, 8.05s/it]
62%|██████▏ | 1696/2757 [3:52:11<2:20:59, 7.97s/it]
62%|██████▏ | 1697/2757 [3:52:19<2:19:02, 7.87s/it]
62%|██████▏ | 1698/2757 [3:52:27<2:20:44, 7.97s/it]
62%|██████▏ | 1699/2757 [3:52:35<2:21:28, 8.02s/it]
62%|██████▏ | 1700/2757 [3:52:43<2:19:49, 7.94s/it]
62%|██████▏ | 1700/2757 [3:52:43<2:19:49, 7.94s/it]
62%|██████▏ | 1701/2757 [3:52:51<2:19:51, 7.95s/it]
62%|██████▏ | 1702/2757 [3:52:59<2:20:10, 7.97s/it]
62%|██████▏ | 1703/2757 [3:53:07<2:19:24, 7.94s/it]
62%|██████▏ | 1704/2757 [3:53:15<2:17:59, 7.86s/it]
62%|██████▏ | 1705/2757 [3:53:23<2:21:14, 8.06s/it]
62%|██████▏ | 1706/2757 [3:53:31<2:21:03, 8.05s/it]
62%|██████▏ | 1707/2757 [3:53:39<2:20:00, 8.00s/it]
62%|██████▏ | 1708/2757 [3:53:47<2:19:10, 7.96s/it]
62%|██████▏ | 1709/2757 [3:53:55<2:18:14, 7.91s/it]
62%|██████▏ | 1710/2757 [3:54:03<2:19:38, 8.00s/it]
62%|██████▏ | 1710/2757 [3:54:03<2:19:38, 8.00s/it]
62%|██████▏ | 1711/2757 [3:54:11<2:20:31, 8.06s/it]
62%|██████▏ | 1712/2757 [3:54:19<2:18:08, 7.93s/it]
62%|██████▏ | 1713/2757 [3:54:27<2:17:38, 7.91s/it]
62%|██████▏ | 1714/2757 [3:54:35<2:18:39, 7.98s/it]
62%|██████▏ | 1715/2757 [3:54:43<2:19:08, 8.01s/it]
62%|██████▏ | 1716/2757 [3:54:51<2:18:06, 7.96s/it]
62%|██████▏ | 1717/2757 [3:54:58<2:16:32, 7.88s/it]
62%|██████▏ | 1718/2757 [3:55:06<2:15:58, 7.85s/it]
62%|██████▏ | 1719/2757 [3:55:14<2:13:35, 7.72s/it]
62%|██████▏ | 1720/2757 [3:55:21<2:12:15, 7.65s/it]
62%|██████▏ | 1720/2757 [3:55:21<2:12:15, 7.65s/it]
62%|██████▏ | 1721/2757 [3:55:29<2:12:56, 7.70s/it]
62%|██████▏ | 1722/2757 [3:55:37<2:15:56, 7.88s/it]
62%|██████▏ | 1723/2757 [3:55:45<2:15:04, 7.84s/it]
63%|██████▎ | 1724/2757 [3:55:53<2:16:23, 7.92s/it]
63%|██████▎ | 1725/2757 [3:56:01<2:18:17, 8.04s/it]
63%|██████▎ | 1726/2757 [3:56:09<2:16:00, 7.92s/it]
63%|██████▎ | 1727/2757 [3:56:17<2:16:01, 7.92s/it]
63%|██████▎ | 1728/2757 [3:56:25<2:14:28, 7.84s/it]
63%|██████▎ | 1729/2757 [3:56:33<2:16:04, 7.94s/it]
63%|██████▎ | 1730/2757 [3:56:41<2:15:57, 7.94s/it]
63%|██████▎ | 1730/2757 [3:56:41<2:15:57, 7.94s/it]
63%|██████▎ | 1731/2757 [3:56:49<2:15:07, 7.90s/it]
63%|██████▎ | 1732/2757 [3:56:57<2:15:15, 7.92s/it]
63%|██████▎ | 1733/2757 [3:57:04<2:14:42, 7.89s/it]
63%|██████▎ | 1734/2757 [3:57:12<2:13:17, 7.82s/it]
63%|██████▎ | 1735/2757 [3:57:20<2:15:50, 7.98s/it]
63%|██████▎ | 1736/2757 [3:57:29<2:16:56, 8.05s/it]
63%|██████▎ | 1737/2757 [3:57:37<2:16:39, 8.04s/it]
63%|██████▎ | 1738/2757 [3:57:45<2:17:39, 8.11s/it]
63%|██████▎ | 1739/2757 [3:57:53<2:16:36, 8.05s/it]
63%|██████▎ | 1740/2757 [3:58:01<2:16:50, 8.07s/it]
63%|██████▎ | 1740/2757 [3:58:01<2:16:50, 8.07s/it]
63%|██████▎ | 1741/2757 [3:58:09<2:15:32, 8.00s/it]
63%|██████▎ | 1742/2757 [3:58:17<2:15:19, 8.00s/it]
63%|██████▎ | 1743/2757 [3:58:25<2:17:43, 8.15s/it]
63%|██████▎ | 1744/2757 [3:58:33<2:17:05, 8.12s/it]
63%|██████▎ | 1745/2757 [3:58:42<2:17:22, 8.14s/it]
63%|██████▎ | 1746/2757 [3:58:49<2:16:07, 8.08s/it]
63%|██████▎ | 1747/2757 [3:58:58<2:16:30, 8.11s/it]
63%|██████▎ | 1748/2757 [3:59:06<2:15:49, 8.08s/it]
63%|██████▎ | 1749/2757 [3:59:14<2:16:40, 8.14s/it]
63%|██████▎ | 1750/2757 [3:59:23<2:20:57, 8.40s/it]
{'eval_loss': '0.4785', 'eval_runtime': '106', 'eval_samples_per_second': '22.64', 'eval_steps_per_second': '2.83', 'eval_rewards/chosen': '-1.847', 'eval_rewards/rejected': '-2.994', 'eval_rewards/accuracies': '0.7079', 'eval_rewards/margins': '1.147', 'eval_logps/chosen': '-98.07', 'eval_logps/rejected': '-115.8', 'eval_logits/chosen': '-4.041', 'eval_logits/rejected': '-3.988', 'epoch': '1.632'}
{'loss': '0.3725', 'grad_norm': '0.6329', 'learning_rate': '9.334e-05', 'rewards/chosen': '-1.795', 'rewards/rejected': '-3.295', 'rewards/accuracies': '0.775', 'rewards/margins': '1.5', 'logps/chosen': '-97.45', 'logps/rejected': '-118.5', 'logits/chosen': '-4.012', 'logits/rejected': '-3.945', 'epoch': '1.643'}
{'loss': '0.3885', 'grad_norm': '0.8267', 'learning_rate': '9.26e-05', 'rewards/chosen': '-2.057', 'rewards/rejected': '-3.51', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.453', 'logps/chosen': '-99.85', 'logps/rejected': '-118.9', 'logits/chosen': '-4.003', 'logits/rejected': '-3.953', 'epoch': '1.654'}
{'loss': '0.3619', 'grad_norm': '0.6179', 'learning_rate': '9.185e-05', 'rewards/chosen': '-1.993', 'rewards/rejected': '-3.517', 'rewards/accuracies': '0.7969', 'rewards/margins': '1.524', 'logps/chosen': '-100.3', 'logps/rejected': '-121', 'logits/chosen': '-3.809', 'logits/rejected': '-3.735', 'epoch': '1.665'}
{'loss': '0.3576', 'grad_norm': '0.6443', 'learning_rate': '9.11e-05', 'rewards/chosen': '-2.11', 'rewards/rejected': '-3.644', 'rewards/accuracies': '0.7937', 'rewards/margins': '1.534', 'logps/chosen': '-103.8', 'logps/rejected': '-123.3', 'logits/chosen': '-3.819', 'logits/rejected': '-3.749', 'epoch': '1.676'}
{'loss': '0.374', 'grad_norm': '0.7384', 'learning_rate': '9.035e-05', 'rewards/chosen': '-2.21', 'rewards/rejected': '-3.729', 'rewards/accuracies': '0.768', 'rewards/margins': '1.519', 'logps/chosen': '-103.8', 'logps/rejected': '-122.9', 'logits/chosen': '-3.885', 'logits/rejected': '-3.787', 'epoch': '1.687'}
{'loss': '0.3584', 'grad_norm': '0.5654', 'learning_rate': '8.96e-05', 'rewards/chosen': '-2.004', 'rewards/rejected': '-3.579', 'rewards/accuracies': '0.7852', 'rewards/margins': '1.574', 'logps/chosen': '-100.5', 'logps/rejected': '-120.8', 'logits/chosen': '-3.935', 'logits/rejected': '-3.844', 'epoch': '1.698'}
{'loss': '0.3536', 'grad_norm': '0.8251', 'learning_rate': '8.886e-05', 'rewards/chosen': '-2.175', 'rewards/rejected': '-3.769', 'rewards/accuracies': '0.8062', 'rewards/margins': '1.595', 'logps/chosen': '-102.4', 'logps/rejected': '-124.2', 'logits/chosen': '-4.114', 'logits/rejected': '-4.016', 'epoch': '1.709'}
{'loss': '0.3635', 'grad_norm': '0.7069', 'learning_rate': '8.811e-05', 'rewards/chosen': '-2.133', 'rewards/rejected': '-3.733', 'rewards/accuracies': '0.7836', 'rewards/margins': '1.6', 'logps/chosen': '-100.6', 'logps/rejected': '-121.4', 'logits/chosen': '-4.129', 'logits/rejected': '-4.013', 'epoch': '1.719'}
{'loss': '0.3753', 'grad_norm': '0.8969', 'learning_rate': '8.736e-05', 'rewards/chosen': '-2.151', 'rewards/rejected': '-3.742', 'rewards/accuracies': '0.7781', 'rewards/margins': '1.591', 'logps/chosen': '-101.7', 'logps/rejected': '-124.9', 'logits/chosen': '-4.027', 'logits/rejected': '-4.019', 'epoch': '1.73'}
{'loss': '0.3766', 'grad_norm': '0.7352', 'learning_rate': '8.661e-05', 'rewards/chosen': '-2.379', 'rewards/rejected': '-3.885', 'rewards/accuracies': '0.7742', 'rewards/margins': '1.506', 'logps/chosen': '-103.6', 'logps/rejected': '-124.1', 'logits/chosen': '-3.981', 'logits/rejected': '-3.917', 'epoch': '1.741'}
{'loss': '0.3653', 'grad_norm': '0.5647', 'learning_rate': '8.586e-05', 'rewards/chosen': '-2.353', 'rewards/rejected': '-3.948', 'rewards/accuracies': '0.7844', 'rewards/margins': '1.595', 'logps/chosen': '-103.6', 'logps/rejected': '-125.9', 'logits/chosen': '-4.083', 'logits/rejected': '-4.028', 'epoch': '1.752'}
{'loss': '0.3691', 'grad_norm': '0.7188', 'learning_rate': '8.512e-05', 'rewards/chosen': '-2.336', 'rewards/rejected': '-3.85', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.515', 'logps/chosen': '-104.9', 'logps/rejected': '-125', 'logits/chosen': '-3.872', 'logits/rejected': '-3.833', 'epoch': '1.763'}
{'loss': '0.3572', 'grad_norm': '0.6679', 'learning_rate': '8.437e-05', 'rewards/chosen': '-2.258', 'rewards/rejected': '-3.829', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.571', 'logps/chosen': '-103.7', 'logps/rejected': '-125', 'logits/chosen': '-4.03', 'logits/rejected': '-3.973', 'epoch': '1.774'}
{'loss': '0.3459', 'grad_norm': '0.6013', 'learning_rate': '8.362e-05', 'rewards/chosen': '-2.236', 'rewards/rejected': '-3.897', 'rewards/accuracies': '0.8023', 'rewards/margins': '1.661', 'logps/chosen': '-100.5', 'logps/rejected': '-123.6', 'logits/chosen': '-3.995', 'logits/rejected': '-3.911', 'epoch': '1.785'}
{'loss': '0.3556', 'grad_norm': '0.6696', 'learning_rate': '8.287e-05', 'rewards/chosen': '-2.186', 'rewards/rejected': '-3.82', 'rewards/accuracies': '0.793', 'rewards/margins': '1.634', 'logps/chosen': '-100.8', 'logps/rejected': '-123.3', 'logits/chosen': '-3.975', 'logits/rejected': '-3.9', 'epoch': '1.796'}
{'loss': '0.3668', 'grad_norm': '0.5696', 'learning_rate': '8.212e-05', 'rewards/chosen': '-2.07', 'rewards/rejected': '-3.639', 'rewards/accuracies': '0.7891', 'rewards/margins': '1.569', 'logps/chosen': '-99.68', 'logps/rejected': '-120.6', 'logits/chosen': '-4.077', 'logits/rejected': '-3.98', 'epoch': '1.807'}
{'loss': '0.377', 'grad_norm': '0.6719', 'learning_rate': '8.138e-05', 'rewards/chosen': '-2.11', 'rewards/rejected': '-3.654', 'rewards/accuracies': '0.7898', 'rewards/margins': '1.544', 'logps/chosen': '-103.2', 'logps/rejected': '-124.1', 'logits/chosen': '-4.046', 'logits/rejected': '-3.934', 'epoch': '1.817'}
{'loss': '0.3561', 'grad_norm': '0.6737', 'learning_rate': '8.063e-05', 'rewards/chosen': '-2.13', 'rewards/rejected': '-3.775', 'rewards/accuracies': '0.7891', 'rewards/margins': '1.645', 'logps/chosen': '-99.72', 'logps/rejected': '-121.9', 'logits/chosen': '-4.077', 'logits/rejected': '-4.007', 'epoch': '1.828'}
{'loss': '0.3536', 'grad_norm': '0.7005', 'learning_rate': '7.988e-05', 'rewards/chosen': '-2.293', 'rewards/rejected': '-3.948', 'rewards/accuracies': '0.8031', 'rewards/margins': '1.655', 'logps/chosen': '-105.8', 'logps/rejected': '-128', 'logits/chosen': '-3.916', 'logits/rejected': '-3.911', 'epoch': '1.839'}
{'loss': '0.3671', 'grad_norm': '0.5833', 'learning_rate': '7.913e-05', 'rewards/chosen': '-2.238', 'rewards/rejected': '-3.842', 'rewards/accuracies': '0.7609', 'rewards/margins': '1.604', 'logps/chosen': '-98.68', 'logps/rejected': '-118.4', 'logits/chosen': '-4.208', 'logits/rejected': '-4.131', 'epoch': '1.85'}
{'loss': '0.3505', 'grad_norm': '0.5559', 'learning_rate': '7.838e-05', 'rewards/chosen': '-2.235', 'rewards/rejected': '-3.849', 'rewards/accuracies': '0.8094', 'rewards/margins': '1.614', 'logps/chosen': '-99.44', 'logps/rejected': '-122.8', 'logits/chosen': '-4.181', 'logits/rejected': '-4.108', 'epoch': '1.861'}
{'loss': '0.3608', 'grad_norm': '0.5495', 'learning_rate': '7.764e-05', 'rewards/chosen': '-2.364', 'rewards/rejected': '-3.972', 'rewards/accuracies': '0.7828', 'rewards/margins': '1.608', 'logps/chosen': '-104.2', 'logps/rejected': '-125.3', 'logits/chosen': '-4.158', 'logits/rejected': '-4.096', 'epoch': '1.872'}
{'loss': '0.355', 'grad_norm': '0.7562', 'learning_rate': '7.689e-05', 'rewards/chosen': '-2.264', 'rewards/rejected': '-3.919', 'rewards/accuracies': '0.7906', 'rewards/margins': '1.655', 'logps/chosen': '-102.7', 'logps/rejected': '-123.5', 'logits/chosen': '-4.054', 'logits/rejected': '-3.934', 'epoch': '1.883'}
{'loss': '0.3495', 'grad_norm': '0.6611', 'learning_rate': '7.614e-05', 'rewards/chosen': '-2.171', 'rewards/rejected': '-3.777', 'rewards/accuracies': '0.7953', 'rewards/margins': '1.607', 'logps/chosen': '-104.4', 'logps/rejected': '-125.9', 'logits/chosen': '-3.961', 'logits/rejected': '-3.83', 'epoch': '1.894'}
{'loss': '0.3612', 'grad_norm': '0.6473', 'learning_rate': '7.539e-05', 'rewards/chosen': '-2.068', 'rewards/rejected': '-3.683', 'rewards/accuracies': '0.7836', 'rewards/margins': '1.615', 'logps/chosen': '-106', 'logps/rejected': '-125.6', 'logits/chosen': '-3.913', 'logits/rejected': '-3.869', 'epoch': '1.904'}
63%|██████▎ | 1750/2757 [3:59:23<2:20:57, 8.40s/it]
64%|██████▎ | 1751/2757 [3:59:37<2:48:24, 10.04s/it]
64%|██████▎ | 1752/2757 [3:59:45<2:37:55, 9.43s/it]
64%|██████▎ | 1753/2757 [3:59:53<2:33:59, 9.20s/it]
64%|██████▎ | 1754/2757 [4:00:02<2:28:18, 8.87s/it]
64%|██████▎ | 1755/2757 [4:00:09<2:21:56, 8.50s/it]
64%|██████▎ | 1756/2757 [4:00:17<2:18:21, 8.29s/it]
64%|██████▎ | 1757/2757 [4:00:26<2:20:33, 8.43s/it]
64%|██████▍ | 1758/2757 [4:00:34<2:17:39, 8.27s/it]
64%|██████▍ | 1759/2757 [4:00:42<2:17:16, 8.25s/it]
64%|██████▍ | 1760/2757 [4:00:50<2:17:25, 8.27s/it]
64%|██████▍ | 1760/2757 [4:00:50<2:17:25, 8.27s/it]
64%|██████▍ | 1761/2757 [4:00:58<2:16:03, 8.20s/it]
64%|██████▍ | 1762/2757 [4:01:06<2:13:52, 8.07s/it]
64%|██████▍ | 1763/2757 [4:01:14<2:12:44, 8.01s/it]
64%|██████▍ | 1764/2757 [4:01:21<2:10:42, 7.90s/it]
64%|██████▍ | 1765/2757 [4:01:29<2:09:45, 7.85s/it]
64%|██████▍ | 1766/2757 [4:01:37<2:10:22, 7.89s/it]
64%|██████▍ | 1767/2757 [4:01:45<2:11:38, 7.98s/it]
64%|██████▍ | 1768/2757 [4:01:54<2:12:42, 8.05s/it]
64%|██████▍ | 1769/2757 [4:02:01<2:11:29, 7.99s/it]
64%|██████▍ | 1770/2757 [4:02:10<2:11:45, 8.01s/it]
64%|██████▍ | 1770/2757 [4:02:10<2:11:45, 8.01s/it]
64%|██████▍ | 1771/2757 [4:02:17<2:11:17, 7.99s/it]
64%|██████▍ | 1772/2757 [4:02:25<2:11:17, 8.00s/it]
64%|██████▍ | 1773/2757 [4:02:33<2:10:10, 7.94s/it]
64%|██████▍ | 1774/2757 [4:02:41<2:09:21, 7.90s/it]
64%|██████▍ | 1775/2757 [4:02:49<2:10:24, 7.97s/it]
64%|██████▍ | 1776/2757 [4:02:57<2:08:37, 7.87s/it]
64%|██████▍ | 1777/2757 [4:03:05<2:08:46, 7.88s/it]
64%|██████▍ | 1778/2757 [4:03:13<2:08:29, 7.88s/it]
65%|██████▍ | 1779/2757 [4:03:21<2:08:30, 7.88s/it]
65%|██████▍ | 1780/2757 [4:03:29<2:08:59, 7.92s/it]
65%|██████▍ | 1780/2757 [4:03:29<2:08:59, 7.92s/it]
65%|██████▍ | 1781/2757 [4:03:37<2:09:10, 7.94s/it]
65%|██████▍ | 1782/2757 [4:03:44<2:07:23, 7.84s/it]
65%|██████▍ | 1783/2757 [4:03:52<2:07:50, 7.88s/it]
65%|██████▍ | 1784/2757 [4:04:00<2:07:14, 7.85s/it]
65%|██████▍ | 1785/2757 [4:04:08<2:08:56, 7.96s/it]
65%|██████▍ | 1786/2757 [4:04:16<2:11:05, 8.10s/it]
65%|██████▍ | 1787/2757 [4:04:25<2:10:45, 8.09s/it]
65%|██████▍ | 1788/2757 [4:04:33<2:10:11, 8.06s/it]
65%|██████▍ | 1789/2757 [4:04:40<2:08:11, 7.95s/it]
65%|██████▍ | 1790/2757 [4:04:48<2:05:59, 7.82s/it]
65%|██████▍ | 1790/2757 [4:04:48<2:05:59, 7.82s/it]
65%|██████▍ | 1791/2757 [4:04:56<2:07:23, 7.91s/it]
65%|██████▍ | 1792/2757 [4:05:04<2:09:32, 8.05s/it]
65%|██████▌ | 1793/2757 [4:05:13<2:11:49, 8.21s/it]
65%|██████▌ | 1794/2757 [4:05:21<2:11:10, 8.17s/it]
65%|██████▌ | 1795/2757 [4:05:29<2:09:08, 8.05s/it]
65%|██████▌ | 1796/2757 [4:05:36<2:06:32, 7.90s/it]
65%|██████▌ | 1797/2757 [4:05:44<2:07:22, 7.96s/it]
65%|██████▌ | 1798/2757 [4:05:52<2:07:04, 7.95s/it]
65%|██████▌ | 1799/2757 [4:06:00<2:05:05, 7.83s/it]
65%|██████▌ | 1800/2757 [4:06:08<2:04:31, 7.81s/it]
65%|██████▌ | 1800/2757 [4:06:08<2:04:31, 7.81s/it]
65%|██████▌ | 1801/2757 [4:06:15<2:04:46, 7.83s/it]
65%|██████▌ | 1802/2757 [4:06:24<2:06:22, 7.94s/it]
65%|██████▌ | 1803/2757 [4:06:32<2:07:22, 8.01s/it]
65%|██████▌ | 1804/2757 [4:06:40<2:06:41, 7.98s/it]
65%|██████▌ | 1805/2757 [4:06:47<2:04:06, 7.82s/it]
66%|██████▌ | 1806/2757 [4:06:55<2:06:11, 7.96s/it]
66%|██████▌ | 1807/2757 [4:07:04<2:07:15, 8.04s/it]
66%|██████▌ | 1808/2757 [4:07:12<2:07:48, 8.08s/it]
66%|██████▌ | 1809/2757 [4:07:20<2:07:06, 8.04s/it]
66%|██████▌ | 1810/2757 [4:07:28<2:07:38, 8.09s/it]
66%|██████▌ | 1810/2757 [4:07:28<2:07:38, 8.09s/it]
66%|██████▌ | 1811/2757 [4:07:36<2:06:59, 8.05s/it]
66%|██████▌ | 1812/2757 [4:07:44<2:05:10, 7.95s/it]
66%|██████▌ | 1813/2757 [4:07:52<2:04:27, 7.91s/it]
66%|██████▌ | 1814/2757 [4:07:59<2:03:28, 7.86s/it]
66%|██████▌ | 1815/2757 [4:08:07<2:03:01, 7.84s/it]
66%|██████▌ | 1816/2757 [4:08:15<2:03:38, 7.88s/it]
66%|██████▌ | 1817/2757 [4:08:23<2:02:31, 7.82s/it]
66%|██████▌ | 1818/2757 [4:08:31<2:04:24, 7.95s/it]
66%|██████▌ | 1819/2757 [4:08:39<2:06:11, 8.07s/it]
66%|██████▌ | 1820/2757 [4:08:47<2:05:42, 8.05s/it]
66%|██████▌ | 1820/2757 [4:08:47<2:05:42, 8.05s/it]
66%|██████▌ | 1821/2757 [4:08:55<2:05:30, 8.05s/it]
66%|██████▌ | 1822/2757 [4:09:03<2:04:18, 7.98s/it]
66%|██████▌ | 1823/2757 [4:09:11<2:02:28, 7.87s/it]
66%|██████▌ | 1824/2757 [4:09:19<2:04:37, 8.01s/it]
66%|██████▌ | 1825/2757 [4:09:27<2:04:53, 8.04s/it]
66%|██████▌ | 1826/2757 [4:09:35<2:05:38, 8.10s/it]
66%|██████▋ | 1827/2757 [4:09:43<2:03:48, 7.99s/it]
66%|██████▋ | 1828/2757 [4:09:51<2:03:12, 7.96s/it]
66%|██████▋ | 1829/2757 [4:09:59<2:02:39, 7.93s/it]
66%|██████▋ | 1830/2757 [4:10:07<2:03:18, 7.98s/it]
66%|██████▋ | 1830/2757 [4:10:07<2:03:18, 7.98s/it]
66%|██████▋ | 1831/2757 [4:10:16<2:06:03, 8.17s/it]
66%|██████▋ | 1832/2757 [4:10:24<2:05:08, 8.12s/it]
66%|██████▋ | 1833/2757 [4:10:32<2:04:45, 8.10s/it]
67%|██████▋ | 1834/2757 [4:10:40<2:05:18, 8.15s/it]
67%|██████▋ | 1835/2757 [4:10:48<2:04:47, 8.12s/it]
67%|██████▋ | 1836/2757 [4:10:56<2:04:35, 8.12s/it]
67%|██████▋ | 1837/2757 [4:11:04<2:05:03, 8.16s/it]
67%|██████▋ | 1838/2757 [4:11:10<1:55:31, 7.54s/it]
{'loss': '0.3603', 'grad_norm': '0.7833', 'learning_rate': '7.464e-05', 'rewards/chosen': '-1.853', 'rewards/rejected': '-3.423', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.57', 'logps/chosen': '-97.6', 'logps/rejected': '-119.1', 'logits/chosen': '-3.939', 'logits/rejected': '-3.906', 'epoch': '1.915'}
{'loss': '0.3666', 'grad_norm': '0.72', 'learning_rate': '7.39e-05', 'rewards/chosen': '-2.322', 'rewards/rejected': '-3.87', 'rewards/accuracies': '0.7812', 'rewards/margins': '1.548', 'logps/chosen': '-104.4', 'logps/rejected': '-122.6', 'logits/chosen': '-3.854', 'logits/rejected': '-3.82', 'epoch': '1.926'}
{'loss': '0.3637', 'grad_norm': '0.6587', 'learning_rate': '7.315e-05', 'rewards/chosen': '-2.446', 'rewards/rejected': '-4.051', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.605', 'logps/chosen': '-106', 'logps/rejected': '-127.8', 'logits/chosen': '-3.952', 'logits/rejected': '-4.003', 'epoch': '1.937'}
{'loss': '0.3613', 'grad_norm': '0.5735', 'learning_rate': '7.24e-05', 'rewards/chosen': '-2.576', 'rewards/rejected': '-4.142', 'rewards/accuracies': '0.7812', 'rewards/margins': '1.566', 'logps/chosen': '-100.7', 'logps/rejected': '-121.9', 'logits/chosen': '-4', 'logits/rejected': '-3.923', 'epoch': '1.948'}
{'loss': '0.3469', 'grad_norm': '0.6537', 'learning_rate': '7.165e-05', 'rewards/chosen': '-2.413', 'rewards/rejected': '-4.107', 'rewards/accuracies': '0.8195', 'rewards/margins': '1.694', 'logps/chosen': '-103.4', 'logps/rejected': '-125.7', 'logits/chosen': '-3.983', 'logits/rejected': '-3.873', 'epoch': '1.959'}
{'loss': '0.3626', 'grad_norm': '0.6989', 'learning_rate': '7.091e-05', 'rewards/chosen': '-2.356', 'rewards/rejected': '-3.993', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.637', 'logps/chosen': '-106.2', 'logps/rejected': '-126.7', 'logits/chosen': '-3.916', 'logits/rejected': '-3.87', 'epoch': '1.97'}
{'loss': '0.3597', 'grad_norm': '0.7157', 'learning_rate': '7.016e-05', 'rewards/chosen': '-2.092', 'rewards/rejected': '-3.722', 'rewards/accuracies': '0.7781', 'rewards/margins': '1.63', 'logps/chosen': '-100.3', 'logps/rejected': '-121.5', 'logits/chosen': '-3.931', 'logits/rejected': '-3.873', 'epoch': '1.981'}
{'loss': '0.3599', 'grad_norm': '0.6526', 'learning_rate': '6.941e-05', 'rewards/chosen': '-1.991', 'rewards/rejected': '-3.627', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.635', 'logps/chosen': '-100', 'logps/rejected': '-122', 'logits/chosen': '-3.948', 'logits/rejected': '-3.869', 'epoch': '1.992'}
67%|██████▋ | 1839/2757 [4:11:19<2:00:17, 7.86s/it]
67%|██████▋ | 1840/2757 [4:11:27<2:00:20, 7.87s/it]
67%|██████▋ | 1840/2757 [4:11:27<2:00:20, 7.87s/it]
67%|██████▋ | 1841/2757 [4:11:35<2:00:20, 7.88s/it]
67%|██████▋ | 1842/2757 [4:11:43<2:02:53, 8.06s/it]
67%|██████▋ | 1843/2757 [4:11:51<2:03:00, 8.07s/it]
67%|██████▋ | 1844/2757 [4:11:59<2:02:30, 8.05s/it]
67%|██████▋ | 1845/2757 [4:12:08<2:03:20, 8.11s/it]
67%|██████▋ | 1846/2757 [4:12:16<2:04:00, 8.17s/it]
67%|██████▋ | 1847/2757 [4:12:24<2:03:45, 8.16s/it]
67%|██████▋ | 1848/2757 [4:12:32<2:03:20, 8.14s/it]
67%|██████▋ | 1849/2757 [4:12:40<2:02:34, 8.10s/it]
67%|██████▋ | 1850/2757 [4:12:49<2:03:26, 8.17s/it]
67%|██████▋ | 1850/2757 [4:12:49<2:03:26, 8.17s/it]
67%|██████▋ | 1851/2757 [4:12:57<2:02:05, 8.09s/it]
67%|██████▋ | 1852/2757 [4:13:05<2:02:39, 8.13s/it]
67%|██████▋ | 1853/2757 [4:13:13<2:01:21, 8.05s/it]
67%|██████▋ | 1854/2757 [4:13:20<1:59:41, 7.95s/it]
67%|██████▋ | 1855/2757 [4:13:28<1:59:20, 7.94s/it]
67%|██████▋ | 1856/2757 [4:13:36<1:58:53, 7.92s/it]
67%|██████▋ | 1857/2757 [4:13:44<1:59:03, 7.94s/it]
67%|██████▋ | 1858/2757 [4:13:53<2:01:11, 8.09s/it]
67%|██████▋ | 1859/2757 [4:14:01<2:02:45, 8.20s/it]
67%|██████▋ | 1860/2757 [4:14:09<2:01:36, 8.13s/it]
67%|██████▋ | 1860/2757 [4:14:09<2:01:36, 8.13s/it]
68%|██████▊ | 1861/2757 [4:14:17<1:59:31, 8.00s/it]
68%|██████▊ | 1862/2757 [4:14:25<2:01:12, 8.13s/it]
68%|██████▊ | 1863/2757 [4:14:33<2:00:43, 8.10s/it]
68%|██████▊ | 1864/2757 [4:14:41<1:59:42, 8.04s/it]
68%|██████▊ | 1865/2757 [4:14:49<1:57:59, 7.94s/it]
68%|██████▊ | 1866/2757 [4:14:57<1:57:20, 7.90s/it]
68%|██████▊ | 1867/2757 [4:15:04<1:55:58, 7.82s/it]
68%|██████▊ | 1868/2757 [4:15:12<1:55:11, 7.77s/it]
68%|██████▊ | 1869/2757 [4:15:20<1:55:19, 7.79s/it]
68%|██████▊ | 1870/2757 [4:15:28<1:57:58, 7.98s/it]
68%|██████▊ | 1870/2757 [4:15:28<1:57:58, 7.98s/it]
68%|██████▊ | 1871/2757 [4:15:36<1:58:00, 7.99s/it]
68%|██████▊ | 1872/2757 [4:15:44<1:58:45, 8.05s/it]
68%|██████▊ | 1873/2757 [4:15:52<1:57:26, 7.97s/it]
68%|██████▊ | 1874/2757 [4:16:00<1:56:33, 7.92s/it]
68%|██████▊ | 1875/2757 [4:16:08<1:57:30, 7.99s/it]
68%|██████▊ | 1876/2757 [4:16:16<1:56:10, 7.91s/it]
68%|██████▊ | 1877/2757 [4:16:24<1:55:26, 7.87s/it]
68%|██████▊ | 1878/2757 [4:16:32<1:57:22, 8.01s/it]
68%|██████▊ | 1879/2757 [4:16:40<1:57:48, 8.05s/it]
68%|██████▊ | 1880/2757 [4:16:48<1:58:01, 8.07s/it]
68%|██████▊ | 1880/2757 [4:16:48<1:58:01, 8.07s/it]
68%|██████▊ | 1881/2757 [4:16:56<1:57:07, 8.02s/it]
68%|██████▊ | 1882/2757 [4:17:05<1:59:05, 8.17s/it]
68%|██████▊ | 1883/2757 [4:17:13<1:58:48, 8.16s/it]
68%|██████▊ | 1884/2757 [4:17:21<1:57:17, 8.06s/it]
68%|██████▊ | 1885/2757 [4:17:28<1:56:14, 8.00s/it]
68%|██████▊ | 1886/2757 [4:17:36<1:54:23, 7.88s/it]
68%|██████▊ | 1887/2757 [4:17:44<1:52:45, 7.78s/it]
68%|██████▊ | 1888/2757 [4:17:52<1:55:10, 7.95s/it]
69%|██████▊ | 1889/2757 [4:18:00<1:55:26, 7.98s/it]
69%|██████▊ | 1890/2757 [4:18:08<1:54:32, 7.93s/it]
69%|██████▊ | 1890/2757 [4:18:08<1:54:32, 7.93s/it]
69%|██████▊ | 1891/2757 [4:18:16<1:56:40, 8.08s/it]
69%|██████▊ | 1892/2757 [4:18:24<1:55:58, 8.04s/it]
69%|██████▊ | 1893/2757 [4:18:33<1:58:03, 8.20s/it]
69%|██████▊ | 1894/2757 [4:18:41<1:57:52, 8.19s/it]
69%|██████▊ | 1895/2757 [4:18:49<1:56:48, 8.13s/it]
69%|██████▉ | 1896/2757 [4:18:57<1:55:24, 8.04s/it]
69%|██████▉ | 1897/2757 [4:19:05<1:55:36, 8.07s/it]
69%|██████▉ | 1898/2757 [4:19:13<1:54:16, 7.98s/it]
69%|██████▉ | 1899/2757 [4:19:20<1:52:46, 7.89s/it]
69%|██████▉ | 1900/2757 [4:19:28<1:52:51, 7.90s/it]
69%|██████▉ | 1900/2757 [4:19:28<1:52:51, 7.90s/it]
69%|██████▉ | 1901/2757 [4:19:36<1:53:16, 7.94s/it]
69%|██████▉ | 1902/2757 [4:19:44<1:53:22, 7.96s/it]
69%|██████▉ | 1903/2757 [4:19:53<1:55:36, 8.12s/it]
69%|██████▉ | 1904/2757 [4:20:01<1:56:44, 8.21s/it]
69%|██████▉ | 1905/2757 [4:20:09<1:53:19, 7.98s/it]
69%|██████▉ | 1906/2757 [4:20:16<1:52:27, 7.93s/it]
69%|██████▉ | 1907/2757 [4:20:24<1:52:39, 7.95s/it]
69%|██████▉ | 1908/2757 [4:20:32<1:50:43, 7.82s/it]
69%|██████▉ | 1909/2757 [4:20:40<1:53:00, 8.00s/it]
69%|██████▉ | 1910/2757 [4:20:48<1:50:36, 7.83s/it]
69%|██████▉ | 1910/2757 [4:20:48<1:50:36, 7.83s/it]
69%|██████▉ | 1911/2757 [4:20:56<1:50:34, 7.84s/it]
69%|██████▉ | 1912/2757 [4:21:04<1:53:38, 8.07s/it]
69%|██████▉ | 1913/2757 [4:21:12<1:52:51, 8.02s/it]
69%|██████▉ | 1914/2757 [4:21:20<1:51:46, 7.96s/it]
69%|██████▉ | 1915/2757 [4:21:28<1:52:38, 8.03s/it]
69%|██████▉ | 1916/2757 [4:21:36<1:53:39, 8.11s/it]
70%|██████▉ | 1917/2757 [4:21:44<1:51:56, 8.00s/it]
70%|██████▉ | 1918/2757 [4:21:52<1:52:33, 8.05s/it]
70%|██████▉ | 1919/2757 [4:22:00<1:50:30, 7.91s/it]
70%|██████▉ | 1920/2757 [4:22:08<1:49:20, 7.84s/it]
70%|██████▉ | 1920/2757 [4:22:08<1:49:20, 7.84s/it]
70%|██████▉ | 1921/2757 [4:22:16<1:49:55, 7.89s/it]
70%|██████▉ | 1922/2757 [4:22:24<1:50:57, 7.97s/it]
70%|██████▉ | 1923/2757 [4:22:32<1:52:03, 8.06s/it]
70%|██████▉ | 1924/2757 [4:22:40<1:51:21, 8.02s/it]
70%|██████▉ | 1925/2757 [4:22:48<1:51:08, 8.01s/it]
70%|██████▉ | 1926/2757 [4:22:56<1:50:32, 7.98s/it]
70%|██████▉ | 1927/2757 [4:23:03<1:48:46, 7.86s/it]
70%|██████▉ | 1928/2757 [4:23:12<1:49:20, 7.91s/it]
70%|██████▉ | 1929/2757 [4:23:19<1:49:17, 7.92s/it]
70%|███████ | 1930/2757 [4:23:27<1:48:58, 7.91s/it]
70%|███████ | 1930/2757 [4:23:27<1:48:58, 7.91s/it]
70%|███████ | 1931/2757 [4:23:35<1:48:09, 7.86s/it]
70%|███████ | 1932/2757 [4:23:43<1:49:22, 7.95s/it]
70%|███████ | 1933/2757 [4:23:51<1:48:10, 7.88s/it]
70%|███████ | 1934/2757 [4:23:59<1:47:18, 7.82s/it]
70%|███████ | 1935/2757 [4:24:06<1:46:53, 7.80s/it]
70%|███████ | 1936/2757 [4:24:15<1:48:23, 7.92s/it]
70%|███████ | 1937/2757 [4:24:23<1:51:09, 8.13s/it]
70%|███████ | 1938/2757 [4:24:31<1:50:27, 8.09s/it]
70%|███████ | 1939/2757 [4:24:39<1:49:31, 8.03s/it]
70%|███████ | 1940/2757 [4:24:47<1:49:07, 8.01s/it]
70%|███████ | 1940/2757 [4:24:47<1:49:07, 8.01s/it]
70%|███████ | 1941/2757 [4:24:56<1:50:50, 8.15s/it]
70%|███████ | 1942/2757 [4:25:03<1:49:34, 8.07s/it]
70%|███████ | 1943/2757 [4:25:11<1:48:53, 8.03s/it]
71%|███████ | 1944/2757 [4:25:19<1:47:29, 7.93s/it]
71%|███████ | 1945/2757 [4:25:27<1:48:19, 8.00s/it]
71%|███████ | 1946/2757 [4:25:35<1:48:14, 8.01s/it]
71%|███████ | 1947/2757 [4:25:43<1:48:05, 8.01s/it]
71%|███████ | 1948/2757 [4:25:51<1:46:40, 7.91s/it]
71%|███████ | 1949/2757 [4:25:59<1:46:48, 7.93s/it]
71%|███████ | 1950/2757 [4:26:06<1:44:57, 7.80s/it]
71%|███████ | 1950/2757 [4:26:06<1:44:57, 7.80s/it]
71%|███████ | 1951/2757 [4:26:15<1:48:40, 8.09s/it]
71%|███████ | 1952/2757 [4:26:23<1:47:47, 8.03s/it]
71%|███████ | 1953/2757 [4:26:31<1:46:20, 7.94s/it]
71%|███████ | 1954/2757 [4:26:39<1:46:59, 7.99s/it]
71%|███████ | 1955/2757 [4:26:47<1:45:09, 7.87s/it]
71%|███████ | 1956/2757 [4:26:55<1:46:05, 7.95s/it]
71%|███████ | 1957/2757 [4:27:02<1:44:34, 7.84s/it]
71%|███████ | 1958/2757 [4:27:09<1:41:23, 7.61s/it]
71%|███████ | 1959/2757 [4:27:18<1:44:21, 7.85s/it]
71%|███████ | 1960/2757 [4:27:26<1:46:44, 8.04s/it]
71%|███████ | 1960/2757 [4:27:26<1:46:44, 8.04s/it]
71%|███████ | 1961/2757 [4:27:34<1:46:18, 8.01s/it]
71%|███████ | 1962/2757 [4:27:42<1:46:09, 8.01s/it]
71%|███████ | 1963/2757 [4:27:50<1:45:45, 7.99s/it]
71%|███████ | 1964/2757 [4:27:58<1:45:40, 8.00s/it]
71%|███████▏ | 1965/2757 [4:28:06<1:44:06, 7.89s/it]
71%|███████▏ | 1966/2757 [4:28:14<1:45:06, 7.97s/it]
71%|███████▏ | 1967/2757 [4:28:22<1:46:10, 8.06s/it]
71%|███████▏ | 1968/2757 [4:28:30<1:43:44, 7.89s/it]
71%|███████▏ | 1969/2757 [4:28:38<1:44:43, 7.97s/it]
71%|███████▏ | 1970/2757 [4:28:46<1:45:39, 8.06s/it]
71%|███████▏ | 1970/2757 [4:28:46<1:45:39, 8.06s/it]
71%|███████▏ | 1971/2757 [4:28:54<1:44:57, 8.01s/it]
72%|███████▏ | 1972/2757 [4:29:01<1:42:40, 7.85s/it]
72%|███████▏ | 1973/2757 [4:29:10<1:43:27, 7.92s/it]
72%|███████▏ | 1974/2757 [4:29:18<1:45:16, 8.07s/it]
72%|███████▏ | 1975/2757 [4:29:26<1:45:11, 8.07s/it]
72%|███████▏ | 1976/2757 [4:29:34<1:44:24, 8.02s/it]
72%|███████▏ | 1977/2757 [4:29:42<1:43:11, 7.94s/it]
72%|███████▏ | 1978/2757 [4:29:50<1:42:34, 7.90s/it]
72%|███████▏ | 1979/2757 [4:29:58<1:44:23, 8.05s/it]
72%|███████▏ | 1980/2757 [4:30:06<1:46:02, 8.19s/it]
72%|███████▏ | 1980/2757 [4:30:06<1:46:02, 8.19s/it]
72%|███████▏ | 1981/2757 [4:30:14<1:44:37, 8.09s/it]
72%|███████▏ | 1982/2757 [4:30:22<1:44:49, 8.12s/it]
72%|███████▏ | 1983/2757 [4:30:31<1:45:02, 8.14s/it]
72%|███████▏ | 1984/2757 [4:30:39<1:44:00, 8.07s/it]
72%|███████▏ | 1985/2757 [4:30:47<1:43:56, 8.08s/it]
72%|███████▏ | 1986/2757 [4:30:55<1:43:54, 8.09s/it]
72%|███████▏ | 1987/2757 [4:31:03<1:42:39, 8.00s/it]
72%|███████▏ | 1988/2757 [4:31:11<1:42:57, 8.03s/it]
72%|███████▏ | 1989/2757 [4:31:19<1:43:49, 8.11s/it]
72%|███████▏ | 1990/2757 [4:31:27<1:42:50, 8.05s/it]
72%|███████▏ | 1990/2757 [4:31:27<1:42:50, 8.05s/it]
72%|███████▏ | 1991/2757 [4:31:35<1:42:40, 8.04s/it]
72%|███████▏ | 1992/2757 [4:31:43<1:43:28, 8.12s/it]
72%|███████▏ | 1993/2757 [4:31:51<1:42:57, 8.09s/it]
72%|███████▏ | 1994/2757 [4:31:59<1:41:24, 7.97s/it]
72%|███████▏ | 1995/2757 [4:32:07<1:41:30, 7.99s/it]
72%|███████▏ | 1996/2757 [4:32:15<1:41:40, 8.02s/it]
72%|███████▏ | 1997/2757 [4:32:24<1:43:45, 8.19s/it]
72%|███████▏ | 1998/2757 [4:32:32<1:42:34, 8.11s/it]
73%|███████▎ | 1999/2757 [4:32:39<1:41:01, 8.00s/it]
73%|███████▎ | 2000/2757 [4:32:47<1:39:59, 7.93s/it]
73%|███████▎ | 2000/2757 [4:32:47<1:39:59, 7.93s/it]{'loss': '0.3285', 'grad_norm': '0.4907', 'learning_rate': '6.866e-05', 'rewards/chosen': '-2.08', 'rewards/rejected': '-3.827', 'rewards/accuracies': '0.8221', 'rewards/margins': '1.747', 'logps/chosen': '-103.3', 'logps/rejected': '-127.7', 'logits/chosen': '-3.946', 'logits/rejected': '-3.856', 'epoch': '2.002'}
{'loss': '0.2799', 'grad_norm': '0.5088', 'learning_rate': '6.791e-05', 'rewards/chosen': '-2.049', 'rewards/rejected': '-3.981', 'rewards/accuracies': '0.8391', 'rewards/margins': '1.933', 'logps/chosen': '-101.1', 'logps/rejected': '-126.3', 'logits/chosen': '-3.941', 'logits/rejected': '-3.925', 'epoch': '2.013'}
{'loss': '0.2695', 'grad_norm': '0.5052', 'learning_rate': '6.717e-05', 'rewards/chosen': '-2.205', 'rewards/rejected': '-4.158', 'rewards/accuracies': '0.8586', 'rewards/margins': '1.953', 'logps/chosen': '-104.2', 'logps/rejected': '-128.5', 'logits/chosen': '-4.173', 'logits/rejected': '-4.081', 'epoch': '2.024'}
{'loss': '0.271', 'grad_norm': '0.4783', 'learning_rate': '6.642e-05', 'rewards/chosen': '-2.17', 'rewards/rejected': '-4.234', 'rewards/accuracies': '0.8555', 'rewards/margins': '2.064', 'logps/chosen': '-104.7', 'logps/rejected': '-129.3', 'logits/chosen': '-4.187', 'logits/rejected': '-4.043', 'epoch': '2.035'}
{'loss': '0.2763', 'grad_norm': '0.543', 'learning_rate': '6.567e-05', 'rewards/chosen': '-2.28', 'rewards/rejected': '-4.33', 'rewards/accuracies': '0.8445', 'rewards/margins': '2.05', 'logps/chosen': '-102.9', 'logps/rejected': '-129.7', 'logits/chosen': '-4.226', 'logits/rejected': '-4.173', 'epoch': '2.046'}
{'loss': '0.2582', 'grad_norm': '0.5694', 'learning_rate': '6.492e-05', 'rewards/chosen': '-2.344', 'rewards/rejected': '-4.545', 'rewards/accuracies': '0.8687', 'rewards/margins': '2.201', 'logps/chosen': '-103.5', 'logps/rejected': '-131.2', 'logits/chosen': '-4.404', 'logits/rejected': '-4.277', 'epoch': '2.057'}
{'loss': '0.2655', 'grad_norm': '0.4885', 'learning_rate': '6.417e-05', 'rewards/chosen': '-2.716', 'rewards/rejected': '-4.911', 'rewards/accuracies': '0.8445', 'rewards/margins': '2.195', 'logps/chosen': '-108.2', 'logps/rejected': '-136', 'logits/chosen': '-4.375', 'logits/rejected': '-4.236', 'epoch': '2.067'}
{'loss': '0.2693', 'grad_norm': '0.5405', 'learning_rate': '6.343e-05', 'rewards/chosen': '-2.618', 'rewards/rejected': '-4.733', 'rewards/accuracies': '0.8461', 'rewards/margins': '2.115', 'logps/chosen': '-108.1', 'logps/rejected': '-135.2', 'logits/chosen': '-4.263', 'logits/rejected': '-4.204', 'epoch': '2.078'}
{'loss': '0.2746', 'grad_norm': '0.5154', 'learning_rate': '6.268e-05', 'rewards/chosen': '-2.565', 'rewards/rejected': '-4.722', 'rewards/accuracies': '0.8398', 'rewards/margins': '2.157', 'logps/chosen': '-102.6', 'logps/rejected': '-129.8', 'logits/chosen': '-4.375', 'logits/rejected': '-4.328', 'epoch': '2.089'}
{'loss': '0.2775', 'grad_norm': '0.5592', 'learning_rate': '6.193e-05', 'rewards/chosen': '-2.541', 'rewards/rejected': '-4.662', 'rewards/accuracies': '0.8219', 'rewards/margins': '2.121', 'logps/chosen': '-107.2', 'logps/rejected': '-131.6', 'logits/chosen': '-4.222', 'logits/rejected': '-4.156', 'epoch': '2.1'}
{'loss': '0.2666', 'grad_norm': '0.4701', 'learning_rate': '6.118e-05', 'rewards/chosen': '-2.732', 'rewards/rejected': '-4.826', 'rewards/accuracies': '0.8531', 'rewards/margins': '2.093', 'logps/chosen': '-109.3', 'logps/rejected': '-134.7', 'logits/chosen': '-4.261', 'logits/rejected': '-4.192', 'epoch': '2.111'}
{'loss': '0.2717', 'grad_norm': '0.5637', 'learning_rate': '6.043e-05', 'rewards/chosen': '-2.671', 'rewards/rejected': '-4.832', 'rewards/accuracies': '0.8391', 'rewards/margins': '2.162', 'logps/chosen': '-107.6', 'logps/rejected': '-132.3', 'logits/chosen': '-4.387', 'logits/rejected': '-4.331', 'epoch': '2.122'}
{'loss': '0.2677', 'grad_norm': '0.574', 'learning_rate': '5.969e-05', 'rewards/chosen': '-2.751', 'rewards/rejected': '-4.891', 'rewards/accuracies': '0.8555', 'rewards/margins': '2.14', 'logps/chosen': '-110.6', 'logps/rejected': '-136.7', 'logits/chosen': '-4.307', 'logits/rejected': '-4.232', 'epoch': '2.133'}
{'loss': '0.2648', 'grad_norm': '0.501', 'learning_rate': '5.894e-05', 'rewards/chosen': '-2.646', 'rewards/rejected': '-4.777', 'rewards/accuracies': '0.8492', 'rewards/margins': '2.131', 'logps/chosen': '-108.6', 'logps/rejected': '-132.8', 'logits/chosen': '-4.332', 'logits/rejected': '-4.311', 'epoch': '2.144'}
{'loss': '0.2541', 'grad_norm': '0.6321', 'learning_rate': '5.819e-05', 'rewards/chosen': '-2.826', 'rewards/rejected': '-5.047', 'rewards/accuracies': '0.857', 'rewards/margins': '2.221', 'logps/chosen': '-107.1', 'logps/rejected': '-135', 'logits/chosen': '-4.377', 'logits/rejected': '-4.221', 'epoch': '2.155'}
{'loss': '0.2612', 'grad_norm': '0.5602', 'learning_rate': '5.744e-05', 'rewards/chosen': '-3.04', 'rewards/rejected': '-5.265', 'rewards/accuracies': '0.8609', 'rewards/margins': '2.225', 'logps/chosen': '-110.8', 'logps/rejected': '-138.8', 'logits/chosen': '-4.365', 'logits/rejected': '-4.279', 'epoch': '2.165'}
{'loss': '0.2652', 'grad_norm': '0.5245', 'learning_rate': '5.669e-05', 'rewards/chosen': '-3.134', 'rewards/rejected': '-5.365', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.231', 'logps/chosen': '-113.5', 'logps/rejected': '-140.8', 'logits/chosen': '-4.356', 'logits/rejected': '-4.298', 'epoch': '2.176'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
0%| | 0/300 [00:00<?, ?it/s]
1%| | 2/300 [00:00<00:52, 5.68it/s]
1%| | 3/300 [00:00<01:48, 2.73it/s]
1%|▏ | 4/300 [00:01<01:35, 3.09it/s]
2%|▏ | 5/300 [00:01<01:45, 2.79it/s]
2%|▏ | 6/300 [00:01<01:40, 2.91it/s]
2%|▏ | 7/300 [00:02<01:54, 2.55it/s]
3%|▎ | 8/300 [00:02<01:50, 2.63it/s]
3%|▎ | 9/300 [00:03<01:52, 2.59it/s]
3%|▎ | 10/300 [00:03<01:48, 2.67it/s]
4%|▎ | 11/300 [00:04<02:06, 2.29it/s]
4%|▍ | 12/300 [00:04<01:48, 2.65it/s]
4%|▍ | 13/300 [00:04<01:49, 2.61it/s]
5%|▍ | 14/300 [00:05<01:40, 2.85it/s]
5%|▌ | 15/300 [00:05<01:49, 2.61it/s]
5%|▌ | 16/300 [00:05<01:36, 2.95it/s]
6%|▌ | 17/300 [00:06<01:39, 2.83it/s]
6%|▌ | 18/300 [00:06<01:29, 3.14it/s]
6%|▋ | 19/300 [00:06<01:40, 2.81it/s]
7%|▋ | 20/300 [00:07<01:32, 3.04it/s]
7%|▋ | 21/300 [00:07<01:41, 2.74it/s]
7%|▋ | 22/300 [00:07<01:30, 3.07it/s]
8%|▊ | 23/300 [00:08<01:35, 2.91it/s]
8%|▊ | 24/300 [00:08<01:25, 3.23it/s]
8%|▊ | 25/300 [00:08<01:37, 2.83it/s]
9%|▊ | 26/300 [00:09<01:29, 3.05it/s]
9%|▉ | 27/300 [00:09<01:38, 2.78it/s]
9%|▉ | 28/300 [00:09<01:27, 3.12it/s]
10%|▉ | 29/300 [00:10<01:29, 3.01it/s]
10%|█ | 30/300 [00:10<01:21, 3.31it/s]
10%|█ | 31/300 [00:10<01:24, 3.19it/s]
11%|█ | 32/300 [00:10<01:17, 3.45it/s]
11%|█ | 33/300 [00:11<01:20, 3.33it/s]
11%|█▏ | 34/300 [00:11<01:15, 3.50it/s]
12%|█▏ | 35/300 [00:11<01:22, 3.23it/s]
12%|█▏ | 36/300 [00:12<01:17, 3.39it/s]
12%|█▏ | 37/300 [00:12<01:27, 2.99it/s]
13%|█▎ | 38/300 [00:12<01:18, 3.33it/s]
13%|█▎ | 39/300 [00:13<01:26, 3.03it/s]
13%|█▎ | 40/300 [00:13<01:18, 3.32it/s]
14%|█▎ | 41/300 [00:13<01:25, 3.03it/s]
14%|█▍ | 42/300 [00:14<01:17, 3.33it/s]
14%|█▍ | 43/300 [00:14<01:23, 3.08it/s]
15%|█▍ | 44/300 [00:14<01:16, 3.35it/s]
15%|█▌ | 45/300 [00:15<01:22, 3.08it/s]
15%|█▌ | 46/300 [00:15<01:24, 3.02it/s]
16%|█▌ | 47/300 [00:15<01:29, 2.81it/s]
16%|█▌ | 48/300 [00:16<01:22, 3.05it/s]
16%|█▋ | 49/300 [00:16<01:23, 3.02it/s]
17%|█▋ | 50/300 [00:16<01:14, 3.36it/s]
17%|█▋ | 51/300 [00:16<01:17, 3.19it/s]
17%|█▋ | 52/300 [00:17<01:14, 3.31it/s]
18%|█▊ | 53/300 [00:17<01:27, 2.81it/s]
18%|█▊ | 54/300 [00:17<01:19, 3.10it/s]
18%|█▊ | 55/300 [00:18<01:26, 2.82it/s]
19%|█▊ | 56/300 [00:18<01:15, 3.25it/s]
19%|█▉ | 57/300 [00:18<01:19, 3.07it/s]
19%|█▉ | 58/300 [00:19<01:20, 3.00it/s]
20%|█▉ | 59/300 [00:19<01:25, 2.81it/s]
20%|██ | 60/300 [00:19<01:17, 3.11it/s]
20%|██ | 61/300 [00:20<01:20, 2.96it/s]
21%|██ | 62/300 [00:20<01:20, 2.96it/s]
21%|██ | 63/300 [00:21<01:25, 2.76it/s]
21%|██▏ | 64/300 [00:21<01:24, 2.80it/s]
22%|██▏ | 65/300 [00:21<01:27, 2.69it/s]
22%|██▏ | 66/300 [00:22<01:19, 2.95it/s]
22%|██▏ | 67/300 [00:22<01:26, 2.68it/s]
23%|██▎ | 68/300 [00:22<01:18, 2.97it/s]
23%|██▎ | 69/300 [00:23<01:21, 2.85it/s]
23%|██▎ | 70/300 [00:23<01:12, 3.16it/s]
24%|██▎ | 71/300 [00:23<01:15, 3.02it/s]
24%|██▍ | 72/300 [00:24<01:09, 3.27it/s]
24%|██▍ | 73/300 [00:24<01:12, 3.13it/s]
25%|██▍ | 74/300 [00:24<01:08, 3.31it/s]
25%|██▌ | 75/300 [00:25<01:13, 3.07it/s]
25%|██▌ | 76/300 [00:25<01:14, 3.00it/s]
26%|██▌ | 77/300 [00:25<01:19, 2.81it/s]
26%|██▌ | 78/300 [00:26<01:13, 3.04it/s]
26%|██▋ | 79/300 [00:26<01:15, 2.93it/s]
27%|██▋ | 80/300 [00:26<01:11, 3.09it/s]
27%|██▋ | 81/300 [00:27<01:19, 2.76it/s]
27%|██▋ | 82/300 [00:27<01:11, 3.07it/s]
28%|██▊ | 83/300 [00:27<01:13, 2.97it/s]
28%|██▊ | 84/300 [00:28<01:13, 2.93it/s]
28%|██▊ | 85/300 [00:28<01:18, 2.74it/s]
29%|██▊ | 86/300 [00:28<01:10, 3.04it/s]
29%|██▉ | 87/300 [00:29<01:12, 2.94it/s]
29%|██▉ | 88/300 [00:29<01:06, 3.19it/s]
30%|██▉ | 89/300 [00:29<01:10, 2.98it/s]
30%|███ | 90/300 [00:30<01:05, 3.21it/s]
30%|███ | 91/300 [00:30<01:09, 3.02it/s]
31%|███ | 92/300 [00:30<01:03, 3.28it/s]
31%|███ | 93/300 [00:31<01:09, 3.00it/s]
31%|███▏ | 94/300 [00:31<01:02, 3.30it/s]
32%|███▏ | 95/300 [00:31<01:07, 3.03it/s]
32%|███▏ | 96/300 [00:31<01:03, 3.22it/s]
32%|███▏ | 97/300 [00:32<01:05, 3.11it/s]
33%|███▎ | 98/300 [00:32<01:01, 3.27it/s]
33%|███▎ | 99/300 [00:33<01:08, 2.92it/s]
33%|███▎ | 100/300 [00:33<01:04, 3.10it/s]
34%|███▎ | 101/300 [00:33<01:07, 2.93it/s]
34%|███▍ | 102/300 [00:34<01:06, 3.00it/s]
34%|███▍ | 103/300 [00:34<01:18, 2.51it/s]
35%|███▍ | 104/300 [00:34<01:08, 2.86it/s]
35%|███▌ | 105/300 [00:35<01:09, 2.81it/s]
35%|███▌ | 106/300 [00:35<01:03, 3.05it/s]
36%|███▌ | 107/300 [00:35<01:09, 2.77it/s]
36%|███▌ | 108/300 [00:36<01:05, 2.91it/s]
36%|███▋ | 109/300 [00:36<01:08, 2.80it/s]
37%|███▋ | 110/300 [00:36<01:03, 2.98it/s]
37%|███▋ | 111/300 [00:37<01:12, 2.62it/s]
37%|███▋ | 112/300 [00:37<01:06, 2.83it/s]
38%|███▊ | 113/300 [00:38<01:08, 2.75it/s]
38%|███▊ | 114/300 [00:38<01:07, 2.77it/s]
38%|███▊ | 115/300 [00:38<01:09, 2.65it/s]
39%|███▊ | 116/300 [00:39<01:01, 2.97it/s]
39%|███▉ | 117/300 [00:39<01:03, 2.89it/s]
39%|███▉ | 118/300 [00:39<00:58, 3.12it/s]
40%|███▉ | 119/300 [00:40<01:01, 2.95it/s]
40%|████ | 120/300 [00:40<01:00, 2.99it/s]
40%|████ | 121/300 [00:40<01:02, 2.87it/s]
41%|████ | 122/300 [00:40<00:57, 3.10it/s]
41%|████ | 123/300 [00:41<01:01, 2.89it/s]
41%|████▏ | 124/300 [00:41<00:57, 3.06it/s]
42%|████▏ | 125/300 [00:42<00:59, 2.94it/s]
42%|████▏ | 126/300 [00:42<00:59, 2.92it/s]
42%|████▏ | 127/300 [00:42<01:01, 2.79it/s]
43%|████▎ | 128/300 [00:43<00:57, 2.99it/s]
43%|████▎ | 129/300 [00:43<01:00, 2.84it/s]
43%|████▎ | 130/300 [00:43<00:54, 3.13it/s]
44%|████▎ | 131/300 [00:44<01:00, 2.78it/s]
44%|████▍ | 132/300 [00:44<00:55, 3.02it/s]
44%|████▍ | 133/300 [00:44<01:02, 2.67it/s]
45%|████▍ | 134/300 [00:45<00:56, 2.93it/s]
45%|████▌ | 135/300 [00:45<00:58, 2.81it/s]
45%|████▌ | 136/300 [00:45<00:52, 3.10it/s]
46%|████▌ | 137/300 [00:46<00:53, 3.04it/s]
46%|████▌ | 138/300 [00:46<00:55, 2.92it/s]
46%|████▋ | 139/300 [00:46<00:54, 2.96it/s]
47%|████▋ | 140/300 [00:47<00:49, 3.25it/s]
47%|████▋ | 141/300 [00:47<00:50, 3.15it/s]
47%|████▋ | 142/300 [00:47<00:51, 3.05it/s]
48%|████▊ | 143/300 [00:48<00:54, 2.89it/s]
48%|████▊ | 144/300 [00:48<00:48, 3.19it/s]
48%|████▊ | 145/300 [00:48<00:49, 3.14it/s]
49%|████▊ | 146/300 [00:48<00:46, 3.30it/s]
49%|████▉ | 147/300 [00:49<00:53, 2.87it/s]
49%|████▉ | 148/300 [00:49<00:50, 3.03it/s]
50%|████▉ | 149/300 [00:50<00:59, 2.56it/s]
50%|█████ | 150/300 [00:50<00:53, 2.80it/s]
50%|█████ | 151/300 [00:50<00:54, 2.75it/s]
51%|█████ | 152/300 [00:51<00:48, 3.08it/s]
51%|█████ | 153/300 [00:51<00:48, 3.01it/s]
51%|█████▏ | 154/300 [00:51<00:44, 3.30it/s]
52%|█████▏ | 155/300 [00:52<00:46, 3.14it/s]
52%|█████▏ | 156/300 [00:52<00:41, 3.47it/s]
52%|█████▏ | 157/300 [00:52<00:45, 3.12it/s]
53%|█████▎ | 158/300 [00:52<00:43, 3.26it/s]
53%|█████▎ | 159/300 [00:53<00:51, 2.72it/s]
53%|█████▎ | 160/300 [00:53<00:47, 2.96it/s]
54%|█████▎ | 161/300 [00:54<00:50, 2.73it/s]
54%|█████▍ | 162/300 [00:54<00:49, 2.77it/s]
54%|█████▍ | 163/300 [00:55<00:55, 2.45it/s]
55%|█████▍ | 164/300 [00:55<00:53, 2.56it/s]
55%|█████▌ | 165/300 [00:55<00:52, 2.55it/s]
55%|█████▌ | 166/300 [00:56<00:50, 2.66it/s]
56%|█████▌ | 167/300 [00:56<00:54, 2.44it/s]
56%|█████▌ | 168/300 [00:56<00:47, 2.76it/s]
56%|█████▋ | 169/300 [00:57<00:52, 2.49it/s]
57%|█████▋ | 170/300 [00:57<00:50, 2.59it/s]
57%|█████▋ | 171/300 [00:58<00:50, 2.54it/s]
57%|█████▋ | 172/300 [00:58<00:44, 2.88it/s]
58%|█████▊ | 173/300 [00:58<00:46, 2.73it/s]
58%|█████▊ | 174/300 [00:59<00:41, 3.06it/s]
58%|█████▊ | 175/300 [00:59<00:44, 2.79it/s]
59%|█████▊ | 176/300 [00:59<00:39, 3.12it/s]
59%|█████▉ | 177/300 [01:00<00:44, 2.78it/s]
59%|█████▉ | 178/300 [01:00<00:38, 3.20it/s]
60%|█████▉ | 179/300 [01:00<00:41, 2.94it/s]
60%|██████ | 180/300 [01:00<00:37, 3.22it/s]
60%|██████ | 181/300 [01:01<00:41, 2.89it/s]
61%|██████ | 182/300 [01:01<00:36, 3.20it/s]
61%|██████ | 183/300 [01:02<00:40, 2.87it/s]
61%|██████▏ | 184/300 [01:02<00:40, 2.86it/s]
62%|██████▏ | 185/300 [01:02<00:42, 2.68it/s]
62%|██████▏ | 186/300 [01:03<00:39, 2.88it/s]
62%|██████▏ | 187/300 [01:03<00:39, 2.87it/s]
63%|██████▎ | 188/300 [01:03<00:35, 3.15it/s]
63%|██████▎ | 189/300 [01:04<00:38, 2.85it/s]
63%|██████▎ | 190/300 [01:04<00:35, 3.09it/s]
64%|██████▎ | 191/300 [01:04<00:41, 2.64it/s]
64%|██████▍ | 192/300 [01:05<00:39, 2.75it/s]
64%|██████▍ | 193/300 [01:05<00:43, 2.46it/s]
65%|██████▍ | 194/300 [01:06<00:41, 2.57it/s]
65%|██████▌ | 195/300 [01:06<00:42, 2.49it/s]
65%|██████▌ | 196/300 [01:06<00:38, 2.73it/s]
66%|██████▌ | 197/300 [01:07<00:38, 2.65it/s]
66%|██████▌ | 198/300 [01:07<00:35, 2.90it/s]
66%|██████▋ | 199/300 [01:07<00:36, 2.80it/s]
67%|██████▋ | 200/300 [01:08<00:33, 2.99it/s]
67%|██████▋ | 201/300 [01:08<00:35, 2.80it/s]
67%|██████▋ | 202/300 [01:08<00:31, 3.09it/s]
68%|██████▊ | 203/300 [01:09<00:33, 2.94it/s]
68%|██████▊ | 204/300 [01:09<00:29, 3.23it/s]
68%|██████▊ | 205/300 [01:09<00:31, 3.02it/s]
69%|██████▊ | 206/300 [01:10<00:27, 3.41it/s]
69%|██████▉ | 207/300 [01:10<00:31, 2.94it/s]
69%|██████▉ | 208/300 [01:10<00:31, 2.95it/s]
70%|██████▉ | 209/300 [01:11<00:32, 2.80it/s]
70%|███████ | 210/300 [01:11<00:29, 3.02it/s]
70%|███████ | 211/300 [01:11<00:33, 2.67it/s]
71%|███████ | 212/300 [01:12<00:29, 2.98it/s]
71%|███████ | 213/300 [01:12<00:31, 2.74it/s]
71%|███████▏ | 214/300 [01:12<00:28, 3.00it/s]
72%|███████▏ | 215/300 [01:13<00:30, 2.80it/s]
72%|███████▏ | 216/300 [01:13<00:27, 3.09it/s]
72%|███████▏ | 217/300 [01:14<00:31, 2.64it/s]
73%|███████▎ | 218/300 [01:14<00:26, 3.07it/s]
73%|███████▎ | 219/300 [01:14<00:28, 2.81it/s]
73%|███████▎ | 220/300 [01:14<00:25, 3.11it/s]
74%|███████▎ | 221/300 [01:15<00:28, 2.78it/s]
74%|███████▍ | 222/300 [01:15<00:25, 3.09it/s]
74%|███████▍ | 223/300 [01:16<00:27, 2.75it/s]
75%|███████▍ | 224/300 [01:16<00:25, 2.96it/s]
75%|███████▌ | 225/300 [01:16<00:26, 2.81it/s]
75%|███████▌ | 226/300 [01:17<00:24, 3.05it/s]
76%|███████▌ | 227/300 [01:17<00:26, 2.80it/s]
76%|███████▌ | 228/300 [01:17<00:23, 3.13it/s]
76%|███████▋ | 229/300 [01:18<00:25, 2.75it/s]
77%|███████▋ | 230/300 [01:18<00:24, 2.86it/s]
77%|███████▋ | 231/300 [01:18<00:25, 2.70it/s]
77%|███████▋ | 232/300 [01:19<00:24, 2.76it/s]
78%|███████▊ | 233/300 [01:19<00:27, 2.45it/s]
78%|███████▊ | 234/300 [01:19<00:23, 2.78it/s]
78%|███████▊ | 235/300 [01:20<00:24, 2.60it/s]
79%|███████▊ | 236/300 [01:20<00:23, 2.67it/s]
79%|███████▉ | 237/300 [01:21<00:24, 2.53it/s]
79%|███████▉ | 238/300 [01:21<00:21, 2.84it/s]
80%|███████▉ | 239/300 [01:21<00:21, 2.83it/s]
80%|████████ | 240/300 [01:22<00:19, 3.15it/s]
80%|████████ | 241/300 [01:22<00:19, 2.96it/s]
81%|████████ | 242/300 [01:22<00:19, 3.00it/s]
81%|████████ | 243/300 [01:23<00:21, 2.68it/s]
81%|████████▏ | 244/300 [01:23<00:20, 2.79it/s]
82%|████████▏ | 245/300 [01:24<00:22, 2.48it/s]
82%|████████▏ | 246/300 [01:24<00:20, 2.63it/s]
82%|████████▏ | 247/300 [01:24<00:21, 2.50it/s]
83%|████████▎ | 248/300 [01:25<00:18, 2.85it/s]
83%|████████▎ | 249/300 [01:25<00:18, 2.73it/s]
83%|████████▎ | 250/300 [01:25<00:17, 2.85it/s]
84%|████████▎ | 251/300 [01:26<00:17, 2.76it/s]
84%|████████▍ | 252/300 [01:26<00:16, 2.96it/s]
84%|████████▍ | 253/300 [01:26<00:16, 2.81it/s]
85%|████████▍ | 254/300 [01:27<00:14, 3.18it/s]
85%|████████▌ | 255/300 [01:27<00:16, 2.75it/s]
85%|████████▌ | 256/300 [01:27<00:14, 3.09it/s]
86%|████████▌ | 257/300 [01:28<00:14, 2.89it/s]
86%|████████▌ | 258/300 [01:28<00:13, 3.16it/s]
86%|████████▋ | 259/300 [01:28<00:13, 2.94it/s]
87%|████████▋ | 260/300 [01:29<00:12, 3.14it/s]
87%|████████▋ | 261/300 [01:29<00:13, 2.97it/s]
87%|████████▋ | 262/300 [01:29<00:11, 3.26it/s]
88%|████████▊ | 263/300 [01:30<00:12, 3.00it/s]
88%|████████▊ | 264/300 [01:30<00:12, 2.99it/s]
88%|████████▊ | 265/300 [01:30<00:13, 2.59it/s]
89%|████████▊ | 266/300 [01:31<00:12, 2.68it/s]
89%|████████▉ | 267/300 [01:31<00:13, 2.42it/s]
89%|████████▉ | 268/300 [01:32<00:11, 2.78it/s]
90%|████████▉ | 269/300 [01:32<00:12, 2.46it/s]
90%|█████████ | 270/300 [01:32<00:11, 2.63it/s]
90%|█████████ | 271/300 [01:33<00:12, 2.37it/s]
91%|█████████ | 272/300 [01:33<00:10, 2.62it/s]
91%|█████████ | 273/300 [01:34<00:11, 2.33it/s]
91%|█████████▏| 274/300 [01:34<00:09, 2.67it/s]
92%|█████████▏| 275/300 [01:34<00:10, 2.42it/s]
92%|█████████▏| 276/300 [01:35<00:09, 2.60it/s]
92%|█████████▏| 277/300 [01:35<00:09, 2.47it/s]
93%|█████████▎| 278/300 [01:36<00:07, 2.79it/s]
93%|█████████▎| 279/300 [01:36<00:07, 2.63it/s]
93%|█████████▎| 280/300 [01:36<00:07, 2.75it/s]
94%|█████████▎| 281/300 [01:37<00:07, 2.63it/s]
94%|█████████▍| 282/300 [01:37<00:06, 2.95it/s]
94%|█████████▍| 283/300 [01:37<00:06, 2.75it/s]
95%|█████████▍| 284/300 [01:38<00:05, 3.03it/s]
95%|█████████▌| 285/300 [01:38<00:05, 2.64it/s]
95%|█████████▌| 286/300 [01:38<00:05, 2.70it/s]
96%|█████████▌| 287/300 [01:39<00:04, 2.63it/s]
96%|█████████▌| 288/300 [01:39<00:03, 3.05it/s]
96%|█████████▋| 289/300 [01:39<00:03, 2.80it/s]
97%|█████████▋| 290/300 [01:40<00:03, 2.82it/s]
97%|█████████▋| 291/300 [01:40<00:03, 2.62it/s]
97%|█████████▋| 292/300 [01:41<00:02, 2.94it/s]
98%|█████████▊| 293/300 [01:41<00:02, 2.78it/s]
98%|█████████▊| 294/300 [01:41<00:01, 3.02it/s]
98%|█████████▊| 295/300 [01:42<00:01, 2.95it/s]
99%|█████████▊| 296/300 [01:42<00:01, 3.15it/s]
99%|█████████▉| 297/300 [01:42<00:01, 2.92it/s]
99%|█████████▉| 298/300 [01:42<00:00, 3.28it/s]
100%|█████████▉| 299/300 [01:43<00:00, 2.88it/s]
100%|██████████| 300/300 [01:43<00:00, 2.75it/s]

73%|███████▎ | 2000/2757 [4:34:32<1:39:59, 7.93s/it]
100%|██████████| 300/300 [01:43<00:00, 2.75it/s]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-2000/tokenizer_config.json.
73%|███████▎ | 2001/2757 [4:34:46<8:39:03, 41.20s/it]
73%|███████▎ | 2002/2757 [4:34:54<6:34:09, 31.32s/it]
73%|███████▎ | 2003/2757 [4:35:02<5:04:47, 24.25s/it]
73%|███████▎ | 2004/2757 [4:35:10<4:02:00, 19.28s/it]
73%|███████▎ | 2005/2757 [4:35:17<3:17:12, 15.74s/it]
73%|███████▎ | 2006/2757 [4:35:25<2:46:16, 13.28s/it]
73%|███████▎ | 2007/2757 [4:35:33<2:27:25, 11.79s/it]
73%|███████▎ | 2008/2757 [4:35:41<2:12:05, 10.58s/it]
73%|███████▎ | 2009/2757 [4:35:48<2:00:45, 9.69s/it]
73%|███████▎ | 2010/2757 [4:35:57<1:55:55, 9.31s/it]
73%|███████▎ | 2010/2757 [4:35:57<1:55:55, 9.31s/it]
73%|███████▎ | 2011/2757 [4:36:04<1:50:02, 8.85s/it]
73%|███████▎ | 2012/2757 [4:36:12<1:45:06, 8.47s/it]
73%|███████▎ | 2013/2757 [4:36:20<1:42:05, 8.23s/it]
73%|███████▎ | 2014/2757 [4:36:28<1:40:27, 8.11s/it]
73%|███████▎ | 2015/2757 [4:36:35<1:38:49, 7.99s/it]
73%|███████▎ | 2016/2757 [4:36:44<1:40:22, 8.13s/it]
73%|███████▎ | 2017/2757 [4:36:52<1:39:20, 8.05s/it]
73%|███████▎ | 2018/2757 [4:37:00<1:39:26, 8.07s/it]
73%|███████▎ | 2019/2757 [4:37:08<1:39:02, 8.05s/it]
73%|███████▎ | 2020/2757 [4:37:15<1:37:06, 7.91s/it]
73%|███████▎ | 2020/2757 [4:37:15<1:37:06, 7.91s/it]
73%|███████▎ | 2021/2757 [4:37:24<1:38:40, 8.04s/it]
73%|███████▎ | 2022/2757 [4:37:31<1:37:17, 7.94s/it]
73%|███████▎ | 2023/2757 [4:37:39<1:35:33, 7.81s/it]
73%|███████▎ | 2024/2757 [4:37:47<1:35:55, 7.85s/it]
73%|███████▎ | 2025/2757 [4:37:55<1:36:24, 7.90s/it]
73%|███████▎ | 2026/2757 [4:38:03<1:37:26, 8.00s/it]
74%|███████▎ | 2027/2757 [4:38:11<1:37:17, 8.00s/it]
74%|███████▎ | 2028/2757 [4:38:19<1:37:57, 8.06s/it]
74%|███████▎ | 2029/2757 [4:38:27<1:37:00, 8.00s/it]
74%|███████▎ | 2030/2757 [4:38:35<1:37:14, 8.03s/it]
74%|███████▎ | 2030/2757 [4:38:35<1:37:14, 8.03s/it]
74%|███████▎ | 2031/2757 [4:38:43<1:37:23, 8.05s/it]
74%|███████▎ | 2032/2757 [4:38:51<1:35:32, 7.91s/it]
74%|███████▎ | 2033/2757 [4:38:59<1:34:31, 7.83s/it]
74%|███████▍ | 2034/2757 [4:39:06<1:34:28, 7.84s/it]
74%|███████▍ | 2035/2757 [4:39:15<1:37:06, 8.07s/it]
74%|███████▍ | 2036/2757 [4:39:23<1:36:27, 8.03s/it]
74%|███████▍ | 2037/2757 [4:39:31<1:35:18, 7.94s/it]
74%|███████▍ | 2038/2757 [4:39:39<1:36:29, 8.05s/it]
74%|███████▍ | 2039/2757 [4:39:47<1:37:51, 8.18s/it]
74%|███████▍ | 2040/2757 [4:39:56<1:37:33, 8.16s/it]
74%|███████▍ | 2040/2757 [4:39:56<1:37:33, 8.16s/it]
74%|███████▍ | 2041/2757 [4:40:04<1:36:30, 8.09s/it]
74%|███████▍ | 2042/2757 [4:40:11<1:35:19, 8.00s/it]
74%|███████▍ | 2043/2757 [4:40:19<1:34:26, 7.94s/it]
74%|███████▍ | 2044/2757 [4:40:27<1:33:36, 7.88s/it]
74%|███████▍ | 2045/2757 [4:40:35<1:34:41, 7.98s/it]
74%|███████▍ | 2046/2757 [4:40:43<1:33:36, 7.90s/it]
74%|███████▍ | 2047/2757 [4:40:51<1:35:05, 8.04s/it]
74%|███████▍ | 2048/2757 [4:40:59<1:34:23, 7.99s/it]
74%|███████▍ | 2049/2757 [4:41:07<1:33:49, 7.95s/it]
74%|███████▍ | 2050/2757 [4:41:15<1:33:52, 7.97s/it]
74%|███████▍ | 2050/2757 [4:41:15<1:33:52, 7.97s/it]
74%|███████▍ | 2051/2757 [4:41:23<1:32:49, 7.89s/it]
74%|███████▍ | 2052/2757 [4:41:30<1:31:56, 7.82s/it]
74%|███████▍ | 2053/2757 [4:41:38<1:32:01, 7.84s/it]
75%|███████▍ | 2054/2757 [4:41:46<1:33:03, 7.94s/it]
75%|███████▍ | 2055/2757 [4:41:54<1:32:18, 7.89s/it]
75%|███████▍ | 2056/2757 [4:42:02<1:33:59, 8.04s/it]
75%|███████▍ | 2057/2757 [4:42:10<1:32:52, 7.96s/it]
75%|███████▍ | 2058/2757 [4:42:18<1:33:46, 8.05s/it]
75%|███████▍ | 2059/2757 [4:42:27<1:35:33, 8.21s/it]
75%|███████▍ | 2060/2757 [4:42:35<1:33:53, 8.08s/it]
75%|███████▍ | 2060/2757 [4:42:35<1:33:53, 8.08s/it]
75%|███████▍ | 2061/2757 [4:42:43<1:33:35, 8.07s/it]
75%|███████▍ | 2062/2757 [4:42:51<1:32:25, 7.98s/it]
75%|███████▍ | 2063/2757 [4:42:59<1:32:39, 8.01s/it]
75%|███████▍ | 2064/2757 [4:43:07<1:34:47, 8.21s/it]
75%|███████▍ | 2065/2757 [4:43:15<1:33:17, 8.09s/it]
75%|███████▍ | 2066/2757 [4:43:23<1:32:11, 8.01s/it]
75%|███████▍ | 2067/2757 [4:43:31<1:32:51, 8.07s/it]
75%|███████▌ | 2068/2757 [4:43:39<1:32:08, 8.02s/it]
75%|███████▌ | 2069/2757 [4:43:47<1:31:40, 8.00s/it]
75%|███████▌ | 2070/2757 [4:43:55<1:29:27, 7.81s/it]
75%|███████▌ | 2070/2757 [4:43:55<1:29:27, 7.81s/it]
75%|███████▌ | 2071/2757 [4:44:02<1:28:34, 7.75s/it]
75%|███████▌ | 2072/2757 [4:44:10<1:28:14, 7.73s/it]
75%|███████▌ | 2073/2757 [4:44:18<1:29:21, 7.84s/it]
75%|███████▌ | 2074/2757 [4:44:26<1:29:43, 7.88s/it]
75%|███████▌ | 2075/2757 [4:44:34<1:28:56, 7.82s/it]
75%|███████▌ | 2076/2757 [4:44:41<1:28:27, 7.79s/it]
75%|███████▌ | 2077/2757 [4:44:49<1:29:43, 7.92s/it]
75%|███████▌ | 2078/2757 [4:44:57<1:29:33, 7.91s/it]
75%|███████▌ | 2079/2757 [4:45:06<1:31:28, 8.09s/it]
75%|███████▌ | 2080/2757 [4:45:14<1:32:30, 8.20s/it]
75%|███████▌ | 2080/2757 [4:45:14<1:32:30, 8.20s/it]
75%|███████▌ | 2081/2757 [4:45:22<1:31:40, 8.14s/it]
76%|███████▌ | 2082/2757 [4:45:30<1:31:11, 8.11s/it]
76%|███████▌ | 2083/2757 [4:45:38<1:29:14, 7.94s/it]
76%|███████▌ | 2084/2757 [4:45:46<1:29:24, 7.97s/it]
76%|███████▌ | 2085/2757 [4:45:54<1:29:18, 7.97s/it]
76%|███████▌ | 2086/2757 [4:46:02<1:28:52, 7.95s/it]
76%|███████▌ | 2087/2757 [4:46:10<1:29:15, 7.99s/it]
76%|███████▌ | 2088/2757 [4:46:18<1:29:00, 7.98s/it]
76%|███████▌ | 2089/2757 [4:46:26<1:30:03, 8.09s/it]
76%|███████▌ | 2090/2757 [4:46:34<1:29:46, 8.08s/it]
76%|███████▌ | 2090/2757 [4:46:34<1:29:46, 8.08s/it]
76%|███████▌ | 2091/2757 [4:46:43<1:30:13, 8.13s/it]
76%|███████▌ | 2092/2757 [4:46:51<1:30:40, 8.18s/it]
76%|███████▌ | 2093/2757 [4:46:59<1:30:30, 8.18s/it]
76%|███████▌ | 2094/2757 [4:47:07<1:29:10, 8.07s/it]
76%|███████▌ | 2095/2757 [4:47:15<1:30:09, 8.17s/it]
76%|███████▌ | 2096/2757 [4:47:23<1:29:20, 8.11s/it]
76%|███████▌ | 2097/2757 [4:47:31<1:29:03, 8.10s/it]
76%|███████▌ | 2098/2757 [4:47:40<1:29:45, 8.17s/it]
76%|███████▌ | 2099/2757 [4:47:48<1:30:00, 8.21s/it]
76%|███████▌ | 2100/2757 [4:47:56<1:30:44, 8.29s/it]
76%|███████▌ | 2100/2757 [4:47:56<1:30:44, 8.29s/it]
76%|███████▌ | 2101/2757 [4:48:05<1:31:07, 8.33s/it]
76%|███████▌ | 2102/2757 [4:48:13<1:31:02, 8.34s/it]
76%|███████▋ | 2103/2757 [4:48:21<1:29:23, 8.20s/it]
76%|███████▋ | 2104/2757 [4:48:29<1:27:44, 8.06s/it]
76%|███████▋ | 2105/2757 [4:48:37<1:28:13, 8.12s/it]
76%|███████▋ | 2106/2757 [4:48:45<1:27:48, 8.09s/it]
76%|███████▋ | 2107/2757 [4:48:53<1:26:48, 8.01s/it]
76%|███████▋ | 2108/2757 [4:49:01<1:26:17, 7.98s/it]
76%|███████▋ | 2109/2757 [4:49:08<1:24:56, 7.87s/it]
77%|███████▋ | 2110/2757 [4:49:16<1:24:39, 7.85s/it]
77%|███████▋ | 2110/2757 [4:49:16<1:24:39, 7.85s/it]
77%|███████▋ | 2111/2757 [4:49:24<1:24:25, 7.84s/it]
77%|███████▋ | 2112/2757 [4:49:33<1:26:33, 8.05s/it]
77%|███████▋ | 2113/2757 [4:49:40<1:25:08, 7.93s/it]
77%|███████▋ | 2114/2757 [4:49:49<1:26:25, 8.06s/it]
77%|███████▋ | 2115/2757 [4:49:56<1:25:10, 7.96s/it]
77%|███████▋ | 2116/2757 [4:50:05<1:26:09, 8.06s/it]
77%|███████▋ | 2117/2757 [4:50:13<1:26:07, 8.07s/it]
77%|███████▋ | 2118/2757 [4:50:20<1:24:51, 7.97s/it]
77%|███████▋ | 2119/2757 [4:50:28<1:24:10, 7.92s/it]
77%|███████▋ | 2120/2757 [4:50:36<1:25:09, 8.02s/it]
77%|███████▋ | 2120/2757 [4:50:36<1:25:09, 8.02s/it]
77%|███████▋ | 2121/2757 [4:50:45<1:25:16, 8.04s/it]
77%|███████▋ | 2122/2757 [4:50:53<1:24:42, 8.00s/it]
77%|███████▋ | 2123/2757 [4:51:00<1:23:08, 7.87s/it]
77%|███████▋ | 2124/2757 [4:51:08<1:22:55, 7.86s/it]
77%|███████▋ | 2125/2757 [4:51:15<1:21:56, 7.78s/it]
77%|███████▋ | 2126/2757 [4:51:23<1:21:47, 7.78s/it]
77%|███████▋ | 2127/2757 [4:51:31<1:22:25, 7.85s/it]
77%|███████▋ | 2128/2757 [4:51:39<1:22:42, 7.89s/it]
77%|███████▋ | 2129/2757 [4:51:47<1:23:13, 7.95s/it]
77%|███████▋ | 2130/2757 [4:51:55<1:23:08, 7.96s/it]
77%|███████▋ | 2130/2757 [4:51:55<1:23:08, 7.96s/it]
77%|███████▋ | 2131/2757 [4:52:03<1:21:52, 7.85s/it]
77%|███████▋ | 2132/2757 [4:52:11<1:22:26, 7.91s/it]
77%|███████▋ | 2133/2757 [4:52:19<1:22:12, 7.90s/it]
77%|███████▋ | 2134/2757 [4:52:27<1:22:31, 7.95s/it]
77%|███████▋ | 2135/2757 [4:52:35<1:21:40, 7.88s/it]
77%|███████▋ | 2136/2757 [4:52:43<1:21:49, 7.91s/it]
78%|███████▊ | 2137/2757 [4:52:50<1:21:08, 7.85s/it]
78%|███████▊ | 2138/2757 [4:52:58<1:21:46, 7.93s/it]
78%|███████▊ | 2139/2757 [4:53:07<1:23:06, 8.07s/it]
78%|███████▊ | 2140/2757 [4:53:15<1:22:52, 8.06s/it]
78%|███████▊ | 2140/2757 [4:53:15<1:22:52, 8.06s/it]
78%|███████▊ | 2141/2757 [4:53:22<1:21:03, 7.90s/it]
78%|███████▊ | 2142/2757 [4:53:31<1:22:17, 8.03s/it]
78%|███████▊ | 2143/2757 [4:53:38<1:21:05, 7.92s/it]
78%|███████▊ | 2144/2757 [4:53:46<1:19:51, 7.82s/it]
78%|███████▊ | 2145/2757 [4:53:54<1:19:54, 7.83s/it]
78%|███████▊ | 2146/2757 [4:54:02<1:21:42, 8.02s/it]
78%|███████▊ | 2147/2757 [4:54:10<1:21:30, 8.02s/it]
78%|███████▊ | 2148/2757 [4:54:18<1:20:00, 7.88s/it]
78%|███████▊ | 2149/2757 [4:54:25<1:18:54, 7.79s/it]
78%|███████▊ | 2150/2757 [4:54:33<1:19:21, 7.84s/it]
78%|███████▊ | 2150/2757 [4:54:33<1:19:21, 7.84s/it]
78%|███████▊ | 2151/2757 [4:54:42<1:20:12, 7.94s/it]
78%|███████▊ | 2152/2757 [4:54:49<1:19:31, 7.89s/it]
78%|███████▊ | 2153/2757 [4:54:58<1:20:33, 8.00s/it]
78%|███████▊ | 2154/2757 [4:55:05<1:19:42, 7.93s/it]
78%|███████▊ | 2155/2757 [4:55:14<1:20:50, 8.06s/it]
78%|███████▊ | 2156/2757 [4:55:21<1:19:16, 7.91s/it]
78%|███████▊ | 2157/2757 [4:55:30<1:20:44, 8.07s/it]
78%|███████▊ | 2158/2757 [4:55:38<1:21:00, 8.11s/it]
78%|███████▊ | 2159/2757 [4:55:46<1:20:15, 8.05s/it]
78%|███████▊ | 2160/2757 [4:55:54<1:20:42, 8.11s/it]
78%|███████▊ | 2160/2757 [4:55:54<1:20:42, 8.11s/it]
78%|███████▊ | 2161/2757 [4:56:02<1:19:43, 8.03s/it]
78%|███████▊ | 2162/2757 [4:56:09<1:17:32, 7.82s/it]
78%|███████▊ | 2163/2757 [4:56:17<1:18:05, 7.89s/it]
78%|███████▊ | 2164/2757 [4:56:25<1:18:23, 7.93s/it]
79%|███████▊ | 2165/2757 [4:56:33<1:18:33, 7.96s/it]
79%|███████▊ | 2166/2757 [4:56:41<1:18:43, 7.99s/it]
79%|███████▊ | 2167/2757 [4:56:50<1:19:22, 8.07s/it]
79%|███████▊ | 2168/2757 [4:56:58<1:18:50, 8.03s/it]
79%|███████▊ | 2169/2757 [4:57:06<1:19:22, 8.10s/it]
79%|███████▊ | 2170/2757 [4:57:14<1:19:06, 8.09s/it]
79%|███████▊ | 2170/2757 [4:57:14<1:19:06, 8.09s/it]
79%|███████▊ | 2171/2757 [4:57:22<1:17:26, 7.93s/it]
79%|███████▉ | 2172/2757 [4:57:29<1:17:10, 7.92s/it]
79%|███████▉ | 2173/2757 [4:57:37<1:17:19, 7.94s/it]
79%|███████▉ | 2174/2757 [4:57:45<1:17:15, 7.95s/it]
79%|███████▉ | 2175/2757 [4:57:54<1:17:36, 8.00s/it]
79%|███████▉ | 2176/2757 [4:58:02<1:18:31, 8.11s/it]
79%|███████▉ | 2177/2757 [4:58:10<1:17:17, 8.00s/it]
79%|███████▉ | 2178/2757 [4:58:17<1:16:41, 7.95s/it]
79%|███████▉ | 2179/2757 [4:58:25<1:16:30, 7.94s/it]
79%|███████▉ | 2180/2757 [4:58:33<1:16:24, 7.95s/it]
79%|███████▉ | 2180/2757 [4:58:33<1:16:24, 7.95s/it]
79%|███████▉ | 2181/2757 [4:58:41<1:16:24, 7.96s/it]
79%|███████▉ | 2182/2757 [4:58:49<1:15:31, 7.88s/it]
79%|███████▉ | 2183/2757 [4:58:57<1:15:36, 7.90s/it]
79%|███████▉ | 2184/2757 [4:59:05<1:14:41, 7.82s/it]
79%|███████▉ | 2185/2757 [4:59:12<1:14:44, 7.84s/it]
79%|███████▉ | 2186/2757 [4:59:21<1:15:06, 7.89s/it]
79%|███████▉ | 2187/2757 [4:59:28<1:14:44, 7.87s/it]
79%|███████▉ | 2188/2757 [4:59:36<1:15:22, 7.95s/it]
79%|███████▉ | 2189/2757 [4:59:45<1:15:42, 8.00s/it]
79%|███████▉ | 2190/2757 [4:59:53<1:16:37, 8.11s/it]
79%|███████▉ | 2190/2757 [4:59:53<1:16:37, 8.11s/it]
79%|███████▉ | 2191/2757 [5:00:01<1:17:15, 8.19s/it]
80%|███████▉ | 2192/2757 [5:00:09<1:16:08, 8.09s/it]
80%|███████▉ | 2193/2757 [5:00:17<1:15:57, 8.08s/it]
80%|███████▉ | 2194/2757 [5:00:25<1:14:20, 7.92s/it]
80%|███████▉ | 2195/2757 [5:00:33<1:14:51, 7.99s/it]
80%|███████▉ | 2196/2757 [5:00:41<1:13:53, 7.90s/it]
80%|███████▉ | 2197/2757 [5:00:48<1:12:26, 7.76s/it]
80%|███████▉ | 2198/2757 [5:00:56<1:11:58, 7.73s/it]
80%|███████▉ | 2199/2757 [5:01:04<1:13:18, 7.88s/it]
80%|███████▉ | 2200/2757 [5:01:12<1:14:14, 8.00s/it]
80%|███████▉ | 2200/2757 [5:01:12<1:14:14, 8.00s/it]
80%|███████▉ | 2201/2757 [5:01:20<1:14:17, 8.02s/it]
80%|███████▉ | 2202/2757 [5:01:28<1:13:41, 7.97s/it]
80%|███████▉ | 2203/2757 [5:01:36<1:13:42, 7.98s/it]
80%|███████▉ | 2204/2757 [5:01:44<1:13:17, 7.95s/it]
80%|███████▉ | 2205/2757 [5:01:52<1:13:31, 7.99s/it]
80%|████████ | 2206/2757 [5:02:00<1:13:11, 7.97s/it]
80%|████████ | 2207/2757 [5:02:07<1:11:31, 7.80s/it]
80%|████████ | 2208/2757 [5:02:15<1:12:03, 7.88s/it]
80%|████████ | 2209/2757 [5:02:23<1:12:16, 7.91s/it]
80%|████████ | 2210/2757 [5:02:31<1:12:14, 7.92s/it]
80%|████████ | 2210/2757 [5:02:31<1:12:14, 7.92s/it]
80%|████████ | 2211/2757 [5:02:40<1:12:32, 7.97s/it]
80%|████████ | 2212/2757 [5:02:48<1:13:27, 8.09s/it]
80%|████████ | 2213/2757 [5:02:56<1:12:39, 8.01s/it]
80%|████████ | 2214/2757 [5:03:04<1:12:31, 8.01s/it]
80%|████████ | 2215/2757 [5:03:11<1:11:04, 7.87s/it]
80%|████████ | 2216/2757 [5:03:19<1:11:23, 7.92s/it]
80%|████████ | 2217/2757 [5:03:27<1:11:13, 7.91s/it]
80%|████████ | 2218/2757 [5:03:35<1:10:38, 7.86s/it]
80%|████████ | 2219/2757 [5:03:43<1:09:58, 7.80s/it]
81%|████████ | 2220/2757 [5:03:51<1:10:13, 7.85s/it]
81%|████████ | 2220/2757 [5:03:51<1:10:13, 7.85s/it]
81%|████████ | 2221/2757 [5:03:59<1:11:15, 7.98s/it]
81%|████████ | 2222/2757 [5:04:07<1:11:11, 7.98s/it]
81%|████████ | 2223/2757 [5:04:14<1:09:54, 7.85s/it]
81%|████████ | 2224/2757 [5:04:22<1:10:24, 7.93s/it]
81%|████████ | 2225/2757 [5:04:30<1:09:48, 7.87s/it]
81%|████████ | 2226/2757 [5:04:39<1:11:20, 8.06s/it]
81%|████████ | 2227/2757 [5:04:46<1:09:31, 7.87s/it]
81%|████████ | 2228/2757 [5:04:54<1:10:10, 7.96s/it]
81%|████████ | 2229/2757 [5:05:02<1:08:40, 7.80s/it]
81%|████████ | 2230/2757 [5:05:10<1:09:17, 7.89s/it]
81%|████████ | 2230/2757 [5:05:10<1:09:17, 7.89s/it]
81%|████████ | 2231/2757 [5:05:18<1:09:15, 7.90s/it]
81%|████████ | 2232/2757 [5:05:26<1:10:00, 8.00s/it]
81%|████████ | 2233/2757 [5:05:34<1:09:49, 7.99s/it]
81%|████████ | 2234/2757 [5:05:42<1:09:22, 7.96s/it]
81%|████████ | 2235/2757 [5:05:50<1:08:30, 7.87s/it]
81%|████████ | 2236/2757 [5:05:57<1:08:12, 7.86s/it]
81%|████████ | 2237/2757 [5:06:06<1:09:01, 7.96s/it]
81%|████████ | 2238/2757 [5:06:14<1:08:51, 7.96s/it]
81%|████████ | 2239/2757 [5:06:22<1:09:11, 8.02s/it]
81%|████████ | 2240/2757 [5:06:30<1:09:41, 8.09s/it]
81%|████████ | 2240/2757 [5:06:30<1:09:41, 8.09s/it]
81%|████████▏ | 2241/2757 [5:06:38<1:08:56, 8.02s/it]
81%|████████▏ | 2242/2757 [5:06:46<1:08:38, 8.00s/it]
81%|████████▏ | 2243/2757 [5:06:53<1:07:24, 7.87s/it]
81%|████████▏ | 2244/2757 [5:07:01<1:06:20, 7.76s/it]
81%|████████▏ | 2245/2757 [5:07:09<1:06:17, 7.77s/it]
81%|████████▏ | 2246/2757 [5:07:16<1:06:23, 7.79s/it]
82%|████████▏ | 2247/2757 [5:07:25<1:07:44, 7.97s/it]
82%|████████▏ | 2248/2757 [5:07:33<1:07:41, 7.98s/it]
82%|████████▏ | 2249/2757 [5:07:41<1:07:59, 8.03s/it]
82%|████████▏ | 2250/2757 [5:07:49<1:08:09, 8.07s/it]
{'eval_loss': '0.4653', 'eval_runtime': '104.8', 'eval_samples_per_second': '22.91', 'eval_steps_per_second': '2.863', 'eval_rewards/chosen': '-3.413', 'eval_rewards/rejected': '-4.97', 'eval_rewards/accuracies': '0.7125', 'eval_rewards/margins': '1.556', 'eval_logps/chosen': '-113.7', 'eval_logps/rejected': '-135.5', 'eval_logits/chosen': '-4.393', 'eval_logits/rejected': '-4.345', 'epoch': '2.176'}
{'loss': '0.2672', 'grad_norm': '0.64', 'learning_rate': '5.595e-05', 'rewards/chosen': '-3.084', 'rewards/rejected': '-5.276', 'rewards/accuracies': '0.8438', 'rewards/margins': '2.193', 'logps/chosen': '-115.2', 'logps/rejected': '-140.9', 'logits/chosen': '-4.258', 'logits/rejected': '-4.195', 'epoch': '2.187'}
{'loss': '0.2488', 'grad_norm': '0.5022', 'learning_rate': '5.52e-05', 'rewards/chosen': '-2.958', 'rewards/rejected': '-5.24', 'rewards/accuracies': '0.8687', 'rewards/margins': '2.282', 'logps/chosen': '-108.5', 'logps/rejected': '-137.5', 'logits/chosen': '-4.378', 'logits/rejected': '-4.255', 'epoch': '2.198'}
{'loss': '0.2526', 'grad_norm': '0.4892', 'learning_rate': '5.445e-05', 'rewards/chosen': '-2.979', 'rewards/rejected': '-5.242', 'rewards/accuracies': '0.8508', 'rewards/margins': '2.262', 'logps/chosen': '-108.2', 'logps/rejected': '-137.7', 'logits/chosen': '-4.429', 'logits/rejected': '-4.317', 'epoch': '2.209'}
{'loss': '0.2479', 'grad_norm': '0.5125', 'learning_rate': '5.37e-05', 'rewards/chosen': '-2.978', 'rewards/rejected': '-5.298', 'rewards/accuracies': '0.8516', 'rewards/margins': '2.321', 'logps/chosen': '-110.8', 'logps/rejected': '-140.5', 'logits/chosen': '-4.358', 'logits/rejected': '-4.336', 'epoch': '2.22'}
{'loss': '0.2748', 'grad_norm': '0.5904', 'learning_rate': '5.295e-05', 'rewards/chosen': '-2.839', 'rewards/rejected': '-4.998', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.159', 'logps/chosen': '-108.7', 'logps/rejected': '-136.3', 'logits/chosen': '-4.396', 'logits/rejected': '-4.374', 'epoch': '2.231'}
{'loss': '0.2631', 'grad_norm': '0.5153', 'learning_rate': '5.221e-05', 'rewards/chosen': '-2.878', 'rewards/rejected': '-5.143', 'rewards/accuracies': '0.8406', 'rewards/margins': '2.264', 'logps/chosen': '-108.4', 'logps/rejected': '-137.3', 'logits/chosen': '-4.383', 'logits/rejected': '-4.335', 'epoch': '2.242'}
{'loss': '0.2632', 'grad_norm': '0.5087', 'learning_rate': '5.146e-05', 'rewards/chosen': '-2.793', 'rewards/rejected': '-4.991', 'rewards/accuracies': '0.8516', 'rewards/margins': '2.198', 'logps/chosen': '-107.8', 'logps/rejected': '-135.7', 'logits/chosen': '-4.283', 'logits/rejected': '-4.199', 'epoch': '2.253'}
{'loss': '0.2556', 'grad_norm': '0.5399', 'learning_rate': '5.071e-05', 'rewards/chosen': '-2.866', 'rewards/rejected': '-5.147', 'rewards/accuracies': '0.857', 'rewards/margins': '2.281', 'logps/chosen': '-106.8', 'logps/rejected': '-136.2', 'logits/chosen': '-4.345', 'logits/rejected': '-4.317', 'epoch': '2.263'}
{'loss': '0.2625', 'grad_norm': '0.4645', 'learning_rate': '4.996e-05', 'rewards/chosen': '-2.911', 'rewards/rejected': '-5.203', 'rewards/accuracies': '0.8477', 'rewards/margins': '2.292', 'logps/chosen': '-106.5', 'logps/rejected': '-136.2', 'logits/chosen': '-4.257', 'logits/rejected': '-4.259', 'epoch': '2.274'}
{'loss': '0.2609', 'grad_norm': '0.3866', 'learning_rate': '4.921e-05', 'rewards/chosen': '-2.991', 'rewards/rejected': '-5.275', 'rewards/accuracies': '0.8461', 'rewards/margins': '2.285', 'logps/chosen': '-111.6', 'logps/rejected': '-139.2', 'logits/chosen': '-4.299', 'logits/rejected': '-4.223', 'epoch': '2.285'}
{'loss': '0.2628', 'grad_norm': '0.5244', 'learning_rate': '4.847e-05', 'rewards/chosen': '-2.894', 'rewards/rejected': '-5.102', 'rewards/accuracies': '0.8578', 'rewards/margins': '2.207', 'logps/chosen': '-105.7', 'logps/rejected': '-132.3', 'logits/chosen': '-4.305', 'logits/rejected': '-4.244', 'epoch': '2.296'}
{'loss': '0.2798', 'grad_norm': '0.5264', 'learning_rate': '4.772e-05', 'rewards/chosen': '-2.842', 'rewards/rejected': '-4.989', 'rewards/accuracies': '0.8211', 'rewards/margins': '2.147', 'logps/chosen': '-107.4', 'logps/rejected': '-135.3', 'logits/chosen': '-4.359', 'logits/rejected': '-4.288', 'epoch': '2.307'}
{'loss': '0.2712', 'grad_norm': '0.619', 'learning_rate': '4.697e-05', 'rewards/chosen': '-2.925', 'rewards/rejected': '-5.061', 'rewards/accuracies': '0.8414', 'rewards/margins': '2.136', 'logps/chosen': '-109.6', 'logps/rejected': '-136.1', 'logits/chosen': '-4.337', 'logits/rejected': '-4.261', 'epoch': '2.318'}
{'loss': '0.2454', 'grad_norm': '0.4767', 'learning_rate': '4.622e-05', 'rewards/chosen': '-2.863', 'rewards/rejected': '-5.176', 'rewards/accuracies': '0.8625', 'rewards/margins': '2.314', 'logps/chosen': '-110.6', 'logps/rejected': '-139.7', 'logits/chosen': '-4.277', 'logits/rejected': '-4.208', 'epoch': '2.329'}
{'loss': '0.2712', 'grad_norm': '0.4791', 'learning_rate': '4.547e-05', 'rewards/chosen': '-2.719', 'rewards/rejected': '-4.824', 'rewards/accuracies': '0.8414', 'rewards/margins': '2.106', 'logps/chosen': '-105.8', 'logps/rejected': '-131.3', 'logits/chosen': '-4.355', 'logits/rejected': '-4.288', 'epoch': '2.34'}
{'loss': '0.2594', 'grad_norm': '0.6803', 'learning_rate': '4.473e-05', 'rewards/chosen': '-2.693', 'rewards/rejected': '-4.999', 'rewards/accuracies': '0.8469', 'rewards/margins': '2.306', 'logps/chosen': '-105.4', 'logps/rejected': '-134.7', 'logits/chosen': '-4.417', 'logits/rejected': '-4.399', 'epoch': '2.35'}
{'loss': '0.2457', 'grad_norm': '0.5753', 'learning_rate': '4.398e-05', 'rewards/chosen': '-2.723', 'rewards/rejected': '-5.072', 'rewards/accuracies': '0.8555', 'rewards/margins': '2.349', 'logps/chosen': '-108.5', 'logps/rejected': '-138', 'logits/chosen': '-4.324', 'logits/rejected': '-4.302', 'epoch': '2.361'}
{'loss': '0.2549', 'grad_norm': '0.5278', 'learning_rate': '4.323e-05', 'rewards/chosen': '-2.749', 'rewards/rejected': '-5.012', 'rewards/accuracies': '0.8547', 'rewards/margins': '2.263', 'logps/chosen': '-109.8', 'logps/rejected': '-138.1', 'logits/chosen': '-4.221', 'logits/rejected': '-4.199', 'epoch': '2.372'}
{'loss': '0.2544', 'grad_norm': '0.6214', 'learning_rate': '4.248e-05', 'rewards/chosen': '-2.788', 'rewards/rejected': '-5.146', 'rewards/accuracies': '0.8523', 'rewards/margins': '2.358', 'logps/chosen': '-109.3', 'logps/rejected': '-137.8', 'logits/chosen': '-4.419', 'logits/rejected': '-4.302', 'epoch': '2.383'}
{'loss': '0.2642', 'grad_norm': '0.6792', 'learning_rate': '4.174e-05', 'rewards/chosen': '-2.842', 'rewards/rejected': '-5.078', 'rewards/accuracies': '0.8438', 'rewards/margins': '2.235', 'logps/chosen': '-107.8', 'logps/rejected': '-135.4', 'logits/chosen': '-4.336', 'logits/rejected': '-4.239', 'epoch': '2.394'}
{'loss': '0.254', 'grad_norm': '0.5071', 'learning_rate': '4.099e-05', 'rewards/chosen': '-2.94', 'rewards/rejected': '-5.256', 'rewards/accuracies': '0.85', 'rewards/margins': '2.316', 'logps/chosen': '-108.2', 'logps/rejected': '-135.4', 'logits/chosen': '-4.275', 'logits/rejected': '-4.161', 'epoch': '2.405'}
{'loss': '0.2531', 'grad_norm': '0.5598', 'learning_rate': '4.024e-05', 'rewards/chosen': '-2.943', 'rewards/rejected': '-5.273', 'rewards/accuracies': '0.8453', 'rewards/margins': '2.33', 'logps/chosen': '-111.8', 'logps/rejected': '-139.2', 'logits/chosen': '-4.343', 'logits/rejected': '-4.282', 'epoch': '2.416'}
{'loss': '0.2632', 'grad_norm': '0.4813', 'learning_rate': '3.949e-05', 'rewards/chosen': '-2.964', 'rewards/rejected': '-5.288', 'rewards/accuracies': '0.8422', 'rewards/margins': '2.324', 'logps/chosen': '-108.8', 'logps/rejected': '-136.8', 'logits/chosen': '-4.238', 'logits/rejected': '-4.215', 'epoch': '2.427'}
{'loss': '0.2669', 'grad_norm': '0.5383', 'learning_rate': '3.874e-05', 'rewards/chosen': '-3.067', 'rewards/rejected': '-5.306', 'rewards/accuracies': '0.8453', 'rewards/margins': '2.239', 'logps/chosen': '-113.3', 'logps/rejected': '-140.9', 'logits/chosen': '-4.169', 'logits/rejected': '-4.063', 'epoch': '2.438'}
{'loss': '0.2641', 'grad_norm': '0.4986', 'learning_rate': '3.8e-05', 'rewards/chosen': '-2.952', 'rewards/rejected': '-5.159', 'rewards/accuracies': '0.8508', 'rewards/margins': '2.207', 'logps/chosen': '-110.6', 'logps/rejected': '-138.2', 'logits/chosen': '-4.118', 'logits/rejected': '-4.037', 'epoch': '2.448'}
82%|████████▏ | 2250/2757 [5:07:49<1:08:09, 8.07s/it]
82%|████████▏ | 2251/2757 [5:08:00<1:15:07, 8.91s/it]
82%|████████▏ | 2252/2757 [5:08:08<1:12:37, 8.63s/it]
82%|████████▏ | 2253/2757 [5:08:16<1:10:38, 8.41s/it]
82%|████████▏ | 2254/2757 [5:08:24<1:09:14, 8.26s/it]
82%|████████▏ | 2255/2757 [5:08:32<1:09:02, 8.25s/it]
82%|████████▏ | 2256/2757 [5:08:40<1:08:28, 8.20s/it]
82%|████████▏ | 2257/2757 [5:08:48<1:06:47, 8.02s/it]
82%|████████▏ | 2258/2757 [5:08:55<1:06:07, 7.95s/it]
82%|████████▏ | 2259/2757 [5:09:04<1:06:13, 7.98s/it]
82%|████████▏ | 2260/2757 [5:09:12<1:06:55, 8.08s/it]
82%|████████▏ | 2260/2757 [5:09:12<1:06:55, 8.08s/it]
82%|████████▏ | 2261/2757 [5:09:20<1:06:34, 8.05s/it]
82%|████████▏ | 2262/2757 [5:09:27<1:05:07, 7.89s/it]
82%|████████▏ | 2263/2757 [5:09:36<1:06:13, 8.04s/it]
82%|████████▏ | 2264/2757 [5:09:44<1:06:15, 8.06s/it]
82%|████████▏ | 2265/2757 [5:09:52<1:05:48, 8.03s/it]
82%|████████▏ | 2266/2757 [5:10:00<1:05:27, 8.00s/it]
82%|████████▏ | 2267/2757 [5:10:08<1:04:50, 7.94s/it]
82%|████████▏ | 2268/2757 [5:10:15<1:04:26, 7.91s/it]
82%|████████▏ | 2269/2757 [5:10:23<1:04:13, 7.90s/it]
82%|████████▏ | 2270/2757 [5:10:31<1:02:34, 7.71s/it]
82%|████████▏ | 2270/2757 [5:10:31<1:02:34, 7.71s/it]
82%|████████▏ | 2271/2757 [5:10:38<1:01:38, 7.61s/it]
82%|████████▏ | 2272/2757 [5:10:45<1:01:13, 7.57s/it]
82%|████████▏ | 2273/2757 [5:10:54<1:02:48, 7.79s/it]
82%|████████▏ | 2274/2757 [5:11:02<1:04:28, 8.01s/it]
83%|████████▎ | 2275/2757 [5:11:10<1:04:45, 8.06s/it]
83%|████████▎ | 2276/2757 [5:11:18<1:04:19, 8.02s/it]
83%|████████▎ | 2277/2757 [5:11:26<1:04:27, 8.06s/it]
83%|████████▎ | 2278/2757 [5:11:35<1:04:59, 8.14s/it]
83%|████████▎ | 2279/2757 [5:11:43<1:04:21, 8.08s/it]
83%|████████▎ | 2280/2757 [5:11:51<1:04:12, 8.08s/it]
83%|████████▎ | 2280/2757 [5:11:51<1:04:12, 8.08s/it]
83%|████████▎ | 2281/2757 [5:11:59<1:03:54, 8.05s/it]
83%|████████▎ | 2282/2757 [5:12:07<1:03:00, 7.96s/it]
83%|████████▎ | 2283/2757 [5:12:15<1:02:54, 7.96s/it]
83%|████████▎ | 2284/2757 [5:12:22<1:02:50, 7.97s/it]
83%|████████▎ | 2285/2757 [5:12:30<1:02:05, 7.89s/it]
83%|████████▎ | 2286/2757 [5:12:38<1:02:25, 7.95s/it]
83%|████████▎ | 2287/2757 [5:12:46<1:01:28, 7.85s/it]
83%|████████▎ | 2288/2757 [5:12:54<1:01:03, 7.81s/it]
83%|████████▎ | 2289/2757 [5:13:02<1:01:49, 7.93s/it]
83%|████████▎ | 2290/2757 [5:13:10<1:01:44, 7.93s/it]
83%|████████▎ | 2290/2757 [5:13:10<1:01:44, 7.93s/it]
83%|████████▎ | 2291/2757 [5:13:18<1:02:49, 8.09s/it]
83%|████████▎ | 2292/2757 [5:13:26<1:02:01, 8.00s/it]
83%|████████▎ | 2293/2757 [5:13:34<1:02:34, 8.09s/it]
83%|████████▎ | 2294/2757 [5:13:42<1:02:07, 8.05s/it]
83%|████████▎ | 2295/2757 [5:13:50<1:01:44, 8.02s/it]
83%|████████▎ | 2296/2757 [5:13:58<1:01:28, 8.00s/it]
83%|████████▎ | 2297/2757 [5:14:06<1:00:39, 7.91s/it]
83%|████████▎ | 2298/2757 [5:14:14<1:00:32, 7.91s/it]
83%|████████▎ | 2299/2757 [5:14:22<1:00:18, 7.90s/it]
83%|████████▎ | 2300/2757 [5:14:30<1:00:44, 7.98s/it]
83%|████████▎ | 2300/2757 [5:14:30<1:00:44, 7.98s/it]
83%|████████▎ | 2301/2757 [5:14:38<1:00:50, 8.00s/it]
83%|████████▎ | 2302/2757 [5:14:46<1:00:56, 8.04s/it]
84%|████████▎ | 2303/2757 [5:14:54<1:00:58, 8.06s/it]
84%|████████▎ | 2304/2757 [5:15:02<1:00:06, 7.96s/it]
84%|████████▎ | 2305/2757 [5:15:10<1:00:21, 8.01s/it]
84%|████████▎ | 2306/2757 [5:15:18<1:00:55, 8.11s/it]
84%|████████▎ | 2307/2757 [5:15:26<1:00:22, 8.05s/it]
84%|████████▎ | 2308/2757 [5:15:34<1:00:12, 8.05s/it]
84%|████████▍ | 2309/2757 [5:15:42<59:19, 7.94s/it]
84%|████████▍ | 2310/2757 [5:15:50<59:38, 8.00s/it]
84%|████████▍ | 2310/2757 [5:15:50<59:38, 8.00s/it]
84%|████████▍ | 2311/2757 [5:15:58<59:18, 7.98s/it]
84%|████████▍ | 2312/2757 [5:16:06<59:43, 8.05s/it]
84%|████████▍ | 2313/2757 [5:16:14<58:56, 7.97s/it]
84%|████████▍ | 2314/2757 [5:16:22<59:10, 8.01s/it]
84%|████████▍ | 2315/2757 [5:16:30<59:25, 8.07s/it]
84%|████████▍ | 2316/2757 [5:16:39<1:00:09, 8.18s/it]
84%|████████▍ | 2317/2757 [5:16:47<59:12, 8.07s/it]
84%|████████▍ | 2318/2757 [5:16:55<59:04, 8.07s/it]
84%|████████▍ | 2319/2757 [5:17:03<58:51, 8.06s/it]
84%|████████▍ | 2320/2757 [5:17:11<58:18, 8.01s/it]
84%|████████▍ | 2320/2757 [5:17:11<58:18, 8.01s/it]
84%|████████▍ | 2321/2757 [5:17:18<56:56, 7.84s/it]
84%|████████▍ | 2322/2757 [5:17:26<57:04, 7.87s/it]
84%|████████▍ | 2323/2757 [5:17:33<55:55, 7.73s/it]
84%|████████▍ | 2324/2757 [5:17:42<56:48, 7.87s/it]
84%|████████▍ | 2325/2757 [5:17:49<56:01, 7.78s/it]
84%|████████▍ | 2326/2757 [5:17:58<57:42, 8.03s/it]
84%|████████▍ | 2327/2757 [5:18:06<58:59, 8.23s/it]
84%|████████▍ | 2328/2757 [5:18:15<58:33, 8.19s/it]
84%|████████▍ | 2329/2757 [5:18:23<57:53, 8.12s/it]
85%|████████▍ | 2330/2757 [5:18:31<58:26, 8.21s/it]
85%|████████▍ | 2330/2757 [5:18:31<58:26, 8.21s/it]
85%|████████▍ | 2331/2757 [5:18:39<58:04, 8.18s/it]
85%|████████▍ | 2332/2757 [5:18:47<57:11, 8.08s/it]
85%|████████▍ | 2333/2757 [5:18:55<56:35, 8.01s/it]
85%|████████▍ | 2334/2757 [5:19:03<56:24, 8.00s/it]
85%|████████▍ | 2335/2757 [5:19:11<57:07, 8.12s/it]
85%|████████▍ | 2336/2757 [5:19:19<57:09, 8.15s/it]
85%|████████▍ | 2337/2757 [5:19:27<56:07, 8.02s/it]
85%|████████▍ | 2338/2757 [5:19:35<56:16, 8.06s/it]
85%|████████▍ | 2339/2757 [5:19:43<56:15, 8.08s/it]
85%|████████▍ | 2340/2757 [5:19:52<57:09, 8.22s/it]
85%|████████▍ | 2340/2757 [5:19:52<57:09, 8.22s/it]
85%|████████▍ | 2341/2757 [5:20:00<56:27, 8.14s/it]
85%|████████▍ | 2342/2757 [5:20:08<56:07, 8.11s/it]
85%|████████▍ | 2343/2757 [5:20:16<57:01, 8.26s/it]
85%|████████▌ | 2344/2757 [5:20:24<55:44, 8.10s/it]
85%|████████▌ | 2345/2757 [5:20:32<54:21, 7.92s/it]
85%|████████▌ | 2346/2757 [5:20:40<54:25, 7.95s/it]
85%|████████▌ | 2347/2757 [5:20:48<54:26, 7.97s/it]
85%|████████▌ | 2348/2757 [5:20:56<54:53, 8.05s/it]
85%|████████▌ | 2349/2757 [5:21:04<55:09, 8.11s/it]
85%|████████▌ | 2350/2757 [5:21:12<55:10, 8.13s/it]
85%|████████▌ | 2350/2757 [5:21:12<55:10, 8.13s/it]
85%|████████▌ | 2351/2757 [5:21:20<54:05, 7.99s/it]
85%|████████▌ | 2352/2757 [5:21:28<54:03, 8.01s/it]
85%|████████▌ | 2353/2757 [5:21:37<55:39, 8.27s/it]
85%|████████▌ | 2354/2757 [5:21:45<54:18, 8.09s/it]
85%|████████▌ | 2355/2757 [5:21:53<53:53, 8.04s/it]
85%|████████▌ | 2356/2757 [5:22:01<53:50, 8.06s/it]
85%|████████▌ | 2357/2757 [5:22:09<53:38, 8.05s/it]
86%|████████▌ | 2358/2757 [5:22:17<53:01, 7.97s/it]
86%|████████▌ | 2359/2757 [5:22:24<52:45, 7.95s/it]
86%|████████▌ | 2360/2757 [5:22:32<52:49, 7.98s/it]
86%|████████▌ | 2360/2757 [5:22:32<52:49, 7.98s/it]
86%|████████▌ | 2361/2757 [5:22:40<52:27, 7.95s/it]
86%|████████▌ | 2362/2757 [5:22:48<52:05, 7.91s/it]
86%|████████▌ | 2363/2757 [5:22:56<52:16, 7.96s/it]
86%|████████▌ | 2364/2757 [5:23:04<52:09, 7.96s/it]
86%|████████▌ | 2365/2757 [5:23:13<53:17, 8.16s/it]
86%|████████▌ | 2366/2757 [5:23:21<52:43, 8.09s/it]
86%|████████▌ | 2367/2757 [5:23:30<53:55, 8.29s/it]
86%|████████▌ | 2368/2757 [5:23:38<53:08, 8.20s/it]
86%|████████▌ | 2369/2757 [5:23:45<52:20, 8.09s/it]
86%|████████▌ | 2370/2757 [5:23:54<52:37, 8.16s/it]
86%|████████▌ | 2370/2757 [5:23:54<52:37, 8.16s/it]
86%|████████▌ | 2371/2757 [5:24:02<52:00, 8.08s/it]
86%|████████▌ | 2372/2757 [5:24:10<52:03, 8.11s/it]
86%|████████▌ | 2373/2757 [5:24:18<51:37, 8.07s/it]
86%|████████▌ | 2374/2757 [5:24:25<50:48, 7.96s/it]
86%|████████▌ | 2375/2757 [5:24:34<51:07, 8.03s/it]
86%|████████▌ | 2376/2757 [5:24:42<51:39, 8.13s/it]
86%|████████▌ | 2377/2757 [5:24:50<51:37, 8.15s/it]
86%|████████▋ | 2378/2757 [5:24:58<51:36, 8.17s/it]
86%|████████▋ | 2379/2757 [5:25:06<51:11, 8.13s/it]
86%|████████▋ | 2380/2757 [5:25:15<51:23, 8.18s/it]
86%|████████▋ | 2380/2757 [5:25:15<51:23, 8.18s/it]
86%|████████▋ | 2381/2757 [5:25:23<50:38, 8.08s/it]
86%|████████▋ | 2382/2757 [5:25:31<50:55, 8.15s/it]
86%|████████▋ | 2383/2757 [5:25:39<50:47, 8.15s/it]
86%|████████▋ | 2384/2757 [5:25:48<51:34, 8.30s/it]
87%|████████▋ | 2385/2757 [5:25:56<51:56, 8.38s/it]
87%|████████▋ | 2386/2757 [5:26:05<51:45, 8.37s/it]
87%|████████▋ | 2387/2757 [5:26:12<50:07, 8.13s/it]
87%|████████▋ | 2388/2757 [5:26:21<50:58, 8.29s/it]
87%|████████▋ | 2389/2757 [5:26:28<49:12, 8.02s/it]
87%|████████▋ | 2390/2757 [5:26:36<48:01, 7.85s/it]
87%|████████▋ | 2390/2757 [5:26:36<48:01, 7.85s/it]
87%|████████▋ | 2391/2757 [5:26:44<48:19, 7.92s/it]
87%|████████▋ | 2392/2757 [5:26:52<48:42, 8.01s/it]
87%|████████▋ | 2393/2757 [5:27:00<48:36, 8.01s/it]
87%|████████▋ | 2394/2757 [5:27:08<48:28, 8.01s/it]
87%|████████▋ | 2395/2757 [5:27:16<48:15, 8.00s/it]
87%|████████▋ | 2396/2757 [5:27:24<48:48, 8.11s/it]
87%|████████▋ | 2397/2757 [5:27:32<48:23, 8.07s/it]
87%|████████▋ | 2398/2757 [5:27:40<47:19, 7.91s/it]
87%|████████▋ | 2399/2757 [5:27:48<46:46, 7.84s/it]
87%|████████▋ | 2400/2757 [5:27:55<46:37, 7.83s/it]
87%|████████▋ | 2400/2757 [5:27:55<46:37, 7.83s/it]
87%|████████▋ | 2401/2757 [5:28:04<47:15, 7.96s/it]
87%|████████▋ | 2402/2757 [5:28:11<46:24, 7.84s/it]
87%|████████▋ | 2403/2757 [5:28:19<46:49, 7.94s/it]
87%|████████▋ | 2404/2757 [5:28:28<47:08, 8.01s/it]
87%|████████▋ | 2405/2757 [5:28:35<46:48, 7.98s/it]
87%|████████▋ | 2406/2757 [5:28:43<46:28, 7.95s/it]
87%|████████▋ | 2407/2757 [5:28:51<46:40, 8.00s/it]
87%|████████▋ | 2408/2757 [5:29:00<46:43, 8.03s/it]
87%|████████▋ | 2409/2757 [5:29:07<45:43, 7.88s/it]
87%|████████▋ | 2410/2757 [5:29:15<45:31, 7.87s/it]
87%|████████▋ | 2410/2757 [5:29:15<45:31, 7.87s/it]
87%|████████▋ | 2411/2757 [5:29:23<45:35, 7.91s/it]
87%|████████▋ | 2412/2757 [5:29:31<45:11, 7.86s/it]
88%|████████▊ | 2413/2757 [5:29:39<45:41, 7.97s/it]
88%|████████▊ | 2414/2757 [5:29:47<45:28, 7.95s/it]
88%|████████▊ | 2415/2757 [5:29:55<45:43, 8.02s/it]
88%|████████▊ | 2416/2757 [5:30:03<45:53, 8.07s/it]
88%|████████▊ | 2417/2757 [5:30:11<45:52, 8.10s/it]
88%|████████▊ | 2418/2757 [5:30:20<46:10, 8.17s/it]
88%|████████▊ | 2419/2757 [5:30:27<45:15, 8.03s/it]
88%|████████▊ | 2420/2757 [5:30:35<45:02, 8.02s/it]
88%|████████▊ | 2420/2757 [5:30:35<45:02, 8.02s/it]
88%|████████▊ | 2421/2757 [5:30:44<45:06, 8.06s/it]
88%|████████▊ | 2422/2757 [5:30:51<44:36, 7.99s/it]
88%|████████▊ | 2423/2757 [5:30:59<43:50, 7.88s/it]
88%|████████▊ | 2424/2757 [5:31:07<43:45, 7.88s/it]
88%|████████▊ | 2425/2757 [5:31:14<42:57, 7.76s/it]
88%|████████▊ | 2426/2757 [5:31:22<42:43, 7.75s/it]
88%|████████▊ | 2427/2757 [5:31:30<43:04, 7.83s/it]
88%|████████▊ | 2428/2757 [5:31:38<42:34, 7.76s/it]
88%|████████▊ | 2429/2757 [5:31:46<42:32, 7.78s/it]
88%|████████▊ | 2430/2757 [5:31:54<42:51, 7.86s/it]
88%|████████▊ | 2430/2757 [5:31:54<42:51, 7.86s/it]
88%|████████▊ | 2431/2757 [5:32:01<42:06, 7.75s/it]
88%|████████▊ | 2432/2757 [5:32:09<41:54, 7.74s/it]
88%|████████▊ | 2433/2757 [5:32:16<41:20, 7.66s/it]
88%|████████▊ | 2434/2757 [5:32:24<42:01, 7.81s/it]
88%|████████▊ | 2435/2757 [5:32:32<41:54, 7.81s/it]
88%|████████▊ | 2436/2757 [5:32:40<42:00, 7.85s/it]
88%|████████▊ | 2437/2757 [5:32:48<42:08, 7.90s/it]
88%|████████▊ | 2438/2757 [5:32:56<42:19, 7.96s/it]
88%|████████▊ | 2439/2757 [5:33:04<42:34, 8.03s/it]
89%|████████▊ | 2440/2757 [5:33:12<42:19, 8.01s/it]
89%|████████▊ | 2440/2757 [5:33:12<42:19, 8.01s/it]
89%|████████▊ | 2441/2757 [5:33:21<42:40, 8.10s/it]
89%|████████▊ | 2442/2757 [5:33:29<42:25, 8.08s/it]
89%|████████▊ | 2443/2757 [5:33:36<40:59, 7.83s/it]
89%|████████▊ | 2444/2757 [5:33:44<41:23, 7.94s/it]
89%|████████▊ | 2445/2757 [5:33:53<41:56, 8.06s/it]
89%|████████▊ | 2446/2757 [5:34:00<40:56, 7.90s/it]
89%|████████▉ | 2447/2757 [5:34:08<41:04, 7.95s/it]
89%|████████▉ | 2448/2757 [5:34:16<40:28, 7.86s/it]
89%|████████▉ | 2449/2757 [5:34:24<40:50, 7.96s/it]
89%|████████▉ | 2450/2757 [5:34:31<39:56, 7.81s/it]
89%|████████▉ | 2450/2757 [5:34:31<39:56, 7.81s/it]
89%|████████▉ | 2451/2757 [5:34:39<39:58, 7.84s/it]
89%|████████▉ | 2452/2757 [5:34:47<40:07, 7.90s/it]
89%|████████▉ | 2453/2757 [5:34:55<40:17, 7.95s/it]
89%|████████▉ | 2454/2757 [5:35:03<39:53, 7.90s/it]
89%|████████▉ | 2455/2757 [5:35:11<39:43, 7.89s/it]
89%|████████▉ | 2456/2757 [5:35:19<39:26, 7.86s/it]
89%|████████▉ | 2457/2757 [5:35:27<39:20, 7.87s/it]
89%|████████▉ | 2458/2757 [5:35:35<39:57, 8.02s/it]
89%|████████▉ | 2459/2757 [5:35:43<39:13, 7.90s/it]
89%|████████▉ | 2460/2757 [5:35:51<39:20, 7.95s/it]
89%|████████▉ | 2460/2757 [5:35:51<39:20, 7.95s/it]
89%|████████▉ | 2461/2757 [5:35:59<39:32, 8.02s/it]
89%|████████▉ | 2462/2757 [5:36:07<39:28, 8.03s/it]
89%|████████▉ | 2463/2757 [5:36:15<39:19, 8.03s/it]
89%|████████▉ | 2464/2757 [5:36:23<39:03, 8.00s/it]
89%|████████▉ | 2465/2757 [5:36:31<38:58, 8.01s/it]
89%|████████▉ | 2466/2757 [5:36:39<38:50, 8.01s/it]
89%|████████▉ | 2467/2757 [5:36:47<38:49, 8.03s/it]
90%|████████▉ | 2468/2757 [5:36:56<39:30, 8.20s/it]
90%|████████▉ | 2469/2757 [5:37:04<38:45, 8.08s/it]
90%|████████▉ | 2470/2757 [5:37:12<39:26, 8.25s/it]
90%|████████▉ | 2470/2757 [5:37:12<39:26, 8.25s/it]
90%|████████▉ | 2471/2757 [5:37:20<38:59, 8.18s/it]
90%|████████▉ | 2472/2757 [5:37:28<38:53, 8.19s/it]
90%|████████▉ | 2473/2757 [5:37:37<38:46, 8.19s/it]
90%|████████▉ | 2474/2757 [5:37:44<37:45, 8.01s/it]
90%|████████▉ | 2475/2757 [5:37:52<37:02, 7.88s/it]
90%|████████▉ | 2476/2757 [5:38:00<37:32, 8.01s/it]
90%|████████▉ | 2477/2757 [5:38:08<37:14, 7.98s/it]
90%|████████▉ | 2478/2757 [5:38:16<37:01, 7.96s/it]
90%|████████▉ | 2479/2757 [5:38:24<36:23, 7.86s/it]
90%|████████▉ | 2480/2757 [5:38:32<36:28, 7.90s/it]
90%|████████▉ | 2480/2757 [5:38:32<36:28, 7.90s/it]
90%|████████▉ | 2481/2757 [5:38:39<36:25, 7.92s/it]
90%|█████████ | 2482/2757 [5:38:47<36:21, 7.93s/it]
90%|█████████ | 2483/2757 [5:38:55<35:33, 7.79s/it]
90%|█████████ | 2484/2757 [5:39:03<35:18, 7.76s/it]
90%|█████████ | 2485/2757 [5:39:11<35:27, 7.82s/it]
90%|█████████ | 2486/2757 [5:39:19<36:05, 7.99s/it]
90%|█████████ | 2487/2757 [5:39:27<36:41, 8.15s/it]
90%|█████████ | 2488/2757 [5:39:36<36:45, 8.20s/it]
90%|█████████ | 2489/2757 [5:39:44<36:13, 8.11s/it]
90%|█████████ | 2490/2757 [5:39:52<35:47, 8.04s/it]
90%|█████████ | 2490/2757 [5:39:52<35:47, 8.04s/it]
90%|█████████ | 2491/2757 [5:39:59<35:26, 7.99s/it]
90%|█████████ | 2492/2757 [5:40:07<34:59, 7.92s/it]
90%|█████████ | 2493/2757 [5:40:15<34:39, 7.88s/it]
90%|█████████ | 2494/2757 [5:40:23<34:43, 7.92s/it]
90%|█████████ | 2495/2757 [5:40:32<35:21, 8.10s/it]
91%|█████████ | 2496/2757 [5:40:40<35:05, 8.07s/it]
91%|█████████ | 2497/2757 [5:40:47<34:46, 8.02s/it]
91%|█████████ | 2498/2757 [5:40:55<33:56, 7.86s/it]
91%|█████████ | 2499/2757 [5:41:03<34:22, 7.99s/it]
91%|█████████ | 2500/2757 [5:41:11<33:59, 7.94s/it]
91%|█████████ | 2500/2757 [5:41:11<33:59, 7.94s/it]
{'loss': '0.2519', 'grad_norm': '0.5483', 'learning_rate': '3.725e-05', 'rewards/chosen': '-3.012', 'rewards/rejected': '-5.305', 'rewards/accuracies': '0.857', 'rewards/margins': '2.293', 'logps/chosen': '-112.8', 'logps/rejected': '-140.4', 'logits/chosen': '-4.164', 'logits/rejected': '-4.179', 'epoch': '2.459'}
{'loss': '0.2518', 'grad_norm': '0.4968', 'learning_rate': '3.65e-05', 'rewards/chosen': '-3.082', 'rewards/rejected': '-5.411', 'rewards/accuracies': '0.8484', 'rewards/margins': '2.329', 'logps/chosen': '-110.1', 'logps/rejected': '-137.6', 'logits/chosen': '-4.339', 'logits/rejected': '-4.249', 'epoch': '2.47'}
{'loss': '0.2498', 'grad_norm': '0.5726', 'learning_rate': '3.575e-05', 'rewards/chosen': '-3.102', 'rewards/rejected': '-5.512', 'rewards/accuracies': '0.8492', 'rewards/margins': '2.41', 'logps/chosen': '-113.6', 'logps/rejected': '-142.6', 'logits/chosen': '-4.268', 'logits/rejected': '-4.18', 'epoch': '2.481'}
{'loss': '0.26', 'grad_norm': '0.5567', 'learning_rate': '3.5e-05', 'rewards/chosen': '-2.999', 'rewards/rejected': '-5.299', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.3', 'logps/chosen': '-107.5', 'logps/rejected': '-135.8', 'logits/chosen': '-4.303', 'logits/rejected': '-4.221', 'epoch': '2.492'}
{'loss': '0.2627', 'grad_norm': '0.4611', 'learning_rate': '3.426e-05', 'rewards/chosen': '-3.027', 'rewards/rejected': '-5.354', 'rewards/accuracies': '0.8414', 'rewards/margins': '2.327', 'logps/chosen': '-109.5', 'logps/rejected': '-138.5', 'logits/chosen': '-4.257', 'logits/rejected': '-4.285', 'epoch': '2.503'}
{'loss': '0.2664', 'grad_norm': '0.5548', 'learning_rate': '3.351e-05', 'rewards/chosen': '-3.159', 'rewards/rejected': '-5.442', 'rewards/accuracies': '0.8406', 'rewards/margins': '2.282', 'logps/chosen': '-112.9', 'logps/rejected': '-140.6', 'logits/chosen': '-4.131', 'logits/rejected': '-4.09', 'epoch': '2.514'}
{'loss': '0.2697', 'grad_norm': '0.6786', 'learning_rate': '3.276e-05', 'rewards/chosen': '-3.175', 'rewards/rejected': '-5.44', 'rewards/accuracies': '0.8383', 'rewards/margins': '2.265', 'logps/chosen': '-113.3', 'logps/rejected': '-140.3', 'logits/chosen': '-4.26', 'logits/rejected': '-4.25', 'epoch': '2.525'}
{'loss': '0.2606', 'grad_norm': '0.5422', 'learning_rate': '3.201e-05', 'rewards/chosen': '-3.227', 'rewards/rejected': '-5.541', 'rewards/accuracies': '0.843', 'rewards/margins': '2.314', 'logps/chosen': '-112.1', 'logps/rejected': '-142.1', 'logits/chosen': '-4.233', 'logits/rejected': '-4.16', 'epoch': '2.536'}
{'loss': '0.2648', 'grad_norm': '0.6207', 'learning_rate': '3.126e-05', 'rewards/chosen': '-3.262', 'rewards/rejected': '-5.532', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.27', 'logps/chosen': '-110.7', 'logps/rejected': '-138.4', 'logits/chosen': '-4.249', 'logits/rejected': '-4.177', 'epoch': '2.546'}
{'loss': '0.2524', 'grad_norm': '0.6249', 'learning_rate': '3.052e-05', 'rewards/chosen': '-3.352', 'rewards/rejected': '-5.683', 'rewards/accuracies': '0.8562', 'rewards/margins': '2.332', 'logps/chosen': '-115.2', 'logps/rejected': '-144.4', 'logits/chosen': '-4.308', 'logits/rejected': '-4.223', 'epoch': '2.557'}
{'loss': '0.2689', 'grad_norm': '0.6504', 'learning_rate': '2.977e-05', 'rewards/chosen': '-3.239', 'rewards/rejected': '-5.564', 'rewards/accuracies': '0.8398', 'rewards/margins': '2.325', 'logps/chosen': '-111.2', 'logps/rejected': '-141.9', 'logits/chosen': '-4.297', 'logits/rejected': '-4.28', 'epoch': '2.568'}
{'loss': '0.2643', 'grad_norm': '0.6845', 'learning_rate': '2.902e-05', 'rewards/chosen': '-3.097', 'rewards/rejected': '-5.37', 'rewards/accuracies': '0.8336', 'rewards/margins': '2.273', 'logps/chosen': '-111', 'logps/rejected': '-138.2', 'logits/chosen': '-4.262', 'logits/rejected': '-4.167', 'epoch': '2.579'}
{'loss': '0.2551', 'grad_norm': '0.4879', 'learning_rate': '2.827e-05', 'rewards/chosen': '-2.942', 'rewards/rejected': '-5.293', 'rewards/accuracies': '0.8484', 'rewards/margins': '2.351', 'logps/chosen': '-106.5', 'logps/rejected': '-137.2', 'logits/chosen': '-4.409', 'logits/rejected': '-4.352', 'epoch': '2.59'}
{'loss': '0.2424', 'grad_norm': '0.5652', 'learning_rate': '2.752e-05', 'rewards/chosen': '-3.074', 'rewards/rejected': '-5.492', 'rewards/accuracies': '0.8578', 'rewards/margins': '2.419', 'logps/chosen': '-108.9', 'logps/rejected': '-137.6', 'logits/chosen': '-4.418', 'logits/rejected': '-4.38', 'epoch': '2.601'}
{'loss': '0.2535', 'grad_norm': '0.5238', 'learning_rate': '2.678e-05', 'rewards/chosen': '-3.111', 'rewards/rejected': '-5.516', 'rewards/accuracies': '0.8562', 'rewards/margins': '2.405', 'logps/chosen': '-112.7', 'logps/rejected': '-142.6', 'logits/chosen': '-4.35', 'logits/rejected': '-4.247', 'epoch': '2.612'}
{'loss': '0.26', 'grad_norm': '0.519', 'learning_rate': '2.603e-05', 'rewards/chosen': '-3.063', 'rewards/rejected': '-5.384', 'rewards/accuracies': '0.8383', 'rewards/margins': '2.322', 'logps/chosen': '-109.9', 'logps/rejected': '-137.5', 'logits/chosen': '-4.415', 'logits/rejected': '-4.263', 'epoch': '2.623'}
{'loss': '0.2537', 'grad_norm': '0.5785', 'learning_rate': '2.528e-05', 'rewards/chosen': '-3.048', 'rewards/rejected': '-5.36', 'rewards/accuracies': '0.8523', 'rewards/margins': '2.312', 'logps/chosen': '-111.9', 'logps/rejected': '-139.8', 'logits/chosen': '-4.251', 'logits/rejected': '-4.197', 'epoch': '2.633'}
{'loss': '0.2483', 'grad_norm': '0.5177', 'learning_rate': '2.453e-05', 'rewards/chosen': '-3.117', 'rewards/rejected': '-5.505', 'rewards/accuracies': '0.8422', 'rewards/margins': '2.388', 'logps/chosen': '-114.2', 'logps/rejected': '-142.7', 'logits/chosen': '-4.407', 'logits/rejected': '-4.32', 'epoch': '2.644'}
{'loss': '0.2676', 'grad_norm': '0.6798', 'learning_rate': '2.378e-05', 'rewards/chosen': '-2.953', 'rewards/rejected': '-5.227', 'rewards/accuracies': '0.8391', 'rewards/margins': '2.275', 'logps/chosen': '-109.7', 'logps/rejected': '-138.4', 'logits/chosen': '-4.313', 'logits/rejected': '-4.273', 'epoch': '2.655'}
{'loss': '0.2508', 'grad_norm': '0.6013', 'learning_rate': '2.304e-05', 'rewards/chosen': '-2.888', 'rewards/rejected': '-5.234', 'rewards/accuracies': '0.8633', 'rewards/margins': '2.347', 'logps/chosen': '-110.6', 'logps/rejected': '-139.8', 'logits/chosen': '-4.369', 'logits/rejected': '-4.297', 'epoch': '2.666'}
{'loss': '0.2731', 'grad_norm': '0.5708', 'learning_rate': '2.229e-05', 'rewards/chosen': '-2.885', 'rewards/rejected': '-5.138', 'rewards/accuracies': '0.8281', 'rewards/margins': '2.253', 'logps/chosen': '-107.8', 'logps/rejected': '-134.8', 'logits/chosen': '-4.391', 'logits/rejected': '-4.356', 'epoch': '2.677'}
{'loss': '0.2608', 'grad_norm': '0.4749', 'learning_rate': '2.154e-05', 'rewards/chosen': '-2.873', 'rewards/rejected': '-5.198', 'rewards/accuracies': '0.8562', 'rewards/margins': '2.325', 'logps/chosen': '-108.3', 'logps/rejected': '-135.5', 'logits/chosen': '-4.335', 'logits/rejected': '-4.284', 'epoch': '2.688'}
{'loss': '0.2541', 'grad_norm': '0.5339', 'learning_rate': '2.079e-05', 'rewards/chosen': '-2.983', 'rewards/rejected': '-5.276', 'rewards/accuracies': '0.8625', 'rewards/margins': '2.292', 'logps/chosen': '-109.3', 'logps/rejected': '-137.2', 'logits/chosen': '-4.364', 'logits/rejected': '-4.272', 'epoch': '2.699'}
{'loss': '0.258', 'grad_norm': '0.533', 'learning_rate': '2.004e-05', 'rewards/chosen': '-3.023', 'rewards/rejected': '-5.346', 'rewards/accuracies': '0.8586', 'rewards/margins': '2.323', 'logps/chosen': '-109.9', 'logps/rejected': '-138.8', 'logits/chosen': '-4.273', 'logits/rejected': '-4.253', 'epoch': '2.71'}
{'loss': '0.2491', 'grad_norm': '0.4829', 'learning_rate': '1.93e-05', 'rewards/chosen': '-3.075', 'rewards/rejected': '-5.44', 'rewards/accuracies': '0.8445', 'rewards/margins': '2.365', 'logps/chosen': '-114.9', 'logps/rejected': '-143.9', 'logits/chosen': '-4.346', 'logits/rejected': '-4.261', 'epoch': '2.721'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
0%| | 0/300 [00:00<?, ?it/s]
1%| | 2/300 [00:00<00:52, 5.72it/s]
1%| | 3/300 [00:00<01:46, 2.79it/s]
1%|▏ | 4/300 [00:01<01:34, 3.12it/s]
2%|▏ | 5/300 [00:01<01:44, 2.83it/s]
2%|▏ | 6/300 [00:01<01:40, 2.93it/s]
2%|▏ | 7/300 [00:02<01:55, 2.55it/s]
3%|▎ | 8/300 [00:02<01:50, 2.64it/s]
3%|▎ | 9/300 [00:03<01:52, 2.60it/s]
3%|▎ | 10/300 [00:03<01:48, 2.67it/s]
4%|▎ | 11/300 [00:03<01:51, 2.59it/s]
4%|▍ | 12/300 [00:04<01:38, 2.93it/s]
4%|▍ | 13/300 [00:04<01:43, 2.78it/s]
5%|▍ | 14/300 [00:04<01:35, 2.98it/s]
5%|▌ | 15/300 [00:05<01:45, 2.69it/s]
5%|▌ | 16/300 [00:05<01:34, 3.01it/s]
6%|▌ | 17/300 [00:05<01:38, 2.87it/s]
6%|▌ | 18/300 [00:06<01:29, 3.17it/s]
6%|▋ | 19/300 [00:06<01:39, 2.82it/s]
7%|▋ | 20/300 [00:06<01:31, 3.05it/s]
7%|▋ | 21/300 [00:07<01:43, 2.70it/s]
7%|▋ | 22/300 [00:07<01:31, 3.02it/s]
8%|▊ | 23/300 [00:07<01:32, 2.99it/s]
8%|▊ | 24/300 [00:08<01:23, 3.30it/s]
8%|▊ | 25/300 [00:08<01:35, 2.87it/s]
9%|▊ | 26/300 [00:08<01:29, 3.06it/s]
9%|▉ | 27/300 [00:09<01:38, 2.78it/s]
9%|▉ | 28/300 [00:09<01:27, 3.12it/s]
10%|▉ | 29/300 [00:09<01:30, 3.00it/s]
10%|█ | 30/300 [00:10<01:21, 3.30it/s]
10%|█ | 31/300 [00:10<01:33, 2.86it/s]
11%|█ | 32/300 [00:10<01:24, 3.17it/s]
11%|█ | 33/300 [00:11<01:25, 3.12it/s]
11%|█▏ | 34/300 [00:11<01:19, 3.35it/s]
12%|█▏ | 35/300 [00:11<01:30, 2.93it/s]
12%|█▏ | 36/300 [00:12<01:23, 3.14it/s]
12%|█▏ | 37/300 [00:12<01:32, 2.84it/s]
13%|█▎ | 38/300 [00:12<01:21, 3.20it/s]
13%|█▎ | 39/300 [00:13<01:28, 2.95it/s]
13%|█▎ | 40/300 [00:13<01:20, 3.22it/s]
14%|█▎ | 41/300 [00:13<01:27, 2.97it/s]
14%|█▍ | 42/300 [00:14<01:19, 3.26it/s]
14%|█▍ | 43/300 [00:14<01:23, 3.09it/s]
15%|█▍ | 44/300 [00:14<01:16, 3.36it/s]
15%|█▌ | 45/300 [00:15<01:23, 3.07it/s]
15%|█▌ | 46/300 [00:15<01:24, 3.02it/s]
16%|█▌ | 47/300 [00:15<01:29, 2.82it/s]
16%|█▌ | 48/300 [00:16<01:22, 3.05it/s]
16%|█▋ | 49/300 [00:16<01:23, 3.01it/s]
17%|█▋ | 50/300 [00:16<01:14, 3.35it/s]
17%|█▋ | 51/300 [00:17<01:18, 3.19it/s]
17%|█▋ | 52/300 [00:17<01:15, 3.30it/s]
18%|█▊ | 53/300 [00:17<01:28, 2.80it/s]
18%|█▊ | 54/300 [00:18<01:19, 3.10it/s]
18%|█▊ | 55/300 [00:18<01:26, 2.82it/s]
19%|█▊ | 56/300 [00:18<01:15, 3.23it/s]
19%|█▉ | 57/300 [00:19<01:19, 3.04it/s]
19%|█▉ | 58/300 [00:19<01:21, 2.98it/s]
20%|█▉ | 59/300 [00:19<01:26, 2.79it/s]
20%|██ | 60/300 [00:20<01:17, 3.10it/s]
20%|██ | 61/300 [00:20<01:21, 2.95it/s]
21%|██ | 62/300 [00:20<01:20, 2.94it/s]
21%|██ | 63/300 [00:21<01:25, 2.76it/s]
21%|██▏ | 64/300 [00:21<01:23, 2.82it/s]
22%|██▏ | 65/300 [00:21<01:27, 2.70it/s]
22%|██▏ | 66/300 [00:22<01:19, 2.95it/s]
22%|██▏ | 67/300 [00:22<01:26, 2.69it/s]
23%|██▎ | 68/300 [00:22<01:18, 2.97it/s]
23%|██▎ | 69/300 [00:23<01:21, 2.84it/s]
23%|██▎ | 70/300 [00:23<01:13, 3.15it/s]
24%|██▎ | 71/300 [00:23<01:15, 3.03it/s]
24%|██▍ | 72/300 [00:24<01:09, 3.29it/s]
24%|██▍ | 73/300 [00:24<01:12, 3.13it/s]
25%|██▍ | 74/300 [00:24<01:08, 3.29it/s]
25%|██▌ | 75/300 [00:25<01:13, 3.06it/s]
25%|██▌ | 76/300 [00:25<01:14, 3.00it/s]
26%|██▌ | 77/300 [00:25<01:23, 2.67it/s]
26%|██▌ | 78/300 [00:26<01:15, 2.93it/s]
26%|██▋ | 79/300 [00:26<01:16, 2.87it/s]
27%|██▋ | 80/300 [00:26<01:12, 3.05it/s]
27%|██▋ | 81/300 [00:27<01:19, 2.74it/s]
27%|██▋ | 82/300 [00:27<01:11, 3.05it/s]
28%|██▊ | 83/300 [00:27<01:14, 2.92it/s]
28%|██▊ | 84/300 [00:28<01:14, 2.90it/s]
28%|██▊ | 85/300 [00:28<01:18, 2.72it/s]
29%|██▊ | 86/300 [00:28<01:10, 3.03it/s]
29%|██▉ | 87/300 [00:29<01:12, 2.93it/s]
29%|██▉ | 88/300 [00:29<01:06, 3.18it/s]
30%|██▉ | 89/300 [00:29<01:10, 2.97it/s]
30%|███ | 90/300 [00:30<01:05, 3.21it/s]
30%|███ | 91/300 [00:30<01:09, 3.02it/s]
31%|███ | 92/300 [00:30<01:03, 3.28it/s]
31%|███ | 93/300 [00:31<01:09, 3.00it/s]
31%|███▏ | 94/300 [00:31<01:02, 3.30it/s]
32%|███▏ | 95/300 [00:31<01:07, 3.02it/s]
32%|███▏ | 96/300 [00:32<01:03, 3.21it/s]
32%|███▏ | 97/300 [00:32<01:05, 3.08it/s]
33%|███▎ | 98/300 [00:32<01:02, 3.25it/s]
33%|███▎ | 99/300 [00:33<01:07, 2.99it/s]
33%|███▎ | 100/300 [00:33<01:03, 3.14it/s]
34%|███▎ | 101/300 [00:33<01:07, 2.95it/s]
34%|███▍ | 102/300 [00:34<01:05, 3.01it/s]
34%|███▍ | 103/300 [00:34<01:09, 2.82it/s]
35%|███▍ | 104/300 [00:34<01:02, 3.14it/s]
35%|███▌ | 105/300 [00:35<01:05, 2.98it/s]
35%|███▌ | 106/300 [00:35<01:00, 3.18it/s]
36%|███▌ | 107/300 [00:35<01:05, 2.96it/s]
36%|███▌ | 108/300 [00:36<01:02, 3.06it/s]
36%|███▋ | 109/300 [00:36<01:06, 2.88it/s]
37%|███▋ | 110/300 [00:36<01:02, 3.03it/s]
37%|███▋ | 111/300 [00:37<01:11, 2.65it/s]
37%|███▋ | 112/300 [00:37<01:05, 2.85it/s]
38%|███▊ | 113/300 [00:37<01:07, 2.77it/s]
38%|███▊ | 114/300 [00:38<01:06, 2.79it/s]
38%|███▊ | 115/300 [00:38<01:09, 2.65it/s]
39%|███▊ | 116/300 [00:38<01:01, 2.98it/s]
39%|███▉ | 117/300 [00:39<01:03, 2.90it/s]
39%|███▉ | 118/300 [00:39<00:58, 3.13it/s]
40%|███▉ | 119/300 [00:39<01:00, 3.00it/s]
40%|████ | 120/300 [00:40<00:59, 3.03it/s]
40%|████ | 121/300 [00:40<01:01, 2.89it/s]
41%|████ | 122/300 [00:40<00:57, 3.08it/s]
41%|████ | 123/300 [00:41<01:01, 2.87it/s]
41%|████▏ | 124/300 [00:41<00:57, 3.04it/s]
42%|████▏ | 125/300 [00:41<00:59, 2.94it/s]
42%|████▏ | 126/300 [00:42<00:59, 2.91it/s]
42%|████▏ | 127/300 [00:42<01:01, 2.79it/s]
43%|████▎ | 128/300 [00:42<00:57, 2.99it/s]
43%|████▎ | 129/300 [00:43<00:59, 2.87it/s]
43%|████▎ | 130/300 [00:43<00:53, 3.15it/s]
44%|████▎ | 131/300 [00:44<01:00, 2.78it/s]
44%|████▍ | 132/300 [00:44<00:55, 3.03it/s]
44%|████▍ | 133/300 [00:44<01:05, 2.54it/s]
45%|████▍ | 134/300 [00:45<00:58, 2.81it/s]
45%|████▌ | 135/300 [00:45<01:00, 2.74it/s]
45%|████▌ | 136/300 [00:45<00:53, 3.04it/s]
46%|████▌ | 137/300 [00:46<00:54, 3.01it/s]
46%|████▌ | 138/300 [00:46<00:50, 3.23it/s]
46%|████▋ | 139/300 [00:46<00:50, 3.17it/s]
47%|████▋ | 140/300 [00:46<00:46, 3.43it/s]
47%|████▋ | 141/300 [00:47<00:48, 3.26it/s]
47%|████▋ | 142/300 [00:47<00:50, 3.12it/s]
48%|████▊ | 143/300 [00:48<00:53, 2.93it/s]
48%|████▊ | 144/300 [00:48<00:48, 3.23it/s]
48%|████▊ | 145/300 [00:48<00:48, 3.17it/s]
49%|████▊ | 146/300 [00:48<00:46, 3.32it/s]
49%|████▉ | 147/300 [00:49<00:53, 2.87it/s]
49%|████▉ | 148/300 [00:49<00:50, 3.04it/s]
50%|████▉ | 149/300 [00:50<00:53, 2.81it/s]
50%|█████ | 150/300 [00:50<00:49, 3.01it/s]
50%|█████ | 151/300 [00:50<00:51, 2.89it/s]
51%|█████ | 152/300 [00:50<00:46, 3.20it/s]
51%|█████ | 153/300 [00:51<00:47, 3.08it/s]
51%|█████▏ | 154/300 [00:51<00:43, 3.35it/s]
52%|█████▏ | 155/300 [00:51<00:46, 3.14it/s]
52%|█████▏ | 156/300 [00:52<00:41, 3.47it/s]
52%|█████▏ | 157/300 [00:52<00:46, 3.09it/s]
53%|█████▎ | 158/300 [00:52<00:43, 3.24it/s]
53%|█████▎ | 159/300 [00:53<00:50, 2.78it/s]
53%|█████▎ | 160/300 [00:53<00:46, 3.01it/s]
54%|█████▎ | 161/300 [00:54<00:54, 2.54it/s]
54%|█████▍ | 162/300 [00:54<00:52, 2.63it/s]
54%|█████▍ | 163/300 [00:54<00:57, 2.38it/s]
55%|█████▍ | 164/300 [00:55<00:54, 2.50it/s]
55%|█████▌ | 165/300 [00:55<00:53, 2.51it/s]
55%|█████▌ | 166/300 [00:55<00:51, 2.63it/s]
56%|█████▌ | 167/300 [00:56<00:54, 2.42it/s]
56%|█████▌ | 168/300 [00:56<00:48, 2.74it/s]
56%|█████▋ | 169/300 [00:57<00:53, 2.47it/s]
57%|█████▋ | 170/300 [00:57<00:50, 2.57it/s]
57%|█████▋ | 171/300 [00:57<00:50, 2.53it/s]
57%|█████▋ | 172/300 [00:58<00:44, 2.87it/s]
58%|█████▊ | 173/300 [00:58<00:46, 2.71it/s]
58%|█████▊ | 174/300 [00:58<00:41, 3.04it/s]
58%|█████▊ | 175/300 [00:59<00:44, 2.78it/s]
59%|█████▊ | 176/300 [00:59<00:39, 3.11it/s]
59%|█████▉ | 177/300 [00:59<00:44, 2.77it/s]
59%|█████▉ | 178/300 [01:00<00:38, 3.20it/s]
60%|█████▉ | 179/300 [01:00<00:41, 2.94it/s]
60%|██████ | 180/300 [01:00<00:37, 3.21it/s]
60%|██████ | 181/300 [01:01<00:41, 2.88it/s]
61%|██████ | 182/300 [01:01<00:37, 3.19it/s]
61%|██████ | 183/300 [01:01<00:40, 2.86it/s]
61%|██████▏ | 184/300 [01:02<00:40, 2.86it/s]
62%|██████▏ | 185/300 [01:02<00:42, 2.68it/s]
62%|██████▏ | 186/300 [01:03<00:39, 2.88it/s]
62%|██████▏ | 187/300 [01:03<00:39, 2.86it/s]
63%|██████▎ | 188/300 [01:03<00:35, 3.14it/s]
63%|██████▎ | 189/300 [01:04<00:38, 2.85it/s]
63%|██████▎ | 190/300 [01:04<00:35, 3.09it/s]
64%|██████▎ | 191/300 [01:04<00:37, 2.94it/s]
64%|██████▍ | 192/300 [01:05<00:36, 2.97it/s]
64%|██████▍ | 193/300 [01:05<00:41, 2.57it/s]
65%|██████▍ | 194/300 [01:05<00:40, 2.65it/s]
65%|██████▌ | 195/300 [01:06<00:41, 2.55it/s]
65%|██████▌ | 196/300 [01:06<00:37, 2.77it/s]
66%|██████▌ | 197/300 [01:07<00:39, 2.62it/s]
66%|██████▌ | 198/300 [01:07<00:35, 2.87it/s]
66%|██████▋ | 199/300 [01:07<00:36, 2.78it/s]
67%|██████▋ | 200/300 [01:07<00:33, 2.98it/s]
67%|██████▋ | 201/300 [01:08<00:35, 2.78it/s]
67%|██████▋ | 202/300 [01:08<00:31, 3.07it/s]
68%|██████▊ | 203/300 [01:08<00:33, 2.93it/s]
68%|██████▊ | 204/300 [01:09<00:29, 3.22it/s]
68%|██████▊ | 205/300 [01:09<00:31, 2.99it/s]
69%|██████▊ | 206/300 [01:09<00:27, 3.39it/s]
69%|██████▉ | 207/300 [01:10<00:32, 2.88it/s]
69%|██████▉ | 208/300 [01:10<00:31, 2.91it/s]
70%|██████▉ | 209/300 [01:11<00:32, 2.77it/s]
70%|███████ | 210/300 [01:11<00:30, 2.94it/s]
70%|███████ | 211/300 [01:11<00:33, 2.62it/s]
71%|███████ | 212/300 [01:12<00:29, 2.94it/s]
71%|███████ | 213/300 [01:12<00:34, 2.53it/s]
71%|███████▏ | 214/300 [01:12<00:30, 2.82it/s]
72%|███████▏ | 215/300 [01:13<00:31, 2.68it/s]
72%|███████▏ | 216/300 [01:13<00:28, 2.99it/s]
72%|███████▏ | 217/300 [01:13<00:32, 2.58it/s]
73%|███████▎ | 218/300 [01:14<00:27, 3.02it/s]
73%|███████▎ | 219/300 [01:14<00:30, 2.63it/s]
73%|███████▎ | 220/300 [01:14<00:27, 2.95it/s]
74%|███████▎ | 221/300 [01:15<00:29, 2.69it/s]
74%|███████▍ | 222/300 [01:15<00:25, 3.00it/s]
74%|███████▍ | 223/300 [01:16<00:28, 2.70it/s]
75%|███████▍ | 224/300 [01:16<00:26, 2.92it/s]
75%|███████▌ | 225/300 [01:16<00:26, 2.79it/s]
75%|███████▌ | 226/300 [01:17<00:24, 3.03it/s]
76%|███████▌ | 227/300 [01:17<00:26, 2.78it/s]
76%|███████▌ | 228/300 [01:17<00:23, 3.11it/s]
76%|███████▋ | 229/300 [01:18<00:25, 2.75it/s]
77%|███████▋ | 230/300 [01:18<00:24, 2.85it/s]
77%|███████▋ | 231/300 [01:18<00:25, 2.70it/s]
77%|███████▋ | 232/300 [01:19<00:24, 2.77it/s]
78%|███████▊ | 233/300 [01:19<00:27, 2.46it/s]
78%|███████▊ | 234/300 [01:19<00:23, 2.82it/s]
78%|███████▊ | 235/300 [01:20<00:24, 2.64it/s]
79%|███████▊ | 236/300 [01:20<00:23, 2.70it/s]
79%|███████▉ | 237/300 [01:21<00:24, 2.60it/s]
79%|███████▉ | 238/300 [01:21<00:21, 2.91it/s]
80%|███████▉ | 239/300 [01:21<00:21, 2.88it/s]
80%|████████ | 240/300 [01:22<00:18, 3.20it/s]
80%|████████ | 241/300 [01:22<00:19, 3.02it/s]
81%|████████ | 242/300 [01:22<00:18, 3.06it/s]
81%|████████ | 243/300 [01:23<00:20, 2.72it/s]
81%|████████▏ | 244/300 [01:23<00:19, 2.82it/s]
82%|████████▏ | 245/300 [01:23<00:20, 2.67it/s]
82%|████████▏ | 246/300 [01:24<00:19, 2.76it/s]
82%|████████▏ | 247/300 [01:24<00:20, 2.59it/s]
83%|████████▎ | 248/300 [01:24<00:17, 2.93it/s]
83%|████████▎ | 249/300 [01:25<00:18, 2.80it/s]
83%|████████▎ | 250/300 [01:25<00:17, 2.92it/s]
84%|████████▎ | 251/300 [01:26<00:18, 2.59it/s]
84%|████████▍ | 252/300 [01:26<00:16, 2.83it/s]
84%|████████▍ | 253/300 [01:26<00:17, 2.74it/s]
85%|████████▍ | 254/300 [01:26<00:14, 3.12it/s]
85%|████████▌ | 255/300 [01:27<00:16, 2.75it/s]
85%|████████▌ | 256/300 [01:27<00:14, 3.10it/s]
86%|████████▌ | 257/300 [01:28<00:14, 2.93it/s]
86%|████████▌ | 258/300 [01:28<00:13, 3.20it/s]
86%|████████▋ | 259/300 [01:28<00:13, 2.97it/s]
87%|████████▋ | 260/300 [01:28<00:12, 3.16it/s]
87%|████████▋ | 261/300 [01:29<00:13, 2.99it/s]
87%|████████▋ | 262/300 [01:29<00:11, 3.26it/s]
88%|████████▊ | 263/300 [01:29<00:12, 3.00it/s]
88%|████████▊ | 264/300 [01:30<00:11, 3.00it/s]
88%|████████▊ | 265/300 [01:30<00:13, 2.59it/s]
89%|████████▊ | 266/300 [01:31<00:12, 2.69it/s]
89%|████████▉ | 267/300 [01:31<00:13, 2.43it/s]
89%|████████▉ | 268/300 [01:31<00:11, 2.79it/s]
90%|████████▉ | 269/300 [01:32<00:12, 2.47it/s]
90%|█████████ | 270/300 [01:32<00:11, 2.65it/s]
90%|█████████ | 271/300 [01:33<00:12, 2.39it/s]
91%|█████████ | 272/300 [01:33<00:10, 2.65it/s]
91%|█████████ | 273/300 [01:34<00:11, 2.39it/s]
91%|█████████▏| 274/300 [01:34<00:09, 2.73it/s]
92%|█████████▏| 275/300 [01:34<00:10, 2.45it/s]
92%|█████████▏| 276/300 [01:35<00:09, 2.63it/s]
92%|█████████▏| 277/300 [01:35<00:09, 2.53it/s]
93%|█████████▎| 278/300 [01:35<00:07, 2.85it/s]
93%|█████████▎| 279/300 [01:36<00:07, 2.81it/s]
93%|█████████▎| 280/300 [01:36<00:06, 2.90it/s]
94%|█████████▎| 281/300 [01:36<00:06, 2.72it/s]
94%|█████████▍| 282/300 [01:37<00:05, 3.03it/s]
94%|█████████▍| 283/300 [01:37<00:06, 2.80it/s]
95%|█████████▍| 284/300 [01:37<00:05, 3.06it/s]
95%|█████████▌| 285/300 [01:38<00:05, 2.65it/s]
95%|█████████▌| 286/300 [01:38<00:05, 2.71it/s]
96%|█████████▌| 287/300 [01:39<00:04, 2.62it/s]
96%|█████████▌| 288/300 [01:39<00:03, 3.04it/s]
96%|█████████▋| 289/300 [01:39<00:03, 2.79it/s]
97%|█████████▋| 290/300 [01:40<00:03, 2.81it/s]
97%|█████████▋| 291/300 [01:40<00:03, 2.61it/s]
97%|█████████▋| 292/300 [01:40<00:02, 2.93it/s]
98%|█████████▊| 293/300 [01:41<00:02, 2.67it/s]
98%|█████████▊| 294/300 [01:41<00:02, 2.91it/s]
98%|█████████▊| 295/300 [01:41<00:01, 2.56it/s]
99%|█████████▊| 296/300 [01:42<00:01, 2.83it/s]
99%|█████████▉| 297/300 [01:42<00:01, 2.72it/s]
99%|█████████▉| 298/300 [01:42<00:00, 3.11it/s]
100%|█████████▉| 299/300 [01:43<00:00, 2.77it/s]
100%|██████████| 300/300 [01:43<00:00, 2.83it/s]

91%|█████████ | 2500/2757 [5:42:56<33:59, 7.94s/it]
100%|██████████| 300/300 [01:43<00:00, 2.83it/s]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-2500/tokenizer_config.json.
91%|█████████ | 2501/2757 [5:43:08<2:53:55, 40.76s/it]
91%|█████████ | 2502/2757 [5:43:17<2:12:09, 31.09s/it]
91%|█████████ | 2503/2757 [5:43:25<1:41:53, 24.07s/it]
91%|█████████ | 2504/2757 [5:43:32<1:20:55, 19.19s/it]
91%|█████████ | 2505/2757 [5:43:41<1:06:48, 15.91s/it]
91%|█████████ | 2506/2757 [5:43:48<56:19, 13.46s/it]
91%|█████████ | 2507/2757 [5:43:56<49:16, 11.82s/it]
91%|█████████ | 2508/2757 [5:44:05<44:48, 10.80s/it]
91%|█████████ | 2509/2757 [5:44:13<41:18, 9.99s/it]
91%|█████████ | 2510/2757 [5:44:21<39:05, 9.50s/it]
91%|█████████ | 2510/2757 [5:44:21<39:05, 9.50s/it]
91%|█████████ | 2511/2757 [5:44:29<36:46, 8.97s/it]
91%|█████████ | 2512/2757 [5:44:37<35:59, 8.81s/it]
91%|█████████ | 2513/2757 [5:44:46<35:07, 8.64s/it]
91%|█████████ | 2514/2757 [5:44:54<34:23, 8.49s/it]
91%|█████████ | 2515/2757 [5:45:02<34:02, 8.44s/it]
91%|█████████▏| 2516/2757 [5:45:10<33:27, 8.33s/it]
91%|█████████▏| 2517/2757 [5:45:18<32:51, 8.21s/it]
91%|█████████▏| 2518/2757 [5:45:26<32:04, 8.05s/it]
91%|█████████▏| 2519/2757 [5:45:34<31:53, 8.04s/it]
91%|█████████▏| 2520/2757 [5:45:42<31:43, 8.03s/it]
91%|█████████▏| 2520/2757 [5:45:42<31:43, 8.03s/it]
91%|█████████▏| 2521/2757 [5:45:50<32:01, 8.14s/it]
91%|█████████▏| 2522/2757 [5:45:59<31:59, 8.17s/it]
92%|█████████▏| 2523/2757 [5:46:06<31:19, 8.03s/it]
92%|█████████▏| 2524/2757 [5:46:14<31:05, 8.01s/it]
92%|█████████▏| 2525/2757 [5:46:22<30:43, 7.94s/it]
92%|█████████▏| 2526/2757 [5:46:30<31:00, 8.06s/it]
92%|█████████▏| 2527/2757 [5:46:38<30:57, 8.08s/it]
92%|█████████▏| 2528/2757 [5:46:46<30:30, 7.99s/it]
92%|█████████▏| 2529/2757 [5:46:54<30:18, 7.98s/it]
92%|█████████▏| 2530/2757 [5:47:01<29:27, 7.79s/it]
92%|█████████▏| 2530/2757 [5:47:01<29:27, 7.79s/it]
92%|█████████▏| 2531/2757 [5:47:09<29:30, 7.83s/it]
92%|█████████▏| 2532/2757 [5:47:17<29:37, 7.90s/it]
92%|█████████▏| 2533/2757 [5:47:26<29:47, 7.98s/it]
92%|█████████▏| 2534/2757 [5:47:33<29:15, 7.87s/it]
92%|█████████▏| 2535/2757 [5:47:41<29:29, 7.97s/it]
92%|█████████▏| 2536/2757 [5:47:49<29:14, 7.94s/it]
92%|█████████▏| 2537/2757 [5:47:57<29:10, 7.96s/it]
92%|█████████▏| 2538/2757 [5:48:05<28:47, 7.89s/it]
92%|█████████▏| 2539/2757 [5:48:13<29:01, 7.99s/it]
92%|█████████▏| 2540/2757 [5:48:21<29:03, 8.03s/it]
92%|█████████▏| 2540/2757 [5:48:21<29:03, 8.03s/it]
92%|█████████▏| 2541/2757 [5:48:30<29:35, 8.22s/it]
92%|█████████▏| 2542/2757 [5:48:37<28:30, 7.96s/it]
92%|█████████▏| 2543/2757 [5:48:45<27:56, 7.83s/it]
92%|█████████▏| 2544/2757 [5:48:53<28:07, 7.92s/it]
92%|█████████▏| 2545/2757 [5:49:01<28:05, 7.95s/it]
92%|█████████▏| 2546/2757 [5:49:09<28:16, 8.04s/it]
92%|█████████▏| 2547/2757 [5:49:17<28:01, 8.01s/it]
92%|█████████▏| 2548/2757 [5:49:26<28:16, 8.12s/it]
92%|█████████▏| 2549/2757 [5:49:34<27:58, 8.07s/it]
92%|█████████▏| 2550/2757 [5:49:42<27:50, 8.07s/it]
92%|█████████▏| 2550/2757 [5:49:42<27:50, 8.07s/it]
93%|█████████▎| 2551/2757 [5:49:50<27:24, 7.99s/it]
93%|█████████▎| 2552/2757 [5:49:58<27:29, 8.05s/it]
93%|█████████▎| 2553/2757 [5:50:06<27:34, 8.11s/it]
93%|█████████▎| 2554/2757 [5:50:14<27:34, 8.15s/it]
93%|█████████▎| 2555/2757 [5:50:22<27:03, 8.04s/it]
93%|█████████▎| 2556/2757 [5:50:30<26:53, 8.03s/it]
93%|█████████▎| 2557/2757 [5:50:38<26:56, 8.08s/it]
93%|█████████▎| 2558/2757 [5:50:46<26:27, 7.98s/it]
93%|█████████▎| 2559/2757 [5:50:54<26:04, 7.90s/it]
93%|█████████▎| 2560/2757 [5:51:02<26:17, 8.01s/it]
93%|█████████▎| 2560/2757 [5:51:02<26:17, 8.01s/it]
93%|█████████▎| 2561/2757 [5:51:10<25:50, 7.91s/it]
93%|█████████▎| 2562/2757 [5:51:18<25:53, 7.96s/it]
93%|█████████▎| 2563/2757 [5:51:26<26:22, 8.16s/it]
93%|█████████▎| 2564/2757 [5:51:35<26:19, 8.18s/it]
93%|█████████▎| 2565/2757 [5:51:42<25:46, 8.05s/it]
93%|█████████▎| 2566/2757 [5:51:50<25:14, 7.93s/it]
93%|█████████▎| 2567/2757 [5:51:58<25:04, 7.92s/it]
93%|█████████▎| 2568/2757 [5:52:06<25:15, 8.02s/it]
93%|█████████▎| 2569/2757 [5:52:14<24:47, 7.91s/it]
93%|█████████▎| 2570/2757 [5:52:22<24:32, 7.88s/it]
93%|█████████▎| 2570/2757 [5:52:22<24:32, 7.88s/it]
93%|█████████▎| 2571/2757 [5:52:29<24:28, 7.90s/it]
93%|█████████▎| 2572/2757 [5:52:37<24:13, 7.86s/it]
93%|█████████▎| 2573/2757 [5:52:45<23:53, 7.79s/it]
93%|█████████▎| 2574/2757 [5:52:53<24:09, 7.92s/it]
93%|█████████▎| 2575/2757 [5:53:01<24:12, 7.98s/it]
93%|█████████▎| 2576/2757 [5:53:09<24:21, 8.08s/it]
93%|█████████▎| 2577/2757 [5:53:18<24:28, 8.16s/it]
94%|█████████▎| 2578/2757 [5:53:26<24:02, 8.06s/it]
94%|█████████▎| 2579/2757 [5:53:34<24:04, 8.12s/it]
94%|█████████▎| 2580/2757 [5:53:42<23:47, 8.07s/it]
94%|█████████▎| 2580/2757 [5:53:42<23:47, 8.07s/it]
94%|█████████▎| 2581/2757 [5:53:50<23:29, 8.01s/it]
94%|█████████▎| 2582/2757 [5:53:58<23:12, 7.96s/it]
94%|█████████▎| 2583/2757 [5:54:06<23:19, 8.05s/it]
94%|█████████▎| 2584/2757 [5:54:14<22:59, 7.97s/it]
94%|█████████▍| 2585/2757 [5:54:21<22:44, 7.93s/it]
94%|█████████▍| 2586/2757 [5:54:29<22:38, 7.94s/it]
94%|█████████▍| 2587/2757 [5:54:38<23:24, 8.26s/it]
94%|█████████▍| 2588/2757 [5:54:47<23:15, 8.25s/it]
94%|█████████▍| 2589/2757 [5:54:55<23:04, 8.24s/it]
94%|█████████▍| 2590/2757 [5:55:03<22:55, 8.23s/it]
94%|█████████▍| 2590/2757 [5:55:03<22:55, 8.23s/it]
94%|█████████▍| 2591/2757 [5:55:11<22:31, 8.14s/it]
94%|█████████▍| 2592/2757 [5:55:19<22:29, 8.18s/it]
94%|█████████▍| 2593/2757 [5:55:27<22:04, 8.08s/it]
94%|█████████▍| 2594/2757 [5:55:35<21:27, 7.90s/it]
94%|█████████▍| 2595/2757 [5:55:42<21:16, 7.88s/it]
94%|█████████▍| 2596/2757 [5:55:50<21:02, 7.84s/it]
94%|█████████▍| 2597/2757 [5:55:58<21:04, 7.90s/it]
94%|█████████▍| 2598/2757 [5:56:07<21:16, 8.03s/it]
94%|█████████▍| 2599/2757 [5:56:14<20:53, 7.93s/it]
94%|█████████▍| 2600/2757 [5:56:23<21:03, 8.05s/it]
94%|█████████▍| 2600/2757 [5:56:23<21:03, 8.05s/it]
94%|█████████▍| 2601/2757 [5:56:31<20:54, 8.04s/it]
94%|█████████▍| 2602/2757 [5:56:39<20:46, 8.04s/it]
94%|█████████▍| 2603/2757 [5:56:47<20:48, 8.11s/it]
94%|█████████▍| 2604/2757 [5:56:55<20:24, 8.00s/it]
94%|█████████▍| 2605/2757 [5:57:03<20:12, 7.98s/it]
95%|█████████▍| 2606/2757 [5:57:11<20:10, 8.02s/it]
95%|█████████▍| 2607/2757 [5:57:18<19:50, 7.94s/it]
95%|█████████▍| 2608/2757 [5:57:26<19:17, 7.77s/it]
95%|█████████▍| 2609/2757 [5:57:34<19:26, 7.88s/it]
95%|█████████▍| 2610/2757 [5:57:42<19:34, 7.99s/it]
95%|█████████▍| 2610/2757 [5:57:42<19:34, 7.99s/it]
95%|█████████▍| 2611/2757 [5:57:51<19:39, 8.08s/it]
95%|█████████▍| 2612/2757 [5:57:59<19:37, 8.12s/it]
95%|█████████▍| 2613/2757 [5:58:07<19:26, 8.10s/it]
95%|█████████▍| 2614/2757 [5:58:15<19:10, 8.04s/it]
95%|█████████▍| 2615/2757 [5:58:22<18:49, 7.96s/it]
95%|█████████▍| 2616/2757 [5:58:30<18:38, 7.93s/it]
95%|█████████▍| 2617/2757 [5:58:38<18:20, 7.86s/it]
95%|█████████▍| 2618/2757 [5:58:46<18:32, 8.00s/it]
95%|█████████▍| 2619/2757 [5:58:55<18:39, 8.12s/it]
95%|█████████▌| 2620/2757 [5:59:02<18:14, 7.99s/it]
95%|█████████▌| 2620/2757 [5:59:02<18:14, 7.99s/it]
95%|█████████▌| 2621/2757 [5:59:10<18:03, 7.96s/it]
95%|█████████▌| 2622/2757 [5:59:19<18:15, 8.11s/it]
95%|█████████▌| 2623/2757 [5:59:27<18:22, 8.23s/it]
95%|█████████▌| 2624/2757 [5:59:36<18:22, 8.29s/it]
95%|█████████▌| 2625/2757 [5:59:44<18:06, 8.23s/it]
95%|█████████▌| 2626/2757 [5:59:51<17:33, 8.04s/it]
95%|█████████▌| 2627/2757 [6:00:00<17:31, 8.09s/it]
95%|█████████▌| 2628/2757 [6:00:07<17:12, 8.01s/it]
95%|█████████▌| 2629/2757 [6:00:16<17:16, 8.10s/it]
95%|█████████▌| 2630/2757 [6:00:24<16:59, 8.03s/it]
95%|█████████▌| 2630/2757 [6:00:24<16:59, 8.03s/it]
95%|█████████▌| 2631/2757 [6:00:32<16:51, 8.03s/it]
95%|█████████▌| 2632/2757 [6:00:40<16:50, 8.08s/it]
96%|█████████▌| 2633/2757 [6:00:48<16:35, 8.03s/it]
96%|█████████▌| 2634/2757 [6:00:56<16:31, 8.06s/it]
96%|█████████▌| 2635/2757 [6:01:03<16:06, 7.92s/it]
96%|█████████▌| 2636/2757 [6:01:11<16:00, 7.94s/it]
96%|█████████▌| 2637/2757 [6:01:20<15:59, 8.00s/it]
96%|█████████▌| 2638/2757 [6:01:28<15:49, 7.98s/it]
96%|█████████▌| 2639/2757 [6:01:35<15:32, 7.90s/it]
96%|█████████▌| 2640/2757 [6:01:43<15:18, 7.85s/it]
96%|█████████▌| 2640/2757 [6:01:43<15:18, 7.85s/it]
96%|█████████▌| 2641/2757 [6:01:51<15:17, 7.91s/it]
96%|█████████▌| 2642/2757 [6:01:59<15:09, 7.91s/it]
96%|█████████▌| 2643/2757 [6:02:07<15:07, 7.96s/it]
96%|█████████▌| 2644/2757 [6:02:15<15:12, 8.07s/it]
96%|█████████▌| 2645/2757 [6:02:23<14:44, 7.90s/it]
96%|█████████▌| 2646/2757 [6:02:31<14:44, 7.97s/it]
96%|█████████▌| 2647/2757 [6:02:39<14:31, 7.92s/it]
96%|█████████▌| 2648/2757 [6:02:47<14:35, 8.03s/it]
96%|█████████▌| 2649/2757 [6:02:55<14:27, 8.04s/it]
96%|█████████▌| 2650/2757 [6:03:03<14:16, 8.00s/it]
96%|█████████▌| 2650/2757 [6:03:03<14:16, 8.00s/it]
96%|█████████▌| 2651/2757 [6:03:11<14:00, 7.93s/it]
96%|█████████▌| 2652/2757 [6:03:19<14:04, 8.04s/it]
96%|█████████▌| 2653/2757 [6:03:27<13:56, 8.04s/it]
96%|█████████▋| 2654/2757 [6:03:35<13:42, 7.99s/it]
96%|█████████▋| 2655/2757 [6:03:44<13:50, 8.14s/it]
96%|█████████▋| 2656/2757 [6:03:52<13:40, 8.12s/it]
96%|█████████▋| 2657/2757 [6:03:59<13:22, 8.03s/it]
96%|█████████▋| 2658/2757 [6:04:08<13:25, 8.14s/it]
96%|█████████▋| 2659/2757 [6:04:16<13:13, 8.09s/it]
96%|█████████▋| 2660/2757 [6:04:23<12:52, 7.96s/it]
96%|█████████▋| 2660/2757 [6:04:23<12:52, 7.96s/it]
97%|█████████▋| 2661/2757 [6:04:31<12:38, 7.90s/it]
97%|█████████▋| 2662/2757 [6:04:39<12:38, 7.99s/it]
97%|█████████▋| 2663/2757 [6:04:47<12:25, 7.93s/it]
97%|█████████▋| 2664/2757 [6:04:55<12:06, 7.81s/it]
97%|█████████▋| 2665/2757 [6:05:03<12:01, 7.84s/it]
97%|█████████▋| 2666/2757 [6:05:11<12:11, 8.04s/it]
97%|█████████▋| 2667/2757 [6:05:20<12:13, 8.15s/it]
97%|█████████▋| 2668/2757 [6:05:27<11:48, 7.96s/it]
97%|█████████▋| 2669/2757 [6:05:35<11:35, 7.90s/it]
97%|█████████▋| 2670/2757 [6:05:43<11:22, 7.84s/it]
97%|█████████▋| 2670/2757 [6:05:43<11:22, 7.84s/it]
97%|█████████▋| 2671/2757 [6:05:50<11:12, 7.82s/it]
97%|█████████▋| 2672/2757 [6:05:58<11:03, 7.80s/it]
97%|█████████▋| 2673/2757 [6:06:06<11:03, 7.89s/it]
97%|█████████▋| 2674/2757 [6:06:14<10:58, 7.93s/it]
97%|█████████▋| 2675/2757 [6:06:22<10:48, 7.91s/it]
97%|█████████▋| 2676/2757 [6:06:30<10:37, 7.87s/it]
97%|█████████▋| 2677/2757 [6:06:38<10:32, 7.90s/it]
97%|█████████▋| 2678/2757 [6:06:46<10:22, 7.88s/it]
97%|█████████▋| 2679/2757 [6:06:54<10:18, 7.93s/it]
97%|█████████▋| 2680/2757 [6:07:02<10:17, 8.02s/it]
97%|█████████▋| 2680/2757 [6:07:02<10:17, 8.02s/it]
97%|█████████▋| 2681/2757 [6:07:10<10:14, 8.09s/it]
97%|█████████▋| 2682/2757 [6:07:18<10:08, 8.12s/it]
97%|█████████▋| 2683/2757 [6:07:26<09:50, 7.97s/it]
97%|█████████▋| 2684/2757 [6:07:34<09:52, 8.11s/it]
97%|█████████▋| 2685/2757 [6:07:43<09:43, 8.11s/it]
97%|█████████▋| 2686/2757 [6:07:51<09:41, 8.19s/it]
97%|█████████▋| 2687/2757 [6:07:59<09:21, 8.02s/it]
97%|█████████▋| 2688/2757 [6:08:06<09:06, 7.92s/it]
98%|█████████▊| 2689/2757 [6:08:14<08:55, 7.88s/it]
98%|█████████▊| 2690/2757 [6:08:22<08:49, 7.91s/it]
98%|█████████▊| 2690/2757 [6:08:22<08:49, 7.91s/it]
98%|█████████▊| 2691/2757 [6:08:30<08:48, 8.01s/it]
98%|█████████▊| 2692/2757 [6:08:38<08:38, 7.98s/it]
98%|█████████▊| 2693/2757 [6:08:46<08:32, 8.01s/it]
98%|█████████▊| 2694/2757 [6:08:54<08:19, 7.93s/it]
98%|█████████▊| 2695/2757 [6:09:02<08:16, 8.02s/it]
98%|█████████▊| 2696/2757 [6:09:10<08:06, 7.98s/it]
98%|█████████▊| 2697/2757 [6:09:18<07:57, 7.95s/it]
98%|█████████▊| 2698/2757 [6:09:26<07:47, 7.92s/it]
98%|█████████▊| 2699/2757 [6:09:34<07:42, 7.97s/it]
98%|█████████▊| 2700/2757 [6:09:42<07:32, 7.95s/it]
98%|█████████▊| 2700/2757 [6:09:42<07:32, 7.95s/it]
98%|█████████▊| 2701/2757 [6:09:50<07:31, 8.07s/it]
98%|█████████▊| 2702/2757 [6:09:58<07:20, 8.01s/it]
98%|█████████▊| 2703/2757 [6:10:06<07:13, 8.03s/it]
98%|█████████▊| 2704/2757 [6:10:14<06:59, 7.91s/it]
98%|█████████▊| 2705/2757 [6:10:22<06:57, 8.04s/it]
98%|█████████▊| 2706/2757 [6:10:30<06:43, 7.91s/it]
98%|█████████▊| 2707/2757 [6:10:38<06:41, 8.02s/it]
98%|█████████▊| 2708/2757 [6:10:46<06:33, 8.03s/it]
98%|█████████▊| 2709/2757 [6:10:54<06:23, 7.99s/it]
98%|█████████▊| 2710/2757 [6:11:02<06:13, 7.94s/it]
98%|█████████▊| 2710/2757 [6:11:02<06:13, 7.94s/it]
98%|█████████▊| 2711/2757 [6:11:10<06:03, 7.90s/it]
98%|█████████▊| 2712/2757 [6:11:17<05:54, 7.87s/it]
98%|█████████▊| 2713/2757 [6:11:25<05:48, 7.91s/it]
98%|█████████▊| 2714/2757 [6:11:34<05:45, 8.04s/it]
98%|█████████▊| 2715/2757 [6:11:41<05:31, 7.90s/it]
99%|█████████▊| 2716/2757 [6:11:49<05:22, 7.88s/it]
99%|█████████▊| 2717/2757 [6:11:57<05:17, 7.94s/it]
99%|█████████▊| 2718/2757 [6:12:05<05:12, 8.00s/it]
99%|█████████▊| 2719/2757 [6:12:13<05:04, 8.02s/it]
99%|█████████▊| 2720/2757 [6:12:21<04:55, 7.99s/it]
99%|█████████▊| 2720/2757 [6:12:21<04:55, 7.99s/it]
99%|█████████▊| 2721/2757 [6:12:29<04:42, 7.84s/it]
99%|█████████▊| 2722/2757 [6:12:36<04:30, 7.73s/it]
99%|█████████▉| 2723/2757 [6:12:44<04:25, 7.81s/it]
99%|█████████▉| 2724/2757 [6:12:52<04:18, 7.84s/it]
99%|█████████▉| 2725/2757 [6:13:00<04:11, 7.86s/it]
99%|█████████▉| 2726/2757 [6:13:08<04:04, 7.89s/it]
99%|█████████▉| 2727/2757 [6:13:16<03:57, 7.92s/it]
99%|█████████▉| 2728/2757 [6:13:24<03:50, 7.95s/it]
99%|█████████▉| 2729/2757 [6:13:32<03:45, 8.04s/it]
99%|█████████▉| 2730/2757 [6:13:40<03:34, 7.95s/it]
99%|█████████▉| 2730/2757 [6:13:40<03:34, 7.95s/it]
99%|█████████▉| 2731/2757 [6:13:48<03:27, 7.99s/it]
99%|█████████▉| 2732/2757 [6:13:56<03:21, 8.07s/it]
99%|█████████▉| 2733/2757 [6:14:04<03:12, 8.00s/it]
99%|█████████▉| 2734/2757 [6:14:12<03:02, 7.93s/it]
99%|█████████▉| 2735/2757 [6:14:20<02:52, 7.86s/it]
99%|█████████▉| 2736/2757 [6:14:27<02:43, 7.77s/it]
99%|█████████▉| 2737/2757 [6:14:36<02:41, 8.09s/it]
99%|█████████▉| 2738/2757 [6:14:44<02:33, 8.07s/it]
99%|█████████▉| 2739/2757 [6:14:52<02:21, 7.88s/it]
99%|█████████▉| 2740/2757 [6:14:59<02:13, 7.84s/it]
99%|█████████▉| 2740/2757 [6:14:59<02:13, 7.84s/it]
99%|█████████▉| 2741/2757 [6:15:07<02:05, 7.86s/it]
99%|█████████▉| 2742/2757 [6:15:15<01:58, 7.87s/it]
99%|█████████▉| 2743/2757 [6:15:23<01:50, 7.92s/it]
100%|█████████▉| 2744/2757 [6:15:31<01:44, 8.03s/it]
100%|█████████▉| 2745/2757 [6:15:39<01:36, 8.01s/it]
100%|█████████▉| 2746/2757 [6:15:48<01:29, 8.15s/it]
100%|█████████▉| 2747/2757 [6:15:56<01:21, 8.18s/it]
100%|█████████▉| 2748/2757 [6:16:03<01:11, 7.94s/it]
100%|█████████▉| 2749/2757 [6:16:12<01:04, 8.10s/it]
100%|█████████▉| 2750/2757 [6:16:20<00:56, 8.05s/it]
{'eval_loss': '0.4573', 'eval_runtime': '104.7', 'eval_samples_per_second': '22.93', 'eval_steps_per_second': '2.867', 'eval_rewards/chosen': '-3.349', 'eval_rewards/rejected': '-4.964', 'eval_rewards/accuracies': '0.7217', 'eval_rewards/margins': '1.615', 'eval_logps/chosen': '-113.1', 'eval_logps/rejected': '-135.5', 'eval_logits/chosen': '-4.366', 'eval_logits/rejected': '-4.316', 'epoch': '2.721'}
{'loss': '0.2396', 'grad_norm': '0.5165', 'learning_rate': '1.855e-05', 'rewards/chosen': '-3.146', 'rewards/rejected': '-5.601', 'rewards/accuracies': '0.8609', 'rewards/margins': '2.456', 'logps/chosen': '-113.9', 'logps/rejected': '-145.7', 'logits/chosen': '-4.321', 'logits/rejected': '-4.34', 'epoch': '2.731'}
{'loss': '0.251', 'grad_norm': '0.5313', 'learning_rate': '1.78e-05', 'rewards/chosen': '-3.095', 'rewards/rejected': '-5.495', 'rewards/accuracies': '0.8547', 'rewards/margins': '2.4', 'logps/chosen': '-112', 'logps/rejected': '-144.2', 'logits/chosen': '-4.337', 'logits/rejected': '-4.252', 'epoch': '2.742'}
{'loss': '0.2669', 'grad_norm': '0.6009', 'learning_rate': '1.705e-05', 'rewards/chosen': '-3.144', 'rewards/rejected': '-5.451', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.306', 'logps/chosen': '-112.2', 'logps/rejected': '-141.5', 'logits/chosen': '-4.345', 'logits/rejected': '-4.265', 'epoch': '2.753'}
{'loss': '0.2554', 'grad_norm': '0.5517', 'learning_rate': '1.631e-05', 'rewards/chosen': '-3.108', 'rewards/rejected': '-5.491', 'rewards/accuracies': '0.8422', 'rewards/margins': '2.383', 'logps/chosen': '-113.3', 'logps/rejected': '-140.6', 'logits/chosen': '-4.365', 'logits/rejected': '-4.302', 'epoch': '2.764'}
{'loss': '0.2582', 'grad_norm': '0.6179', 'learning_rate': '1.556e-05', 'rewards/chosen': '-3.127', 'rewards/rejected': '-5.444', 'rewards/accuracies': '0.8375', 'rewards/margins': '2.317', 'logps/chosen': '-113.7', 'logps/rejected': '-143.8', 'logits/chosen': '-4.218', 'logits/rejected': '-4.297', 'epoch': '2.775'}
{'loss': '0.2668', 'grad_norm': '0.5837', 'learning_rate': '1.481e-05', 'rewards/chosen': '-3.097', 'rewards/rejected': '-5.393', 'rewards/accuracies': '0.8422', 'rewards/margins': '2.297', 'logps/chosen': '-107.7', 'logps/rejected': '-136.1', 'logits/chosen': '-4.289', 'logits/rejected': '-4.224', 'epoch': '2.786'}
{'loss': '0.2648', 'grad_norm': '0.6681', 'learning_rate': '1.406e-05', 'rewards/chosen': '-3.148', 'rewards/rejected': '-5.445', 'rewards/accuracies': '0.8406', 'rewards/margins': '2.297', 'logps/chosen': '-112.1', 'logps/rejected': '-140.3', 'logits/chosen': '-4.223', 'logits/rejected': '-4.183', 'epoch': '2.797'}
{'loss': '0.2467', 'grad_norm': '0.5711', 'learning_rate': '1.331e-05', 'rewards/chosen': '-3.118', 'rewards/rejected': '-5.44', 'rewards/accuracies': '0.8547', 'rewards/margins': '2.322', 'logps/chosen': '-113.7', 'logps/rejected': '-142.5', 'logits/chosen': '-4.21', 'logits/rejected': '-4.185', 'epoch': '2.808'}
{'loss': '0.262', 'grad_norm': '0.5574', 'learning_rate': '1.257e-05', 'rewards/chosen': '-3.077', 'rewards/rejected': '-5.374', 'rewards/accuracies': '0.8438', 'rewards/margins': '2.297', 'logps/chosen': '-109', 'logps/rejected': '-136.3', 'logits/chosen': '-4.284', 'logits/rejected': '-4.183', 'epoch': '2.819'}
{'loss': '0.2469', 'grad_norm': '0.549', 'learning_rate': '1.182e-05', 'rewards/chosen': '-3.052', 'rewards/rejected': '-5.457', 'rewards/accuracies': '0.8555', 'rewards/margins': '2.405', 'logps/chosen': '-111.3', 'logps/rejected': '-140.8', 'logits/chosen': '-4.32', 'logits/rejected': '-4.266', 'epoch': '2.829'}
{'loss': '0.2493', 'grad_norm': '0.4994', 'learning_rate': '1.107e-05', 'rewards/chosen': '-3.06', 'rewards/rejected': '-5.434', 'rewards/accuracies': '0.8508', 'rewards/margins': '2.374', 'logps/chosen': '-108.5', 'logps/rejected': '-137.1', 'logits/chosen': '-4.284', 'logits/rejected': '-4.174', 'epoch': '2.84'}
{'loss': '0.2597', 'grad_norm': '0.596', 'learning_rate': '1.032e-05', 'rewards/chosen': '-3.036', 'rewards/rejected': '-5.337', 'rewards/accuracies': '0.8578', 'rewards/margins': '2.3', 'logps/chosen': '-107.8', 'logps/rejected': '-135.4', 'logits/chosen': '-4.267', 'logits/rejected': '-4.113', 'epoch': '2.851'}
{'loss': '0.2591', 'grad_norm': '0.5377', 'learning_rate': '9.574e-06', 'rewards/chosen': '-3.168', 'rewards/rejected': '-5.573', 'rewards/accuracies': '0.8469', 'rewards/margins': '2.405', 'logps/chosen': '-111.8', 'logps/rejected': '-141.1', 'logits/chosen': '-4.322', 'logits/rejected': '-4.283', 'epoch': '2.862'}
{'loss': '0.2559', 'grad_norm': '0.6059', 'learning_rate': '8.826e-06', 'rewards/chosen': '-3.091', 'rewards/rejected': '-5.442', 'rewards/accuracies': '0.8352', 'rewards/margins': '2.351', 'logps/chosen': '-111.7', 'logps/rejected': '-141', 'logits/chosen': '-4.247', 'logits/rejected': '-4.217', 'epoch': '2.873'}
{'loss': '0.2493', 'grad_norm': '0.6502', 'learning_rate': '8.078e-06', 'rewards/chosen': '-3.102', 'rewards/rejected': '-5.487', 'rewards/accuracies': '0.8523', 'rewards/margins': '2.385', 'logps/chosen': '-112.2', 'logps/rejected': '-142.6', 'logits/chosen': '-4.256', 'logits/rejected': '-4.207', 'epoch': '2.884'}
{'loss': '0.2628', 'grad_norm': '0.524', 'learning_rate': '7.33e-06', 'rewards/chosen': '-3.17', 'rewards/rejected': '-5.479', 'rewards/accuracies': '0.8336', 'rewards/margins': '2.309', 'logps/chosen': '-113', 'logps/rejected': '-140.3', 'logits/chosen': '-4.29', 'logits/rejected': '-4.22', 'epoch': '2.895'}
{'loss': '0.2616', 'grad_norm': '0.6188', 'learning_rate': '6.582e-06', 'rewards/chosen': '-3.144', 'rewards/rejected': '-5.507', 'rewards/accuracies': '0.8484', 'rewards/margins': '2.363', 'logps/chosen': '-111.8', 'logps/rejected': '-140.9', 'logits/chosen': '-4.309', 'logits/rejected': '-4.274', 'epoch': '2.906'}
{'loss': '0.2506', 'grad_norm': '0.66', 'learning_rate': '5.834e-06', 'rewards/chosen': '-3.092', 'rewards/rejected': '-5.493', 'rewards/accuracies': '0.8562', 'rewards/margins': '2.402', 'logps/chosen': '-112.2', 'logps/rejected': '-143.6', 'logits/chosen': '-4.265', 'logits/rejected': '-4.203', 'epoch': '2.916'}
{'loss': '0.2568', 'grad_norm': '0.4364', 'learning_rate': '5.086e-06', 'rewards/chosen': '-3.115', 'rewards/rejected': '-5.444', 'rewards/accuracies': '0.8406', 'rewards/margins': '2.328', 'logps/chosen': '-110', 'logps/rejected': '-139.2', 'logits/chosen': '-4.176', 'logits/rejected': '-4.107', 'epoch': '2.927'}
{'loss': '0.2593', 'grad_norm': '0.5918', 'learning_rate': '4.338e-06', 'rewards/chosen': '-3.071', 'rewards/rejected': '-5.375', 'rewards/accuracies': '0.8438', 'rewards/margins': '2.304', 'logps/chosen': '-112.2', 'logps/rejected': '-139.6', 'logits/chosen': '-4.195', 'logits/rejected': '-4.221', 'epoch': '2.938'}
{'loss': '0.2563', 'grad_norm': '0.4884', 'learning_rate': '3.59e-06', 'rewards/chosen': '-3.157', 'rewards/rejected': '-5.468', 'rewards/accuracies': '0.8375', 'rewards/margins': '2.311', 'logps/chosen': '-113.7', 'logps/rejected': '-140.7', 'logits/chosen': '-4.237', 'logits/rejected': '-4.152', 'epoch': '2.949'}
{'loss': '0.2622', 'grad_norm': '0.6017', 'learning_rate': '2.842e-06', 'rewards/chosen': '-3.124', 'rewards/rejected': '-5.479', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.355', 'logps/chosen': '-112.5', 'logps/rejected': '-141.4', 'logits/chosen': '-4.274', 'logits/rejected': '-4.203', 'epoch': '2.96'}
{'loss': '0.2499', 'grad_norm': '0.6066', 'learning_rate': '2.094e-06', 'rewards/chosen': '-3.07', 'rewards/rejected': '-5.482', 'rewards/accuracies': '0.8547', 'rewards/margins': '2.412', 'logps/chosen': '-112.7', 'logps/rejected': '-142.3', 'logits/chosen': '-4.194', 'logits/rejected': '-4.145', 'epoch': '2.971'}
{'loss': '0.2538', 'grad_norm': '0.6238', 'learning_rate': '1.346e-06', 'rewards/chosen': '-3.098', 'rewards/rejected': '-5.456', 'rewards/accuracies': '0.8492', 'rewards/margins': '2.358', 'logps/chosen': '-111.6', 'logps/rejected': '-141.5', 'logits/chosen': '-4.227', 'logits/rejected': '-4.174', 'epoch': '2.982'}
{'loss': '0.2607', 'grad_norm': '0.6461', 'learning_rate': '5.984e-07', 'rewards/chosen': '-3.128', 'rewards/rejected': '-5.369', 'rewards/accuracies': '0.8492', 'rewards/margins': '2.241', 'logps/chosen': '-109.2', 'logps/rejected': '-137.4', 'logits/chosen': '-4.289', 'logits/rejected': '-4.216', 'epoch': '2.993'}
100%|█████████▉| 2750/2757 [6:16:20<00:56, 8.05s/it]
100%|█████████▉| 2751/2757 [6:16:31<00:54, 9.02s/it]
100%|█████████▉| 2752/2757 [6:16:39<00:43, 8.63s/it]
100%|█████████▉| 2753/2757 [6:16:47<00:33, 8.38s/it]
100%|█████████▉| 2754/2757 [6:16:54<00:24, 8.20s/it]
100%|█████████▉| 2755/2757 [6:17:02<00:16, 8.16s/it]
100%|█████████▉| 2756/2757 [6:17:10<00:08, 8.06s/it]
100%|██████████| 2757/2757 [6:17:16<00:00, 7.44s/it]
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
0%| | 0/300 [00:00<?, ?it/s]
1%| | 2/300 [00:00<00:52, 5.70it/s]
1%| | 3/300 [00:00<01:46, 2.80it/s]
1%|▏ | 4/300 [00:01<01:34, 3.12it/s]
2%|▏ | 5/300 [00:01<01:44, 2.81it/s]
2%|▏ | 6/300 [00:01<01:40, 2.92it/s]
2%|▏ | 7/300 [00:02<01:55, 2.54it/s]
3%|▎ | 8/300 [00:02<01:50, 2.64it/s]
3%|▎ | 9/300 [00:03<01:52, 2.59it/s]
3%|▎ | 10/300 [00:03<01:49, 2.66it/s]
4%|▎ | 11/300 [00:03<01:51, 2.59it/s]
4%|▍ | 12/300 [00:04<01:39, 2.91it/s]
4%|▍ | 13/300 [00:04<01:49, 2.63it/s]
5%|▍ | 14/300 [00:04<01:40, 2.86it/s]
5%|▌ | 15/300 [00:05<01:49, 2.61it/s]
5%|▌ | 16/300 [00:05<01:36, 2.94it/s]
6%|▌ | 17/300 [00:06<01:39, 2.83it/s]
6%|▌ | 18/300 [00:06<01:30, 3.13it/s]
6%|▋ | 19/300 [00:06<01:40, 2.80it/s]
7%|▋ | 20/300 [00:07<01:32, 3.03it/s]
7%|▋ | 21/300 [00:07<01:50, 2.53it/s]
7%|▋ | 22/300 [00:07<01:36, 2.87it/s]
8%|▊ | 23/300 [00:08<01:35, 2.89it/s]
8%|▊ | 24/300 [00:08<01:26, 3.21it/s]
8%|▊ | 25/300 [00:08<01:37, 2.81it/s]
9%|▊ | 26/300 [00:09<01:30, 3.01it/s]
9%|▉ | 27/300 [00:09<01:41, 2.70it/s]
9%|▉ | 28/300 [00:09<01:29, 3.05it/s]
10%|▉ | 29/300 [00:10<01:31, 2.95it/s]
10%|█ | 30/300 [00:10<01:22, 3.26it/s]
10%|█ | 31/300 [00:10<01:25, 3.14it/s]
11%|█ | 32/300 [00:10<01:18, 3.41it/s]
11%|█ | 33/300 [00:11<01:21, 3.27it/s]
11%|█▏ | 34/300 [00:11<01:17, 3.45it/s]
12%|█▏ | 35/300 [00:11<01:23, 3.19it/s]
12%|█▏ | 36/300 [00:12<01:19, 3.34it/s]
12%|█▏ | 37/300 [00:12<01:27, 3.01it/s]
13%|█▎ | 38/300 [00:12<01:18, 3.35it/s]
13%|█▎ | 39/300 [00:13<01:27, 2.97it/s]
13%|█▎ | 40/300 [00:13<01:19, 3.27it/s]
14%|█▎ | 41/300 [00:13<01:26, 3.01it/s]
14%|█▍ | 42/300 [00:14<01:18, 3.30it/s]
14%|█▍ | 43/300 [00:14<01:21, 3.14it/s]
15%|█▍ | 44/300 [00:14<01:15, 3.41it/s]
15%|█▌ | 45/300 [00:15<01:22, 3.10it/s]
15%|█▌ | 46/300 [00:15<01:23, 3.04it/s]
16%|█▌ | 47/300 [00:15<01:29, 2.83it/s]
16%|█▌ | 48/300 [00:16<01:22, 3.06it/s]
16%|█▋ | 49/300 [00:16<01:23, 3.02it/s]
17%|█▋ | 50/300 [00:16<01:14, 3.35it/s]
17%|█▋ | 51/300 [00:17<01:17, 3.21it/s]
17%|█▋ | 52/300 [00:17<01:14, 3.31it/s]
18%|█▊ | 53/300 [00:17<01:27, 2.81it/s]
18%|█▊ | 54/300 [00:18<01:18, 3.11it/s]
18%|█▊ | 55/300 [00:18<01:30, 2.70it/s]
19%|█▊ | 56/300 [00:18<01:18, 3.12it/s]
19%|█▉ | 57/300 [00:19<01:21, 3.00it/s]
19%|█▉ | 58/300 [00:19<01:22, 2.95it/s]
20%|█▉ | 59/300 [00:19<01:27, 2.75it/s]
20%|██ | 60/300 [00:20<01:18, 3.07it/s]
20%|██ | 61/300 [00:20<01:21, 2.93it/s]
21%|██ | 62/300 [00:20<01:26, 2.74it/s]
21%|██ | 63/300 [00:21<01:29, 2.64it/s]
21%|██▏ | 64/300 [00:21<01:26, 2.72it/s]
22%|██▏ | 65/300 [00:22<01:29, 2.62it/s]
22%|██▏ | 66/300 [00:22<01:20, 2.89it/s]
22%|██▏ | 67/300 [00:22<01:27, 2.65it/s]
23%|██▎ | 68/300 [00:23<01:19, 2.94it/s]
23%|██▎ | 69/300 [00:23<01:21, 2.82it/s]
23%|██▎ | 70/300 [00:23<01:13, 3.12it/s]
24%|██▎ | 71/300 [00:23<01:15, 3.02it/s]
24%|██▍ | 72/300 [00:24<01:09, 3.28it/s]
24%|██▍ | 73/300 [00:24<01:12, 3.13it/s]
25%|██▍ | 74/300 [00:24<01:08, 3.31it/s]
25%|██▌ | 75/300 [00:25<01:16, 2.94it/s]
25%|██▌ | 76/300 [00:25<01:16, 2.92it/s]
26%|██▌ | 77/300 [00:26<01:21, 2.74it/s]
26%|██▌ | 78/300 [00:26<01:14, 2.99it/s]
26%|██▋ | 79/300 [00:26<01:16, 2.90it/s]
27%|██▋ | 80/300 [00:26<01:11, 3.06it/s]
27%|██▋ | 81/300 [00:27<01:19, 2.75it/s]
27%|██▋ | 82/300 [00:27<01:11, 3.05it/s]
28%|██▊ | 83/300 [00:28<01:13, 2.94it/s]
28%|██▊ | 84/300 [00:28<01:14, 2.90it/s]
28%|██▊ | 85/300 [00:28<01:19, 2.72it/s]
29%|██▊ | 86/300 [00:29<01:10, 3.01it/s]
29%|██▉ | 87/300 [00:29<01:13, 2.90it/s]
29%|██▉ | 88/300 [00:29<01:07, 3.14it/s]
30%|██▉ | 89/300 [00:30<01:11, 2.95it/s]
30%|███ | 90/300 [00:30<01:06, 3.17it/s]
30%|███ | 91/300 [00:30<01:09, 2.99it/s]
31%|███ | 92/300 [00:30<01:03, 3.26it/s]
31%|███ | 93/300 [00:31<01:09, 2.98it/s]
31%|███▏ | 94/300 [00:31<01:02, 3.27it/s]
32%|███▏ | 95/300 [00:32<01:10, 2.91it/s]
32%|███▏ | 96/300 [00:32<01:05, 3.13it/s]
32%|███▏ | 97/300 [00:32<01:06, 3.04it/s]
33%|███▎ | 98/300 [00:32<01:02, 3.22it/s]
33%|███▎ | 99/300 [00:33<01:07, 2.98it/s]
33%|███▎ | 100/300 [00:33<01:03, 3.15it/s]
34%|███▎ | 101/300 [00:33<01:07, 2.95it/s]
34%|███▍ | 102/300 [00:34<01:05, 3.01it/s]
34%|███▍ | 103/300 [00:34<01:08, 2.88it/s]
35%|███▍ | 104/300 [00:34<01:01, 3.20it/s]
35%|███▌ | 105/300 [00:35<01:04, 3.02it/s]
35%|███▌ | 106/300 [00:35<01:00, 3.22it/s]
36%|███▌ | 107/300 [00:35<01:04, 3.00it/s]
36%|███▌ | 108/300 [00:36<01:02, 3.08it/s]
36%|███▋ | 109/300 [00:36<01:06, 2.89it/s]
37%|███▋ | 110/300 [00:36<01:02, 3.05it/s]
37%|███▋ | 111/300 [00:37<01:11, 2.66it/s]
37%|███▋ | 112/300 [00:37<01:05, 2.86it/s]
38%|███▊ | 113/300 [00:38<01:17, 2.42it/s]
38%|███▊ | 114/300 [00:38<01:13, 2.53it/s]
38%|███▊ | 115/300 [00:39<01:16, 2.43it/s]
39%|███▊ | 116/300 [00:39<01:06, 2.77it/s]
39%|███▉ | 117/300 [00:39<01:10, 2.59it/s]
39%|███▉ | 118/300 [00:39<01:03, 2.87it/s]
40%|███▉ | 119/300 [00:40<01:04, 2.82it/s]
40%|████ | 120/300 [00:40<01:02, 2.89it/s]
40%|████ | 121/300 [00:41<01:03, 2.82it/s]
41%|████ | 122/300 [00:41<00:58, 3.05it/s]
41%|████ | 123/300 [00:41<01:02, 2.85it/s]
41%|████▏ | 124/300 [00:42<00:58, 3.03it/s]
42%|████▏ | 125/300 [00:42<00:59, 2.93it/s]
42%|████▏ | 126/300 [00:42<00:59, 2.91it/s]
42%|████▏ | 127/300 [00:43<01:02, 2.77it/s]
43%|████▎ | 128/300 [00:43<00:57, 2.98it/s]
43%|████▎ | 129/300 [00:43<00:59, 2.86it/s]
43%|████▎ | 130/300 [00:44<00:54, 3.14it/s]
44%|████▎ | 131/300 [00:44<01:00, 2.78it/s]
44%|████▍ | 132/300 [00:44<00:55, 3.03it/s]
44%|████▍ | 133/300 [00:45<01:02, 2.68it/s]
45%|████▍ | 134/300 [00:45<00:56, 2.93it/s]
45%|████▌ | 135/300 [00:45<00:58, 2.80it/s]
45%|████▌ | 136/300 [00:46<00:52, 3.09it/s]
46%|████▌ | 137/300 [00:46<00:53, 3.03it/s]
46%|████▌ | 138/300 [00:46<00:49, 3.25it/s]
46%|████▋ | 139/300 [00:47<00:50, 3.19it/s]
47%|████▋ | 140/300 [00:47<00:46, 3.44it/s]
47%|████▋ | 141/300 [00:47<00:48, 3.27it/s]
47%|████▋ | 142/300 [00:47<00:50, 3.13it/s]
48%|████▊ | 143/300 [00:48<00:53, 2.94it/s]
48%|████▊ | 144/300 [00:48<00:48, 3.24it/s]
48%|████▊ | 145/300 [00:48<00:48, 3.18it/s]
49%|████▊ | 146/300 [00:49<00:46, 3.33it/s]
49%|████▉ | 147/300 [00:49<00:53, 2.88it/s]
49%|████▉ | 148/300 [00:49<00:49, 3.04it/s]
50%|████▉ | 149/300 [00:50<00:53, 2.80it/s]
50%|█████ | 150/300 [00:50<00:50, 3.00it/s]
50%|█████ | 151/300 [00:51<00:51, 2.88it/s]
51%|█████ | 152/300 [00:51<00:46, 3.19it/s]
51%|█████ | 153/300 [00:51<00:47, 3.09it/s]
51%|█████▏ | 154/300 [00:51<00:43, 3.36it/s]
52%|█████▏ | 155/300 [00:52<00:45, 3.17it/s]
52%|█████▏ | 156/300 [00:52<00:41, 3.49it/s]
52%|█████▏ | 157/300 [00:52<00:45, 3.12it/s]
53%|█████▎ | 158/300 [00:53<00:43, 3.26it/s]
53%|█████▎ | 159/300 [00:53<00:50, 2.79it/s]
53%|█████▎ | 160/300 [00:53<00:46, 3.02it/s]
54%|█████▎ | 161/300 [00:54<00:50, 2.76it/s]
54%|█████▍ | 162/300 [00:54<00:49, 2.79it/s]
54%|█████▍ | 163/300 [00:55<00:55, 2.47it/s]
55%|█████▍ | 164/300 [00:55<00:52, 2.58it/s]
55%|█████▌ | 165/300 [00:55<00:52, 2.56it/s]
55%|█████▌ | 166/300 [00:56<00:50, 2.67it/s]
56%|█████▌ | 167/300 [00:56<00:54, 2.44it/s]
56%|█████▌ | 168/300 [00:56<00:48, 2.75it/s]
56%|█████▋ | 169/300 [00:57<00:52, 2.48it/s]
57%|█████▋ | 170/300 [00:57<00:50, 2.58it/s]
57%|█████▋ | 171/300 [00:58<00:50, 2.54it/s]
57%|█████▋ | 172/300 [00:58<00:44, 2.87it/s]
58%|█████▊ | 173/300 [00:58<00:46, 2.71it/s]
58%|█████▊ | 174/300 [00:59<00:41, 3.05it/s]
58%|█████▊ | 175/300 [00:59<00:46, 2.69it/s]
59%|█████▊ | 176/300 [00:59<00:40, 3.03it/s]
59%|█████▉ | 177/300 [01:00<00:44, 2.73it/s]
59%|█████▉ | 178/300 [01:00<00:38, 3.16it/s]
60%|█████▉ | 179/300 [01:00<00:41, 2.92it/s]
60%|██████ | 180/300 [01:01<00:37, 3.19it/s]
60%|██████ | 181/300 [01:01<00:41, 2.88it/s]
61%|██████ | 182/300 [01:01<00:37, 3.18it/s]
61%|██████ | 183/300 [01:02<00:40, 2.91it/s]
61%|██████▏ | 184/300 [01:02<00:40, 2.90it/s]
62%|██████▏ | 185/300 [01:02<00:42, 2.71it/s]
62%|██████▏ | 186/300 [01:03<00:39, 2.90it/s]
62%|██████▏ | 187/300 [01:03<00:39, 2.87it/s]
63%|██████▎ | 188/300 [01:03<00:35, 3.15it/s]
63%|██████▎ | 189/300 [01:04<00:38, 2.86it/s]
63%|██████▎ | 190/300 [01:04<00:35, 3.10it/s]
64%|██████▎ | 191/300 [01:04<00:37, 2.89it/s]
64%|██████▍ | 192/300 [01:05<00:36, 2.94it/s]
64%|██████▍ | 193/300 [01:05<00:41, 2.56it/s]
65%|██████▍ | 194/300 [01:06<00:40, 2.64it/s]
65%|██████▌ | 195/300 [01:06<00:43, 2.42it/s]
65%|██████▌ | 196/300 [01:06<00:39, 2.66it/s]
66%|██████▌ | 197/300 [01:07<00:39, 2.64it/s]
66%|██████▌ | 198/300 [01:07<00:35, 2.87it/s]
66%|██████▋ | 199/300 [01:07<00:37, 2.73it/s]
67%|██████▋ | 200/300 [01:08<00:34, 2.93it/s]
67%|██████▋ | 201/300 [01:08<00:37, 2.63it/s]
67%|██████▋ | 202/300 [01:08<00:33, 2.93it/s]
68%|██████▊ | 203/300 [01:09<00:34, 2.85it/s]
68%|██████▊ | 204/300 [01:09<00:30, 3.15it/s]
68%|██████▊ | 205/300 [01:09<00:32, 2.97it/s]
69%|██████▊ | 206/300 [01:10<00:28, 3.35it/s]
69%|██████▉ | 207/300 [01:10<00:31, 2.91it/s]
69%|██████▉ | 208/300 [01:10<00:31, 2.93it/s]
70%|██████▉ | 209/300 [01:11<00:32, 2.78it/s]
70%|███████ | 210/300 [01:11<00:29, 3.01it/s]
70%|███████ | 211/300 [01:12<00:33, 2.66it/s]
71%|███████ | 212/300 [01:12<00:29, 2.97it/s]
71%|███████ | 213/300 [01:12<00:31, 2.72it/s]
71%|███████▏ | 214/300 [01:13<00:28, 2.97it/s]
72%|███████▏ | 215/300 [01:13<00:30, 2.77it/s]
72%|███████▏ | 216/300 [01:13<00:27, 3.07it/s]
72%|███████▏ | 217/300 [01:14<00:30, 2.75it/s]
73%|███████▎ | 218/300 [01:14<00:26, 3.15it/s]
73%|███████▎ | 219/300 [01:14<00:28, 2.86it/s]
73%|███████▎ | 220/300 [01:15<00:25, 3.14it/s]
74%|███████▎ | 221/300 [01:15<00:28, 2.80it/s]
74%|███████▍ | 222/300 [01:15<00:25, 3.10it/s]
74%|███████▍ | 223/300 [01:16<00:27, 2.75it/s]
75%|███████▍ | 224/300 [01:16<00:25, 2.97it/s]
75%|███████▌ | 225/300 [01:16<00:26, 2.82it/s]
75%|███████▌ | 226/300 [01:17<00:24, 3.05it/s]
76%|███████▌ | 227/300 [01:17<00:26, 2.80it/s]
76%|███████▌ | 228/300 [01:17<00:23, 3.13it/s]
76%|███████▋ | 229/300 [01:18<00:25, 2.76it/s]
77%|███████▋ | 230/300 [01:18<00:24, 2.86it/s]
77%|███████▋ | 231/300 [01:19<00:25, 2.71it/s]
77%|███████▋ | 232/300 [01:19<00:24, 2.77it/s]
78%|███████▊ | 233/300 [01:19<00:27, 2.47it/s]
78%|███████▊ | 234/300 [01:20<00:23, 2.82it/s]
78%|███████▊ | 235/300 [01:20<00:24, 2.63it/s]
79%|███████▊ | 236/300 [01:20<00:23, 2.69it/s]
79%|███████▉ | 237/300 [01:21<00:24, 2.58it/s]
79%|███████▉ | 238/300 [01:21<00:21, 2.91it/s]
80%|███████▉ | 239/300 [01:21<00:21, 2.83it/s]
80%|████████ | 240/300 [01:22<00:18, 3.16it/s]
80%|████████ | 241/300 [01:22<00:19, 3.05it/s]
81%|████████ | 242/300 [01:22<00:18, 3.09it/s]
81%|████████ | 243/300 [01:23<00:20, 2.73it/s]
81%|████████▏ | 244/300 [01:23<00:19, 2.82it/s]
82%|████████▏ | 245/300 [01:24<00:20, 2.67it/s]
82%|████████▏ | 246/300 [01:24<00:19, 2.77it/s]
82%|████████▏ | 247/300 [01:24<00:20, 2.59it/s]
83%|████████▎ | 248/300 [01:25<00:17, 2.92it/s]
83%|████████▎ | 249/300 [01:25<00:18, 2.78it/s]
83%|████████▎ | 250/300 [01:25<00:17, 2.89it/s]
84%|████████▎ | 251/300 [01:26<00:17, 2.78it/s]
84%|████████▍ | 252/300 [01:26<00:16, 2.99it/s]
84%|████████▍ | 253/300 [01:26<00:17, 2.73it/s]
85%|████████▍ | 254/300 [01:27<00:14, 3.11it/s]
85%|████████▌ | 255/300 [01:27<00:16, 2.80it/s]
85%|████████▌ | 256/300 [01:27<00:14, 3.14it/s]
86%|████████▌ | 257/300 [01:28<00:14, 2.95it/s]
86%|████████▌ | 258/300 [01:28<00:13, 3.21it/s]
86%|████████▋ | 259/300 [01:28<00:14, 2.93it/s]
87%|████████▋ | 260/300 [01:29<00:12, 3.13it/s]
87%|████████▋ | 261/300 [01:29<00:13, 2.96it/s]
87%|████████▋ | 262/300 [01:29<00:11, 3.26it/s]
88%|████████▊ | 263/300 [01:30<00:12, 3.00it/s]
88%|████████▊ | 264/300 [01:30<00:12, 2.99it/s]
88%|████████▊ | 265/300 [01:30<00:13, 2.59it/s]
89%|████████▊ | 266/300 [01:31<00:12, 2.68it/s]
89%|████████▉ | 267/300 [01:31<00:13, 2.42it/s]
89%|████████▉ | 268/300 [01:32<00:11, 2.78it/s]
90%|████████▉ | 269/300 [01:32<00:12, 2.46it/s]
90%|█████████ | 270/300 [01:32<00:11, 2.62it/s]
90%|█████████ | 271/300 [01:33<00:12, 2.37it/s]
91%|█████████ | 272/300 [01:33<00:10, 2.63it/s]
91%|█████████ | 273/300 [01:34<00:11, 2.38it/s]
91%|█████████▏| 274/300 [01:34<00:09, 2.72it/s]
92%|█████████▏| 275/300 [01:34<00:10, 2.44it/s]
92%|█████████▏| 276/300 [01:35<00:09, 2.62it/s]
92%|█████████▏| 277/300 [01:35<00:09, 2.53it/s]
93%|█████████▎| 278/300 [01:35<00:07, 2.85it/s]
93%|█████████▎| 279/300 [01:36<00:07, 2.80it/s]
93%|█████████▎| 280/300 [01:36<00:06, 2.89it/s]
94%|█████████▎| 281/300 [01:37<00:06, 2.72it/s]
94%|█████████▍| 282/300 [01:37<00:05, 3.02it/s]
94%|█████████▍| 283/300 [01:37<00:06, 2.79it/s]
95%|█████████▍| 284/300 [01:37<00:05, 3.06it/s]
95%|█████████▌| 285/300 [01:38<00:05, 2.66it/s]
95%|█████████▌| 286/300 [01:38<00:05, 2.71it/s]
96%|█████████▌| 287/300 [01:39<00:05, 2.59it/s]
96%|█████████▌| 288/300 [01:39<00:03, 3.01it/s]
96%|█████████▋| 289/300 [01:39<00:04, 2.68it/s]
97%|█████████▋| 290/300 [01:40<00:03, 2.73it/s]
97%|█████████▋| 291/300 [01:40<00:03, 2.47it/s]
97%|█████████▋| 292/300 [01:40<00:02, 2.80it/s]
98%|█████████▊| 293/300 [01:41<00:02, 2.70it/s]
98%|█████████▊| 294/300 [01:41<00:02, 2.95it/s]
98%|█████████▊| 295/300 [01:42<00:01, 2.89it/s]
99%|█████████▊| 296/300 [01:42<00:01, 3.11it/s]
99%|█████████▉| 297/300 [01:42<00:01, 2.88it/s]
99%|█████████▉| 298/300 [01:42<00:00, 3.25it/s]
100%|█████████▉| 299/300 [01:43<00:00, 2.53it/s]
100%|██████████| 300/300 [01:43<00:00, 2.57it/s]

100%|██████████| 2757/2757 [6:19:01<00:00, 7.44s/it]
100%|██████████| 300/300 [01:44<00:00, 2.57it/s]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-2757/tokenizer_config.json.
100%|██████████| 2757/2757 [6:19:31<00:00, 7.44s/it]
100%|██████████| 2757/2757 [6:19:32<00:00, 8.26s/it]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/final/tokenizer_config.json.
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/merged/tokenizer_config.json.
{'eval_loss': '0.4576', 'eval_runtime': '104.9', 'eval_samples_per_second': '22.88', 'eval_steps_per_second': '2.86', 'eval_rewards/chosen': '-3.332', 'eval_rewards/rejected': '-4.978', 'eval_rewards/accuracies': '0.7233', 'eval_rewards/margins': '1.646', 'eval_logps/chosen': '-112.9', 'eval_logps/rejected': '-135.6', 'eval_logits/chosen': '-4.33', 'eval_logits/rejected': '-4.281', 'epoch': '3'}
{'train_runtime': '2.277e+04', 'train_samples_per_second': '15.49', 'train_steps_per_second': '0.121', 'train_loss': '0.4124', 'epoch': '3'}
Saved adapter to models/dpo_default/final
Found HuggingFace hub cache directory: /workspace/.cache/huggingface/hub
Checking cache directory for required files...
Unsloth: Copying 1 files from cache to `models/dpo_default/merged`: 0%| | 0/1 [00:00<?, ?it/s]
Unsloth: Copying 1 files from cache to `models/dpo_default/merged`: 100%|██████████| 1/1 [00:02<00:00, 2.20s/it]
Unsloth: Copying 1 files from cache to `models/dpo_default/merged`: 100%|██████████| 1/1 [00:02<00:00, 2.20s/it]
Successfully copied all 1 files from cache to `models/dpo_default/merged`
Checking cache directory for required files...
Cache check failed: tokenizer.model not found in local cache.
Not all required files found in cache. Will proceed with downloading.
Unsloth: Preparing safetensor model files: 0%| | 0/1 [00:00<?, ?it/s]
Unsloth: Preparing safetensor model files: 100%|██████████| 1/1 [00:00<00:00, 37117.73it/s]
Unsloth: Merging weights into 16bit: 0%| | 0/1 [00:00<?, ?it/s]
Unsloth: Merging weights into 16bit: 100%|██████████| 1/1 [00:02<00:00, 2.43s/it]
Unsloth: Merging weights into 16bit: 100%|██████████| 1/1 [00:02<00:00, 2.43s/it]
Unsloth: Merge process complete. Saved to `/workspace/post-training-experiments/dpo/DPO_SmolLM135M/models/dpo_default/merged`
Saved merged model to models/dpo_default/merged
== Step 2: post-dpo diversity (compare to baseline above) ==
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
`torch_dtype` is deprecated! Use `dtype` instead!
Skipping import of cpp extensions due to incompatible torch version. Please upgrade to torch >= 2.11.0 (found 2.10.0+cu128).
Loading weights: 0%| | 0/272 [00:00<?, ?it/s]
Loading weights: 100%|██████████| 272/272 [00:00<00:00, 3064.64it/s]
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
[1/100] 16 responses avg_len=815 chars
t=0.3 ### Q1 **Question:** What metrics are used to control page layout in the dataset...
t=0.3 ### Q1 **Question:** What metrics are used to control page layout in the dataset...
t=0.3 ### Q1 **Question:** What metrics are used to control page layout in the dataset...
t=0.3 ### Q1 **Question:** What does Figure 4 and 5 illustrate? **Answer:** Figure 4 ...
t=0.5 ### Q1 **Question:** What metrics are used to control page layout in the dataset...
t=0.5 ### Q1 **Question:** What does Figure 4 and Table 4 illustrate? **Answer:** Fig...
t=0.5 ### Q1 **Question:** What does Figure 4 and 5 illustrate? **Answer:** Figure 4 ...
t=0.5 ### Q1 **Question:** What does Figure 4 and Figure 5 illustrate? **Answer:** Fi...
t=0.7 **Question:** What metrics are used to control page layout in the dataset? **An...
t=0.7 *'default' is a value that is empty if none. *Figure 4 and Figure 5 provide an o...
t=0.7 ### Q1 **Question:** What metrics are used to control the page layout of the dat...
t=0.7 *['How is the 'default' value defined in the context of the API format?'] *What ...
t=1.0 ### Q1 **Question:** What metrics are used to control page layout in the dataset...
t=1.0 ### Q1 **Question:** What metrics are used to control page layout in the dataset...
t=1.0 *'default' represents the default value, empty if none *Figure 5 illustrates the...
t=1.0 ### Q1 **Question:** What does Figure 4 and Table 4 illustrate? **Answer:** Fig...
[2/100] 16 responses avg_len=240 chars
t=0.3 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
t=0.3 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {" timedelt...
t=0.3 [{"subject": "AoA", "relation": "is a", "object": "second entry of path estimate...
t=0.3 [{"subject": "AoA", "relation": "is a", "object": "second entry of path estimate...
t=0.5 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
t=0.5 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {" timedelt...
t=0.5 [{"subject": "AoA", "relation": "is a", "object": "second entry of path estimate...
t=0.5 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {" timedelt...
t=0.7 [{"subject": "Option", "relation": "is a declaration in RLang", "object": "presy...
t=0.7 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
t=0.7 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"subject":...
t=0.7 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
t=1.0 [{"subject": "Option", "relation": "describes", "object": "sequence of actions s...
t=1.0 [{"subject": "AoA", "relation": "is a specification of", " object": "sequences o...
t=1.0 [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
t=1.0 [{"subject": "Option", "relation": "is a declaration in RLang", "object": "pecul...
[3/100] 16 responses avg_len=218 chars
[4/100] 16 responses avg_len=823 chars
[5/100] 16 responses avg_len=880 chars
[6/100] 16 responses avg_len=276 chars
[7/100] 16 responses avg_len=152 chars
[8/100] 16 responses avg_len=295 chars
[9/100] 16 responses avg_len=230 chars
[10/100] 16 responses avg_len=311 chars
[11/100] 16 responses avg_len=741 chars
[12/100] 16 responses avg_len=865 chars
[13/100] 16 responses avg_len=133 chars
[14/100] 16 responses avg_len=736 chars
[15/100] 16 responses avg_len=327 chars
[16/100] 16 responses avg_len=1030 chars
[17/100] 16 responses avg_len=702 chars
[18/100] 16 responses avg_len=1030 chars
[19/100] 16 responses avg_len=297 chars
[20/100] 16 responses avg_len=319 chars
[21/100] 16 responses avg_len=420 chars
[22/100] 16 responses avg_len=70 chars
[23/100] 16 responses avg_len=320 chars
[24/100] 16 responses avg_len=25 chars
[25/100] 16 responses avg_len=68 chars
[26/100] 16 responses avg_len=299 chars
[27/100] 16 responses avg_len=720 chars
[28/100] 16 responses avg_len=1254 chars
[29/100] 16 responses avg_len=137 chars
[30/100] 16 responses avg_len=353 chars
[31/100] 16 responses avg_len=1093 chars
[32/100] 16 responses avg_len=603 chars
[33/100] 16 responses avg_len=75 chars
[34/100] 16 responses avg_len=109 chars
[35/100] 16 responses avg_len=1307 chars
[36/100] 16 responses avg_len=750 chars
[37/100] 16 responses avg_len=714 chars
[38/100] 16 responses avg_len=179 chars
[39/100] 16 responses avg_len=284 chars
[40/100] 16 responses avg_len=120 chars
[41/100] 16 responses avg_len=765 chars
[42/100] 16 responses avg_len=259 chars
[43/100] 16 responses avg_len=481 chars
[44/100] 16 responses avg_len=241 chars
[45/100] 16 responses avg_len=95 chars
[46/100] 16 responses avg_len=320 chars
[47/100] 16 responses avg_len=391 chars
[48/100] 16 responses avg_len=1030 chars
[49/100] 16 responses avg_len=283 chars
[50/100] 16 responses avg_len=822 chars
[51/100] 16 responses avg_len=788 chars
[52/100] 16 responses avg_len=741 chars
[53/100] 16 responses avg_len=294 chars
[54/100] 16 responses avg_len=120 chars
[55/100] 16 responses avg_len=550 chars
[56/100] 16 responses avg_len=1174 chars
[57/100] 16 responses avg_len=205 chars
[58/100] 16 responses avg_len=190 chars
[59/100] 16 responses avg_len=373 chars
[60/100] 16 responses avg_len=674 chars
[61/100] 16 responses avg_len=1170 chars
[62/100] 16 responses avg_len=693 chars
[63/100] 16 responses avg_len=245 chars
[64/100] 16 responses avg_len=705 chars
[65/100] 16 responses avg_len=161 chars
[66/100] 16 responses avg_len=247 chars
[67/100] 16 responses avg_len=846 chars
[68/100] 16 responses avg_len=816 chars
[69/100] 16 responses avg_len=276 chars
[70/100] 16 responses avg_len=527 chars
[71/100] 16 responses avg_len=253 chars
[72/100] 16 responses avg_len=1074 chars
[73/100] 16 responses avg_len=324 chars
[74/100] 16 responses avg_len=110 chars
[75/100] 16 responses avg_len=337 chars
[76/100] 16 responses avg_len=703 chars
[77/100] 16 responses avg_len=1396 chars
[78/100] 16 responses avg_len=449 chars
[79/100] 16 responses avg_len=592 chars
[80/100] 16 responses avg_len=156 chars
[81/100] 16 responses avg_len=243 chars
[82/100] 16 responses avg_len=581 chars
[83/100] 16 responses avg_len=272 chars
[84/100] 16 responses avg_len=152 chars
[85/100] 16 responses avg_len=75 chars
[86/100] 16 responses avg_len=199 chars
[87/100] 16 responses avg_len=306 chars
[88/100] 16 responses avg_len=306 chars
[89/100] 16 responses avg_len=1197 chars
[90/100] 16 responses avg_len=689 chars
[91/100] 16 responses avg_len=597 chars
[92/100] 16 responses avg_len=677 chars
[93/100] 16 responses avg_len=165 chars
[94/100] 16 responses avg_len=274 chars
[95/100] 16 responses avg_len=416 chars
[96/100] 16 responses avg_len=287 chars
[97/100] 16 responses avg_len=308 chars
[98/100] 16 responses avg_len=62 chars
[99/100] 16 responses avg_len=69 chars
[100/100] 16 responses avg_len=129 chars
Saved 100 records to evals/dpo_default_n100_r4.jsonl
Skipping import of cpp extensions due to incompatible torch version. Please upgrade to torch >= 2.11.0 (found 2.10.0+cu128).
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Loading SBERT model: sentence-transformers/all-MiniLM-L6-v2 ...
Loading weights: 0%| | 0/103 [00:00<?, ?it/s]
Loading weights: 46%|████▌ | 47/103 [00:00<00:00, 409.92it/s]
Loading weights: 91%|█████████▏| 94/103 [00:00<00:00, 414.36it/s]
Loading weights: 100%|██████████| 103/103 [00:00<00:00, 450.62it/s]
BertModel LOAD REPORT from: sentence-transformers/all-MiniLM-L6-v2
Key | Status | |
------------------------+------------+--+-
embeddings.position_ids | UNEXPECTED | |
Notes:
- UNEXPECTED: can be ignored when loading from different task/architecture; not ok if you expect identical arch.
Scoring 100 records ...
[1/100] ead=0.2215 sbert=0.4202 vendi=4.7004
[2/100] ead=0.1539 sbert=0.3031 vendi=3.0089
[3/100] ead=0.1791 sbert=0.3479 vendi=3.7807
[4/100] ead=0.2852 sbert=0.5217 vendi=6.8363
[5/100] ead=0.2914 sbert=0.5311 vendi=6.9277
[6/100] ead=0.0645 sbert=0.1331 vendi=1.4444
[7/100] ead=0.1304 sbert=0.2600 vendi=2.8699
[8/100] ead=0.0776 sbert=0.1590 vendi=1.9956
[9/100] ead=0.0983 sbert=0.1994 vendi=2.3667
[10/100] ead=0.0731 sbert=0.1503 vendi=1.9737
[11/100] ead=0.0827 sbert=0.1691 vendi=2.0416
[12/100] ead=0.0821 sbert=0.1679 vendi=2.0866
[13/100] ead=0.0655 sbert=0.1352 vendi=1.7483
[14/100] ead=0.0807 sbert=0.1652 vendi=2.1535
[15/100] ead=0.1073 sbert=0.2166 vendi=2.4434
[16/100] ead=0.3385 sbert=0.5999 vendi=8.1126
[17/100] ead=0.0539 sbert=0.1119 vendi=1.7672
[18/100] ead=0.3339 sbert=0.5934 vendi=8.0090
[19/100] ead=0.2089 sbert=0.3991 vendi=3.5623
[20/100] ead=0.1009 sbert=0.2043 vendi=2.3543
[21/100] ead=0.0806 sbert=0.1650 vendi=1.9491
[22/100] ead=0.1957 sbert=0.3766 vendi=2.7339
[23/100] ead=0.3114 sbert=0.5609 vendi=5.9426
[24/100] ead=0.1103 sbert=0.2223 vendi=1.7700
[25/100] ead=0.0882 sbert=0.1798 vendi=1.6982
[26/100] ead=0.1494 sbert=0.2950 vendi=3.0703
[27/100] ead=0.1351 sbert=0.2688 vendi=3.2119
[28/100] ead=0.2144 sbert=0.4084 vendi=5.0096
[29/100] ead=0.0538 sbert=0.1117 vendi=1.5520
[30/100] ead=0.0033 sbert=0.0070 vendi=1.0495
[31/100] ead=0.2240 sbert=0.4243 vendi=5.3786
[32/100] ead=0.2644 sbert=0.4895 vendi=5.5140
[33/100] ead=0.1268 sbert=0.2533 vendi=2.8508
[34/100] ead=0.0572 sbert=0.1185 vendi=1.5461
[35/100] ead=0.2483 sbert=0.4640 vendi=4.7925
[36/100] ead=0.0421 sbert=0.0880 vendi=1.5689
[37/100] ead=0.0832 sbert=0.1701 vendi=2.1962
[38/100] ead=0.0302 sbert=0.0634 vendi=1.3600
[39/100] ead=0.0455 sbert=0.0948 vendi=1.6311
[40/100] ead=0.0178 sbert=0.0376 vendi=1.1938
[41/100] ead=0.0478 sbert=0.0996 vendi=1.6916
[42/100] ead=0.0623 sbert=0.1289 vendi=1.8174
[43/100] ead=0.0325 sbert=0.0681 vendi=1.4654
[44/100] ead=0.1723 sbert=0.3359 vendi=3.1290
[45/100] ead=0.1127 sbert=0.2269 vendi=2.2373
[46/100] ead=0.0968 sbert=0.1966 vendi=2.1951
[47/100] ead=0.0870 sbert=0.1775 vendi=2.2215
[48/100] ead=0.1293 sbert=0.2581 vendi=3.1035
[49/100] ead=0.1105 sbert=0.2227 vendi=2.5606
[50/100] ead=0.0722 sbert=0.1485 vendi=2.0503
[51/100] ead=0.0434 sbert=0.0905 vendi=1.6095
[52/100] ead=0.1231 sbert=0.2464 vendi=2.9946
[53/100] ead=0.0189 sbert=0.0399 vendi=1.2115
[54/100] ead=0.2173 sbert=0.4132 vendi=3.7740
[55/100] ead=0.0714 sbert=0.1468 vendi=1.9421
[56/100] ead=0.0795 sbert=0.1629 vendi=2.1548
[57/100] ead=0.0646 sbert=0.1334 vendi=1.8689
[58/100] ead=0.2422 sbert=0.4542 vendi=2.8661
[59/100] ead=0.1103 sbert=0.2224 vendi=2.3809
[60/100] ead=0.0845 sbert=0.1727 vendi=2.2505
[61/100] ead=0.2839 sbert=0.5197 vendi=6.7955
[62/100] ead=0.0455 sbert=0.0948 vendi=1.6370
[63/100] ead=0.0812 sbert=0.1662 vendi=1.9496
[64/100] ead=0.0175 sbert=0.0369 vendi=1.2523
[65/100] ead=0.2150 sbert=0.4094 vendi=2.5259
[66/100] ead=0.1168 sbert=0.2345 vendi=2.2798
[67/100] ead=0.1980 sbert=0.3807 vendi=4.4107
[68/100] ead=0.0632 sbert=0.1306 vendi=1.9128
[69/100] ead=0.0313 sbert=0.0657 vendi=1.3866
[70/100] ead=0.0799 sbert=0.1637 vendi=2.2017
[71/100] ead=0.0780 sbert=0.1600 vendi=2.0474
[72/100] ead=0.2311 sbert=0.4360 vendi=5.4815
[73/100] ead=0.1055 sbert=0.2132 vendi=2.1208
[74/100] ead=0.0918 sbert=0.1869 vendi=1.9718
[75/100] ead=0.0958 sbert=0.1945 vendi=2.1223
[76/100] ead=0.0597 sbert=0.1235 vendi=1.8748
[77/100] ead=0.0671 sbert=0.1384 vendi=1.9527
[78/100] ead=0.1057 sbert=0.2136 vendi=2.4193
[79/100] ead=0.0185 sbert=0.0391 vendi=1.2546
[80/100] ead=0.0971 sbert=0.1972 vendi=2.0882
[81/100] ead=0.1035 sbert=0.2094 vendi=2.3673
[82/100] ead=0.1347 sbert=0.2680 vendi=3.1583
[83/100] ead=0.0241 sbert=0.0508 vendi=1.3213
[84/100] ead=0.0043 sbert=0.0092 vendi=1.0509
[85/100] ead=0.2164 sbert=0.4118 vendi=2.9182
[86/100] ead=0.1350 sbert=0.2685 vendi=2.6217
[87/100] ead=0.0750 sbert=0.1541 vendi=2.0077
[88/100] ead=0.0538 sbert=0.1116 vendi=1.5855
[89/100] ead=0.2717 sbert=0.5008 vendi=6.5359
[90/100] ead=0.0692 sbert=0.1426 vendi=2.0181
[91/100] ead=0.2908 sbert=0.5302 vendi=6.5217
[92/100] ead=0.1777 sbert=0.3454 vendi=3.8662
[93/100] ead=0.2116 sbert=0.4037 vendi=3.1923
[94/100] ead=0.0679 sbert=0.1400 vendi=1.8048
[95/100] ead=0.0586 sbert=0.1213 vendi=1.6027
[96/100] ead=0.0668 sbert=0.1378 vendi=1.7351
[97/100] ead=0.1519 sbert=0.2994 vendi=3.0876
[98/100] ead=0.2075 sbert=0.3967 vendi=2.8306
[99/100] ead=0.0000 sbert=0.0000 vendi=1.0000
[100/100] ead=0.0407 sbert=0.0850 vendi=1.4648
--- summary ---
ead mean=0.1193 min=0.0000 max=0.3385
sbert mean=0.2322 min=0.0000 max=0.5999
vendi mean=2.7410 min=1.0000 max=8.1126
Saved to evals/dpo_default_n100_r4_diversity.jsonl
==========================================
Done. Compare the two diversity summaries:
baseline: evals/baseline_smollm_135M_neuraltxt_v1_n100_r4.jsonl
dpo_default: evals/dpo_default_n100_r4.jsonl
A large drop in SBERT/Vendi => mode collapse.
Merged model: models/dpo_default/merged
==========================================