==========================================
 method=dpo run_id=dpo_default
 base=paperbd/smollm_135M_neuraltxt_v1
 dataset=paperbd/paper_preference_150K-v1
 batch=8 grad_accum=16 (eff ~128)
==========================================
== uv sync ==
Resolved 148 packages in 5ms
Checked 130 packages in 418ms
== Step 0: skipped (RUN_BASELINE=0) ==
== Step 1: dpo training ==
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
Unsloth: Your Flash Attention 2 installation seems to be broken. Using Xformers instead. No performance changes will be seen.
🦥 Unsloth Zoo will now patch everything to make training faster!
Method: DPO
Base model: paperbd/smollm_135M_neuraltxt_v1
Dataset: paperbd/paper_preference_150K-v1
==((====))==  Unsloth 2026.5.9: Fast Llama patching. Transformers: 5.5.0.
   \\   /|    NVIDIA GeForce RTX 3090. Num GPUs = 1. Max memory: 23.559 GB. Platform: Linux.
O^O/ \_/ \    Torch: 2.10.0+cu128. CUDA: 8.6. CUDA Toolkit: 12.8. Triton: 3.6.0
\        /    Bfloat16 = TRUE. FA [Xformers = 0.0.35. FA2 = False]
 "-____-"     Free license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!

Loading weights:   0%|          | 0/272 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/272 [00:00<01:06,  4.07it/s]
Loading weights:   1%|          | 3/272 [00:00<00:46,  5.83it/s]
Loading weights:   8%|▊         | 23/272 [00:00<00:04, 51.42it/s]
Loading weights:  16%|█▌        | 43/272 [00:00<00:02, 88.67it/s]
Loading weights:  23%|██▎       | 63/272 [00:00<00:01, 117.62it/s]
Loading weights:  31%|███       | 84/272 [00:00<00:01, 141.77it/s]
Loading weights:  39%|███▉      | 107/272 [00:01<00:00, 165.90it/s]
Loading weights:  48%|████▊     | 131/272 [00:01<00:00, 186.38it/s]
Loading weights:  58%|█████▊    | 157/272 [00:01<00:00, 206.87it/s]
Loading weights:  67%|██████▋   | 183/272 [00:01<00:00, 216.32it/s]
Loading weights:  76%|███████▌  | 206/272 [00:01<00:00, 199.31it/s]
Loading weights:  83%|████████▎ | 227/272 [00:01<00:00, 195.80it/s]
Loading weights:  91%|█████████ | 248/272 [00:01<00:00, 179.27it/s]
Loading weights: 100%|██████████| 272/272 [00:01<00:00, 147.70it/s]
Unsloth: Will map <|im_end|> to EOS = <|im_end|>.
Unsloth 2026.5.9 patched 30 layers with 30 QKV layers, 30 O layers and 30 MLP layers.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.

Tokenizing train dataset (num_proc=64):   0%|          | 0/117600 [00:00<?, ? examples/s]
Tokenizing train dataset (num_proc=64):   0%|          | 22/117600 [00:03<4:47:59,  6.80 examples/s]
Tokenizing train dataset (num_proc=64):   0%|          | 148/117600 [00:03<32:59, 59.33 examples/s] 
Tokenizing train dataset (num_proc=64):   0%|          | 347/117600 [00:03<11:42, 166.96 examples/s]
Tokenizing train dataset (num_proc=64):   1%|          | 780/117600 [00:03<04:08, 470.67 examples/s]
Tokenizing train dataset (num_proc=64):   1%|          | 1462/117600 [00:03<01:48, 1069.04 examples/s]
Tokenizing train dataset (num_proc=64):   2%|▏         | 2474/117600 [00:03<00:54, 2111.82 examples/s]
Tokenizing train dataset (num_proc=64):   3%|▎         | 3639/117600 [00:03<00:33, 3432.29 examples/s]
Tokenizing train dataset (num_proc=64):   4%|▍         | 5143/117600 [00:04<00:21, 5282.21 examples/s]
Tokenizing train dataset (num_proc=64):   6%|▌         | 6825/117600 [00:04<00:15, 7306.57 examples/s]
Tokenizing train dataset (num_proc=64):   7%|▋         | 8586/117600 [00:04<00:11, 9211.19 examples/s]
Tokenizing train dataset (num_proc=64):   9%|▉         | 10716/117600 [00:04<00:09, 11377.87 examples/s]
Tokenizing train dataset (num_proc=64):  10%|█         | 12137/117600 [00:04<00:08, 12036.98 examples/s]
Tokenizing train dataset (num_proc=64):  12%|█▏        | 13754/117600 [00:04<00:08, 12292.25 examples/s]
Tokenizing train dataset (num_proc=64):  13%|█▎        | 15227/117600 [00:04<00:08, 11921.56 examples/s]
Tokenizing train dataset (num_proc=64):  14%|█▍        | 16688/117600 [00:04<00:08, 12451.34 examples/s]
Tokenizing train dataset (num_proc=64):  15%|█▌        | 18043/117600 [00:04<00:07, 12645.89 examples/s]
Tokenizing train dataset (num_proc=64):  17%|█▋        | 19423/117600 [00:05<00:07, 12668.27 examples/s]
Tokenizing train dataset (num_proc=64):  18%|█▊        | 20782/117600 [00:05<00:07, 12260.18 examples/s]
Tokenizing train dataset (num_proc=64):  19%|█▊        | 22040/117600 [00:05<00:07, 12330.93 examples/s]
Tokenizing train dataset (num_proc=64):  20%|█▉        | 23302/117600 [00:05<00:09, 9851.69 examples/s] 
Tokenizing train dataset (num_proc=64):  21%|██        | 24699/117600 [00:05<00:10, 9103.06 examples/s]
Tokenizing train dataset (num_proc=64):  22%|██▏       | 25699/117600 [00:05<00:10, 8665.93 examples/s]
Tokenizing train dataset (num_proc=64):  23%|██▎       | 27321/117600 [00:05<00:10, 8799.02 examples/s]
Tokenizing train dataset (num_proc=64):  25%|██▍       | 29095/117600 [00:06<00:09, 9167.80 examples/s]
Tokenizing train dataset (num_proc=64):  26%|██▌       | 30038/117600 [00:06<00:09, 8817.06 examples/s]
Tokenizing train dataset (num_proc=64):  30%|██▉       | 35077/117600 [00:06<00:04, 18054.95 examples/s]
Tokenizing train dataset (num_proc=64):  34%|███▍      | 40286/117600 [00:06<00:02, 26251.45 examples/s]
Tokenizing train dataset (num_proc=64):  37%|███▋      | 43355/117600 [00:06<00:03, 21808.34 examples/s]
Tokenizing train dataset (num_proc=64):  39%|███▉      | 45958/117600 [00:06<00:03, 18784.22 examples/s]
Tokenizing train dataset (num_proc=64):  41%|████      | 48188/117600 [00:06<00:03, 19334.06 examples/s]
Tokenizing train dataset (num_proc=64):  43%|████▎     | 50396/117600 [00:07<00:03, 19834.56 examples/s]
Tokenizing train dataset (num_proc=64):  45%|████▍     | 52599/117600 [00:07<00:03, 16618.19 examples/s]
Tokenizing train dataset (num_proc=64):  46%|████▋     | 54538/117600 [00:07<00:03, 17228.98 examples/s]
Tokenizing train dataset (num_proc=64):  48%|████▊     | 56434/117600 [00:07<00:03, 17513.71 examples/s]
Tokenizing train dataset (num_proc=64):  50%|████▉     | 58386/117600 [00:07<00:03, 18020.73 examples/s]
Tokenizing train dataset (num_proc=64):  51%|█████▏    | 60290/117600 [00:07<00:03, 17977.29 examples/s]
Tokenizing train dataset (num_proc=64):  53%|█████▎    | 62177/117600 [00:07<00:03, 18194.95 examples/s]
Tokenizing train dataset (num_proc=64):  54%|█████▍    | 64075/117600 [00:07<00:03, 14800.11 examples/s]
Tokenizing train dataset (num_proc=64):  56%|█████▌    | 65825/117600 [00:08<00:03, 15407.31 examples/s]
Tokenizing train dataset (num_proc=64):  57%|█████▋    | 67479/117600 [00:08<00:03, 15598.71 examples/s]
Tokenizing train dataset (num_proc=64):  59%|█████▉    | 69675/117600 [00:08<00:02, 17298.65 examples/s]
Tokenizing train dataset (num_proc=64):  61%|██████    | 71493/117600 [00:08<00:02, 16907.54 examples/s]
Tokenizing train dataset (num_proc=64):  63%|██████▎   | 73550/117600 [00:08<00:02, 17285.60 examples/s]
Tokenizing train dataset (num_proc=64):  64%|██████▍   | 75319/117600 [00:08<00:02, 14467.41 examples/s]
Tokenizing train dataset (num_proc=64):  65%|██████▌   | 76880/117600 [00:08<00:02, 14465.11 examples/s]
Tokenizing train dataset (num_proc=64):  67%|██████▋   | 79303/117600 [00:08<00:02, 16971.11 examples/s]
Tokenizing train dataset (num_proc=64):  69%|██████▉   | 81103/117600 [00:09<00:02, 13995.97 examples/s]
Tokenizing train dataset (num_proc=64):  71%|███████   | 83301/117600 [00:09<00:02, 15885.74 examples/s]
Tokenizing train dataset (num_proc=64):  72%|███████▏  | 85054/117600 [00:09<00:02, 16077.23 examples/s]
Tokenizing train dataset (num_proc=64):  74%|███████▍  | 86801/117600 [00:09<00:01, 16341.09 examples/s]
Tokenizing train dataset (num_proc=64):  75%|███████▌  | 88523/117600 [00:09<00:01, 16559.18 examples/s]
Tokenizing train dataset (num_proc=64):  77%|███████▋  | 90599/117600 [00:09<00:01, 16250.03 examples/s]
Tokenizing train dataset (num_proc=64):  79%|███████▊  | 92460/117600 [00:09<00:01, 16877.10 examples/s]
Tokenizing train dataset (num_proc=64):  80%|████████  | 94191/117600 [00:09<00:01, 16783.68 examples/s]
Tokenizing train dataset (num_proc=64):  82%|████████▏ | 96019/117600 [00:09<00:01, 17200.72 examples/s]
Tokenizing train dataset (num_proc=64):  83%|████████▎ | 97766/117600 [00:09<00:01, 16754.61 examples/s]
Tokenizing train dataset (num_proc=64):  85%|████████▌ | 100029/117600 [00:10<00:00, 18416.41 examples/s]
Tokenizing train dataset (num_proc=64):  87%|████████▋ | 101924/117600 [00:10<00:00, 16956.50 examples/s]
Tokenizing train dataset (num_proc=64):  88%|████████▊ | 103663/117600 [00:10<00:00, 16863.61 examples/s]
Tokenizing train dataset (num_proc=64):  90%|████████▉ | 105377/117600 [00:10<00:00, 15894.52 examples/s]
Tokenizing train dataset (num_proc=64):  91%|█████████ | 107162/117600 [00:10<00:00, 16425.19 examples/s]
Tokenizing train dataset (num_proc=64):  93%|█████████▎| 108855/117600 [00:10<00:00, 16506.49 examples/s]
Tokenizing train dataset (num_proc=64):  94%|█████████▍| 110730/117600 [00:10<00:00, 17106.29 examples/s]
Tokenizing train dataset (num_proc=64):  96%|█████████▌| 112464/117600 [00:10<00:00, 15181.05 examples/s]
Tokenizing train dataset (num_proc=64):  97%|█████████▋| 114056/117600 [00:11<00:00, 15324.58 examples/s]
Tokenizing train dataset (num_proc=64):  98%|█████████▊| 115637/117600 [00:11<00:00, 13236.85 examples/s]
Tokenizing train dataset (num_proc=64): 100%|█████████▉| 117035/117600 [00:11<00:00, 11759.07 examples/s]
Tokenizing train dataset (num_proc=64): 100%|█████████▉| 117553/117600 [00:23<00:00, 11759.07 examples/s]
Tokenizing train dataset (num_proc=64): 100%|█████████▉| 117581/117600 [00:31<00:00, 212.82 examples/s]  
Tokenizing train dataset (num_proc=64): 100%|█████████▉| 117599/117600 [00:33<00:00, 177.67 examples/s]
Tokenizing train dataset (num_proc=64): 100%|██████████| 117600/117600 [00:37<00:00, 3171.84 examples/s]

Tokenizing eval dataset (num_proc=64):   0%|          | 0/2400 [00:00<?, ? examples/s]
Tokenizing eval dataset (num_proc=64):   1%|          | 23/2400 [00:03<06:04,  6.53 examples/s]
Tokenizing eval dataset (num_proc=64):   4%|▍         | 103/2400 [00:03<01:01, 37.19 examples/s]
Tokenizing eval dataset (num_proc=64):   8%|▊         | 203/2400 [00:03<00:25, 87.05 examples/s]
Tokenizing eval dataset (num_proc=64):  13%|█▎        | 304/2400 [00:03<00:14, 149.51 examples/s]
Tokenizing eval dataset (num_proc=64):  18%|█▊        | 428/2400 [00:03<00:08, 244.40 examples/s]
Tokenizing eval dataset (num_proc=64):  23%|██▎       | 557/2400 [00:04<00:05, 356.08 examples/s]
Tokenizing eval dataset (num_proc=64):  28%|██▊       | 672/2400 [00:04<00:03, 460.08 examples/s]
Tokenizing eval dataset (num_proc=64):  32%|███▏      | 773/2400 [00:04<00:03, 541.31 examples/s]
Tokenizing eval dataset (num_proc=64):  36%|███▋      | 874/2400 [00:04<00:02, 613.89 examples/s]
Tokenizing eval dataset (num_proc=64):  42%|████▏     | 1000/2400 [00:04<00:01, 725.76 examples/s]
Tokenizing eval dataset (num_proc=64):  47%|████▋     | 1119/2400 [00:04<00:01, 813.13 examples/s]
Tokenizing eval dataset (num_proc=64):  52%|█████▏    | 1242/2400 [00:04<00:01, 901.11 examples/s]
Tokenizing eval dataset (num_proc=64):  56%|█████▋    | 1352/2400 [00:04<00:01, 858.55 examples/s]
Tokenizing eval dataset (num_proc=64):  61%|██████    | 1463/2400 [00:04<00:01, 916.66 examples/s]
Tokenizing eval dataset (num_proc=64):  67%|██████▋   | 1599/2400 [00:05<00:00, 950.68 examples/s]
Tokenizing eval dataset (num_proc=64):  72%|███████▏  | 1724/2400 [00:05<00:00, 1025.60 examples/s]
Tokenizing eval dataset (num_proc=64):  76%|███████▋  | 1833/2400 [00:05<00:00, 1038.31 examples/s]
Tokenizing eval dataset (num_proc=64):  82%|████████▏ | 1970/2400 [00:05<00:00, 1037.88 examples/s]
Tokenizing eval dataset (num_proc=64):  88%|████████▊ | 2104/2400 [00:05<00:00, 1107.38 examples/s]
Tokenizing eval dataset (num_proc=64):  93%|█████████▎| 2225/2400 [00:05<00:00, 1102.42 examples/s]
Tokenizing eval dataset (num_proc=64):  98%|█████████▊| 2349/2400 [00:05<00:00, 1102.17 examples/s]
Tokenizing eval dataset (num_proc=64): 100%|██████████| 2400/2400 [00:06<00:00, 363.13 examples/s] 
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': 2}.
==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1
   \\   /|    Num examples = 117,600 | Num Epochs = 3 | Total steps = 2,757
O^O/ \_/ \    Batch size per device = 8 | Gradient accumulation steps = 16
\        /    Data Parallel GPUs = 1 | Total batch size (8 x 16 x 1) = 128
 "-____-"     Trainable parameters = 9,768,960 of 144,283,968 (6.77% trained)

  0%|          | 0/2757 [00:00<?, ?it/s]`use_return_dict` is deprecated! Use `return_dict` instead!

  0%|          | 1/2757 [00:18<14:04:38, 18.39s/it]
  0%|          | 2/2757 [00:26<9:35:41, 12.54s/it] 
  0%|          | 3/2757 [00:35<8:04:08, 10.55s/it]
  0%|          | 4/2757 [00:42<7:14:13,  9.46s/it]
  0%|          | 5/2757 [00:50<6:49:58,  8.94s/it]
  0%|          | 6/2757 [00:58<6:35:17,  8.62s/it]
  0%|          | 7/2757 [01:06<6:25:30,  8.41s/it]
  0%|          | 8/2757 [01:15<6:24:42,  8.40s/it]
  0%|          | 9/2757 [01:23<6:25:05,  8.41s/it]
  0%|          | 10/2757 [01:31<6:14:47,  8.19s/it]
                                                   

  0%|          | 10/2757 [01:31<6:14:47,  8.19s/it]
  0%|          | 11/2757 [01:39<6:16:32,  8.23s/it]
  0%|          | 12/2757 [01:47<6:14:45,  8.19s/it]
  0%|          | 13/2757 [01:55<6:13:45,  8.17s/it]
  1%|          | 14/2757 [02:04<6:13:33,  8.17s/it]
  1%|          | 15/2757 [02:11<6:08:16,  8.06s/it]
  1%|          | 16/2757 [02:19<6:06:42,  8.03s/it]
  1%|          | 17/2757 [02:27<6:07:00,  8.04s/it]
  1%|          | 18/2757 [02:35<6:06:41,  8.03s/it]
  1%|          | 19/2757 [02:43<6:04:08,  7.98s/it]
  1%|          | 20/2757 [02:51<6:03:17,  7.96s/it]
                                                   

  1%|          | 20/2757 [02:51<6:03:17,  7.96s/it]
  1%|          | 21/2757 [02:59<6:03:00,  7.96s/it]
  1%|          | 22/2757 [03:07<6:07:19,  8.06s/it]
  1%|          | 23/2757 [03:16<6:08:49,  8.09s/it]
  1%|          | 24/2757 [03:23<6:04:23,  8.00s/it]
  1%|          | 25/2757 [03:31<5:58:22,  7.87s/it]
  1%|          | 26/2757 [03:40<6:08:21,  8.09s/it]
  1%|          | 27/2757 [03:47<6:05:53,  8.04s/it]
  1%|          | 28/2757 [03:56<6:07:42,  8.08s/it]
  1%|          | 29/2757 [04:04<6:10:32,  8.15s/it]
  1%|          | 30/2757 [04:11<6:01:58,  7.96s/it]
                                                   

  1%|          | 30/2757 [04:11<6:01:58,  7.96s/it]
  1%|          | 31/2757 [04:19<6:01:21,  7.95s/it]
  1%|          | 32/2757 [04:27<6:01:29,  7.96s/it]
  1%|          | 33/2757 [04:36<6:03:54,  8.02s/it]
  1%|          | 34/2757 [04:43<6:03:26,  8.01s/it]
  1%|▏         | 35/2757 [04:52<6:07:57,  8.11s/it]
  1%|▏         | 36/2757 [05:00<6:03:40,  8.02s/it]
  1%|▏         | 37/2757 [05:08<6:01:54,  7.98s/it]
  1%|▏         | 38/2757 [05:16<6:08:12,  8.13s/it]
  1%|▏         | 39/2757 [05:24<6:07:04,  8.10s/it]
  1%|▏         | 40/2757 [05:32<6:00:19,  7.96s/it]
                                                   

  1%|▏         | 40/2757 [05:32<6:00:19,  7.96s/it]
  1%|▏         | 41/2757 [05:39<5:54:11,  7.82s/it]
  2%|▏         | 42/2757 [05:47<5:58:43,  7.93s/it]
  2%|▏         | 43/2757 [05:55<5:55:50,  7.87s/it]
  2%|▏         | 44/2757 [06:03<5:55:02,  7.85s/it]
  2%|▏         | 45/2757 [06:11<5:58:09,  7.92s/it]
  2%|▏         | 46/2757 [06:19<6:05:15,  8.08s/it]
  2%|▏         | 47/2757 [06:27<6:01:18,  8.00s/it]
  2%|▏         | 48/2757 [06:35<6:04:18,  8.07s/it]
  2%|▏         | 49/2757 [06:43<6:01:21,  8.01s/it]
  2%|▏         | 50/2757 [06:51<5:57:05,  7.91s/it]
                                                   

  2%|▏         | 50/2757 [06:51<5:57:05,  7.91s/it]
  2%|▏         | 51/2757 [06:59<5:55:45,  7.89s/it]
  2%|▏         | 52/2757 [07:07<6:04:10,  8.08s/it]
  2%|▏         | 53/2757 [07:15<5:55:19,  7.88s/it]
  2%|▏         | 54/2757 [07:23<6:03:22,  8.07s/it]
  2%|▏         | 55/2757 [07:32<6:07:54,  8.17s/it]
  2%|▏         | 56/2757 [07:39<5:59:55,  8.00s/it]
  2%|▏         | 57/2757 [07:48<6:06:46,  8.15s/it]
  2%|▏         | 58/2757 [07:56<6:05:33,  8.13s/it]
  2%|▏         | 59/2757 [08:04<6:08:12,  8.19s/it]
  2%|▏         | 60/2757 [08:12<6:04:13,  8.10s/it]
                                                   

  2%|▏         | 60/2757 [08:12<6:04:13,  8.10s/it]
  2%|▏         | 61/2757 [08:20<6:02:48,  8.07s/it]
  2%|▏         | 62/2757 [08:28<5:59:11,  8.00s/it]
  2%|▏         | 63/2757 [08:36<6:02:13,  8.07s/it]
  2%|▏         | 64/2757 [08:44<6:00:34,  8.03s/it]
  2%|▏         | 65/2757 [08:52<6:00:39,  8.04s/it]
  2%|▏         | 66/2757 [09:00<5:52:05,  7.85s/it]
  2%|▏         | 67/2757 [09:08<5:53:32,  7.89s/it]
  2%|▏         | 68/2757 [09:15<5:52:41,  7.87s/it]
  3%|▎         | 69/2757 [09:24<5:57:26,  7.98s/it]
  3%|▎         | 70/2757 [09:32<5:56:28,  7.96s/it]
                                                   

  3%|▎         | 70/2757 [09:32<5:56:28,  7.96s/it]
  3%|▎         | 71/2757 [09:39<5:55:12,  7.93s/it]
  3%|▎         | 72/2757 [09:47<5:54:07,  7.91s/it]
  3%|▎         | 73/2757 [09:55<5:56:56,  7.98s/it]
  3%|▎         | 74/2757 [10:03<5:55:35,  7.95s/it]
  3%|▎         | 75/2757 [10:11<5:54:36,  7.93s/it]
  3%|▎         | 76/2757 [10:19<5:52:56,  7.90s/it]
  3%|▎         | 77/2757 [10:27<5:48:11,  7.80s/it]
  3%|▎         | 78/2757 [10:34<5:47:47,  7.79s/it]
  3%|▎         | 79/2757 [10:42<5:51:32,  7.88s/it]
  3%|▎         | 80/2757 [10:50<5:49:30,  7.83s/it]
                                                   

  3%|▎         | 80/2757 [10:50<5:49:30,  7.83s/it]
  3%|▎         | 81/2757 [10:58<5:47:39,  7.79s/it]
  3%|▎         | 82/2757 [11:06<5:52:02,  7.90s/it]
  3%|▎         | 83/2757 [11:14<5:52:07,  7.90s/it]
  3%|▎         | 84/2757 [11:22<5:52:55,  7.92s/it]
  3%|▎         | 85/2757 [11:30<5:52:10,  7.91s/it]
  3%|▎         | 86/2757 [11:38<5:50:13,  7.87s/it]
  3%|▎         | 87/2757 [11:45<5:45:48,  7.77s/it]
  3%|▎         | 88/2757 [11:53<5:52:14,  7.92s/it]
  3%|▎         | 89/2757 [12:02<5:59:16,  8.08s/it]
  3%|▎         | 90/2757 [12:10<6:03:47,  8.18s/it]
                                                   

  3%|▎         | 90/2757 [12:10<6:03:47,  8.18s/it]
  3%|▎         | 91/2757 [12:18<5:58:38,  8.07s/it]
  3%|▎         | 92/2757 [12:26<5:56:40,  8.03s/it]
  3%|▎         | 93/2757 [12:34<5:57:56,  8.06s/it]
  3%|▎         | 94/2757 [12:42<6:02:15,  8.16s/it]
  3%|▎         | 95/2757 [12:50<5:57:54,  8.07s/it]
  3%|▎         | 96/2757 [12:58<5:56:07,  8.03s/it]
  4%|▎         | 97/2757 [13:06<5:57:03,  8.05s/it]
  4%|▎         | 98/2757 [13:14<5:57:09,  8.06s/it]
  4%|▎         | 99/2757 [13:22<5:54:14,  8.00s/it]
  4%|▎         | 100/2757 [13:30<5:53:32,  7.98s/it]
                                                    

  4%|▎         | 100/2757 [13:30<5:53:32,  7.98s/it]
  4%|▎         | 101/2757 [13:38<5:50:59,  7.93s/it]
  4%|▎         | 102/2757 [13:46<5:48:13,  7.87s/it]
  4%|▎         | 103/2757 [13:54<5:54:47,  8.02s/it]
  4%|▍         | 104/2757 [14:02<5:57:02,  8.07s/it]
  4%|▍         | 105/2757 [14:10<5:53:03,  7.99s/it]
  4%|▍         | 106/2757 [14:18<5:54:04,  8.01s/it]
  4%|▍         | 107/2757 [14:26<5:55:47,  8.06s/it]
  4%|▍         | 108/2757 [14:35<5:58:09,  8.11s/it]
  4%|▍         | 109/2757 [14:43<5:58:13,  8.12s/it]
  4%|▍         | 110/2757 [14:51<5:56:32,  8.08s/it]
                                                    

  4%|▍         | 110/2757 [14:51<5:56:32,  8.08s/it]
  4%|▍         | 111/2757 [14:58<5:46:26,  7.86s/it]
  4%|▍         | 112/2757 [15:06<5:48:07,  7.90s/it]
  4%|▍         | 113/2757 [15:14<5:50:11,  7.95s/it]
  4%|▍         | 114/2757 [15:22<5:43:15,  7.79s/it]
  4%|▍         | 115/2757 [15:30<5:47:38,  7.90s/it]
  4%|▍         | 116/2757 [15:37<5:44:38,  7.83s/it]
  4%|▍         | 117/2757 [15:45<5:46:51,  7.88s/it]
  4%|▍         | 118/2757 [15:53<5:45:31,  7.86s/it]
  4%|▍         | 119/2757 [16:01<5:43:45,  7.82s/it]
  4%|▍         | 120/2757 [16:09<5:43:17,  7.81s/it]
                                                    

  4%|▍         | 120/2757 [16:09<5:43:17,  7.81s/it]
  4%|▍         | 121/2757 [16:17<5:43:40,  7.82s/it]
  4%|▍         | 122/2757 [16:25<5:48:11,  7.93s/it]
  4%|▍         | 123/2757 [16:32<5:44:34,  7.85s/it]
  4%|▍         | 124/2757 [16:40<5:45:00,  7.86s/it]
  5%|▍         | 125/2757 [16:48<5:39:24,  7.74s/it]
  5%|▍         | 126/2757 [16:56<5:41:44,  7.79s/it]
  5%|▍         | 127/2757 [17:04<5:43:58,  7.85s/it]
  5%|▍         | 128/2757 [17:11<5:43:42,  7.84s/it]
  5%|▍         | 129/2757 [17:20<5:50:34,  8.00s/it]
  5%|▍         | 130/2757 [17:28<5:49:17,  7.98s/it]
                                                    

  5%|▍         | 130/2757 [17:28<5:49:17,  7.98s/it]
  5%|▍         | 131/2757 [17:36<5:52:59,  8.07s/it]
  5%|▍         | 132/2757 [17:44<5:47:31,  7.94s/it]
  5%|▍         | 133/2757 [17:52<5:50:43,  8.02s/it]
  5%|▍         | 134/2757 [18:00<5:56:00,  8.14s/it]
  5%|▍         | 135/2757 [18:08<5:56:06,  8.15s/it]
  5%|▍         | 136/2757 [18:16<5:53:39,  8.10s/it]
  5%|▍         | 137/2757 [18:25<5:58:56,  8.22s/it]
  5%|▌         | 138/2757 [18:33<5:53:32,  8.10s/it]
  5%|▌         | 139/2757 [18:41<5:51:06,  8.05s/it]
  5%|▌         | 140/2757 [18:49<5:48:37,  7.99s/it]
                                                    

  5%|▌         | 140/2757 [18:49<5:48:37,  7.99s/it]
  5%|▌         | 141/2757 [18:56<5:46:16,  7.94s/it]
  5%|▌         | 142/2757 [19:04<5:47:20,  7.97s/it]
  5%|▌         | 143/2757 [19:13<5:48:47,  8.01s/it]
  5%|▌         | 144/2757 [19:21<5:51:13,  8.06s/it]
  5%|▌         | 145/2757 [19:29<5:54:24,  8.14s/it]
  5%|▌         | 146/2757 [19:37<5:57:37,  8.22s/it]
  5%|▌         | 147/2757 [19:45<5:54:26,  8.15s/it]
  5%|▌         | 148/2757 [19:53<5:52:32,  8.11s/it]
  5%|▌         | 149/2757 [20:01<5:50:33,  8.07s/it]
  5%|▌         | 150/2757 [20:09<5:48:17,  8.02s/it]
                                                    

  5%|▌         | 150/2757 [20:09<5:48:17,  8.02s/it]
  5%|▌         | 151/2757 [20:18<5:53:52,  8.15s/it]
  6%|▌         | 152/2757 [20:26<5:54:13,  8.16s/it]
  6%|▌         | 153/2757 [20:34<5:46:55,  7.99s/it]
  6%|▌         | 154/2757 [20:41<5:45:24,  7.96s/it]
  6%|▌         | 155/2757 [20:50<5:48:40,  8.04s/it]
  6%|▌         | 156/2757 [20:58<5:52:52,  8.14s/it]
  6%|▌         | 157/2757 [21:06<5:45:25,  7.97s/it]
  6%|▌         | 158/2757 [21:14<5:47:00,  8.01s/it]
  6%|▌         | 159/2757 [21:21<5:42:50,  7.92s/it]
  6%|▌         | 160/2757 [21:30<5:50:19,  8.09s/it]
                                                    

  6%|▌         | 160/2757 [21:30<5:50:19,  8.09s/it]
  6%|▌         | 161/2757 [21:38<5:43:46,  7.95s/it]
  6%|▌         | 162/2757 [21:46<5:49:53,  8.09s/it]
  6%|▌         | 163/2757 [21:54<5:53:17,  8.17s/it]
  6%|▌         | 164/2757 [22:03<5:53:42,  8.18s/it]
  6%|▌         | 165/2757 [22:11<5:51:56,  8.15s/it]
  6%|▌         | 166/2757 [22:18<5:44:45,  7.98s/it]
  6%|▌         | 167/2757 [22:26<5:41:45,  7.92s/it]
  6%|▌         | 168/2757 [22:34<5:42:09,  7.93s/it]
  6%|▌         | 169/2757 [22:42<5:49:06,  8.09s/it]
  6%|▌         | 170/2757 [22:50<5:48:26,  8.08s/it]
                                                    

  6%|▌         | 170/2757 [22:50<5:48:26,  8.08s/it]
  6%|▌         | 171/2757 [22:59<5:48:44,  8.09s/it]
  6%|▌         | 172/2757 [23:07<5:49:53,  8.12s/it]
  6%|▋         | 173/2757 [23:15<5:45:13,  8.02s/it]
  6%|▋         | 174/2757 [23:22<5:39:45,  7.89s/it]
  6%|▋         | 175/2757 [23:30<5:41:57,  7.95s/it]
  6%|▋         | 176/2757 [23:38<5:45:40,  8.04s/it]
  6%|▋         | 177/2757 [23:46<5:44:38,  8.01s/it]
  6%|▋         | 178/2757 [23:55<5:46:14,  8.06s/it]
  6%|▋         | 179/2757 [24:02<5:42:59,  7.98s/it]
  7%|▋         | 180/2757 [24:11<5:47:18,  8.09s/it]
                                                    

  7%|▋         | 180/2757 [24:11<5:47:18,  8.09s/it]
  7%|▋         | 181/2757 [24:19<5:45:26,  8.05s/it]
  7%|▋         | 182/2757 [24:27<5:44:17,  8.02s/it]
  7%|▋         | 183/2757 [24:35<5:42:33,  7.99s/it]
  7%|▋         | 184/2757 [24:42<5:41:50,  7.97s/it]
  7%|▋         | 185/2757 [24:51<5:43:10,  8.01s/it]
  7%|▋         | 186/2757 [24:58<5:42:01,  7.98s/it]
  7%|▋         | 187/2757 [25:07<5:44:24,  8.04s/it]
  7%|▋         | 188/2757 [25:15<5:47:33,  8.12s/it]
  7%|▋         | 189/2757 [25:23<5:50:12,  8.18s/it]
  7%|▋         | 190/2757 [25:31<5:46:29,  8.10s/it]
                                                    

  7%|▋         | 190/2757 [25:31<5:46:29,  8.10s/it]
  7%|▋         | 191/2757 [25:39<5:41:36,  7.99s/it]
  7%|▋         | 192/2757 [25:47<5:37:46,  7.90s/it]
  7%|▋         | 193/2757 [25:55<5:38:33,  7.92s/it]
  7%|▋         | 194/2757 [26:03<5:38:24,  7.92s/it]
  7%|▋         | 195/2757 [26:11<5:43:54,  8.05s/it]
  7%|▋         | 196/2757 [26:19<5:40:39,  7.98s/it]
  7%|▋         | 197/2757 [26:27<5:43:10,  8.04s/it]
  7%|▋         | 198/2757 [26:35<5:43:13,  8.05s/it]
  7%|▋         | 199/2757 [26:43<5:46:33,  8.13s/it]
  7%|▋         | 200/2757 [26:51<5:47:05,  8.14s/it]
                                                    

  7%|▋         | 200/2757 [26:51<5:47:05,  8.14s/it]
  7%|▋         | 201/2757 [27:00<5:52:34,  8.28s/it]
  7%|▋         | 202/2757 [27:08<5:46:34,  8.14s/it]
  7%|▋         | 203/2757 [27:16<5:41:01,  8.01s/it]
  7%|▋         | 204/2757 [27:24<5:46:03,  8.13s/it]
  7%|▋         | 205/2757 [27:32<5:43:59,  8.09s/it]
  7%|▋         | 206/2757 [27:40<5:38:35,  7.96s/it]
  8%|▊         | 207/2757 [27:48<5:44:00,  8.09s/it]
  8%|▊         | 208/2757 [27:56<5:43:58,  8.10s/it]
  8%|▊         | 209/2757 [28:04<5:38:30,  7.97s/it]
  8%|▊         | 210/2757 [28:12<5:38:50,  7.98s/it]
                                                    

  8%|▊         | 210/2757 [28:12<5:38:50,  7.98s/it]
  8%|▊         | 211/2757 [28:20<5:37:48,  7.96s/it]
  8%|▊         | 212/2757 [28:28<5:46:09,  8.16s/it]
  8%|▊         | 213/2757 [28:36<5:41:16,  8.05s/it]
  8%|▊         | 214/2757 [28:45<5:50:48,  8.28s/it]
  8%|▊         | 215/2757 [28:53<5:51:39,  8.30s/it]
  8%|▊         | 216/2757 [29:01<5:46:50,  8.19s/it]
  8%|▊         | 217/2757 [29:09<5:47:46,  8.21s/it]
  8%|▊         | 218/2757 [29:18<5:52:22,  8.33s/it]
  8%|▊         | 219/2757 [29:26<5:47:59,  8.23s/it]
  8%|▊         | 220/2757 [29:34<5:44:29,  8.15s/it]
                                                    

  8%|▊         | 220/2757 [29:34<5:44:29,  8.15s/it]
  8%|▊         | 221/2757 [29:42<5:44:06,  8.14s/it]
  8%|▊         | 222/2757 [29:50<5:44:42,  8.16s/it]
  8%|▊         | 223/2757 [29:58<5:43:59,  8.15s/it]
  8%|▊         | 224/2757 [30:06<5:38:55,  8.03s/it]
  8%|▊         | 225/2757 [30:14<5:31:42,  7.86s/it]
  8%|▊         | 226/2757 [30:21<5:29:38,  7.81s/it]
  8%|▊         | 227/2757 [30:30<5:34:35,  7.93s/it]
  8%|▊         | 228/2757 [30:38<5:35:18,  7.96s/it]
  8%|▊         | 229/2757 [30:46<5:35:13,  7.96s/it]
  8%|▊         | 230/2757 [30:53<5:32:12,  7.89s/it]
                                                    

  8%|▊         | 230/2757 [30:53<5:32:12,  7.89s/it]
  8%|▊         | 231/2757 [31:01<5:34:40,  7.95s/it]
  8%|▊         | 232/2757 [31:09<5:36:14,  7.99s/it]
  8%|▊         | 233/2757 [31:18<5:41:15,  8.11s/it]
  8%|▊         | 234/2757 [31:27<5:49:52,  8.32s/it]
  9%|▊         | 235/2757 [31:35<5:47:49,  8.27s/it]
  9%|▊         | 236/2757 [31:43<5:42:59,  8.16s/it]
  9%|▊         | 237/2757 [31:51<5:42:50,  8.16s/it]
  9%|▊         | 238/2757 [31:59<5:38:35,  8.06s/it]
  9%|▊         | 239/2757 [32:07<5:40:33,  8.12s/it]
  9%|▊         | 240/2757 [32:15<5:42:59,  8.18s/it]
                                                    

  9%|▊         | 240/2757 [32:15<5:42:59,  8.18s/it]
  9%|▊         | 241/2757 [32:24<5:46:28,  8.26s/it]
  9%|▉         | 242/2757 [32:32<5:50:12,  8.35s/it]
  9%|▉         | 243/2757 [32:41<5:49:12,  8.33s/it]
  9%|▉         | 244/2757 [32:49<5:47:50,  8.31s/it]
  9%|▉         | 245/2757 [32:57<5:46:34,  8.28s/it]
  9%|▉         | 246/2757 [33:05<5:42:16,  8.18s/it]
  9%|▉         | 247/2757 [33:13<5:43:12,  8.20s/it]
  9%|▉         | 248/2757 [33:22<5:44:39,  8.24s/it]
  9%|▉         | 249/2757 [33:30<5:48:51,  8.35s/it]
  9%|▉         | 250/2757 [33:38<5:40:57,  8.16s/it]
                                                    

  9%|▉         | 250/2757 [33:38<5:40:57,  8.16s/it]
  9%|▉         | 251/2757 [33:50<6:24:38,  9.21s/it]
  9%|▉         | 252/2757 [33:57<6:01:28,  8.66s/it]
  9%|▉         | 253/2757 [34:04<5:45:47,  8.29s/it]
  9%|▉         | 254/2757 [34:12<5:40:16,  8.16s/it]
  9%|▉         | 255/2757 [34:20<5:34:10,  8.01s/it]
  9%|▉         | 256/2757 [34:28<5:30:07,  7.92s/it]
  9%|▉         | 257/2757 [34:36<5:30:47,  7.94s/it]
  9%|▉         | 258/2757 [34:44<5:34:20,  8.03s/it]
  9%|▉         | 259/2757 [34:52<5:36:06,  8.07s/it]
  9%|▉         | 260/2757 [35:00<5:38:48,  8.14s/it]
                                                    
Unsloth: Will smartly offload gradients to save VRAM!
Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
{'loss': '0.693', 'grad_norm': '0.7077', 'learning_rate': '2.169e-05', 'rewards/chosen': '0.001227', 'rewards/rejected': '-0.00209', 'rewards/accuracies': '0.3492', 'rewards/margins': '0.003317', 'logps/chosen': '-76.98', 'logps/rejected': '-83.68', 'logits/chosen': '-2.212', 'logits/rejected': '-2.102', 'epoch': '0.01088'}
{'loss': '0.6921', 'grad_norm': '0.9575', 'learning_rate': '4.578e-05', 'rewards/chosen': '0.0005895', 'rewards/rejected': '-0.005486', 'rewards/accuracies': '0.4672', 'rewards/margins': '0.006076', 'logps/chosen': '-79.9', 'logps/rejected': '-86.09', 'logits/chosen': '-2.361', 'logits/rejected': '-2.318', 'epoch': '0.02177'}
{'loss': '0.6918', 'grad_norm': '0.6923', 'learning_rate': '6.988e-05', 'rewards/chosen': '-0.01929', 'rewards/rejected': '-0.02695', 'rewards/accuracies': '0.4766', 'rewards/margins': '0.007659', 'logps/chosen': '-80.27', 'logps/rejected': '-87.22', 'logits/chosen': '-2.296', 'logits/rejected': '-2.237', 'epoch': '0.03265'}
{'loss': '0.6888', 'grad_norm': '0.6989', 'learning_rate': '9.398e-05', 'rewards/chosen': '-0.08199', 'rewards/rejected': '-0.0995', 'rewards/accuracies': '0.4875', 'rewards/margins': '0.01751', 'logps/chosen': '-84.66', 'logps/rejected': '-91.65', 'logits/chosen': '-2.305', 'logits/rejected': '-2.264', 'epoch': '0.04354'}
{'loss': '0.6795', 'grad_norm': '0.7515', 'learning_rate': '0.0001181', 'rewards/chosen': '-0.1621', 'rewards/rejected': '-0.2124', 'rewards/accuracies': '0.5016', 'rewards/margins': '0.05032', 'logps/chosen': '-86.99', 'logps/rejected': '-93.27', 'logits/chosen': '-2.576', 'logits/rejected': '-2.516', 'epoch': '0.05442'}
{'loss': '0.6819', 'grad_norm': '0.864', 'learning_rate': '0.0001422', 'rewards/chosen': '-0.202', 'rewards/rejected': '-0.2602', 'rewards/accuracies': '0.5117', 'rewards/margins': '0.05819', 'logps/chosen': '-83.06', 'logps/rejected': '-88.82', 'logits/chosen': '-2.496', 'logits/rejected': '-2.35', 'epoch': '0.06531'}
{'loss': '0.6768', 'grad_norm': '0.8207', 'learning_rate': '0.0001663', 'rewards/chosen': '-0.1923', 'rewards/rejected': '-0.2661', 'rewards/accuracies': '0.5203', 'rewards/margins': '0.07378', 'logps/chosen': '-81.74', 'logps/rejected': '-87.19', 'logits/chosen': '-2.699', 'logits/rejected': '-2.621', 'epoch': '0.07619'}
{'loss': '0.6668', 'grad_norm': '0.8357', 'learning_rate': '0.0001904', 'rewards/chosen': '-0.2063', 'rewards/rejected': '-0.32', 'rewards/accuracies': '0.5289', 'rewards/margins': '0.1137', 'logps/chosen': '-82.09', 'logps/rejected': '-89.19', 'logits/chosen': '-2.595', 'logits/rejected': '-2.484', 'epoch': '0.08707'}
{'loss': '0.6667', 'grad_norm': '0.7759', 'learning_rate': '0.0001996', 'rewards/chosen': '-0.1989', 'rewards/rejected': '-0.3181', 'rewards/accuracies': '0.5469', 'rewards/margins': '0.1192', 'logps/chosen': '-80.27', 'logps/rejected': '-88.54', 'logits/chosen': '-2.37', 'logits/rejected': '-2.255', 'epoch': '0.09796'}
{'loss': '0.6667', 'grad_norm': '0.7275', 'learning_rate': '0.0001988', 'rewards/chosen': '-0.2421', 'rewards/rejected': '-0.3706', 'rewards/accuracies': '0.5492', 'rewards/margins': '0.1285', 'logps/chosen': '-85.48', 'logps/rejected': '-92.23', 'logits/chosen': '-2.458', 'logits/rejected': '-2.306', 'epoch': '0.1088'}
{'loss': '0.6598', 'grad_norm': '0.8685', 'learning_rate': '0.0001981', 'rewards/chosen': '-0.1943', 'rewards/rejected': '-0.3449', 'rewards/accuracies': '0.5523', 'rewards/margins': '0.1507', 'logps/chosen': '-81.67', 'logps/rejected': '-89.74', 'logits/chosen': '-2.194', 'logits/rejected': '-2.128', 'epoch': '0.1197'}
{'loss': '0.6632', 'grad_norm': '0.7957', 'learning_rate': '0.0001973', 'rewards/chosen': '-0.1917', 'rewards/rejected': '-0.3336', 'rewards/accuracies': '0.5422', 'rewards/margins': '0.1418', 'logps/chosen': '-84.79', 'logps/rejected': '-90.5', 'logits/chosen': '-2.447', 'logits/rejected': '-2.352', 'epoch': '0.1306'}
{'loss': '0.6509', 'grad_norm': '0.6242', 'learning_rate': '0.0001966', 'rewards/chosen': '-0.1273', 'rewards/rejected': '-0.2984', 'rewards/accuracies': '0.5508', 'rewards/margins': '0.1711', 'logps/chosen': '-79.19', 'logps/rejected': '-85.88', 'logits/chosen': '-2.458', 'logits/rejected': '-2.388', 'epoch': '0.1415'}
{'loss': '0.6572', 'grad_norm': '0.6959', 'learning_rate': '0.0001958', 'rewards/chosen': '-0.1161', 'rewards/rejected': '-0.282', 'rewards/accuracies': '0.5641', 'rewards/margins': '0.166', 'logps/chosen': '-82.53', 'logps/rejected': '-89.38', 'logits/chosen': '-2.645', 'logits/rejected': '-2.575', 'epoch': '0.1524'}
{'loss': '0.6591', 'grad_norm': '0.704', 'learning_rate': '0.0001951', 'rewards/chosen': '-0.0406', 'rewards/rejected': '-0.2036', 'rewards/accuracies': '0.5641', 'rewards/margins': '0.163', 'logps/chosen': '-84.36', 'logps/rejected': '-91.33', 'logits/chosen': '-2.601', 'logits/rejected': '-2.53', 'epoch': '0.1633'}
{'loss': '0.6441', 'grad_norm': '0.6618', 'learning_rate': '0.0001943', 'rewards/chosen': '-0.01285', 'rewards/rejected': '-0.201', 'rewards/accuracies': '0.5883', 'rewards/margins': '0.1881', 'logps/chosen': '-83.15', 'logps/rejected': '-89.39', 'logits/chosen': '-2.715', 'logits/rejected': '-2.597', 'epoch': '0.1741'}
{'loss': '0.653', 'grad_norm': '0.6484', 'learning_rate': '0.0001936', 'rewards/chosen': '-0.05956', 'rewards/rejected': '-0.2501', 'rewards/accuracies': '0.5641', 'rewards/margins': '0.1905', 'logps/chosen': '-79.66', 'logps/rejected': '-85.64', 'logits/chosen': '-2.808', 'logits/rejected': '-2.71', 'epoch': '0.185'}
{'loss': '0.671', 'grad_norm': '0.6358', 'learning_rate': '0.0001928', 'rewards/chosen': '-0.1839', 'rewards/rejected': '-0.3243', 'rewards/accuracies': '0.557', 'rewards/margins': '0.1405', 'logps/chosen': '-86.25', 'logps/rejected': '-91.76', 'logits/chosen': '-2.726', 'logits/rejected': '-2.632', 'epoch': '0.1959'}
{'loss': '0.6496', 'grad_norm': '0.7087', 'learning_rate': '0.0001921', 'rewards/chosen': '-0.259', 'rewards/rejected': '-0.4433', 'rewards/accuracies': '0.5711', 'rewards/margins': '0.1842', 'logps/chosen': '-83.1', 'logps/rejected': '-90.81', 'logits/chosen': '-2.575', 'logits/rejected': '-2.476', 'epoch': '0.2068'}
{'loss': '0.6528', 'grad_norm': '0.7277', 'learning_rate': '0.0001913', 'rewards/chosen': '-0.2878', 'rewards/rejected': '-0.4894', 'rewards/accuracies': '0.5523', 'rewards/margins': '0.2016', 'logps/chosen': '-84.37', 'logps/rejected': '-92.43', 'logits/chosen': '-2.665', 'logits/rejected': '-2.642', 'epoch': '0.2177'}
{'loss': '0.6556', 'grad_norm': '0.7041', 'learning_rate': '0.0001906', 'rewards/chosen': '-0.175', 'rewards/rejected': '-0.3603', 'rewards/accuracies': '0.5703', 'rewards/margins': '0.1853', 'logps/chosen': '-83.11', 'logps/rejected': '-90.62', 'logits/chosen': '-2.796', 'logits/rejected': '-2.704', 'epoch': '0.2286'}
{'loss': '0.6463', 'grad_norm': '0.6894', 'learning_rate': '0.0001898', 'rewards/chosen': '0.06025', 'rewards/rejected': '-0.1298', 'rewards/accuracies': '0.5805', 'rewards/margins': '0.19', 'logps/chosen': '-78.92', 'logps/rejected': '-84.79', 'logits/chosen': '-2.694', 'logits/rejected': '-2.68', 'epoch': '0.2395'}
{'loss': '0.6312', 'grad_norm': '0.6098', 'learning_rate': '0.0001891', 'rewards/chosen': '0.03963', 'rewards/rejected': '-0.1992', 'rewards/accuracies': '0.5922', 'rewards/margins': '0.2389', 'logps/chosen': '-80.18', 'logps/rejected': '-86.77', 'logits/chosen': '-2.949', 'logits/rejected': '-2.881', 'epoch': '0.2503'}
{'loss': '0.6405', 'grad_norm': '0.6649', 'learning_rate': '0.0001883', 'rewards/chosen': '-0.1528', 'rewards/rejected': '-0.3926', 'rewards/accuracies': '0.593', 'rewards/margins': '0.2397', 'logps/chosen': '-82.76', 'logps/rejected': '-92.11', 'logits/chosen': '-3.144', 'logits/rejected': '-3.156', 'epoch': '0.2612'}
{'loss': '0.629', 'grad_norm': '0.7529', 'learning_rate': '0.0001876', 'rewards/chosen': '-0.3856', 'rewards/rejected': '-0.6723', 'rewards/accuracies': '0.5914', 'rewards/margins': '0.2867', 'logps/chosen': '-88.3', 'logps/rejected': '-97.05', 'logits/chosen': '-3.041', 'logits/rejected': '-3.004', 'epoch': '0.2721'}
{'loss': '0.62', 'grad_norm': '0.6279', 'learning_rate': '0.0001868', 'rewards/chosen': '-0.438', 'rewards/rejected': '-0.7219', 'rewards/accuracies': '0.5859', 'rewards/margins': '0.284', 'logps/chosen': '-87.58', 'logps/rejected': '-93.7', 'logits/chosen': '-3.091', 'logits/rejected': '-2.95', 'epoch': '0.283'}
  9%|▉         | 260/2757 [35:00<5:38:48,  8.14s/it]
  9%|▉         | 261/2757 [35:08<5:36:28,  8.09s/it]
 10%|▉         | 262/2757 [35:16<5:35:03,  8.06s/it]
 10%|▉         | 263/2757 [35:24<5:34:47,  8.05s/it]
 10%|▉         | 264/2757 [35:32<5:35:24,  8.07s/it]
 10%|▉         | 265/2757 [35:41<5:40:45,  8.20s/it]
 10%|▉         | 266/2757 [35:49<5:38:07,  8.14s/it]
 10%|▉         | 267/2757 [35:57<5:41:59,  8.24s/it]
 10%|▉         | 268/2757 [36:05<5:31:09,  7.98s/it]
 10%|▉         | 269/2757 [36:13<5:32:00,  8.01s/it]
 10%|▉         | 270/2757 [36:20<5:25:24,  7.85s/it]
                                                    

 10%|▉         | 270/2757 [36:20<5:25:24,  7.85s/it]
 10%|▉         | 271/2757 [36:28<5:27:27,  7.90s/it]
 10%|▉         | 272/2757 [36:36<5:29:00,  7.94s/it]
 10%|▉         | 273/2757 [36:45<5:34:02,  8.07s/it]
 10%|▉         | 274/2757 [36:53<5:33:43,  8.06s/it]
 10%|▉         | 275/2757 [37:01<5:31:46,  8.02s/it]
 10%|█         | 276/2757 [37:08<5:25:47,  7.88s/it]
 10%|█         | 277/2757 [37:17<5:36:08,  8.13s/it]
 10%|█         | 278/2757 [37:25<5:34:53,  8.11s/it]
 10%|█         | 279/2757 [37:34<5:39:02,  8.21s/it]
 10%|█         | 280/2757 [37:42<5:36:54,  8.16s/it]
                                                    

 10%|█         | 280/2757 [37:42<5:36:54,  8.16s/it]
 10%|█         | 281/2757 [37:49<5:30:11,  8.00s/it]
 10%|█         | 282/2757 [37:57<5:32:12,  8.05s/it]
 10%|█         | 283/2757 [38:06<5:35:45,  8.14s/it]
 10%|█         | 284/2757 [38:14<5:34:14,  8.11s/it]
 10%|█         | 285/2757 [38:21<5:29:31,  8.00s/it]
 10%|█         | 286/2757 [38:30<5:35:09,  8.14s/it]
 10%|█         | 287/2757 [38:38<5:28:24,  7.98s/it]
 10%|█         | 288/2757 [38:46<5:36:57,  8.19s/it]
 10%|█         | 289/2757 [38:54<5:32:11,  8.08s/it]
 11%|█         | 290/2757 [39:02<5:33:59,  8.12s/it]
                                                    

 11%|█         | 290/2757 [39:02<5:33:59,  8.12s/it]
 11%|█         | 291/2757 [39:10<5:27:12,  7.96s/it]
 11%|█         | 292/2757 [39:18<5:25:56,  7.93s/it]
 11%|█         | 293/2757 [39:25<5:20:08,  7.80s/it]
 11%|█         | 294/2757 [39:33<5:21:08,  7.82s/it]
 11%|█         | 295/2757 [39:42<5:30:09,  8.05s/it]
 11%|█         | 296/2757 [39:49<5:26:05,  7.95s/it]
 11%|█         | 297/2757 [39:57<5:25:10,  7.93s/it]
 11%|█         | 298/2757 [40:06<5:28:38,  8.02s/it]
 11%|█         | 299/2757 [40:13<5:27:05,  7.98s/it]
 11%|█         | 300/2757 [40:22<5:29:03,  8.04s/it]
                                                    

 11%|█         | 300/2757 [40:22<5:29:03,  8.04s/it]
 11%|█         | 301/2757 [40:30<5:36:05,  8.21s/it]
 11%|█         | 302/2757 [40:38<5:29:15,  8.05s/it]
 11%|█         | 303/2757 [40:46<5:28:38,  8.04s/it]
 11%|█         | 304/2757 [40:54<5:25:46,  7.97s/it]
 11%|█         | 305/2757 [41:02<5:25:20,  7.96s/it]
 11%|█         | 306/2757 [41:10<5:25:19,  7.96s/it]
 11%|█         | 307/2757 [41:18<5:28:42,  8.05s/it]
 11%|█         | 308/2757 [41:26<5:32:15,  8.14s/it]
 11%|█         | 309/2757 [41:34<5:29:04,  8.07s/it]
 11%|█         | 310/2757 [41:43<5:34:50,  8.21s/it]
                                                    

 11%|█         | 310/2757 [41:43<5:34:50,  8.21s/it]
 11%|█▏        | 311/2757 [41:51<5:34:07,  8.20s/it]
 11%|█▏        | 312/2757 [41:59<5:29:01,  8.07s/it]
 11%|█▏        | 313/2757 [42:07<5:28:13,  8.06s/it]
 11%|█▏        | 314/2757 [42:15<5:30:44,  8.12s/it]
 11%|█▏        | 315/2757 [42:23<5:27:30,  8.05s/it]
 11%|█▏        | 316/2757 [42:31<5:24:03,  7.97s/it]
 11%|█▏        | 317/2757 [42:39<5:28:08,  8.07s/it]
 12%|█▏        | 318/2757 [42:47<5:24:56,  7.99s/it]
 12%|█▏        | 319/2757 [42:55<5:28:55,  8.09s/it]
 12%|█▏        | 320/2757 [43:03<5:25:58,  8.03s/it]
                                                    

 12%|█▏        | 320/2757 [43:03<5:25:58,  8.03s/it]
 12%|█▏        | 321/2757 [43:10<5:20:37,  7.90s/it]
 12%|█▏        | 322/2757 [43:18<5:19:10,  7.86s/it]
 12%|█▏        | 323/2757 [43:27<5:25:30,  8.02s/it]
 12%|█▏        | 324/2757 [43:35<5:24:09,  7.99s/it]
 12%|█▏        | 325/2757 [43:42<5:20:58,  7.92s/it]
 12%|█▏        | 326/2757 [43:50<5:18:49,  7.87s/it]
 12%|█▏        | 327/2757 [43:58<5:19:37,  7.89s/it]
 12%|█▏        | 328/2757 [44:06<5:22:48,  7.97s/it]
 12%|█▏        | 329/2757 [44:14<5:26:50,  8.08s/it]
 12%|█▏        | 330/2757 [44:23<5:26:45,  8.08s/it]
                                                    

 12%|█▏        | 330/2757 [44:23<5:26:45,  8.08s/it]
 12%|█▏        | 331/2757 [44:31<5:25:44,  8.06s/it]
 12%|█▏        | 332/2757 [44:39<5:28:03,  8.12s/it]
 12%|█▏        | 333/2757 [44:46<5:19:24,  7.91s/it]
 12%|█▏        | 334/2757 [44:54<5:20:36,  7.94s/it]
 12%|█▏        | 335/2757 [45:02<5:24:13,  8.03s/it]
 12%|█▏        | 336/2757 [45:11<5:24:53,  8.05s/it]
 12%|█▏        | 337/2757 [45:19<5:24:41,  8.05s/it]
 12%|█▏        | 338/2757 [45:26<5:21:47,  7.98s/it]
 12%|█▏        | 339/2757 [45:34<5:20:56,  7.96s/it]
 12%|█▏        | 340/2757 [45:42<5:18:59,  7.92s/it]
                                                    

 12%|█▏        | 340/2757 [45:42<5:18:59,  7.92s/it]
 12%|█▏        | 341/2757 [45:50<5:17:05,  7.87s/it]
 12%|█▏        | 342/2757 [45:58<5:19:30,  7.94s/it]
 12%|█▏        | 343/2757 [46:06<5:18:26,  7.91s/it]
 12%|█▏        | 344/2757 [46:14<5:16:16,  7.86s/it]
 13%|█▎        | 345/2757 [46:22<5:19:59,  7.96s/it]
 13%|█▎        | 346/2757 [46:30<5:16:48,  7.88s/it]
 13%|█▎        | 347/2757 [46:38<5:24:02,  8.07s/it]
 13%|█▎        | 348/2757 [46:46<5:24:11,  8.07s/it]
 13%|█▎        | 349/2757 [46:54<5:27:11,  8.15s/it]
 13%|█▎        | 350/2757 [47:02<5:24:27,  8.09s/it]
                                                    

 13%|█▎        | 350/2757 [47:02<5:24:27,  8.09s/it]
 13%|█▎        | 351/2757 [47:10<5:18:21,  7.94s/it]
 13%|█▎        | 352/2757 [47:17<5:12:46,  7.80s/it]
 13%|█▎        | 353/2757 [47:25<5:13:10,  7.82s/it]
 13%|█▎        | 354/2757 [47:33<5:14:14,  7.85s/it]
 13%|█▎        | 355/2757 [47:42<5:19:11,  7.97s/it]
 13%|█▎        | 356/2757 [47:49<5:16:20,  7.91s/it]
 13%|█▎        | 357/2757 [47:57<5:18:02,  7.95s/it]
 13%|█▎        | 358/2757 [48:05<5:19:20,  7.99s/it]
 13%|█▎        | 359/2757 [48:13<5:19:04,  7.98s/it]
 13%|█▎        | 360/2757 [48:22<5:21:16,  8.04s/it]
                                                    

 13%|█▎        | 360/2757 [48:22<5:21:16,  8.04s/it]
 13%|█▎        | 361/2757 [48:30<5:25:30,  8.15s/it]
 13%|█▎        | 362/2757 [48:38<5:22:37,  8.08s/it]
 13%|█▎        | 363/2757 [48:46<5:23:13,  8.10s/it]
 13%|█▎        | 364/2757 [48:54<5:24:42,  8.14s/it]
 13%|█▎        | 365/2757 [49:02<5:23:44,  8.12s/it]
 13%|█▎        | 366/2757 [49:10<5:22:04,  8.08s/it]
 13%|█▎        | 367/2757 [49:18<5:22:52,  8.11s/it]
 13%|█▎        | 368/2757 [49:26<5:17:57,  7.99s/it]
 13%|█▎        | 369/2757 [49:34<5:21:07,  8.07s/it]
 13%|█▎        | 370/2757 [49:42<5:18:35,  8.01s/it]
                                                    

 13%|█▎        | 370/2757 [49:42<5:18:35,  8.01s/it]
 13%|█▎        | 371/2757 [49:50<5:18:15,  8.00s/it]
 13%|█▎        | 372/2757 [49:59<5:20:53,  8.07s/it]
 14%|█▎        | 373/2757 [50:07<5:23:28,  8.14s/it]
 14%|█▎        | 374/2757 [50:15<5:21:53,  8.10s/it]
 14%|█▎        | 375/2757 [50:23<5:21:22,  8.10s/it]
 14%|█▎        | 376/2757 [50:31<5:17:58,  8.01s/it]
 14%|█▎        | 377/2757 [50:39<5:22:25,  8.13s/it]
 14%|█▎        | 378/2757 [50:47<5:20:49,  8.09s/it]
 14%|█▎        | 379/2757 [50:55<5:21:13,  8.11s/it]
 14%|█▍        | 380/2757 [51:03<5:12:13,  7.88s/it]
                                                    

 14%|█▍        | 380/2757 [51:03<5:12:13,  7.88s/it]
 14%|█▍        | 381/2757 [51:11<5:15:14,  7.96s/it]
 14%|█▍        | 382/2757 [51:18<5:11:11,  7.86s/it]
 14%|█▍        | 383/2757 [51:27<5:17:57,  8.04s/it]
 14%|█▍        | 384/2757 [51:35<5:18:56,  8.06s/it]
 14%|█▍        | 385/2757 [51:42<5:11:05,  7.87s/it]
 14%|█▍        | 386/2757 [51:50<5:10:02,  7.85s/it]
 14%|█▍        | 387/2757 [51:58<5:14:48,  7.97s/it]
 14%|█▍        | 388/2757 [52:06<5:08:57,  7.82s/it]
 14%|█▍        | 389/2757 [52:14<5:11:59,  7.91s/it]
 14%|█▍        | 390/2757 [52:22<5:18:05,  8.06s/it]
                                                    

 14%|█▍        | 390/2757 [52:22<5:18:05,  8.06s/it]
 14%|█▍        | 391/2757 [52:30<5:14:31,  7.98s/it]
 14%|█▍        | 392/2757 [52:39<5:18:29,  8.08s/it]
 14%|█▍        | 393/2757 [52:46<5:12:04,  7.92s/it]
 14%|█▍        | 394/2757 [52:55<5:19:01,  8.10s/it]
 14%|█▍        | 395/2757 [53:02<5:13:11,  7.96s/it]
 14%|█▍        | 396/2757 [53:10<5:09:18,  7.86s/it]
 14%|█▍        | 397/2757 [53:18<5:09:50,  7.88s/it]
 14%|█▍        | 398/2757 [53:26<5:14:45,  8.01s/it]
 14%|█▍        | 399/2757 [53:34<5:15:25,  8.03s/it]
 15%|█▍        | 400/2757 [53:42<5:15:01,  8.02s/it]
                                                    

 15%|█▍        | 400/2757 [53:42<5:15:01,  8.02s/it]
 15%|█▍        | 401/2757 [53:50<5:16:02,  8.05s/it]
 15%|█▍        | 402/2757 [53:58<5:14:40,  8.02s/it]
 15%|█▍        | 403/2757 [54:06<5:11:09,  7.93s/it]
 15%|█▍        | 404/2757 [54:15<5:20:32,  8.17s/it]
 15%|█▍        | 405/2757 [54:23<5:18:59,  8.14s/it]
 15%|█▍        | 406/2757 [54:31<5:19:48,  8.16s/it]
 15%|█▍        | 407/2757 [54:39<5:20:58,  8.19s/it]
 15%|█▍        | 408/2757 [54:48<5:22:04,  8.23s/it]
 15%|█▍        | 409/2757 [54:56<5:22:25,  8.24s/it]
 15%|█▍        | 410/2757 [55:04<5:15:49,  8.07s/it]
                                                    

 15%|█▍        | 410/2757 [55:04<5:15:49,  8.07s/it]
 15%|█▍        | 411/2757 [55:12<5:17:32,  8.12s/it]
 15%|█▍        | 412/2757 [55:20<5:14:54,  8.06s/it]
 15%|█▍        | 413/2757 [55:28<5:15:28,  8.08s/it]
 15%|█▌        | 414/2757 [55:36<5:13:26,  8.03s/it]
 15%|█▌        | 415/2757 [55:44<5:13:15,  8.03s/it]
 15%|█▌        | 416/2757 [55:52<5:11:22,  7.98s/it]
 15%|█▌        | 417/2757 [55:59<5:07:04,  7.87s/it]
 15%|█▌        | 418/2757 [56:07<5:06:17,  7.86s/it]
 15%|█▌        | 419/2757 [56:15<5:07:04,  7.88s/it]
 15%|█▌        | 420/2757 [56:23<5:12:47,  8.03s/it]
                                                    

 15%|█▌        | 420/2757 [56:23<5:12:47,  8.03s/it]
 15%|█▌        | 421/2757 [56:31<5:11:26,  8.00s/it]
 15%|█▌        | 422/2757 [56:39<5:05:53,  7.86s/it]
 15%|█▌        | 423/2757 [56:47<5:05:54,  7.86s/it]
 15%|█▌        | 424/2757 [56:55<5:07:56,  7.92s/it]
 15%|█▌        | 425/2757 [57:03<5:10:41,  7.99s/it]
 15%|█▌        | 426/2757 [57:11<5:13:28,  8.07s/it]
 15%|█▌        | 427/2757 [57:19<5:06:31,  7.89s/it]
 16%|█▌        | 428/2757 [57:26<5:02:23,  7.79s/it]
 16%|█▌        | 429/2757 [57:34<5:02:58,  7.81s/it]
 16%|█▌        | 430/2757 [57:42<5:01:14,  7.77s/it]
                                                    

 16%|█▌        | 430/2757 [57:42<5:01:14,  7.77s/it]
 16%|█▌        | 431/2757 [57:49<4:57:04,  7.66s/it]
 16%|█▌        | 432/2757 [57:57<5:04:20,  7.85s/it]
 16%|█▌        | 433/2757 [58:05<5:02:16,  7.80s/it]
 16%|█▌        | 434/2757 [58:13<5:04:47,  7.87s/it]
 16%|█▌        | 435/2757 [58:21<5:08:18,  7.97s/it]
 16%|█▌        | 436/2757 [58:29<5:07:47,  7.96s/it]
 16%|█▌        | 437/2757 [58:37<5:08:24,  7.98s/it]
 16%|█▌        | 438/2757 [58:45<5:05:49,  7.91s/it]
 16%|█▌        | 439/2757 [58:53<5:05:28,  7.91s/it]
 16%|█▌        | 440/2757 [59:01<5:11:12,  8.06s/it]
                                                    

 16%|█▌        | 440/2757 [59:01<5:11:12,  8.06s/it]
 16%|█▌        | 441/2757 [59:09<5:11:28,  8.07s/it]
 16%|█▌        | 442/2757 [59:18<5:12:07,  8.09s/it]
 16%|█▌        | 443/2757 [59:25<5:09:21,  8.02s/it]
 16%|█▌        | 444/2757 [59:34<5:13:29,  8.13s/it]
 16%|█▌        | 445/2757 [59:42<5:13:48,  8.14s/it]
 16%|█▌        | 446/2757 [59:50<5:15:04,  8.18s/it]
 16%|█▌        | 447/2757 [59:58<5:11:52,  8.10s/it]
 16%|█▌        | 448/2757 [1:00:06<5:12:55,  8.13s/it]
 16%|█▋        | 449/2757 [1:00:14<5:07:45,  8.00s/it]
 16%|█▋        | 450/2757 [1:00:22<5:07:50,  8.01s/it]
                                                      

 16%|█▋        | 450/2757 [1:00:22<5:07:50,  8.01s/it]
 16%|█▋        | 451/2757 [1:00:30<5:04:26,  7.92s/it]
 16%|█▋        | 452/2757 [1:00:38<5:03:37,  7.90s/it]
 16%|█▋        | 453/2757 [1:00:46<5:05:07,  7.95s/it]
 16%|█▋        | 454/2757 [1:00:54<5:06:05,  7.97s/it]
 17%|█▋        | 455/2757 [1:01:02<5:10:57,  8.10s/it]
 17%|█▋        | 456/2757 [1:01:10<5:05:31,  7.97s/it]
 17%|█▋        | 457/2757 [1:01:18<5:06:38,  8.00s/it]
 17%|█▋        | 458/2757 [1:01:26<5:02:25,  7.89s/it]
 17%|█▋        | 459/2757 [1:01:33<4:59:22,  7.82s/it]
 17%|█▋        | 460/2757 [1:01:41<4:58:28,  7.80s/it]
                                                      

 17%|█▋        | 460/2757 [1:01:41<4:58:28,  7.80s/it]
 17%|█▋        | 461/2757 [1:01:49<4:59:33,  7.83s/it]
 17%|█▋        | 462/2757 [1:01:56<4:55:28,  7.72s/it]
 17%|█▋        | 463/2757 [1:02:04<4:59:32,  7.83s/it]
 17%|█▋        | 464/2757 [1:02:12<5:01:20,  7.89s/it]
 17%|█▋        | 465/2757 [1:02:20<5:00:44,  7.87s/it]
 17%|█▋        | 466/2757 [1:02:28<5:03:07,  7.94s/it]
 17%|█▋        | 467/2757 [1:02:37<5:07:49,  8.07s/it]
 17%|█▋        | 468/2757 [1:02:45<5:08:24,  8.08s/it]
 17%|█▋        | 469/2757 [1:02:52<5:01:37,  7.91s/it]
 17%|█▋        | 470/2757 [1:03:00<4:59:47,  7.87s/it]
                                                      

 17%|█▋        | 470/2757 [1:03:00<4:59:47,  7.87s/it]
 17%|█▋        | 471/2757 [1:03:08<5:03:42,  7.97s/it]
 17%|█▋        | 472/2757 [1:03:17<5:06:44,  8.05s/it]
 17%|█▋        | 473/2757 [1:03:25<5:06:40,  8.06s/it]
 17%|█▋        | 474/2757 [1:03:32<5:04:02,  7.99s/it]
 17%|█▋        | 475/2757 [1:03:41<5:10:18,  8.16s/it]
 17%|█▋        | 476/2757 [1:03:49<5:10:22,  8.16s/it]
 17%|█▋        | 477/2757 [1:03:57<5:05:12,  8.03s/it]
 17%|█▋        | 478/2757 [1:04:05<5:04:56,  8.03s/it]
 17%|█▋        | 479/2757 [1:04:13<5:09:02,  8.14s/it]
 17%|█▋        | 480/2757 [1:04:21<5:09:00,  8.14s/it]
                                                      

 17%|█▋        | 480/2757 [1:04:21<5:09:00,  8.14s/it]
 17%|█▋        | 481/2757 [1:04:29<5:04:30,  8.03s/it]
 17%|█▋        | 482/2757 [1:04:38<5:07:14,  8.10s/it]
 18%|█▊        | 483/2757 [1:04:45<5:03:11,  8.00s/it]
 18%|█▊        | 484/2757 [1:04:53<5:04:33,  8.04s/it]
 18%|█▊        | 485/2757 [1:05:02<5:06:42,  8.10s/it]
 18%|█▊        | 486/2757 [1:05:10<5:07:29,  8.12s/it]
 18%|█▊        | 487/2757 [1:05:18<5:07:07,  8.12s/it]
 18%|█▊        | 488/2757 [1:05:26<5:02:58,  8.01s/it]
 18%|█▊        | 489/2757 [1:05:34<5:01:55,  7.99s/it]
 18%|█▊        | 490/2757 [1:05:42<5:00:30,  7.95s/it]
                                                      

 18%|█▊        | 490/2757 [1:05:42<5:00:30,  7.95s/it]
 18%|█▊        | 491/2757 [1:05:50<5:00:43,  7.96s/it]
 18%|█▊        | 492/2757 [1:05:57<4:59:23,  7.93s/it]
 18%|█▊        | 493/2757 [1:06:05<4:59:31,  7.94s/it]
 18%|█▊        | 494/2757 [1:06:14<5:02:51,  8.03s/it]
 18%|█▊        | 495/2757 [1:06:22<5:04:03,  8.07s/it]
 18%|█▊        | 496/2757 [1:06:30<5:01:46,  8.01s/it]
 18%|█▊        | 497/2757 [1:06:38<5:06:35,  8.14s/it]
 18%|█▊        | 498/2757 [1:06:46<5:07:37,  8.17s/it]
 18%|█▊        | 499/2757 [1:06:54<5:05:40,  8.12s/it]
 18%|█▊        | 500/2757 [1:07:02<5:03:35,  8.07s/it]
                                                      

 18%|█▊        | 500/2757 [1:07:02<5:03:35,  8.07s/it]
{'loss': '0.6296', 'grad_norm': '0.5629', 'learning_rate': '0.0001861', 'rewards/chosen': '-0.3549', 'rewards/rejected': '-0.6289', 'rewards/accuracies': '0.5961', 'rewards/margins': '0.274', 'logps/chosen': '-80.02', 'logps/rejected': '-88.44', 'logits/chosen': '-3.056', 'logits/rejected': '-2.951', 'epoch': '0.2939'}
{'loss': '0.626', 'grad_norm': '0.6267', 'learning_rate': '0.0001853', 'rewards/chosen': '-0.3159', 'rewards/rejected': '-0.5841', 'rewards/accuracies': '0.5914', 'rewards/margins': '0.2681', 'logps/chosen': '-79.61', 'logps/rejected': '-87.73', 'logits/chosen': '-3.073', 'logits/rejected': '-2.979', 'epoch': '0.3048'}
{'loss': '0.6292', 'grad_norm': '0.6236', 'learning_rate': '0.0001846', 'rewards/chosen': '-0.253', 'rewards/rejected': '-0.538', 'rewards/accuracies': '0.5852', 'rewards/margins': '0.285', 'logps/chosen': '-82', 'logps/rejected': '-90.57', 'logits/chosen': '-3.13', 'logits/rejected': '-3.074', 'epoch': '0.3156'}
{'loss': '0.6195', 'grad_norm': '0.6637', 'learning_rate': '0.0001838', 'rewards/chosen': '-0.08129', 'rewards/rejected': '-0.3927', 'rewards/accuracies': '0.6008', 'rewards/margins': '0.3114', 'logps/chosen': '-80.13', 'logps/rejected': '-88.26', 'logits/chosen': '-3.198', 'logits/rejected': '-3.114', 'epoch': '0.3265'}
{'loss': '0.6184', 'grad_norm': '0.6673', 'learning_rate': '0.0001831', 'rewards/chosen': '-0.3788', 'rewards/rejected': '-0.6948', 'rewards/accuracies': '0.5984', 'rewards/margins': '0.316', 'logps/chosen': '-86.59', 'logps/rejected': '-95.23', 'logits/chosen': '-2.986', 'logits/rejected': '-2.906', 'epoch': '0.3374'}
{'loss': '0.6248', 'grad_norm': '0.6171', 'learning_rate': '0.0001823', 'rewards/chosen': '-0.5991', 'rewards/rejected': '-0.8997', 'rewards/accuracies': '0.5844', 'rewards/margins': '0.3006', 'logps/chosen': '-88.97', 'logps/rejected': '-97.9', 'logits/chosen': '-3.004', 'logits/rejected': '-2.952', 'epoch': '0.3483'}
{'loss': '0.6249', 'grad_norm': '0.7893', 'learning_rate': '0.0001816', 'rewards/chosen': '-0.639', 'rewards/rejected': '-0.9381', 'rewards/accuracies': '0.5859', 'rewards/margins': '0.2991', 'logps/chosen': '-84.33', 'logps/rejected': '-94.08', 'logits/chosen': '-3.131', 'logits/rejected': '-3.113', 'epoch': '0.3592'}
{'loss': '0.6234', 'grad_norm': '0.5814', 'learning_rate': '0.0001809', 'rewards/chosen': '-0.6253', 'rewards/rejected': '-0.9446', 'rewards/accuracies': '0.5922', 'rewards/margins': '0.3192', 'logps/chosen': '-86.04', 'logps/rejected': '-93.89', 'logits/chosen': '-3.178', 'logits/rejected': '-3.083', 'epoch': '0.3701'}
{'loss': '0.6068', 'grad_norm': '0.6354', 'learning_rate': '0.0001801', 'rewards/chosen': '-0.5151', 'rewards/rejected': '-0.8645', 'rewards/accuracies': '0.6133', 'rewards/margins': '0.3494', 'logps/chosen': '-89.97', 'logps/rejected': '-98.78', 'logits/chosen': '-2.931', 'logits/rejected': '-2.912', 'epoch': '0.381'}
{'loss': '0.6245', 'grad_norm': '0.6956', 'learning_rate': '0.0001794', 'rewards/chosen': '-0.5855', 'rewards/rejected': '-0.8816', 'rewards/accuracies': '0.6016', 'rewards/margins': '0.2961', 'logps/chosen': '-84.48', 'logps/rejected': '-91.43', 'logits/chosen': '-2.768', 'logits/rejected': '-2.695', 'epoch': '0.3918'}
{'loss': '0.623', 'grad_norm': '0.5492', 'learning_rate': '0.0001786', 'rewards/chosen': '-0.6255', 'rewards/rejected': '-0.9144', 'rewards/accuracies': '0.6023', 'rewards/margins': '0.2889', 'logps/chosen': '-88.09', 'logps/rejected': '-94.99', 'logits/chosen': '-2.877', 'logits/rejected': '-2.792', 'epoch': '0.4027'}
{'loss': '0.6103', 'grad_norm': '0.6273', 'learning_rate': '0.0001779', 'rewards/chosen': '-0.5627', 'rewards/rejected': '-0.8772', 'rewards/accuracies': '0.6078', 'rewards/margins': '0.3145', 'logps/chosen': '-89.07', 'logps/rejected': '-97.89', 'logits/chosen': '-2.693', 'logits/rejected': '-2.596', 'epoch': '0.4136'}
{'loss': '0.6128', 'grad_norm': '0.6782', 'learning_rate': '0.0001771', 'rewards/chosen': '-0.5093', 'rewards/rejected': '-0.8476', 'rewards/accuracies': '0.6219', 'rewards/margins': '0.3384', 'logps/chosen': '-85.96', 'logps/rejected': '-94.58', 'logits/chosen': '-2.902', 'logits/rejected': '-2.792', 'epoch': '0.4245'}
{'loss': '0.6213', 'grad_norm': '0.6319', 'learning_rate': '0.0001764', 'rewards/chosen': '-0.3547', 'rewards/rejected': '-0.6653', 'rewards/accuracies': '0.6102', 'rewards/margins': '0.3106', 'logps/chosen': '-82.64', 'logps/rejected': '-93.14', 'logits/chosen': '-2.978', 'logits/rejected': '-2.966', 'epoch': '0.4354'}
{'loss': '0.6141', 'grad_norm': '0.6016', 'learning_rate': '0.0001756', 'rewards/chosen': '-0.3532', 'rewards/rejected': '-0.698', 'rewards/accuracies': '0.5945', 'rewards/margins': '0.3448', 'logps/chosen': '-84.97', 'logps/rejected': '-93.02', 'logits/chosen': '-3.077', 'logits/rejected': '-3.008', 'epoch': '0.4463'}
{'loss': '0.6004', 'grad_norm': '0.6052', 'learning_rate': '0.0001749', 'rewards/chosen': '-0.2189', 'rewards/rejected': '-0.587', 'rewards/accuracies': '0.6258', 'rewards/margins': '0.3681', 'logps/chosen': '-79.92', 'logps/rejected': '-88.4', 'logits/chosen': '-3.087', 'logits/rejected': '-2.981', 'epoch': '0.4571'}
{'loss': '0.62', 'grad_norm': '0.73', 'learning_rate': '0.0001741', 'rewards/chosen': '-0.2221', 'rewards/rejected': '-0.5518', 'rewards/accuracies': '0.6047', 'rewards/margins': '0.3297', 'logps/chosen': '-84.05', 'logps/rejected': '-91.39', 'logits/chosen': '-3.061', 'logits/rejected': '-3.027', 'epoch': '0.468'}
{'loss': '0.6032', 'grad_norm': '0.5996', 'learning_rate': '0.0001734', 'rewards/chosen': '-0.5063', 'rewards/rejected': '-0.86', 'rewards/accuracies': '0.6305', 'rewards/margins': '0.3537', 'logps/chosen': '-88.16', 'logps/rejected': '-94.57', 'logits/chosen': '-2.93', 'logits/rejected': '-2.835', 'epoch': '0.4789'}
{'loss': '0.5989', 'grad_norm': '0.6094', 'learning_rate': '0.0001726', 'rewards/chosen': '-0.5802', 'rewards/rejected': '-0.9776', 'rewards/accuracies': '0.6117', 'rewards/margins': '0.3973', 'logps/chosen': '-85.01', 'logps/rejected': '-94.11', 'logits/chosen': '-3.055', 'logits/rejected': '-2.933', 'epoch': '0.4898'}
{'loss': '0.6101', 'grad_norm': '0.6395', 'learning_rate': '0.0001719', 'rewards/chosen': '-0.5156', 'rewards/rejected': '-0.8643', 'rewards/accuracies': '0.6172', 'rewards/margins': '0.3487', 'logps/chosen': '-86.3', 'logps/rejected': '-94.5', 'logits/chosen': '-2.982', 'logits/rejected': '-2.935', 'epoch': '0.5007'}
{'loss': '0.6019', 'grad_norm': '0.5491', 'learning_rate': '0.0001711', 'rewards/chosen': '-0.3541', 'rewards/rejected': '-0.7315', 'rewards/accuracies': '0.6062', 'rewards/margins': '0.3774', 'logps/chosen': '-81.23', 'logps/rejected': '-88.91', 'logits/chosen': '-2.927', 'logits/rejected': '-2.875', 'epoch': '0.5116'}
{'loss': '0.5966', 'grad_norm': '0.6346', 'learning_rate': '0.0001704', 'rewards/chosen': '-0.242', 'rewards/rejected': '-0.6034', 'rewards/accuracies': '0.6055', 'rewards/margins': '0.3614', 'logps/chosen': '-82.53', 'logps/rejected': '-91.44', 'logits/chosen': '-2.846', 'logits/rejected': '-2.819', 'epoch': '0.5224'}
{'loss': '0.5878', 'grad_norm': '0.6204', 'learning_rate': '0.0001696', 'rewards/chosen': '-0.3545', 'rewards/rejected': '-0.7606', 'rewards/accuracies': '0.643', 'rewards/margins': '0.4061', 'logps/chosen': '-83.22', 'logps/rejected': '-91.02', 'logits/chosen': '-3.038', 'logits/rejected': '-2.952', 'epoch': '0.5333'}
{'loss': '0.5975', 'grad_norm': '0.6997', 'learning_rate': '0.0001689', 'rewards/chosen': '-0.6454', 'rewards/rejected': '-1.051', 'rewards/accuracies': '0.6109', 'rewards/margins': '0.4053', 'logps/chosen': '-88.25', 'logps/rejected': '-98.36', 'logits/chosen': '-3.002', 'logits/rejected': '-2.958', 'epoch': '0.5442'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)


  0%|          | 0/300 [00:00<?, ?it/s]

  1%|          | 2/300 [00:00<00:52,  5.64it/s]

  1%|          | 3/300 [00:00<01:46,  2.79it/s]

  1%|▏         | 4/300 [00:01<01:34,  3.14it/s]

  2%|▏         | 5/300 [00:01<01:44,  2.82it/s]

  2%|▏         | 6/300 [00:01<01:40,  2.93it/s]

  2%|▏         | 7/300 [00:02<01:54,  2.56it/s]

  3%|▎         | 8/300 [00:02<01:50,  2.64it/s]

  3%|▎         | 9/300 [00:03<01:53,  2.57it/s]

  3%|▎         | 10/300 [00:03<01:49,  2.66it/s]

  4%|▎         | 11/300 [00:03<01:51,  2.59it/s]

  4%|▍         | 12/300 [00:04<01:38,  2.92it/s]

  4%|▍         | 13/300 [00:04<01:45,  2.72it/s]

  5%|▍         | 14/300 [00:04<01:37,  2.93it/s]

  5%|▌         | 15/300 [00:05<01:47,  2.66it/s]

  5%|▌         | 16/300 [00:05<01:34,  2.99it/s]

  6%|▌         | 17/300 [00:05<01:38,  2.87it/s]

  6%|▌         | 18/300 [00:06<01:28,  3.18it/s]

  6%|▋         | 19/300 [00:06<01:39,  2.83it/s]

  7%|▋         | 20/300 [00:06<01:31,  3.05it/s]

  7%|▋         | 21/300 [00:07<01:41,  2.74it/s]

  7%|▋         | 22/300 [00:07<01:30,  3.06it/s]

  8%|▊         | 23/300 [00:07<01:32,  3.00it/s]

  8%|▊         | 24/300 [00:08<01:23,  3.31it/s]

  8%|▊         | 25/300 [00:08<01:35,  2.87it/s]

  9%|▊         | 26/300 [00:08<01:29,  3.08it/s]

  9%|▉         | 27/300 [00:09<01:37,  2.80it/s]

  9%|▉         | 28/300 [00:09<01:26,  3.15it/s]

 10%|▉         | 29/300 [00:09<01:29,  3.03it/s]

 10%|█         | 30/300 [00:10<01:21,  3.33it/s]

 10%|█         | 31/300 [00:10<01:23,  3.22it/s]

 11%|█         | 32/300 [00:10<01:17,  3.48it/s]

 11%|█         | 33/300 [00:11<01:19,  3.34it/s]

 11%|█▏        | 34/300 [00:11<01:15,  3.52it/s]

 12%|█▏        | 35/300 [00:11<01:22,  3.23it/s]

 12%|█▏        | 36/300 [00:11<01:17,  3.39it/s]

 12%|█▏        | 37/300 [00:12<01:26,  3.04it/s]

 13%|█▎        | 38/300 [00:12<01:17,  3.36it/s]

 13%|█▎        | 39/300 [00:13<01:40,  2.60it/s]

 13%|█▎        | 40/300 [00:13<01:28,  2.94it/s]

 14%|█▎        | 41/300 [00:13<01:32,  2.81it/s]

 14%|█▍        | 42/300 [00:14<01:22,  3.14it/s]

 14%|█▍        | 43/300 [00:14<01:25,  3.02it/s]

 15%|█▍        | 44/300 [00:14<01:17,  3.30it/s]

 15%|█▌        | 45/300 [00:15<01:23,  3.04it/s]

 15%|█▌        | 46/300 [00:15<01:24,  3.00it/s]

 16%|█▌        | 47/300 [00:15<01:30,  2.80it/s]

 16%|█▌        | 48/300 [00:16<01:22,  3.04it/s]

 16%|█▋        | 49/300 [00:16<01:23,  3.02it/s]

 17%|█▋        | 50/300 [00:16<01:14,  3.36it/s]

 17%|█▋        | 51/300 [00:16<01:18,  3.19it/s]

 17%|█▋        | 52/300 [00:17<01:15,  3.29it/s]

 18%|█▊        | 53/300 [00:17<01:28,  2.80it/s]

 18%|█▊        | 54/300 [00:17<01:19,  3.09it/s]

 18%|█▊        | 55/300 [00:18<01:29,  2.73it/s]

 19%|█▊        | 56/300 [00:18<01:17,  3.15it/s]

 19%|█▉        | 57/300 [00:18<01:20,  3.01it/s]

 19%|█▉        | 58/300 [00:19<01:21,  2.96it/s]

 20%|█▉        | 59/300 [00:19<01:26,  2.79it/s]

 20%|██        | 60/300 [00:20<01:18,  3.08it/s]

 20%|██        | 61/300 [00:20<01:27,  2.72it/s]

 21%|██        | 62/300 [00:20<01:25,  2.78it/s]

 21%|██        | 63/300 [00:21<01:32,  2.56it/s]

 21%|██▏       | 64/300 [00:21<01:28,  2.66it/s]

 22%|██▏       | 65/300 [00:22<01:30,  2.59it/s]

 22%|██▏       | 66/300 [00:22<01:21,  2.87it/s]

 22%|██▏       | 67/300 [00:22<01:29,  2.62it/s]

 23%|██▎       | 68/300 [00:22<01:19,  2.91it/s]

 23%|██▎       | 69/300 [00:23<01:22,  2.81it/s]

 23%|██▎       | 70/300 [00:23<01:13,  3.12it/s]

 24%|██▎       | 71/300 [00:23<01:15,  3.03it/s]

 24%|██▍       | 72/300 [00:24<01:09,  3.29it/s]

 24%|██▍       | 73/300 [00:24<01:12,  3.14it/s]

 25%|██▍       | 74/300 [00:24<01:08,  3.32it/s]

 25%|██▌       | 75/300 [00:25<01:15,  2.98it/s]

 25%|██▌       | 76/300 [00:25<01:15,  2.95it/s]

 26%|██▌       | 77/300 [00:26<01:20,  2.76it/s]

 26%|██▌       | 78/300 [00:26<01:13,  3.01it/s]

 26%|██▋       | 79/300 [00:26<01:15,  2.91it/s]

 27%|██▋       | 80/300 [00:26<01:11,  3.08it/s]

 27%|██▋       | 81/300 [00:27<01:19,  2.76it/s]

 27%|██▋       | 82/300 [00:27<01:11,  3.07it/s]

 28%|██▊       | 83/300 [00:27<01:13,  2.97it/s]

 28%|██▊       | 84/300 [00:28<01:13,  2.93it/s]

 28%|██▊       | 85/300 [00:28<01:18,  2.74it/s]

 29%|██▊       | 86/300 [00:28<01:10,  3.04it/s]

 29%|██▉       | 87/300 [00:29<01:12,  2.94it/s]

 29%|██▉       | 88/300 [00:29<01:06,  3.19it/s]

 30%|██▉       | 89/300 [00:29<01:10,  2.98it/s]

 30%|███       | 90/300 [00:30<01:05,  3.22it/s]

 30%|███       | 91/300 [00:30<01:09,  3.03it/s]

 31%|███       | 92/300 [00:30<01:03,  3.28it/s]

 31%|███       | 93/300 [00:31<01:09,  2.99it/s]

 31%|███▏      | 94/300 [00:31<01:02,  3.29it/s]

 32%|███▏      | 95/300 [00:31<01:07,  3.02it/s]

 32%|███▏      | 96/300 [00:32<01:03,  3.21it/s]

 32%|███▏      | 97/300 [00:32<01:05,  3.09it/s]

 33%|███▎      | 98/300 [00:32<01:02,  3.25it/s]

 33%|███▎      | 99/300 [00:33<01:07,  3.00it/s]

 33%|███▎      | 100/300 [00:33<01:03,  3.16it/s]

 34%|███▎      | 101/300 [00:33<01:07,  2.93it/s]

 34%|███▍      | 102/300 [00:34<01:06,  3.00it/s]

 34%|███▍      | 103/300 [00:34<01:09,  2.85it/s]

 35%|███▍      | 104/300 [00:34<01:01,  3.18it/s]

 35%|███▌      | 105/300 [00:35<01:04,  3.01it/s]

 35%|███▌      | 106/300 [00:35<01:00,  3.21it/s]

 36%|███▌      | 107/300 [00:35<01:05,  2.96it/s]

 36%|███▌      | 108/300 [00:36<01:02,  3.06it/s]

 36%|███▋      | 109/300 [00:36<01:06,  2.87it/s]

 37%|███▋      | 110/300 [00:36<01:02,  3.03it/s]

 37%|███▋      | 111/300 [00:37<01:11,  2.65it/s]

 37%|███▋      | 112/300 [00:37<01:05,  2.85it/s]

 38%|███▊      | 113/300 [00:37<01:07,  2.75it/s]

 38%|███▊      | 114/300 [00:38<01:06,  2.79it/s]

 38%|███▊      | 115/300 [00:38<01:09,  2.65it/s]

 39%|███▊      | 116/300 [00:38<01:01,  2.98it/s]

 39%|███▉      | 117/300 [00:39<01:03,  2.90it/s]

 39%|███▉      | 118/300 [00:39<00:58,  3.12it/s]

 40%|███▉      | 119/300 [00:39<01:00,  2.99it/s]

 40%|████      | 120/300 [00:40<00:59,  3.01it/s]

 40%|████      | 121/300 [00:40<01:03,  2.83it/s]

 41%|████      | 122/300 [00:40<00:57,  3.07it/s]

 41%|████      | 123/300 [00:41<01:01,  2.87it/s]

 41%|████▏     | 124/300 [00:41<00:57,  3.04it/s]

 42%|████▏     | 125/300 [00:42<00:59,  2.94it/s]

 42%|████▏     | 126/300 [00:42<00:59,  2.91it/s]

 42%|████▏     | 127/300 [00:42<01:02,  2.76it/s]

 43%|████▎     | 128/300 [00:43<00:58,  2.97it/s]

 43%|████▎     | 129/300 [00:43<01:09,  2.46it/s]

 43%|████▎     | 130/300 [00:43<01:00,  2.79it/s]

 44%|████▎     | 131/300 [00:44<01:05,  2.58it/s]

 44%|████▍     | 132/300 [00:44<00:58,  2.86it/s]

 44%|████▍     | 133/300 [00:45<01:05,  2.56it/s]

 45%|████▍     | 134/300 [00:45<00:58,  2.84it/s]

 45%|████▌     | 135/300 [00:45<01:02,  2.65it/s]

 45%|████▌     | 136/300 [00:46<00:55,  2.96it/s]

 46%|████▌     | 137/300 [00:46<00:55,  2.93it/s]

 46%|████▌     | 138/300 [00:46<00:51,  3.17it/s]

 46%|████▋     | 139/300 [00:46<00:51,  3.12it/s]

 47%|████▋     | 140/300 [00:47<00:47,  3.37it/s]

 47%|████▋     | 141/300 [00:47<00:49,  3.20it/s]

 47%|████▋     | 142/300 [00:47<00:51,  3.09it/s]

 48%|████▊     | 143/300 [00:48<00:54,  2.89it/s]

 48%|████▊     | 144/300 [00:48<00:48,  3.20it/s]

 48%|████▊     | 145/300 [00:48<00:50,  3.09it/s]

 49%|████▊     | 146/300 [00:49<00:47,  3.26it/s]

 49%|████▉     | 147/300 [00:49<00:53,  2.84it/s]

 49%|████▉     | 148/300 [00:49<00:50,  3.01it/s]

 50%|████▉     | 149/300 [00:50<00:54,  2.79it/s]

 50%|█████     | 150/300 [00:50<00:50,  2.99it/s]

 50%|█████     | 151/300 [00:50<00:51,  2.88it/s]

 51%|█████     | 152/300 [00:51<00:46,  3.20it/s]

 51%|█████     | 153/300 [00:51<00:47,  3.09it/s]

 51%|█████▏    | 154/300 [00:51<00:43,  3.37it/s]

 52%|█████▏    | 155/300 [00:52<00:45,  3.17it/s]

 52%|█████▏    | 156/300 [00:52<00:41,  3.50it/s]

 52%|█████▏    | 157/300 [00:52<00:48,  2.97it/s]

 53%|█████▎    | 158/300 [00:53<00:45,  3.14it/s]

 53%|█████▎    | 159/300 [00:53<00:51,  2.72it/s]

 53%|█████▎    | 160/300 [00:53<00:47,  2.96it/s]

 54%|█████▎    | 161/300 [00:54<00:51,  2.72it/s]

 54%|█████▍    | 162/300 [00:54<00:50,  2.76it/s]

 54%|█████▍    | 163/300 [00:55<00:55,  2.45it/s]

 55%|█████▍    | 164/300 [00:55<00:53,  2.56it/s]

 55%|█████▌    | 165/300 [00:55<00:52,  2.56it/s]

 55%|█████▌    | 166/300 [00:56<00:50,  2.67it/s]

 56%|█████▌    | 167/300 [00:56<00:54,  2.43it/s]

 56%|█████▌    | 168/300 [00:56<00:47,  2.75it/s]

 56%|█████▋    | 169/300 [00:57<00:52,  2.48it/s]

 57%|█████▋    | 170/300 [00:57<00:50,  2.58it/s]

 57%|█████▋    | 171/300 [00:58<00:50,  2.55it/s]

 57%|█████▋    | 172/300 [00:58<00:44,  2.88it/s]

 58%|█████▊    | 173/300 [00:58<00:46,  2.72it/s]

 58%|█████▊    | 174/300 [00:59<00:41,  3.04it/s]

 58%|█████▊    | 175/300 [00:59<00:45,  2.73it/s]

 59%|█████▊    | 176/300 [00:59<00:40,  3.06it/s]

 59%|█████▉    | 177/300 [01:00<00:44,  2.74it/s]

 59%|█████▉    | 178/300 [01:00<00:38,  3.15it/s]

 60%|█████▉    | 179/300 [01:00<00:41,  2.93it/s]

 60%|██████    | 180/300 [01:01<00:37,  3.21it/s]

 60%|██████    | 181/300 [01:01<00:41,  2.88it/s]

 61%|██████    | 182/300 [01:01<00:36,  3.19it/s]

 61%|██████    | 183/300 [01:02<00:39,  2.93it/s]

 61%|██████▏   | 184/300 [01:02<00:39,  2.90it/s]

 62%|██████▏   | 185/300 [01:02<00:42,  2.72it/s]

 62%|██████▏   | 186/300 [01:03<00:39,  2.91it/s]

 62%|██████▏   | 187/300 [01:03<00:39,  2.87it/s]

 63%|██████▎   | 188/300 [01:03<00:35,  3.15it/s]

 63%|██████▎   | 189/300 [01:04<00:38,  2.87it/s]

 63%|██████▎   | 190/300 [01:04<00:35,  3.10it/s]

 64%|██████▎   | 191/300 [01:04<00:36,  2.96it/s]

 64%|██████▍   | 192/300 [01:05<00:36,  2.98it/s]

 64%|██████▍   | 193/300 [01:05<00:41,  2.58it/s]

 65%|██████▍   | 194/300 [01:06<00:39,  2.66it/s]

 65%|██████▌   | 195/300 [01:06<00:42,  2.50it/s]

 65%|██████▌   | 196/300 [01:06<00:38,  2.73it/s]

 66%|██████▌   | 197/300 [01:07<00:37,  2.71it/s]

 66%|██████▌   | 198/300 [01:07<00:34,  2.94it/s]

 66%|██████▋   | 199/300 [01:07<00:35,  2.83it/s]

 67%|██████▋   | 200/300 [01:08<00:33,  3.01it/s]

 67%|██████▋   | 201/300 [01:08<00:35,  2.82it/s]

 67%|██████▋   | 202/300 [01:08<00:31,  3.11it/s]

 68%|██████▊   | 203/300 [01:09<00:32,  2.96it/s]

 68%|██████▊   | 204/300 [01:09<00:29,  3.24it/s]

 68%|██████▊   | 205/300 [01:09<00:31,  3.03it/s]

 69%|██████▊   | 206/300 [01:10<00:27,  3.42it/s]

 69%|██████▉   | 207/300 [01:10<00:31,  2.94it/s]

 69%|██████▉   | 208/300 [01:10<00:31,  2.96it/s]

 70%|██████▉   | 209/300 [01:11<00:32,  2.81it/s]

 70%|███████   | 210/300 [01:11<00:29,  3.03it/s]

 70%|███████   | 211/300 [01:11<00:33,  2.67it/s]

 71%|███████   | 212/300 [01:12<00:29,  2.99it/s]

 71%|███████   | 213/300 [01:12<00:31,  2.75it/s]

 71%|███████▏  | 214/300 [01:12<00:28,  3.00it/s]

 72%|███████▏  | 215/300 [01:13<00:31,  2.70it/s]

 72%|███████▏  | 216/300 [01:13<00:27,  3.01it/s]

 72%|███████▏  | 217/300 [01:14<00:34,  2.41it/s]

 73%|███████▎  | 218/300 [01:14<00:28,  2.85it/s]

 73%|███████▎  | 219/300 [01:14<00:30,  2.69it/s]

 73%|███████▎  | 220/300 [01:15<00:26,  3.00it/s]

 74%|███████▎  | 221/300 [01:15<00:28,  2.73it/s]

 74%|███████▍  | 222/300 [01:15<00:25,  3.04it/s]

 74%|███████▍  | 223/300 [01:16<00:28,  2.73it/s]

 75%|███████▍  | 224/300 [01:16<00:25,  2.94it/s]

 75%|███████▌  | 225/300 [01:16<00:26,  2.81it/s]

 75%|███████▌  | 226/300 [01:17<00:24,  3.05it/s]

 76%|███████▌  | 227/300 [01:17<00:26,  2.78it/s]

 76%|███████▌  | 228/300 [01:17<00:23,  3.11it/s]

 76%|███████▋  | 229/300 [01:18<00:25,  2.75it/s]

 77%|███████▋  | 230/300 [01:18<00:24,  2.86it/s]

 77%|███████▋  | 231/300 [01:18<00:25,  2.70it/s]

 77%|███████▋  | 232/300 [01:19<00:24,  2.77it/s]

 78%|███████▊  | 233/300 [01:19<00:27,  2.45it/s]

 78%|███████▊  | 234/300 [01:20<00:23,  2.81it/s]

 78%|███████▊  | 235/300 [01:20<00:24,  2.63it/s]

 79%|███████▊  | 236/300 [01:20<00:23,  2.70it/s]

 79%|███████▉  | 237/300 [01:21<00:25,  2.49it/s]

 79%|███████▉  | 238/300 [01:21<00:21,  2.82it/s]

 80%|███████▉  | 239/300 [01:21<00:21,  2.83it/s]

 80%|████████  | 240/300 [01:22<00:18,  3.16it/s]

 80%|████████  | 241/300 [01:22<00:19,  3.04it/s]

 81%|████████  | 242/300 [01:22<00:18,  3.08it/s]

 81%|████████  | 243/300 [01:23<00:20,  2.73it/s]

 81%|████████▏ | 244/300 [01:23<00:19,  2.83it/s]

 82%|████████▏ | 245/300 [01:24<00:20,  2.67it/s]

 82%|████████▏ | 246/300 [01:24<00:19,  2.77it/s]

 82%|████████▏ | 247/300 [01:24<00:20,  2.60it/s]

 83%|████████▎ | 248/300 [01:25<00:17,  2.94it/s]

 83%|████████▎ | 249/300 [01:25<00:18,  2.80it/s]

 83%|████████▎ | 250/300 [01:25<00:17,  2.92it/s]

 84%|████████▎ | 251/300 [01:26<00:17,  2.81it/s]

 84%|████████▍ | 252/300 [01:26<00:15,  3.01it/s]

 84%|████████▍ | 253/300 [01:26<00:17,  2.74it/s]

 85%|████████▍ | 254/300 [01:27<00:14,  3.12it/s]

 85%|████████▌ | 255/300 [01:27<00:16,  2.81it/s]

 85%|████████▌ | 256/300 [01:27<00:13,  3.15it/s]

 86%|████████▌ | 257/300 [01:28<00:14,  2.96it/s]

 86%|████████▌ | 258/300 [01:28<00:13,  3.22it/s]

 86%|████████▋ | 259/300 [01:28<00:13,  2.94it/s]

 87%|████████▋ | 260/300 [01:29<00:12,  3.14it/s]

 87%|████████▋ | 261/300 [01:29<00:13,  2.99it/s]

 87%|████████▋ | 262/300 [01:29<00:11,  3.28it/s]

 88%|████████▊ | 263/300 [01:30<00:12,  3.02it/s]

 88%|████████▊ | 264/300 [01:30<00:11,  3.01it/s]

 88%|████████▊ | 265/300 [01:30<00:13,  2.60it/s]

 89%|████████▊ | 266/300 [01:31<00:12,  2.70it/s]

 89%|████████▉ | 267/300 [01:31<00:13,  2.44it/s]

 89%|████████▉ | 268/300 [01:31<00:11,  2.80it/s]

 90%|████████▉ | 269/300 [01:32<00:12,  2.47it/s]

 90%|█████████ | 270/300 [01:32<00:11,  2.64it/s]

 90%|█████████ | 271/300 [01:33<00:12,  2.39it/s]

 91%|█████████ | 272/300 [01:33<00:10,  2.65it/s]

 91%|█████████ | 273/300 [01:34<00:11,  2.39it/s]

 91%|█████████▏| 274/300 [01:34<00:09,  2.74it/s]

 92%|█████████▏| 275/300 [01:34<00:10,  2.28it/s]

 92%|█████████▏| 276/300 [01:35<00:09,  2.49it/s]

 92%|█████████▏| 277/300 [01:35<00:09,  2.45it/s]

 93%|█████████▎| 278/300 [01:35<00:07,  2.78it/s]

 93%|█████████▎| 279/300 [01:36<00:07,  2.77it/s]

 93%|█████████▎| 280/300 [01:36<00:06,  2.87it/s]

 94%|█████████▎| 281/300 [01:37<00:07,  2.71it/s]

 94%|█████████▍| 282/300 [01:37<00:05,  3.02it/s]

 94%|█████████▍| 283/300 [01:37<00:06,  2.77it/s]

 95%|█████████▍| 284/300 [01:37<00:05,  3.03it/s]

 95%|█████████▌| 285/300 [01:38<00:05,  2.62it/s]

 95%|█████████▌| 286/300 [01:38<00:05,  2.69it/s]

 96%|█████████▌| 287/300 [01:39<00:04,  2.62it/s]

 96%|█████████▌| 288/300 [01:39<00:03,  3.05it/s]

 96%|█████████▋| 289/300 [01:39<00:03,  2.80it/s]

 97%|█████████▋| 290/300 [01:40<00:03,  2.82it/s]

 97%|█████████▋| 291/300 [01:40<00:03,  2.47it/s]

 97%|█████████▋| 292/300 [01:40<00:02,  2.81it/s]

 98%|█████████▊| 293/300 [01:41<00:02,  2.70it/s]

 98%|█████████▊| 294/300 [01:41<00:02,  2.96it/s]

 98%|█████████▊| 295/300 [01:41<00:01,  2.92it/s]

 99%|█████████▊| 296/300 [01:42<00:01,  3.13it/s]

 99%|█████████▉| 297/300 [01:42<00:01,  2.90it/s]

 99%|█████████▉| 298/300 [01:42<00:00,  3.27it/s]

100%|█████████▉| 299/300 [01:43<00:00,  2.87it/s]

100%|██████████| 300/300 [01:43<00:00,  2.76it/s]
                                                      

                                                 

 18%|█▊        | 500/2757 [1:08:47<5:03:35,  8.07s/it]

100%|██████████| 300/300 [01:43<00:00,  2.76it/s]

                                                 Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-500/tokenizer_config.json.

 18%|█▊        | 501/2757 [1:09:01<25:49:43, 41.22s/it]
 18%|█▊        | 502/2757 [1:09:09<19:33:58, 31.24s/it]
 18%|█▊        | 503/2757 [1:09:17<15:09:23, 24.21s/it]
 18%|█▊        | 504/2757 [1:09:24<11:57:50, 19.12s/it]
 18%|█▊        | 505/2757 [1:09:32<9:49:08, 15.70s/it] 
 18%|█▊        | 506/2757 [1:09:39<8:19:28, 13.31s/it]
 18%|█▊        | 507/2757 [1:09:47<7:20:56, 11.76s/it]
 18%|█▊        | 508/2757 [1:09:56<6:40:22, 10.68s/it]
 18%|█▊        | 509/2757 [1:10:03<6:07:33,  9.81s/it]
 18%|█▊        | 510/2757 [1:10:11<5:41:59,  9.13s/it]
                                                      

 18%|█▊        | 510/2757 [1:10:11<5:41:59,  9.13s/it]
 19%|█▊        | 511/2757 [1:10:19<5:25:41,  8.70s/it]
 19%|█▊        | 512/2757 [1:10:27<5:17:03,  8.47s/it]
 19%|█▊        | 513/2757 [1:10:35<5:12:13,  8.35s/it]
 19%|█▊        | 514/2757 [1:10:43<5:07:24,  8.22s/it]
 19%|█▊        | 515/2757 [1:10:50<4:59:38,  8.02s/it]
 19%|█▊        | 516/2757 [1:10:58<5:01:44,  8.08s/it]
 19%|█▉        | 517/2757 [1:11:07<5:03:39,  8.13s/it]
 19%|█▉        | 518/2757 [1:11:14<5:00:38,  8.06s/it]
 19%|█▉        | 519/2757 [1:11:23<5:04:44,  8.17s/it]
 19%|█▉        | 520/2757 [1:11:31<4:59:29,  8.03s/it]
                                                      

 19%|█▉        | 520/2757 [1:11:31<4:59:29,  8.03s/it]
 19%|█▉        | 521/2757 [1:11:39<5:02:53,  8.13s/it]
 19%|█▉        | 522/2757 [1:11:47<5:02:38,  8.12s/it]
 19%|█▉        | 523/2757 [1:11:55<4:59:37,  8.05s/it]
 19%|█▉        | 524/2757 [1:12:03<4:56:07,  7.96s/it]
 19%|█▉        | 525/2757 [1:12:10<4:51:43,  7.84s/it]
 19%|█▉        | 526/2757 [1:12:18<4:54:56,  7.93s/it]
 19%|█▉        | 527/2757 [1:12:26<4:53:28,  7.90s/it]
 19%|█▉        | 528/2757 [1:12:34<4:55:12,  7.95s/it]
 19%|█▉        | 529/2757 [1:12:42<4:54:24,  7.93s/it]
 19%|█▉        | 530/2757 [1:12:50<4:54:31,  7.94s/it]
                                                      

 19%|█▉        | 530/2757 [1:12:50<4:54:31,  7.94s/it]
 19%|█▉        | 531/2757 [1:12:58<4:55:44,  7.97s/it]
 19%|█▉        | 532/2757 [1:13:06<4:55:46,  7.98s/it]
 19%|█▉        | 533/2757 [1:13:14<4:52:19,  7.89s/it]
 19%|█▉        | 534/2757 [1:13:22<4:53:03,  7.91s/it]
 19%|█▉        | 535/2757 [1:13:30<4:51:32,  7.87s/it]
 19%|█▉        | 536/2757 [1:13:38<4:54:57,  7.97s/it]
 19%|█▉        | 537/2757 [1:13:45<4:51:27,  7.88s/it]
 20%|█▉        | 538/2757 [1:13:53<4:53:09,  7.93s/it]
 20%|█▉        | 539/2757 [1:14:01<4:50:54,  7.87s/it]
 20%|█▉        | 540/2757 [1:14:09<4:53:03,  7.93s/it]
                                                      

 20%|█▉        | 540/2757 [1:14:09<4:53:03,  7.93s/it]
 20%|█▉        | 541/2757 [1:14:17<4:54:32,  7.98s/it]
 20%|█▉        | 542/2757 [1:14:25<4:53:00,  7.94s/it]
 20%|█▉        | 543/2757 [1:14:34<4:57:26,  8.06s/it]
 20%|█▉        | 544/2757 [1:14:41<4:51:29,  7.90s/it]
 20%|█▉        | 545/2757 [1:14:49<4:54:20,  7.98s/it]
 20%|█▉        | 546/2757 [1:14:58<4:58:13,  8.09s/it]
 20%|█▉        | 547/2757 [1:15:05<4:54:39,  8.00s/it]
 20%|█▉        | 548/2757 [1:15:14<4:56:26,  8.05s/it]
 20%|█▉        | 549/2757 [1:15:22<5:00:59,  8.18s/it]
 20%|█▉        | 550/2757 [1:15:30<5:02:26,  8.22s/it]
                                                      

 20%|█▉        | 550/2757 [1:15:30<5:02:26,  8.22s/it]
 20%|█▉        | 551/2757 [1:15:39<5:03:46,  8.26s/it]
 20%|██        | 552/2757 [1:15:46<4:57:16,  8.09s/it]
 20%|██        | 553/2757 [1:15:54<4:56:03,  8.06s/it]
 20%|██        | 554/2757 [1:16:02<4:55:32,  8.05s/it]
 20%|██        | 555/2757 [1:16:11<4:59:00,  8.15s/it]
 20%|██        | 556/2757 [1:16:19<5:00:26,  8.19s/it]
 20%|██        | 557/2757 [1:16:27<4:54:49,  8.04s/it]
 20%|██        | 558/2757 [1:16:35<4:53:26,  8.01s/it]
 20%|██        | 559/2757 [1:16:43<4:54:02,  8.03s/it]
 20%|██        | 560/2757 [1:16:52<5:03:55,  8.30s/it]
                                                      

 20%|██        | 560/2757 [1:16:52<5:03:55,  8.30s/it]
 20%|██        | 561/2757 [1:17:00<5:02:05,  8.25s/it]
 20%|██        | 562/2757 [1:17:08<5:00:18,  8.21s/it]
 20%|██        | 563/2757 [1:17:16<4:59:14,  8.18s/it]
 20%|██        | 564/2757 [1:17:24<4:56:18,  8.11s/it]
 20%|██        | 565/2757 [1:17:33<5:01:51,  8.26s/it]
 21%|██        | 566/2757 [1:17:41<5:00:49,  8.24s/it]
 21%|██        | 567/2757 [1:17:49<4:55:55,  8.11s/it]
 21%|██        | 568/2757 [1:17:57<4:55:24,  8.10s/it]
 21%|██        | 569/2757 [1:18:05<4:52:35,  8.02s/it]
 21%|██        | 570/2757 [1:18:13<4:56:29,  8.13s/it]
                                                      

 21%|██        | 570/2757 [1:18:13<4:56:29,  8.13s/it]
 21%|██        | 571/2757 [1:18:21<4:59:14,  8.21s/it]
 21%|██        | 572/2757 [1:18:29<4:58:35,  8.20s/it]
 21%|██        | 573/2757 [1:18:38<5:00:35,  8.26s/it]
 21%|██        | 574/2757 [1:18:46<4:57:10,  8.17s/it]
 21%|██        | 575/2757 [1:18:54<4:52:03,  8.03s/it]
 21%|██        | 576/2757 [1:19:02<4:54:31,  8.10s/it]
 21%|██        | 577/2757 [1:19:10<4:52:34,  8.05s/it]
 21%|██        | 578/2757 [1:19:17<4:49:02,  7.96s/it]
 21%|██        | 579/2757 [1:19:25<4:43:39,  7.81s/it]
 21%|██        | 580/2757 [1:19:33<4:43:06,  7.80s/it]
                                                      

 21%|██        | 580/2757 [1:19:33<4:43:06,  7.80s/it]
 21%|██        | 581/2757 [1:19:41<4:46:11,  7.89s/it]
 21%|██        | 582/2757 [1:19:49<4:47:43,  7.94s/it]
 21%|██        | 583/2757 [1:19:57<4:46:39,  7.91s/it]
 21%|██        | 584/2757 [1:20:05<4:45:10,  7.87s/it]
 21%|██        | 585/2757 [1:20:12<4:45:50,  7.90s/it]
 21%|██▏       | 586/2757 [1:20:21<4:47:26,  7.94s/it]
 21%|██▏       | 587/2757 [1:20:28<4:46:32,  7.92s/it]
 21%|██▏       | 588/2757 [1:20:36<4:48:21,  7.98s/it]
 21%|██▏       | 589/2757 [1:20:44<4:44:12,  7.87s/it]
 21%|██▏       | 590/2757 [1:20:52<4:40:45,  7.77s/it]
                                                      

 21%|██▏       | 590/2757 [1:20:52<4:40:45,  7.77s/it]
 21%|██▏       | 591/2757 [1:21:00<4:43:43,  7.86s/it]
 21%|██▏       | 592/2757 [1:21:08<4:43:31,  7.86s/it]
 22%|██▏       | 593/2757 [1:21:16<4:47:21,  7.97s/it]
 22%|██▏       | 594/2757 [1:21:24<4:45:34,  7.92s/it]
 22%|██▏       | 595/2757 [1:21:32<4:47:49,  7.99s/it]
 22%|██▏       | 596/2757 [1:21:40<4:47:23,  7.98s/it]
 22%|██▏       | 597/2757 [1:21:48<4:48:15,  8.01s/it]
 22%|██▏       | 598/2757 [1:21:56<4:49:02,  8.03s/it]
 22%|██▏       | 599/2757 [1:22:04<4:47:48,  8.00s/it]
 22%|██▏       | 600/2757 [1:22:12<4:44:38,  7.92s/it]
                                                      

 22%|██▏       | 600/2757 [1:22:12<4:44:38,  7.92s/it]
 22%|██▏       | 601/2757 [1:22:20<4:46:12,  7.97s/it]
 22%|██▏       | 602/2757 [1:22:28<4:49:08,  8.05s/it]
 22%|██▏       | 603/2757 [1:22:36<4:44:57,  7.94s/it]
 22%|██▏       | 604/2757 [1:22:44<4:45:59,  7.97s/it]
 22%|██▏       | 605/2757 [1:22:52<4:50:52,  8.11s/it]
 22%|██▏       | 606/2757 [1:23:00<4:53:05,  8.18s/it]
 22%|██▏       | 607/2757 [1:23:08<4:51:14,  8.13s/it]
 22%|██▏       | 608/2757 [1:23:16<4:48:07,  8.04s/it]
 22%|██▏       | 609/2757 [1:23:24<4:48:23,  8.06s/it]
 22%|██▏       | 610/2757 [1:23:32<4:49:19,  8.09s/it]
                                                      

 22%|██▏       | 610/2757 [1:23:32<4:49:19,  8.09s/it]
 22%|██▏       | 611/2757 [1:23:41<4:50:24,  8.12s/it]
 22%|██▏       | 612/2757 [1:23:49<4:54:34,  8.24s/it]
 22%|██▏       | 613/2757 [1:23:57<4:51:55,  8.17s/it]
 22%|██▏       | 614/2757 [1:24:05<4:49:26,  8.10s/it]
 22%|██▏       | 615/2757 [1:24:13<4:50:31,  8.14s/it]
 22%|██▏       | 616/2757 [1:24:22<4:52:21,  8.19s/it]
 22%|██▏       | 617/2757 [1:24:30<4:51:10,  8.16s/it]
 22%|██▏       | 618/2757 [1:24:38<4:47:57,  8.08s/it]
 22%|██▏       | 619/2757 [1:24:46<4:48:07,  8.09s/it]
 22%|██▏       | 620/2757 [1:24:53<4:42:34,  7.93s/it]
                                                      

 22%|██▏       | 620/2757 [1:24:53<4:42:34,  7.93s/it]
 23%|██▎       | 621/2757 [1:25:01<4:40:57,  7.89s/it]
 23%|██▎       | 622/2757 [1:25:09<4:44:11,  7.99s/it]
 23%|██▎       | 623/2757 [1:25:17<4:36:22,  7.77s/it]
 23%|██▎       | 624/2757 [1:25:25<4:41:35,  7.92s/it]
 23%|██▎       | 625/2757 [1:25:33<4:43:06,  7.97s/it]
 23%|██▎       | 626/2757 [1:25:41<4:43:43,  7.99s/it]
 23%|██▎       | 627/2757 [1:25:49<4:44:41,  8.02s/it]
 23%|██▎       | 628/2757 [1:25:57<4:44:18,  8.01s/it]
 23%|██▎       | 629/2757 [1:26:05<4:43:39,  8.00s/it]
 23%|██▎       | 630/2757 [1:26:13<4:47:36,  8.11s/it]
                                                      

 23%|██▎       | 630/2757 [1:26:13<4:47:36,  8.11s/it]
 23%|██▎       | 631/2757 [1:26:22<4:47:46,  8.12s/it]
 23%|██▎       | 632/2757 [1:26:30<4:54:37,  8.32s/it]
 23%|██▎       | 633/2757 [1:26:38<4:48:41,  8.16s/it]
 23%|██▎       | 634/2757 [1:26:47<4:52:48,  8.28s/it]
 23%|██▎       | 635/2757 [1:26:55<4:51:05,  8.23s/it]
 23%|██▎       | 636/2757 [1:27:03<4:48:05,  8.15s/it]
 23%|██▎       | 637/2757 [1:27:11<4:52:19,  8.27s/it]
 23%|██▎       | 638/2757 [1:27:20<4:51:59,  8.27s/it]
 23%|██▎       | 639/2757 [1:27:28<4:50:23,  8.23s/it]
 23%|██▎       | 640/2757 [1:27:35<4:43:36,  8.04s/it]
                                                      

 23%|██▎       | 640/2757 [1:27:35<4:43:36,  8.04s/it]
 23%|██▎       | 641/2757 [1:27:43<4:40:02,  7.94s/it]
 23%|██▎       | 642/2757 [1:27:51<4:38:05,  7.89s/it]
 23%|██▎       | 643/2757 [1:27:59<4:39:02,  7.92s/it]
 23%|██▎       | 644/2757 [1:28:07<4:41:22,  7.99s/it]
 23%|██▎       | 645/2757 [1:28:16<4:50:53,  8.26s/it]
 23%|██▎       | 646/2757 [1:28:24<4:50:04,  8.24s/it]
 23%|██▎       | 647/2757 [1:28:32<4:44:08,  8.08s/it]
 24%|██▎       | 648/2757 [1:28:40<4:42:29,  8.04s/it]
 24%|██▎       | 649/2757 [1:28:48<4:41:44,  8.02s/it]
 24%|██▎       | 650/2757 [1:28:56<4:42:18,  8.04s/it]
                                                      

 24%|██▎       | 650/2757 [1:28:56<4:42:18,  8.04s/it]
 24%|██▎       | 651/2757 [1:29:04<4:44:25,  8.10s/it]
 24%|██▎       | 652/2757 [1:29:12<4:47:45,  8.20s/it]
 24%|██▎       | 653/2757 [1:29:21<4:47:57,  8.21s/it]
 24%|██▎       | 654/2757 [1:29:29<4:47:35,  8.21s/it]
 24%|██▍       | 655/2757 [1:29:37<4:43:45,  8.10s/it]
 24%|██▍       | 656/2757 [1:29:45<4:43:11,  8.09s/it]
 24%|██▍       | 657/2757 [1:29:53<4:47:40,  8.22s/it]
 24%|██▍       | 658/2757 [1:30:01<4:41:47,  8.06s/it]
 24%|██▍       | 659/2757 [1:30:09<4:40:42,  8.03s/it]
 24%|██▍       | 660/2757 [1:30:17<4:38:10,  7.96s/it]
                                                      

 24%|██▍       | 660/2757 [1:30:17<4:38:10,  7.96s/it]
 24%|██▍       | 661/2757 [1:30:24<4:31:12,  7.76s/it]
 24%|██▍       | 662/2757 [1:30:31<4:27:54,  7.67s/it]
 24%|██▍       | 663/2757 [1:30:40<4:33:08,  7.83s/it]
 24%|██▍       | 664/2757 [1:30:48<4:34:16,  7.86s/it]
 24%|██▍       | 665/2757 [1:30:56<4:40:29,  8.04s/it]
 24%|██▍       | 666/2757 [1:31:04<4:41:44,  8.08s/it]
 24%|██▍       | 667/2757 [1:31:13<4:45:48,  8.21s/it]
 24%|██▍       | 668/2757 [1:31:21<4:41:54,  8.10s/it]
 24%|██▍       | 669/2757 [1:31:28<4:38:12,  7.99s/it]
 24%|██▍       | 670/2757 [1:31:36<4:39:08,  8.03s/it]
                                                      

 24%|██▍       | 670/2757 [1:31:36<4:39:08,  8.03s/it]
 24%|██▍       | 671/2757 [1:31:45<4:40:37,  8.07s/it]
 24%|██▍       | 672/2757 [1:31:52<4:35:39,  7.93s/it]
 24%|██▍       | 673/2757 [1:32:00<4:32:02,  7.83s/it]
 24%|██▍       | 674/2757 [1:32:08<4:34:47,  7.92s/it]
 24%|██▍       | 675/2757 [1:32:15<4:31:05,  7.81s/it]
 25%|██▍       | 676/2757 [1:32:23<4:30:35,  7.80s/it]
 25%|██▍       | 677/2757 [1:32:32<4:36:49,  7.99s/it]
 25%|██▍       | 678/2757 [1:32:40<4:41:07,  8.11s/it]
 25%|██▍       | 679/2757 [1:32:48<4:41:14,  8.12s/it]
 25%|██▍       | 680/2757 [1:32:56<4:38:54,  8.06s/it]
                                                      

 25%|██▍       | 680/2757 [1:32:56<4:38:54,  8.06s/it]
 25%|██▍       | 681/2757 [1:33:04<4:31:58,  7.86s/it]
 25%|██▍       | 682/2757 [1:33:12<4:33:14,  7.90s/it]
 25%|██▍       | 683/2757 [1:33:19<4:31:25,  7.85s/it]
 25%|██▍       | 684/2757 [1:33:27<4:34:15,  7.94s/it]
 25%|██▍       | 685/2757 [1:33:35<4:33:51,  7.93s/it]
 25%|██▍       | 686/2757 [1:33:43<4:33:15,  7.92s/it]
 25%|██▍       | 687/2757 [1:33:51<4:33:59,  7.94s/it]
 25%|██▍       | 688/2757 [1:33:59<4:27:13,  7.75s/it]
 25%|██▍       | 689/2757 [1:34:06<4:28:54,  7.80s/it]
 25%|██▌       | 690/2757 [1:34:15<4:33:13,  7.93s/it]
                                                      

 25%|██▌       | 690/2757 [1:34:15<4:33:13,  7.93s/it]
 25%|██▌       | 691/2757 [1:34:22<4:29:59,  7.84s/it]
 25%|██▌       | 692/2757 [1:34:30<4:29:43,  7.84s/it]
 25%|██▌       | 693/2757 [1:34:38<4:33:57,  7.96s/it]
 25%|██▌       | 694/2757 [1:34:47<4:36:22,  8.04s/it]
 25%|██▌       | 695/2757 [1:34:55<4:40:55,  8.17s/it]
 25%|██▌       | 696/2757 [1:35:03<4:36:55,  8.06s/it]
 25%|██▌       | 697/2757 [1:35:11<4:35:05,  8.01s/it]
 25%|██▌       | 698/2757 [1:35:19<4:36:50,  8.07s/it]
 25%|██▌       | 699/2757 [1:35:27<4:36:34,  8.06s/it]
 25%|██▌       | 700/2757 [1:35:35<4:34:19,  8.00s/it]
                                                      

 25%|██▌       | 700/2757 [1:35:35<4:34:19,  8.00s/it]
 25%|██▌       | 701/2757 [1:35:43<4:34:35,  8.01s/it]
 25%|██▌       | 702/2757 [1:35:51<4:33:25,  7.98s/it]
 25%|██▌       | 703/2757 [1:35:59<4:30:29,  7.90s/it]
 26%|██▌       | 704/2757 [1:36:06<4:28:10,  7.84s/it]
 26%|██▌       | 705/2757 [1:36:14<4:27:55,  7.83s/it]
 26%|██▌       | 706/2757 [1:36:23<4:33:58,  8.01s/it]
 26%|██▌       | 707/2757 [1:36:30<4:32:56,  7.99s/it]
 26%|██▌       | 708/2757 [1:36:38<4:32:11,  7.97s/it]
 26%|██▌       | 709/2757 [1:36:46<4:27:26,  7.84s/it]
 26%|██▌       | 710/2757 [1:36:54<4:29:01,  7.89s/it]
                                                      

 26%|██▌       | 710/2757 [1:36:54<4:29:01,  7.89s/it]
 26%|██▌       | 711/2757 [1:37:02<4:31:54,  7.97s/it]
 26%|██▌       | 712/2757 [1:37:10<4:30:56,  7.95s/it]
 26%|██▌       | 713/2757 [1:37:18<4:29:57,  7.92s/it]
 26%|██▌       | 714/2757 [1:37:26<4:30:56,  7.96s/it]
 26%|██▌       | 715/2757 [1:37:34<4:28:21,  7.89s/it]
 26%|██▌       | 716/2757 [1:37:42<4:28:57,  7.91s/it]
 26%|██▌       | 717/2757 [1:37:50<4:33:00,  8.03s/it]
 26%|██▌       | 718/2757 [1:37:57<4:28:44,  7.91s/it]
 26%|██▌       | 719/2757 [1:38:05<4:25:26,  7.81s/it]
 26%|██▌       | 720/2757 [1:38:13<4:25:41,  7.83s/it]
                                                      

 26%|██▌       | 720/2757 [1:38:13<4:25:41,  7.83s/it]
 26%|██▌       | 721/2757 [1:38:20<4:22:46,  7.74s/it]
 26%|██▌       | 722/2757 [1:38:28<4:23:05,  7.76s/it]
 26%|██▌       | 723/2757 [1:38:37<4:30:15,  7.97s/it]
 26%|██▋       | 724/2757 [1:38:44<4:27:46,  7.90s/it]
 26%|██▋       | 725/2757 [1:38:52<4:24:20,  7.81s/it]
 26%|██▋       | 726/2757 [1:39:01<4:31:17,  8.01s/it]
 26%|██▋       | 727/2757 [1:39:09<4:37:03,  8.19s/it]
 26%|██▋       | 728/2757 [1:39:17<4:34:19,  8.11s/it]
 26%|██▋       | 729/2757 [1:39:25<4:32:14,  8.05s/it]
 26%|██▋       | 730/2757 [1:39:33<4:29:33,  7.98s/it]
                                                      

 26%|██▋       | 730/2757 [1:39:33<4:29:33,  7.98s/it]
 27%|██▋       | 731/2757 [1:39:41<4:27:45,  7.93s/it]
 27%|██▋       | 732/2757 [1:39:49<4:28:19,  7.95s/it]
 27%|██▋       | 733/2757 [1:39:57<4:28:42,  7.97s/it]
 27%|██▋       | 734/2757 [1:40:05<4:28:14,  7.96s/it]
 27%|██▋       | 735/2757 [1:40:13<4:33:13,  8.11s/it]
 27%|██▋       | 736/2757 [1:40:21<4:34:56,  8.16s/it]
 27%|██▋       | 737/2757 [1:40:29<4:32:04,  8.08s/it]
 27%|██▋       | 738/2757 [1:40:37<4:29:44,  8.02s/it]
 27%|██▋       | 739/2757 [1:40:45<4:24:21,  7.86s/it]
 27%|██▋       | 740/2757 [1:40:53<4:33:32,  8.14s/it]
                                                      

 27%|██▋       | 740/2757 [1:40:53<4:33:32,  8.14s/it]
 27%|██▋       | 741/2757 [1:41:01<4:32:02,  8.10s/it]
 27%|██▋       | 742/2757 [1:41:09<4:32:17,  8.11s/it]
 27%|██▋       | 743/2757 [1:41:18<4:37:26,  8.27s/it]
 27%|██▋       | 744/2757 [1:41:26<4:32:39,  8.13s/it]
 27%|██▋       | 745/2757 [1:41:34<4:32:33,  8.13s/it]
 27%|██▋       | 746/2757 [1:41:42<4:27:01,  7.97s/it]
 27%|██▋       | 747/2757 [1:41:49<4:22:32,  7.84s/it]
 27%|██▋       | 748/2757 [1:41:57<4:25:00,  7.91s/it]
 27%|██▋       | 749/2757 [1:42:05<4:25:26,  7.93s/it]
 27%|██▋       | 750/2757 [1:42:13<4:25:53,  7.95s/it]
                                                      
{'eval_loss': '0.5771', 'eval_runtime': '104.9', 'eval_samples_per_second': '22.88', 'eval_steps_per_second': '2.86', 'eval_rewards/chosen': '-0.6041', 'eval_rewards/rejected': '-1.069', 'eval_rewards/accuracies': '0.625', 'eval_rewards/margins': '0.4646', 'eval_logps/chosen': '-85.65', 'eval_logps/rejected': '-96.54', 'eval_logits/chosen': '-3.086', 'eval_logits/rejected': '-3.039', 'epoch': '0.5442'}
{'loss': '0.5776', 'grad_norm': '0.5719', 'learning_rate': '0.0001681', 'rewards/chosen': '-0.6292', 'rewards/rejected': '-1.103', 'rewards/accuracies': '0.6281', 'rewards/margins': '0.4741', 'logps/chosen': '-84.31', 'logps/rejected': '-96.12', 'logits/chosen': '-2.981', 'logits/rejected': '-3.005', 'epoch': '0.5551'}
{'loss': '0.5797', 'grad_norm': '0.6719', 'learning_rate': '0.0001674', 'rewards/chosen': '-0.6672', 'rewards/rejected': '-1.152', 'rewards/accuracies': '0.6539', 'rewards/margins': '0.4843', 'logps/chosen': '-84.52', 'logps/rejected': '-95.7', 'logits/chosen': '-3.107', 'logits/rejected': '-3.058', 'epoch': '0.566'}
{'loss': '0.5927', 'grad_norm': '0.6522', 'learning_rate': '0.0001666', 'rewards/chosen': '-0.6613', 'rewards/rejected': '-1.109', 'rewards/accuracies': '0.625', 'rewards/margins': '0.4473', 'logps/chosen': '-85.24', 'logps/rejected': '-94.5', 'logits/chosen': '-3.016', 'logits/rejected': '-3.001', 'epoch': '0.5769'}
{'loss': '0.6019', 'grad_norm': '0.9451', 'learning_rate': '0.0001659', 'rewards/chosen': '-0.6853', 'rewards/rejected': '-1.141', 'rewards/accuracies': '0.6266', 'rewards/margins': '0.4554', 'logps/chosen': '-87.74', 'logps/rejected': '-98.06', 'logits/chosen': '-2.828', 'logits/rejected': '-2.82', 'epoch': '0.5878'}
{'loss': '0.5943', 'grad_norm': '0.6104', 'learning_rate': '0.0001651', 'rewards/chosen': '-0.7626', 'rewards/rejected': '-1.175', 'rewards/accuracies': '0.6313', 'rewards/margins': '0.4125', 'logps/chosen': '-86.3', 'logps/rejected': '-96.72', 'logits/chosen': '-2.712', 'logits/rejected': '-2.636', 'epoch': '0.5986'}
{'loss': '0.5997', 'grad_norm': '0.5953', 'learning_rate': '0.0001644', 'rewards/chosen': '-0.7641', 'rewards/rejected': '-1.156', 'rewards/accuracies': '0.632', 'rewards/margins': '0.3917', 'logps/chosen': '-85.59', 'logps/rejected': '-94.78', 'logits/chosen': '-2.773', 'logits/rejected': '-2.675', 'epoch': '0.6095'}
{'loss': '0.5831', 'grad_norm': '0.6583', 'learning_rate': '0.0001636', 'rewards/chosen': '-0.7877', 'rewards/rejected': '-1.254', 'rewards/accuracies': '0.6258', 'rewards/margins': '0.4662', 'logps/chosen': '-90.34', 'logps/rejected': '-99.95', 'logits/chosen': '-2.784', 'logits/rejected': '-2.651', 'epoch': '0.6204'}
{'loss': '0.5975', 'grad_norm': '0.5589', 'learning_rate': '0.0001629', 'rewards/chosen': '-0.5206', 'rewards/rejected': '-0.9202', 'rewards/accuracies': '0.625', 'rewards/margins': '0.3997', 'logps/chosen': '-83.24', 'logps/rejected': '-91.77', 'logits/chosen': '-2.907', 'logits/rejected': '-2.885', 'epoch': '0.6313'}
{'loss': '0.5781', 'grad_norm': '0.6037', 'learning_rate': '0.0001622', 'rewards/chosen': '-0.4098', 'rewards/rejected': '-0.8762', 'rewards/accuracies': '0.6414', 'rewards/margins': '0.4664', 'logps/chosen': '-83.76', 'logps/rejected': '-94.21', 'logits/chosen': '-2.92', 'logits/rejected': '-2.842', 'epoch': '0.6422'}
{'loss': '0.5839', 'grad_norm': '0.6424', 'learning_rate': '0.0001614', 'rewards/chosen': '-0.5493', 'rewards/rejected': '-0.9931', 'rewards/accuracies': '0.6586', 'rewards/margins': '0.4438', 'logps/chosen': '-86.28', 'logps/rejected': '-96.4', 'logits/chosen': '-2.929', 'logits/rejected': '-2.954', 'epoch': '0.6531'}
{'loss': '0.5724', 'grad_norm': '0.5479', 'learning_rate': '0.0001607', 'rewards/chosen': '-0.5622', 'rewards/rejected': '-1.034', 'rewards/accuracies': '0.6438', 'rewards/margins': '0.472', 'logps/chosen': '-87.9', 'logps/rejected': '-96.35', 'logits/chosen': '-2.836', 'logits/rejected': '-2.8', 'epoch': '0.6639'}
{'loss': '0.5745', 'grad_norm': '0.6719', 'learning_rate': '0.0001599', 'rewards/chosen': '-0.7492', 'rewards/rejected': '-1.227', 'rewards/accuracies': '0.6531', 'rewards/margins': '0.478', 'logps/chosen': '-88.12', 'logps/rejected': '-98.25', 'logits/chosen': '-2.963', 'logits/rejected': '-2.848', 'epoch': '0.6748'}
{'loss': '0.5667', 'grad_norm': '0.6403', 'learning_rate': '0.0001592', 'rewards/chosen': '-0.9368', 'rewards/rejected': '-1.455', 'rewards/accuracies': '0.6602', 'rewards/margins': '0.5185', 'logps/chosen': '-90.78', 'logps/rejected': '-103.6', 'logits/chosen': '-2.943', 'logits/rejected': '-2.97', 'epoch': '0.6857'}
{'loss': '0.5546', 'grad_norm': '0.6475', 'learning_rate': '0.0001584', 'rewards/chosen': '-0.9921', 'rewards/rejected': '-1.549', 'rewards/accuracies': '0.6461', 'rewards/margins': '0.557', 'logps/chosen': '-89.43', 'logps/rejected': '-100.7', 'logits/chosen': '-2.979', 'logits/rejected': '-2.938', 'epoch': '0.6966'}
{'loss': '0.574', 'grad_norm': '0.7136', 'learning_rate': '0.0001577', 'rewards/chosen': '-1.028', 'rewards/rejected': '-1.534', 'rewards/accuracies': '0.6375', 'rewards/margins': '0.5058', 'logps/chosen': '-91.74', 'logps/rejected': '-102.1', 'logits/chosen': '-3.021', 'logits/rejected': '-2.974', 'epoch': '0.7075'}
{'loss': '0.56', 'grad_norm': '0.6012', 'learning_rate': '0.0001569', 'rewards/chosen': '-1.082', 'rewards/rejected': '-1.642', 'rewards/accuracies': '0.6555', 'rewards/margins': '0.5598', 'logps/chosen': '-90.38', 'logps/rejected': '-101.6', 'logits/chosen': '-3.157', 'logits/rejected': '-3.125', 'epoch': '0.7184'}
{'loss': '0.5732', 'grad_norm': '0.6408', 'learning_rate': '0.0001562', 'rewards/chosen': '-1.105', 'rewards/rejected': '-1.613', 'rewards/accuracies': '0.6477', 'rewards/margins': '0.5072', 'logps/chosen': '-89.5', 'logps/rejected': '-100.5', 'logits/chosen': '-3.229', 'logits/rejected': '-3.236', 'epoch': '0.7293'}
{'loss': '0.5617', 'grad_norm': '0.6599', 'learning_rate': '0.0001554', 'rewards/chosen': '-0.8735', 'rewards/rejected': '-1.432', 'rewards/accuracies': '0.6641', 'rewards/margins': '0.5582', 'logps/chosen': '-91.57', 'logps/rejected': '-102.2', 'logits/chosen': '-3.211', 'logits/rejected': '-3.173', 'epoch': '0.7401'}
{'loss': '0.5498', 'grad_norm': '0.6686', 'learning_rate': '0.0001547', 'rewards/chosen': '-0.6966', 'rewards/rejected': '-1.305', 'rewards/accuracies': '0.6695', 'rewards/margins': '0.6086', 'logps/chosen': '-90.9', 'logps/rejected': '-102.5', 'logits/chosen': '-3.235', 'logits/rejected': '-3.097', 'epoch': '0.751'}
{'loss': '0.5865', 'grad_norm': '0.7763', 'learning_rate': '0.0001539', 'rewards/chosen': '-0.608', 'rewards/rejected': '-1.119', 'rewards/accuracies': '0.6539', 'rewards/margins': '0.5112', 'logps/chosen': '-85.77', 'logps/rejected': '-96.22', 'logits/chosen': '-3.261', 'logits/rejected': '-3.15', 'epoch': '0.7619'}
{'loss': '0.5637', 'grad_norm': '0.6534', 'learning_rate': '0.0001532', 'rewards/chosen': '-0.7708', 'rewards/rejected': '-1.309', 'rewards/accuracies': '0.6508', 'rewards/margins': '0.5377', 'logps/chosen': '-88.75', 'logps/rejected': '-99.82', 'logits/chosen': '-2.995', 'logits/rejected': '-2.962', 'epoch': '0.7728'}
{'loss': '0.5601', 'grad_norm': '0.5064', 'learning_rate': '0.0001524', 'rewards/chosen': '-0.8121', 'rewards/rejected': '-1.338', 'rewards/accuracies': '0.6375', 'rewards/margins': '0.5259', 'logps/chosen': '-84.73', 'logps/rejected': '-96.15', 'logits/chosen': '-3.161', 'logits/rejected': '-3.103', 'epoch': '0.7837'}
{'loss': '0.5589', 'grad_norm': '0.603', 'learning_rate': '0.0001517', 'rewards/chosen': '-0.7111', 'rewards/rejected': '-1.252', 'rewards/accuracies': '0.65', 'rewards/margins': '0.5404', 'logps/chosen': '-85.85', 'logps/rejected': '-96.86', 'logits/chosen': '-3.101', 'logits/rejected': '-3.052', 'epoch': '0.7946'}
{'loss': '0.5412', 'grad_norm': '0.608', 'learning_rate': '0.0001509', 'rewards/chosen': '-0.7364', 'rewards/rejected': '-1.331', 'rewards/accuracies': '0.6672', 'rewards/margins': '0.5946', 'logps/chosen': '-91.16', 'logps/rejected': '-101.2', 'logits/chosen': '-3.177', 'logits/rejected': '-3.198', 'epoch': '0.8054'}
{'loss': '0.5476', 'grad_norm': '0.6249', 'learning_rate': '0.0001502', 'rewards/chosen': '-0.6863', 'rewards/rejected': '-1.317', 'rewards/accuracies': '0.668', 'rewards/margins': '0.6311', 'logps/chosen': '-85.68', 'logps/rejected': '-97.83', 'logits/chosen': '-3.299', 'logits/rejected': '-3.16', 'epoch': '0.8163'}
 27%|██▋       | 750/2757 [1:42:13<4:25:53,  7.95s/it]
 27%|██▋       | 751/2757 [1:42:25<5:02:01,  9.03s/it]
 27%|██▋       | 752/2757 [1:42:33<4:50:08,  8.68s/it]
 27%|██▋       | 753/2757 [1:42:40<4:37:16,  8.30s/it]
 27%|██▋       | 754/2757 [1:42:48<4:35:19,  8.25s/it]
 27%|██▋       | 755/2757 [1:42:57<4:36:26,  8.29s/it]
 27%|██▋       | 756/2757 [1:43:05<4:34:48,  8.24s/it]
 27%|██▋       | 757/2757 [1:43:13<4:35:00,  8.25s/it]
 27%|██▋       | 758/2757 [1:43:21<4:33:42,  8.22s/it]
 28%|██▊       | 759/2757 [1:43:30<4:35:52,  8.28s/it]
 28%|██▊       | 760/2757 [1:43:37<4:31:39,  8.16s/it]
                                                      

 28%|██▊       | 760/2757 [1:43:37<4:31:39,  8.16s/it]
 28%|██▊       | 761/2757 [1:43:45<4:28:49,  8.08s/it]
 28%|██▊       | 762/2757 [1:43:54<4:30:20,  8.13s/it]
 28%|██▊       | 763/2757 [1:44:02<4:31:47,  8.18s/it]
 28%|██▊       | 764/2757 [1:44:09<4:25:25,  7.99s/it]
 28%|██▊       | 765/2757 [1:44:17<4:21:32,  7.88s/it]
 28%|██▊       | 766/2757 [1:44:25<4:22:31,  7.91s/it]
 28%|██▊       | 767/2757 [1:44:33<4:26:36,  8.04s/it]
 28%|██▊       | 768/2757 [1:44:42<4:29:57,  8.14s/it]
 28%|██▊       | 769/2757 [1:44:50<4:26:13,  8.03s/it]
 28%|██▊       | 770/2757 [1:44:57<4:23:28,  7.96s/it]
                                                      

 28%|██▊       | 770/2757 [1:44:57<4:23:28,  7.96s/it]
 28%|██▊       | 771/2757 [1:45:05<4:25:31,  8.02s/it]
 28%|██▊       | 772/2757 [1:45:14<4:27:16,  8.08s/it]
 28%|██▊       | 773/2757 [1:45:22<4:27:16,  8.08s/it]
 28%|██▊       | 774/2757 [1:45:30<4:25:37,  8.04s/it]
 28%|██▊       | 775/2757 [1:45:38<4:25:14,  8.03s/it]
 28%|██▊       | 776/2757 [1:45:46<4:26:07,  8.06s/it]
 28%|██▊       | 777/2757 [1:45:54<4:22:47,  7.96s/it]
 28%|██▊       | 778/2757 [1:46:01<4:20:55,  7.91s/it]
 28%|██▊       | 779/2757 [1:46:09<4:22:31,  7.96s/it]
 28%|██▊       | 780/2757 [1:46:17<4:18:58,  7.86s/it]
                                                      

 28%|██▊       | 780/2757 [1:46:17<4:18:58,  7.86s/it]
 28%|██▊       | 781/2757 [1:46:26<4:25:22,  8.06s/it]
 28%|██▊       | 782/2757 [1:46:34<4:24:28,  8.03s/it]
 28%|██▊       | 783/2757 [1:46:41<4:20:20,  7.91s/it]
 28%|██▊       | 784/2757 [1:46:49<4:20:02,  7.91s/it]
 28%|██▊       | 785/2757 [1:46:57<4:23:30,  8.02s/it]
 29%|██▊       | 786/2757 [1:47:05<4:19:49,  7.91s/it]
 29%|██▊       | 787/2757 [1:47:13<4:19:16,  7.90s/it]
 29%|██▊       | 788/2757 [1:47:21<4:20:24,  7.94s/it]
 29%|██▊       | 789/2757 [1:47:29<4:22:53,  8.02s/it]
 29%|██▊       | 790/2757 [1:47:37<4:18:18,  7.88s/it]
                                                      

 29%|██▊       | 790/2757 [1:47:37<4:18:18,  7.88s/it]
 29%|██▊       | 791/2757 [1:47:44<4:16:32,  7.83s/it]
 29%|██▊       | 792/2757 [1:47:52<4:14:41,  7.78s/it]
 29%|██▉       | 793/2757 [1:48:00<4:13:26,  7.74s/it]
 29%|██▉       | 794/2757 [1:48:07<4:11:02,  7.67s/it]
 29%|██▉       | 795/2757 [1:48:15<4:12:41,  7.73s/it]
 29%|██▉       | 796/2757 [1:48:23<4:15:42,  7.82s/it]
 29%|██▉       | 797/2757 [1:48:31<4:16:44,  7.86s/it]
 29%|██▉       | 798/2757 [1:48:39<4:17:43,  7.89s/it]
 29%|██▉       | 799/2757 [1:48:47<4:17:56,  7.90s/it]
 29%|██▉       | 800/2757 [1:48:55<4:18:34,  7.93s/it]
                                                      

 29%|██▉       | 800/2757 [1:48:55<4:18:34,  7.93s/it]
 29%|██▉       | 801/2757 [1:49:03<4:20:00,  7.98s/it]
 29%|██▉       | 802/2757 [1:49:11<4:20:17,  7.99s/it]
 29%|██▉       | 803/2757 [1:49:19<4:19:35,  7.97s/it]
 29%|██▉       | 804/2757 [1:49:27<4:20:49,  8.01s/it]
 29%|██▉       | 805/2757 [1:49:35<4:21:01,  8.02s/it]
 29%|██▉       | 806/2757 [1:49:43<4:19:24,  7.98s/it]
 29%|██▉       | 807/2757 [1:49:51<4:16:11,  7.88s/it]
 29%|██▉       | 808/2757 [1:49:58<4:12:47,  7.78s/it]
 29%|██▉       | 809/2757 [1:50:06<4:14:54,  7.85s/it]
 29%|██▉       | 810/2757 [1:50:14<4:14:54,  7.86s/it]
                                                      

 29%|██▉       | 810/2757 [1:50:14<4:14:54,  7.86s/it]
 29%|██▉       | 811/2757 [1:50:22<4:17:39,  7.94s/it]
 29%|██▉       | 812/2757 [1:50:30<4:18:44,  7.98s/it]
 29%|██▉       | 813/2757 [1:50:39<4:21:04,  8.06s/it]
 30%|██▉       | 814/2757 [1:50:47<4:20:41,  8.05s/it]
 30%|██▉       | 815/2757 [1:50:55<4:21:15,  8.07s/it]
 30%|██▉       | 816/2757 [1:51:03<4:21:10,  8.07s/it]
 30%|██▉       | 817/2757 [1:51:11<4:21:42,  8.09s/it]
 30%|██▉       | 818/2757 [1:51:19<4:18:45,  8.01s/it]
 30%|██▉       | 819/2757 [1:51:27<4:18:13,  7.99s/it]
 30%|██▉       | 820/2757 [1:51:35<4:16:44,  7.95s/it]
                                                      

 30%|██▉       | 820/2757 [1:51:35<4:16:44,  7.95s/it]
 30%|██▉       | 821/2757 [1:51:43<4:16:28,  7.95s/it]
 30%|██▉       | 822/2757 [1:51:51<4:17:36,  7.99s/it]
 30%|██▉       | 823/2757 [1:51:58<4:14:10,  7.89s/it]
 30%|██▉       | 824/2757 [1:52:06<4:11:44,  7.81s/it]
 30%|██▉       | 825/2757 [1:52:13<4:07:02,  7.67s/it]
 30%|██▉       | 826/2757 [1:52:21<4:10:58,  7.80s/it]
 30%|██▉       | 827/2757 [1:52:29<4:10:36,  7.79s/it]
 30%|███       | 828/2757 [1:52:37<4:11:05,  7.81s/it]
 30%|███       | 829/2757 [1:52:45<4:09:20,  7.76s/it]
 30%|███       | 830/2757 [1:52:52<4:10:41,  7.81s/it]
                                                      

 30%|███       | 830/2757 [1:52:52<4:10:41,  7.81s/it]
 30%|███       | 831/2757 [1:53:01<4:15:33,  7.96s/it]
 30%|███       | 832/2757 [1:53:09<4:12:58,  7.89s/it]
 30%|███       | 833/2757 [1:53:17<4:14:32,  7.94s/it]
 30%|███       | 834/2757 [1:53:24<4:13:02,  7.89s/it]
 30%|███       | 835/2757 [1:53:32<4:12:32,  7.88s/it]
 30%|███       | 836/2757 [1:53:40<4:12:39,  7.89s/it]
 30%|███       | 837/2757 [1:53:48<4:16:51,  8.03s/it]
 30%|███       | 838/2757 [1:53:56<4:13:19,  7.92s/it]
 30%|███       | 839/2757 [1:54:05<4:17:09,  8.04s/it]
 30%|███       | 840/2757 [1:54:13<4:16:53,  8.04s/it]
                                                      

 30%|███       | 840/2757 [1:54:13<4:16:53,  8.04s/it]
 31%|███       | 841/2757 [1:54:21<4:19:30,  8.13s/it]
 31%|███       | 842/2757 [1:54:29<4:20:05,  8.15s/it]
 31%|███       | 843/2757 [1:54:38<4:23:11,  8.25s/it]
 31%|███       | 844/2757 [1:54:45<4:18:12,  8.10s/it]
 31%|███       | 845/2757 [1:54:53<4:16:49,  8.06s/it]
 31%|███       | 846/2757 [1:55:01<4:13:59,  7.97s/it]
 31%|███       | 847/2757 [1:55:09<4:09:02,  7.82s/it]
 31%|███       | 848/2757 [1:55:16<4:08:12,  7.80s/it]
 31%|███       | 849/2757 [1:55:24<4:10:15,  7.87s/it]
 31%|███       | 850/2757 [1:55:32<4:12:21,  7.94s/it]
                                                      

 31%|███       | 850/2757 [1:55:32<4:12:21,  7.94s/it]
 31%|███       | 851/2757 [1:55:40<4:11:58,  7.93s/it]
 31%|███       | 852/2757 [1:55:48<4:12:03,  7.94s/it]
 31%|███       | 853/2757 [1:55:56<4:13:45,  8.00s/it]
 31%|███       | 854/2757 [1:56:04<4:13:21,  7.99s/it]
 31%|███       | 855/2757 [1:56:12<4:14:35,  8.03s/it]
 31%|███       | 856/2757 [1:56:20<4:09:33,  7.88s/it]
 31%|███       | 857/2757 [1:56:28<4:12:43,  7.98s/it]
 31%|███       | 858/2757 [1:56:36<4:08:40,  7.86s/it]
 31%|███       | 859/2757 [1:56:44<4:14:31,  8.05s/it]
 31%|███       | 860/2757 [1:56:52<4:15:06,  8.07s/it]
                                                      

 31%|███       | 860/2757 [1:56:52<4:15:06,  8.07s/it]
 31%|███       | 861/2757 [1:57:01<4:17:50,  8.16s/it]
 31%|███▏      | 862/2757 [1:57:09<4:17:01,  8.14s/it]
 31%|███▏      | 863/2757 [1:57:17<4:17:28,  8.16s/it]
 31%|███▏      | 864/2757 [1:57:25<4:16:48,  8.14s/it]
 31%|███▏      | 865/2757 [1:57:33<4:16:26,  8.13s/it]
 31%|███▏      | 866/2757 [1:57:41<4:16:47,  8.15s/it]
 31%|███▏      | 867/2757 [1:57:49<4:15:30,  8.11s/it]
 31%|███▏      | 868/2757 [1:57:57<4:12:44,  8.03s/it]
 32%|███▏      | 869/2757 [1:58:06<4:14:20,  8.08s/it]
 32%|███▏      | 870/2757 [1:58:13<4:12:29,  8.03s/it]
                                                      

 32%|███▏      | 870/2757 [1:58:13<4:12:29,  8.03s/it]
 32%|███▏      | 871/2757 [1:58:22<4:15:38,  8.13s/it]
 32%|███▏      | 872/2757 [1:58:30<4:11:17,  8.00s/it]
 32%|███▏      | 873/2757 [1:58:37<4:07:54,  7.90s/it]
 32%|███▏      | 874/2757 [1:58:45<4:07:19,  7.88s/it]
 32%|███▏      | 875/2757 [1:58:53<4:06:46,  7.87s/it]
 32%|███▏      | 876/2757 [1:59:01<4:07:48,  7.90s/it]
 32%|███▏      | 877/2757 [1:59:10<4:18:00,  8.23s/it]
 32%|███▏      | 878/2757 [1:59:18<4:14:29,  8.13s/it]
 32%|███▏      | 879/2757 [1:59:26<4:14:30,  8.13s/it]
 32%|███▏      | 880/2757 [1:59:34<4:11:47,  8.05s/it]
                                                      

 32%|███▏      | 880/2757 [1:59:34<4:11:47,  8.05s/it]
 32%|███▏      | 881/2757 [1:59:42<4:12:35,  8.08s/it]
 32%|███▏      | 882/2757 [1:59:50<4:12:27,  8.08s/it]
 32%|███▏      | 883/2757 [1:59:58<4:12:35,  8.09s/it]
 32%|███▏      | 884/2757 [2:00:06<4:11:39,  8.06s/it]
 32%|███▏      | 885/2757 [2:00:14<4:11:25,  8.06s/it]
 32%|███▏      | 886/2757 [2:00:23<4:16:52,  8.24s/it]
 32%|███▏      | 887/2757 [2:00:31<4:15:02,  8.18s/it]
 32%|███▏      | 888/2757 [2:00:39<4:11:31,  8.07s/it]
 32%|███▏      | 889/2757 [2:00:46<4:08:38,  7.99s/it]
 32%|███▏      | 890/2757 [2:00:54<4:05:52,  7.90s/it]
                                                      

 32%|███▏      | 890/2757 [2:00:54<4:05:52,  7.90s/it]
 32%|███▏      | 891/2757 [2:01:02<4:07:10,  7.95s/it]
 32%|███▏      | 892/2757 [2:01:10<4:03:06,  7.82s/it]
 32%|███▏      | 893/2757 [2:01:17<4:02:31,  7.81s/it]
 32%|███▏      | 894/2757 [2:01:25<4:03:47,  7.85s/it]
 32%|███▏      | 895/2757 [2:01:33<4:00:13,  7.74s/it]
 32%|███▏      | 896/2757 [2:01:41<4:05:54,  7.93s/it]
 33%|███▎      | 897/2757 [2:01:49<4:05:23,  7.92s/it]
 33%|███▎      | 898/2757 [2:01:57<4:05:21,  7.92s/it]
 33%|███▎      | 899/2757 [2:02:05<4:06:15,  7.95s/it]
 33%|███▎      | 900/2757 [2:02:14<4:10:53,  8.11s/it]
                                                      

 33%|███▎      | 900/2757 [2:02:14<4:10:53,  8.11s/it]
 33%|███▎      | 901/2757 [2:02:21<4:08:48,  8.04s/it]
 33%|███▎      | 902/2757 [2:02:29<4:05:04,  7.93s/it]
 33%|███▎      | 903/2757 [2:02:37<4:01:10,  7.80s/it]
 33%|███▎      | 904/2757 [2:02:44<3:58:52,  7.74s/it]
 33%|███▎      | 905/2757 [2:02:52<4:03:40,  7.89s/it]
 33%|███▎      | 906/2757 [2:03:00<4:03:28,  7.89s/it]
 33%|███▎      | 907/2757 [2:03:09<4:06:13,  7.99s/it]
 33%|███▎      | 908/2757 [2:03:17<4:08:47,  8.07s/it]
 33%|███▎      | 909/2757 [2:03:25<4:07:38,  8.04s/it]
 33%|███▎      | 910/2757 [2:03:33<4:06:17,  8.00s/it]
                                                      

 33%|███▎      | 910/2757 [2:03:33<4:06:17,  8.00s/it]
 33%|███▎      | 911/2757 [2:03:41<4:08:09,  8.07s/it]
 33%|███▎      | 912/2757 [2:03:49<4:05:18,  7.98s/it]
 33%|███▎      | 913/2757 [2:03:56<4:01:59,  7.87s/it]
 33%|███▎      | 914/2757 [2:04:05<4:08:51,  8.10s/it]
 33%|███▎      | 915/2757 [2:04:13<4:11:25,  8.19s/it]
 33%|███▎      | 916/2757 [2:04:21<4:09:42,  8.14s/it]
 33%|███▎      | 917/2757 [2:04:29<4:04:35,  7.98s/it]
 33%|███▎      | 918/2757 [2:04:37<4:04:56,  7.99s/it]
 33%|███▎      | 919/2757 [2:04:43<3:45:56,  7.38s/it]
{'loss': '0.5396', 'grad_norm': '0.7027', 'learning_rate': '0.0001494', 'rewards/chosen': '-0.8498', 'rewards/rejected': '-1.501', 'rewards/accuracies': '0.6695', 'rewards/margins': '0.6513', 'logps/chosen': '-87.9', 'logps/rejected': '-101.8', 'logits/chosen': '-3.371', 'logits/rejected': '-3.293', 'epoch': '0.8272'}
{'loss': '0.5349', 'grad_norm': '0.6295', 'learning_rate': '0.0001487', 'rewards/chosen': '-1.063', 'rewards/rejected': '-1.742', 'rewards/accuracies': '0.6594', 'rewards/margins': '0.6796', 'logps/chosen': '-89.56', 'logps/rejected': '-102.7', 'logits/chosen': '-3.39', 'logits/rejected': '-3.347', 'epoch': '0.8381'}
{'loss': '0.5414', 'grad_norm': '0.6378', 'learning_rate': '0.0001479', 'rewards/chosen': '-1.214', 'rewards/rejected': '-1.858', 'rewards/accuracies': '0.6672', 'rewards/margins': '0.6445', 'logps/chosen': '-92.15', 'logps/rejected': '-104.5', 'logits/chosen': '-3.227', 'logits/rejected': '-3.154', 'epoch': '0.849'}
{'loss': '0.5625', 'grad_norm': '0.7089', 'learning_rate': '0.0001472', 'rewards/chosen': '-1.144', 'rewards/rejected': '-1.71', 'rewards/accuracies': '0.6352', 'rewards/margins': '0.5667', 'logps/chosen': '-92.22', 'logps/rejected': '-102.2', 'logits/chosen': '-3.079', 'logits/rejected': '-3.059', 'epoch': '0.8599'}
{'loss': '0.5564', 'grad_norm': '0.672', 'learning_rate': '0.0001464', 'rewards/chosen': '-1.028', 'rewards/rejected': '-1.614', 'rewards/accuracies': '0.6586', 'rewards/margins': '0.5857', 'logps/chosen': '-87.96', 'logps/rejected': '-99.78', 'logits/chosen': '-3.219', 'logits/rejected': '-3.18', 'epoch': '0.8707'}
{'loss': '0.5628', 'grad_norm': '0.7438', 'learning_rate': '0.0001457', 'rewards/chosen': '-1.04', 'rewards/rejected': '-1.64', 'rewards/accuracies': '0.6594', 'rewards/margins': '0.5995', 'logps/chosen': '-90.28', 'logps/rejected': '-101.8', 'logits/chosen': '-3.175', 'logits/rejected': '-3.129', 'epoch': '0.8816'}
{'loss': '0.5592', 'grad_norm': '0.7402', 'learning_rate': '0.000145', 'rewards/chosen': '-1.04', 'rewards/rejected': '-1.605', 'rewards/accuracies': '0.6562', 'rewards/margins': '0.5652', 'logps/chosen': '-92.35', 'logps/rejected': '-103.3', 'logits/chosen': '-3.144', 'logits/rejected': '-3.037', 'epoch': '0.8925'}
{'loss': '0.5435', 'grad_norm': '0.6177', 'learning_rate': '0.0001442', 'rewards/chosen': '-0.964', 'rewards/rejected': '-1.562', 'rewards/accuracies': '0.675', 'rewards/margins': '0.5981', 'logps/chosen': '-92.94', 'logps/rejected': '-102.4', 'logits/chosen': '-3.209', 'logits/rejected': '-3.142', 'epoch': '0.9034'}
{'loss': '0.5488', 'grad_norm': '0.6203', 'learning_rate': '0.0001435', 'rewards/chosen': '-0.9898', 'rewards/rejected': '-1.58', 'rewards/accuracies': '0.6789', 'rewards/margins': '0.5904', 'logps/chosen': '-91.24', 'logps/rejected': '-101.4', 'logits/chosen': '-3.352', 'logits/rejected': '-3.235', 'epoch': '0.9143'}
{'loss': '0.5207', 'grad_norm': '0.669', 'learning_rate': '0.0001427', 'rewards/chosen': '-0.9211', 'rewards/rejected': '-1.579', 'rewards/accuracies': '0.6836', 'rewards/margins': '0.6583', 'logps/chosen': '-89.67', 'logps/rejected': '-102.7', 'logits/chosen': '-3.454', 'logits/rejected': '-3.354', 'epoch': '0.9252'}
{'loss': '0.523', 'grad_norm': '0.7382', 'learning_rate': '0.000142', 'rewards/chosen': '-0.9207', 'rewards/rejected': '-1.614', 'rewards/accuracies': '0.6977', 'rewards/margins': '0.6931', 'logps/chosen': '-92.43', 'logps/rejected': '-103.6', 'logits/chosen': '-3.535', 'logits/rejected': '-3.453', 'epoch': '0.9361'}
{'loss': '0.532', 'grad_norm': '0.7022', 'learning_rate': '0.0001412', 'rewards/chosen': '-1.145', 'rewards/rejected': '-1.819', 'rewards/accuracies': '0.6953', 'rewards/margins': '0.6743', 'logps/chosen': '-91.38', 'logps/rejected': '-104.2', 'logits/chosen': '-3.44', 'logits/rejected': '-3.353', 'epoch': '0.9469'}
{'loss': '0.5318', 'grad_norm': '0.6528', 'learning_rate': '0.0001405', 'rewards/chosen': '-1.164', 'rewards/rejected': '-1.821', 'rewards/accuracies': '0.6773', 'rewards/margins': '0.6567', 'logps/chosen': '-90.99', 'logps/rejected': '-103.5', 'logits/chosen': '-3.513', 'logits/rejected': '-3.527', 'epoch': '0.9578'}
{'loss': '0.532', 'grad_norm': '0.6701', 'learning_rate': '0.0001397', 'rewards/chosen': '-1.009', 'rewards/rejected': '-1.716', 'rewards/accuracies': '0.6789', 'rewards/margins': '0.7072', 'logps/chosen': '-89.25', 'logps/rejected': '-101.4', 'logits/chosen': '-3.612', 'logits/rejected': '-3.547', 'epoch': '0.9687'}
{'loss': '0.5377', 'grad_norm': '0.6882', 'learning_rate': '0.000139', 'rewards/chosen': '-0.9364', 'rewards/rejected': '-1.605', 'rewards/accuracies': '0.6805', 'rewards/margins': '0.6687', 'logps/chosen': '-88.02', 'logps/rejected': '-99.57', 'logits/chosen': '-3.528', 'logits/rejected': '-3.411', 'epoch': '0.9796'}
{'loss': '0.5362', 'grad_norm': '0.6982', 'learning_rate': '0.0001382', 'rewards/chosen': '-0.8833', 'rewards/rejected': '-1.537', 'rewards/accuracies': '0.6523', 'rewards/margins': '0.654', 'logps/chosen': '-87.5', 'logps/rejected': '-98.83', 'logits/chosen': '-3.544', 'logits/rejected': '-3.5', 'epoch': '0.9905'}

 33%|███▎      | 920/2757 [2:04:52<3:57:58,  7.77s/it]
                                                      

 33%|███▎      | 920/2757 [2:04:52<3:57:58,  7.77s/it]
 33%|███▎      | 921/2757 [2:05:00<4:03:27,  7.96s/it]
 33%|███▎      | 922/2757 [2:05:09<4:10:14,  8.18s/it]
 33%|███▎      | 923/2757 [2:05:17<4:09:05,  8.15s/it]
 34%|███▎      | 924/2757 [2:05:25<4:09:27,  8.17s/it]
 34%|███▎      | 925/2757 [2:05:33<4:10:49,  8.21s/it]
 34%|███▎      | 926/2757 [2:05:41<4:07:28,  8.11s/it]
 34%|███▎      | 927/2757 [2:05:49<4:05:37,  8.05s/it]
 34%|███▎      | 928/2757 [2:05:57<4:03:45,  8.00s/it]
 34%|███▎      | 929/2757 [2:06:05<4:04:49,  8.04s/it]
 34%|███▎      | 930/2757 [2:06:13<4:04:38,  8.03s/it]
                                                      

 34%|███▎      | 930/2757 [2:06:13<4:04:38,  8.03s/it]
 34%|███▍      | 931/2757 [2:06:21<4:06:45,  8.11s/it]
 34%|███▍      | 932/2757 [2:06:30<4:06:35,  8.11s/it]
 34%|███▍      | 933/2757 [2:06:38<4:06:31,  8.11s/it]
 34%|███▍      | 934/2757 [2:06:46<4:05:03,  8.07s/it]
 34%|███▍      | 935/2757 [2:06:53<4:01:31,  7.95s/it]
 34%|███▍      | 936/2757 [2:07:01<3:58:12,  7.85s/it]
 34%|███▍      | 937/2757 [2:07:09<4:01:53,  7.97s/it]
 34%|███▍      | 938/2757 [2:07:18<4:08:38,  8.20s/it]
 34%|███▍      | 939/2757 [2:07:26<4:09:15,  8.23s/it]
 34%|███▍      | 940/2757 [2:07:34<4:06:23,  8.14s/it]
                                                      

 34%|███▍      | 940/2757 [2:07:34<4:06:23,  8.14s/it]
 34%|███▍      | 941/2757 [2:07:42<4:05:38,  8.12s/it]
 34%|███▍      | 942/2757 [2:07:50<4:01:14,  7.97s/it]
 34%|███▍      | 943/2757 [2:07:58<3:58:23,  7.89s/it]
 34%|███▍      | 944/2757 [2:08:06<4:02:31,  8.03s/it]
 34%|███▍      | 945/2757 [2:08:14<4:02:37,  8.03s/it]
 34%|███▍      | 946/2757 [2:08:22<4:04:50,  8.11s/it]
 34%|███▍      | 947/2757 [2:08:30<4:00:51,  7.98s/it]
 34%|███▍      | 948/2757 [2:08:38<4:02:03,  8.03s/it]
 34%|███▍      | 949/2757 [2:08:46<4:05:32,  8.15s/it]
 34%|███▍      | 950/2757 [2:08:55<4:07:08,  8.21s/it]
                                                      

 34%|███▍      | 950/2757 [2:08:55<4:07:08,  8.21s/it]
 34%|███▍      | 951/2757 [2:09:03<4:07:01,  8.21s/it]
 35%|███▍      | 952/2757 [2:09:11<4:06:29,  8.19s/it]
 35%|███▍      | 953/2757 [2:09:19<4:04:44,  8.14s/it]
 35%|███▍      | 954/2757 [2:09:27<4:00:41,  8.01s/it]
 35%|███▍      | 955/2757 [2:09:35<4:04:11,  8.13s/it]
 35%|███▍      | 956/2757 [2:09:44<4:07:09,  8.23s/it]
 35%|███▍      | 957/2757 [2:09:52<4:03:52,  8.13s/it]
 35%|███▍      | 958/2757 [2:10:00<4:08:07,  8.28s/it]
 35%|███▍      | 959/2757 [2:10:08<4:05:23,  8.19s/it]
 35%|███▍      | 960/2757 [2:10:16<4:02:57,  8.11s/it]
                                                      

 35%|███▍      | 960/2757 [2:10:16<4:02:57,  8.11s/it]
 35%|███▍      | 961/2757 [2:10:24<4:01:14,  8.06s/it]
 35%|███▍      | 962/2757 [2:10:32<4:01:22,  8.07s/it]
 35%|███▍      | 963/2757 [2:10:41<4:02:59,  8.13s/it]
 35%|███▍      | 964/2757 [2:10:48<3:58:51,  7.99s/it]
 35%|███▌      | 965/2757 [2:10:56<3:59:00,  8.00s/it]
 35%|███▌      | 966/2757 [2:11:05<4:05:08,  8.21s/it]
 35%|███▌      | 967/2757 [2:11:13<4:02:15,  8.12s/it]
 35%|███▌      | 968/2757 [2:11:21<4:05:00,  8.22s/it]
 35%|███▌      | 969/2757 [2:11:29<4:04:20,  8.20s/it]
 35%|███▌      | 970/2757 [2:11:37<4:00:06,  8.06s/it]
                                                      

 35%|███▌      | 970/2757 [2:11:37<4:00:06,  8.06s/it]
 35%|███▌      | 971/2757 [2:11:46<4:03:06,  8.17s/it]
 35%|███▌      | 972/2757 [2:11:54<4:00:59,  8.10s/it]
 35%|███▌      | 973/2757 [2:12:01<3:57:25,  7.99s/it]
 35%|███▌      | 974/2757 [2:12:09<3:58:25,  8.02s/it]
 35%|███▌      | 975/2757 [2:12:17<3:58:08,  8.02s/it]
 35%|███▌      | 976/2757 [2:12:25<3:57:40,  8.01s/it]
 35%|███▌      | 977/2757 [2:12:33<3:58:22,  8.04s/it]
 35%|███▌      | 978/2757 [2:12:41<3:55:08,  7.93s/it]
 36%|███▌      | 979/2757 [2:12:49<3:57:19,  8.01s/it]
 36%|███▌      | 980/2757 [2:12:57<3:55:22,  7.95s/it]
                                                      

 36%|███▌      | 980/2757 [2:12:57<3:55:22,  7.95s/it]
 36%|███▌      | 981/2757 [2:13:05<3:55:53,  7.97s/it]
 36%|███▌      | 982/2757 [2:13:13<3:55:58,  7.98s/it]
 36%|███▌      | 983/2757 [2:13:22<3:59:21,  8.10s/it]
 36%|███▌      | 984/2757 [2:13:30<3:59:10,  8.09s/it]
 36%|███▌      | 985/2757 [2:13:38<4:03:46,  8.25s/it]
 36%|███▌      | 986/2757 [2:13:47<4:03:57,  8.26s/it]
 36%|███▌      | 987/2757 [2:13:54<3:59:02,  8.10s/it]
 36%|███▌      | 988/2757 [2:14:02<3:59:24,  8.12s/it]
 36%|███▌      | 989/2757 [2:14:10<3:52:49,  7.90s/it]
 36%|███▌      | 990/2757 [2:14:18<3:52:25,  7.89s/it]
                                                      

 36%|███▌      | 990/2757 [2:14:18<3:52:25,  7.89s/it]
 36%|███▌      | 991/2757 [2:14:26<3:54:55,  7.98s/it]
 36%|███▌      | 992/2757 [2:14:34<3:58:39,  8.11s/it]
 36%|███▌      | 993/2757 [2:14:42<3:57:48,  8.09s/it]
 36%|███▌      | 994/2757 [2:14:50<3:57:20,  8.08s/it]
 36%|███▌      | 995/2757 [2:14:58<3:56:39,  8.06s/it]
 36%|███▌      | 996/2757 [2:15:07<3:58:26,  8.12s/it]
 36%|███▌      | 997/2757 [2:15:15<3:56:13,  8.05s/it]
 36%|███▌      | 998/2757 [2:15:23<3:57:57,  8.12s/it]
 36%|███▌      | 999/2757 [2:15:30<3:52:36,  7.94s/it]
 36%|███▋      | 1000/2757 [2:15:39<3:55:40,  8.05s/it]
                                                       

 36%|███▋      | 1000/2757 [2:15:39<3:55:40,  8.05s/it]{'loss': '0.5034', 'grad_norm': '0.4905', 'learning_rate': '0.0001375', 'rewards/chosen': '-0.9122', 'rewards/rejected': '-1.683', 'rewards/accuracies': '0.7075', 'rewards/margins': '0.7706', 'logps/chosen': '-86.75', 'logps/rejected': '-98.81', 'logits/chosen': '-3.483', 'logits/rejected': '-3.456', 'epoch': '1.001'}
{'loss': '0.4122', 'grad_norm': '0.5277', 'learning_rate': '0.0001367', 'rewards/chosen': '-1.053', 'rewards/rejected': '-2.115', 'rewards/accuracies': '0.7789', 'rewards/margins': '1.062', 'logps/chosen': '-92.64', 'logps/rejected': '-107.8', 'logits/chosen': '-3.593', 'logits/rejected': '-3.548', 'epoch': '1.012'}
{'loss': '0.4067', 'grad_norm': '0.5503', 'learning_rate': '0.000136', 'rewards/chosen': '-1.235', 'rewards/rejected': '-2.371', 'rewards/accuracies': '0.7844', 'rewards/margins': '1.136', 'logps/chosen': '-94.07', 'logps/rejected': '-112.1', 'logits/chosen': '-3.677', 'logits/rejected': '-3.588', 'epoch': '1.023'}
{'loss': '0.4037', 'grad_norm': '0.5421', 'learning_rate': '0.0001352', 'rewards/chosen': '-1.13', 'rewards/rejected': '-2.287', 'rewards/accuracies': '0.7844', 'rewards/margins': '1.158', 'logps/chosen': '-90.41', 'logps/rejected': '-106.1', 'logits/chosen': '-3.779', 'logits/rejected': '-3.727', 'epoch': '1.034'}
{'loss': '0.4048', 'grad_norm': '0.5324', 'learning_rate': '0.0001345', 'rewards/chosen': '-0.9678', 'rewards/rejected': '-2.156', 'rewards/accuracies': '0.7937', 'rewards/margins': '1.188', 'logps/chosen': '-92.28', 'logps/rejected': '-108.5', 'logits/chosen': '-3.955', 'logits/rejected': '-3.926', 'epoch': '1.045'}
{'loss': '0.3986', 'grad_norm': '0.4539', 'learning_rate': '0.0001337', 'rewards/chosen': '-0.9384', 'rewards/rejected': '-2.162', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.224', 'logps/chosen': '-90.46', 'logps/rejected': '-109.6', 'logits/chosen': '-3.724', 'logits/rejected': '-3.732', 'epoch': '1.056'}
{'loss': '0.4087', 'grad_norm': '0.5155', 'learning_rate': '0.000133', 'rewards/chosen': '-1.096', 'rewards/rejected': '-2.233', 'rewards/accuracies': '0.7719', 'rewards/margins': '1.137', 'logps/chosen': '-88.56', 'logps/rejected': '-105.9', 'logits/chosen': '-3.808', 'logits/rejected': '-3.751', 'epoch': '1.066'}
{'loss': '0.3877', 'grad_norm': '0.5626', 'learning_rate': '0.0001322', 'rewards/chosen': '-1.119', 'rewards/rejected': '-2.363', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.244', 'logps/chosen': '-93.86', 'logps/rejected': '-112.1', 'logits/chosen': '-3.833', 'logits/rejected': '-3.733', 'epoch': '1.077'}
{'loss': '0.3857', 'grad_norm': '0.4981', 'learning_rate': '0.0001315', 'rewards/chosen': '-1.254', 'rewards/rejected': '-2.53', 'rewards/accuracies': '0.7844', 'rewards/margins': '1.277', 'logps/chosen': '-90.55', 'logps/rejected': '-110.4', 'logits/chosen': '-3.934', 'logits/rejected': '-3.891', 'epoch': '1.088'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)


  0%|          | 0/300 [00:00<?, ?it/s]

  1%|          | 2/300 [00:00<00:51,  5.74it/s]

  1%|          | 3/300 [00:00<01:46,  2.80it/s]

  1%|▏         | 4/300 [00:01<01:34,  3.13it/s]

  2%|▏         | 5/300 [00:01<01:44,  2.83it/s]

  2%|▏         | 6/300 [00:01<01:40,  2.94it/s]

  2%|▏         | 7/300 [00:02<01:54,  2.56it/s]

  3%|▎         | 8/300 [00:02<01:50,  2.65it/s]

  3%|▎         | 9/300 [00:03<01:51,  2.61it/s]

  3%|▎         | 10/300 [00:03<01:48,  2.67it/s]

  4%|▎         | 11/300 [00:03<01:50,  2.61it/s]

  4%|▍         | 12/300 [00:04<01:37,  2.94it/s]

  4%|▍         | 13/300 [00:04<01:42,  2.80it/s]

  5%|▍         | 14/300 [00:04<01:35,  3.00it/s]

  5%|▌         | 15/300 [00:05<01:45,  2.70it/s]

  5%|▌         | 16/300 [00:05<01:33,  3.04it/s]

  6%|▌         | 17/300 [00:05<01:38,  2.89it/s]

  6%|▌         | 18/300 [00:06<01:28,  3.19it/s]

  6%|▋         | 19/300 [00:06<01:38,  2.85it/s]

  7%|▋         | 20/300 [00:06<01:31,  3.06it/s]

  7%|▋         | 21/300 [00:07<01:41,  2.76it/s]

  7%|▋         | 22/300 [00:07<01:30,  3.08it/s]

  8%|▊         | 23/300 [00:07<01:31,  3.04it/s]

  8%|▊         | 24/300 [00:08<01:22,  3.35it/s]

  8%|▊         | 25/300 [00:08<01:35,  2.89it/s]

  9%|▊         | 26/300 [00:08<01:28,  3.10it/s]

  9%|▉         | 27/300 [00:09<01:36,  2.82it/s]

  9%|▉         | 28/300 [00:09<01:26,  3.16it/s]

 10%|▉         | 29/300 [00:09<01:34,  2.85it/s]

 10%|█         | 30/300 [00:10<01:24,  3.18it/s]

 10%|█         | 31/300 [00:10<01:26,  3.12it/s]

 11%|█         | 32/300 [00:10<01:18,  3.40it/s]

 11%|█         | 33/300 [00:11<01:21,  3.29it/s]

 11%|█▏        | 34/300 [00:11<01:16,  3.47it/s]

 12%|█▏        | 35/300 [00:11<01:22,  3.22it/s]

 12%|█▏        | 36/300 [00:11<01:18,  3.38it/s]

 12%|█▏        | 37/300 [00:12<01:26,  3.03it/s]

 13%|█▎        | 38/300 [00:12<01:17,  3.37it/s]

 13%|█▎        | 39/300 [00:12<01:25,  3.05it/s]

 13%|█▎        | 40/300 [00:13<01:17,  3.34it/s]

 14%|█▎        | 41/300 [00:13<01:25,  3.03it/s]

 14%|█▍        | 42/300 [00:13<01:17,  3.32it/s]

 14%|█▍        | 43/300 [00:14<01:21,  3.15it/s]

 15%|█▍        | 44/300 [00:14<01:14,  3.42it/s]

 15%|█▌        | 45/300 [00:14<01:22,  3.11it/s]

 15%|█▌        | 46/300 [00:15<01:23,  3.05it/s]

 16%|█▌        | 47/300 [00:15<01:35,  2.66it/s]

 16%|█▌        | 48/300 [00:15<01:26,  2.92it/s]

 16%|█▋        | 49/300 [00:16<01:40,  2.49it/s]

 17%|█▋        | 50/300 [00:16<01:26,  2.88it/s]

 17%|█▋        | 51/300 [00:17<01:26,  2.88it/s]

 17%|█▋        | 52/300 [00:17<01:21,  3.05it/s]

 18%|█▊        | 53/300 [00:17<01:32,  2.68it/s]

 18%|█▊        | 54/300 [00:18<01:22,  2.99it/s]

 18%|█▊        | 55/300 [00:18<01:29,  2.74it/s]

 19%|█▊        | 56/300 [00:18<01:17,  3.15it/s]

 19%|█▉        | 57/300 [00:19<01:21,  2.98it/s]

 19%|█▉        | 58/300 [00:19<01:22,  2.94it/s]

 20%|█▉        | 59/300 [00:19<01:26,  2.78it/s]

 20%|██        | 60/300 [00:20<01:18,  3.07it/s]

 20%|██        | 61/300 [00:20<01:21,  2.95it/s]

 21%|██        | 62/300 [00:20<01:20,  2.95it/s]

 21%|██        | 63/300 [00:21<01:25,  2.76it/s]

 21%|██▏       | 64/300 [00:21<01:23,  2.81it/s]

 22%|██▏       | 65/300 [00:21<01:27,  2.70it/s]

 22%|██▏       | 66/300 [00:22<01:19,  2.95it/s]

 22%|██▏       | 67/300 [00:22<01:26,  2.69it/s]

 23%|██▎       | 68/300 [00:22<01:17,  2.97it/s]

 23%|██▎       | 69/300 [00:23<01:21,  2.85it/s]

 23%|██▎       | 70/300 [00:23<01:12,  3.15it/s]

 24%|██▎       | 71/300 [00:23<01:15,  3.05it/s]

 24%|██▍       | 72/300 [00:24<01:09,  3.30it/s]

 24%|██▍       | 73/300 [00:24<01:12,  3.14it/s]

 25%|██▍       | 74/300 [00:24<01:08,  3.32it/s]

 25%|██▌       | 75/300 [00:25<01:12,  3.09it/s]

 25%|██▌       | 76/300 [00:25<01:14,  3.02it/s]

 26%|██▌       | 77/300 [00:25<01:20,  2.79it/s]

 26%|██▌       | 78/300 [00:26<01:13,  3.03it/s]

 26%|██▋       | 79/300 [00:26<01:15,  2.93it/s]

 27%|██▋       | 80/300 [00:26<01:11,  3.09it/s]

 27%|██▋       | 81/300 [00:27<01:19,  2.77it/s]

 27%|██▋       | 82/300 [00:27<01:10,  3.07it/s]

 28%|██▊       | 83/300 [00:27<01:13,  2.97it/s]

 28%|██▊       | 84/300 [00:28<01:13,  2.94it/s]

 28%|██▊       | 85/300 [00:28<01:18,  2.75it/s]

 29%|██▊       | 86/300 [00:28<01:10,  3.05it/s]

 29%|██▉       | 87/300 [00:29<01:12,  2.94it/s]

 29%|██▉       | 88/300 [00:29<01:06,  3.19it/s]

 30%|██▉       | 89/300 [00:29<01:10,  2.98it/s]

 30%|███       | 90/300 [00:30<01:05,  3.21it/s]

 30%|███       | 91/300 [00:30<01:09,  3.01it/s]

 31%|███       | 92/300 [00:30<01:03,  3.28it/s]

 31%|███       | 93/300 [00:31<01:09,  3.00it/s]

 31%|███▏      | 94/300 [00:31<01:02,  3.29it/s]

 32%|███▏      | 95/300 [00:31<01:07,  3.02it/s]

 32%|███▏      | 96/300 [00:32<01:03,  3.21it/s]

 32%|███▏      | 97/300 [00:32<01:05,  3.09it/s]

 33%|███▎      | 98/300 [00:32<01:01,  3.26it/s]

 33%|███▎      | 99/300 [00:33<01:06,  3.00it/s]

 33%|███▎      | 100/300 [00:33<01:03,  3.16it/s]

 34%|███▎      | 101/300 [00:33<01:07,  2.97it/s]

 34%|███▍      | 102/300 [00:34<01:05,  3.02it/s]

 34%|███▍      | 103/300 [00:34<01:09,  2.84it/s]

 35%|███▍      | 104/300 [00:34<01:02,  3.16it/s]

 35%|███▌      | 105/300 [00:35<01:05,  2.99it/s]

 35%|███▌      | 106/300 [00:35<01:00,  3.20it/s]

 36%|███▌      | 107/300 [00:35<01:04,  2.98it/s]

 36%|███▌      | 108/300 [00:35<01:02,  3.07it/s]

 36%|███▋      | 109/300 [00:36<01:06,  2.89it/s]

 37%|███▋      | 110/300 [00:36<01:02,  3.04it/s]

 37%|███▋      | 111/300 [00:37<01:11,  2.66it/s]

 37%|███▋      | 112/300 [00:37<01:05,  2.85it/s]

 38%|███▊      | 113/300 [00:37<01:07,  2.75it/s]

 38%|███▊      | 114/300 [00:38<01:06,  2.78it/s]

 38%|███▊      | 115/300 [00:38<01:09,  2.65it/s]

 39%|███▊      | 116/300 [00:38<01:01,  2.97it/s]

 39%|███▉      | 117/300 [00:39<01:03,  2.90it/s]

 39%|███▉      | 118/300 [00:39<00:58,  3.12it/s]

 40%|███▉      | 119/300 [00:39<01:00,  2.97it/s]

 40%|████      | 120/300 [00:40<01:05,  2.76it/s]

 40%|████      | 121/300 [00:40<01:05,  2.72it/s]

 41%|████      | 122/300 [00:40<00:59,  2.97it/s]

 41%|████      | 123/300 [00:41<01:03,  2.77it/s]

 41%|████▏     | 124/300 [00:41<00:59,  2.96it/s]

 42%|████▏     | 125/300 [00:42<01:00,  2.87it/s]

 42%|████▏     | 126/300 [00:42<01:00,  2.87it/s]

 42%|████▏     | 127/300 [00:42<01:03,  2.73it/s]

 43%|████▎     | 128/300 [00:43<00:58,  2.94it/s]

 43%|████▎     | 129/300 [00:43<01:00,  2.83it/s]

 43%|████▎     | 130/300 [00:43<00:54,  3.12it/s]

 44%|████▎     | 131/300 [00:44<01:01,  2.76it/s]

 44%|████▍     | 132/300 [00:44<00:55,  3.01it/s]

 44%|████▍     | 133/300 [00:44<01:05,  2.57it/s]

 45%|████▍     | 134/300 [00:45<00:58,  2.84it/s]

 45%|████▌     | 135/300 [00:45<01:04,  2.57it/s]

 45%|████▌     | 136/300 [00:45<00:56,  2.89it/s]

 46%|████▌     | 137/300 [00:46<00:58,  2.79it/s]

 46%|████▌     | 138/300 [00:46<00:53,  3.05it/s]

 46%|████▋     | 139/300 [00:46<00:53,  3.03it/s]

 47%|████▋     | 140/300 [00:47<00:48,  3.32it/s]

 47%|████▋     | 141/300 [00:47<00:49,  3.18it/s]

 47%|████▋     | 142/300 [00:47<00:51,  3.08it/s]

 48%|████▊     | 143/300 [00:48<00:55,  2.85it/s]

 48%|████▊     | 144/300 [00:48<00:49,  3.16it/s]

 48%|████▊     | 145/300 [00:48<00:49,  3.13it/s]

 49%|████▊     | 146/300 [00:49<00:46,  3.29it/s]

 49%|████▉     | 147/300 [00:49<00:53,  2.86it/s]

 49%|████▉     | 148/300 [00:49<00:50,  3.02it/s]

 50%|████▉     | 149/300 [00:50<00:53,  2.81it/s]

 50%|█████     | 150/300 [00:50<00:49,  3.01it/s]

 50%|█████     | 151/300 [00:50<00:51,  2.89it/s]

 51%|█████     | 152/300 [00:51<00:46,  3.20it/s]

 51%|█████     | 153/300 [00:51<00:47,  3.09it/s]

 51%|█████▏    | 154/300 [00:51<00:43,  3.37it/s]

 52%|█████▏    | 155/300 [00:52<00:45,  3.17it/s]

 52%|█████▏    | 156/300 [00:52<00:41,  3.49it/s]

 52%|█████▏    | 157/300 [00:52<00:45,  3.13it/s]

 53%|█████▎    | 158/300 [00:52<00:43,  3.27it/s]

 53%|█████▎    | 159/300 [00:53<00:52,  2.67it/s]

 53%|█████▎    | 160/300 [00:53<00:48,  2.90it/s]

 54%|█████▎    | 161/300 [00:54<00:51,  2.70it/s]

 54%|█████▍    | 162/300 [00:54<00:50,  2.74it/s]

 54%|█████▍    | 163/300 [00:55<00:56,  2.43it/s]

 55%|█████▍    | 164/300 [00:55<00:53,  2.55it/s]

 55%|█████▌    | 165/300 [00:55<00:53,  2.54it/s]

 55%|█████▌    | 166/300 [00:56<00:50,  2.65it/s]

 56%|█████▌    | 167/300 [00:56<00:54,  2.43it/s]

 56%|█████▌    | 168/300 [00:56<00:47,  2.75it/s]

 56%|█████▋    | 169/300 [00:57<00:52,  2.48it/s]

 57%|█████▋    | 170/300 [00:57<00:50,  2.59it/s]

 57%|█████▋    | 171/300 [00:58<00:50,  2.54it/s]

 57%|█████▋    | 172/300 [00:58<00:44,  2.88it/s]

 58%|█████▊    | 173/300 [00:58<00:48,  2.63it/s]

 58%|█████▊    | 174/300 [00:59<00:42,  2.97it/s]

 58%|█████▊    | 175/300 [00:59<00:45,  2.73it/s]

 59%|█████▊    | 176/300 [00:59<00:40,  3.07it/s]

 59%|█████▉    | 177/300 [01:00<00:46,  2.65it/s]

 59%|█████▉    | 178/300 [01:00<00:39,  3.08it/s]

 60%|█████▉    | 179/300 [01:00<00:42,  2.88it/s]

 60%|██████    | 180/300 [01:01<00:38,  3.16it/s]

 60%|██████    | 181/300 [01:01<00:41,  2.85it/s]

 61%|██████    | 182/300 [01:01<00:37,  3.16it/s]

 61%|██████    | 183/300 [01:02<00:40,  2.90it/s]

 61%|██████▏   | 184/300 [01:02<00:40,  2.89it/s]

 62%|██████▏   | 185/300 [01:02<00:42,  2.70it/s]

 62%|██████▏   | 186/300 [01:03<00:39,  2.89it/s]

 62%|██████▏   | 187/300 [01:03<00:39,  2.87it/s]

 63%|██████▎   | 188/300 [01:03<00:35,  3.15it/s]

 63%|██████▎   | 189/300 [01:04<00:38,  2.86it/s]

 63%|██████▎   | 190/300 [01:04<00:35,  3.09it/s]

 64%|██████▎   | 191/300 [01:04<00:37,  2.94it/s]

 64%|██████▍   | 192/300 [01:05<00:36,  2.97it/s]

 64%|██████▍   | 193/300 [01:05<00:41,  2.56it/s]

 65%|██████▍   | 194/300 [01:06<00:40,  2.64it/s]

 65%|██████▌   | 195/300 [01:06<00:41,  2.54it/s]

 65%|██████▌   | 196/300 [01:06<00:37,  2.75it/s]

 66%|██████▌   | 197/300 [01:07<00:37,  2.72it/s]

 66%|██████▌   | 198/300 [01:07<00:34,  2.94it/s]

 66%|██████▋   | 199/300 [01:07<00:35,  2.81it/s]

 67%|██████▋   | 200/300 [01:08<00:33,  2.99it/s]

 67%|██████▋   | 201/300 [01:08<00:35,  2.79it/s]

 67%|██████▋   | 202/300 [01:08<00:31,  3.08it/s]

 68%|██████▊   | 203/300 [01:09<00:33,  2.93it/s]

 68%|██████▊   | 204/300 [01:09<00:29,  3.22it/s]

 68%|██████▊   | 205/300 [01:09<00:31,  3.01it/s]

 69%|██████▊   | 206/300 [01:09<00:27,  3.40it/s]

 69%|██████▉   | 207/300 [01:10<00:31,  2.92it/s]

 69%|██████▉   | 208/300 [01:10<00:31,  2.93it/s]

 70%|██████▉   | 209/300 [01:11<00:32,  2.79it/s]

 70%|███████   | 210/300 [01:11<00:29,  3.00it/s]

 70%|███████   | 211/300 [01:11<00:33,  2.64it/s]

 71%|███████   | 212/300 [01:12<00:29,  2.96it/s]

 71%|███████   | 213/300 [01:12<00:32,  2.69it/s]

 71%|███████▏  | 214/300 [01:12<00:29,  2.94it/s]

 72%|███████▏  | 215/300 [01:13<00:30,  2.76it/s]

 72%|███████▏  | 216/300 [01:13<00:27,  3.06it/s]

 72%|███████▏  | 217/300 [01:14<00:31,  2.64it/s]

 73%|███████▎  | 218/300 [01:14<00:26,  3.04it/s]

 73%|███████▎  | 219/300 [01:14<00:28,  2.80it/s]

 73%|███████▎  | 220/300 [01:14<00:25,  3.09it/s]

 74%|███████▎  | 221/300 [01:15<00:28,  2.78it/s]

 74%|███████▍  | 222/300 [01:15<00:25,  3.09it/s]

 74%|███████▍  | 223/300 [01:16<00:28,  2.74it/s]

 75%|███████▍  | 224/300 [01:16<00:25,  2.94it/s]

 75%|███████▌  | 225/300 [01:16<00:26,  2.80it/s]

 75%|███████▌  | 226/300 [01:17<00:24,  3.03it/s]

 76%|███████▌  | 227/300 [01:17<00:26,  2.79it/s]

 76%|███████▌  | 228/300 [01:17<00:23,  3.12it/s]

 76%|███████▋  | 229/300 [01:18<00:25,  2.75it/s]

 77%|███████▋  | 230/300 [01:18<00:24,  2.85it/s]

 77%|███████▋  | 231/300 [01:18<00:25,  2.71it/s]

 77%|███████▋  | 232/300 [01:19<00:24,  2.78it/s]

 78%|███████▊  | 233/300 [01:19<00:27,  2.47it/s]

 78%|███████▊  | 234/300 [01:19<00:23,  2.82it/s]

 78%|███████▊  | 235/300 [01:20<00:24,  2.64it/s]

 79%|███████▊  | 236/300 [01:20<00:23,  2.71it/s]

 79%|███████▉  | 237/300 [01:21<00:24,  2.61it/s]

 79%|███████▉  | 238/300 [01:21<00:21,  2.93it/s]

 80%|███████▉  | 239/300 [01:21<00:21,  2.90it/s]

 80%|████████  | 240/300 [01:21<00:18,  3.21it/s]

 80%|████████  | 241/300 [01:22<00:19,  3.07it/s]

 81%|████████  | 242/300 [01:22<00:18,  3.10it/s]

 81%|████████  | 243/300 [01:23<00:20,  2.74it/s]

 81%|████████▏ | 244/300 [01:23<00:19,  2.84it/s]

 82%|████████▏ | 245/300 [01:23<00:20,  2.69it/s]

 82%|████████▏ | 246/300 [01:24<00:19,  2.78it/s]

 82%|████████▏ | 247/300 [01:24<00:20,  2.60it/s]

 83%|████████▎ | 248/300 [01:24<00:17,  2.94it/s]

 83%|████████▎ | 249/300 [01:25<00:18,  2.80it/s]

 83%|████████▎ | 250/300 [01:25<00:17,  2.90it/s]

 84%|████████▎ | 251/300 [01:25<00:17,  2.80it/s]

 84%|████████▍ | 252/300 [01:26<00:16,  2.99it/s]

 84%|████████▍ | 253/300 [01:26<00:16,  2.84it/s]

 85%|████████▍ | 254/300 [01:26<00:14,  3.21it/s]

 85%|████████▌ | 255/300 [01:27<00:15,  2.85it/s]

 85%|████████▌ | 256/300 [01:27<00:13,  3.19it/s]

 86%|████████▌ | 257/300 [01:27<00:14,  2.98it/s]

 86%|████████▌ | 258/300 [01:28<00:12,  3.24it/s]

 86%|████████▋ | 259/300 [01:28<00:13,  3.00it/s]

 87%|████████▋ | 260/300 [01:28<00:12,  3.18it/s]

 87%|████████▋ | 261/300 [01:29<00:12,  3.01it/s]

 87%|████████▋ | 262/300 [01:29<00:11,  3.30it/s]

 88%|████████▊ | 263/300 [01:29<00:12,  3.01it/s]

 88%|████████▊ | 264/300 [01:30<00:12,  3.00it/s]

 88%|████████▊ | 265/300 [01:30<00:13,  2.59it/s]

 89%|████████▊ | 266/300 [01:31<00:12,  2.68it/s]

 89%|████████▉ | 267/300 [01:31<00:13,  2.42it/s]

 89%|████████▉ | 268/300 [01:31<00:11,  2.77it/s]

 90%|████████▉ | 269/300 [01:32<00:12,  2.45it/s]

 90%|█████████ | 270/300 [01:32<00:11,  2.63it/s]

 90%|█████████ | 271/300 [01:33<00:12,  2.38it/s]

 91%|█████████ | 272/300 [01:33<00:10,  2.64it/s]

 91%|█████████ | 273/300 [01:33<00:11,  2.38it/s]

 91%|█████████▏| 274/300 [01:34<00:09,  2.72it/s]

 92%|█████████▏| 275/300 [01:34<00:10,  2.43it/s]

 92%|█████████▏| 276/300 [01:34<00:09,  2.61it/s]

 92%|█████████▏| 277/300 [01:35<00:09,  2.53it/s]

 93%|█████████▎| 278/300 [01:35<00:07,  2.85it/s]

 93%|█████████▎| 279/300 [01:36<00:07,  2.82it/s]

 93%|█████████▎| 280/300 [01:36<00:06,  2.91it/s]

 94%|█████████▎| 281/300 [01:36<00:06,  2.73it/s]

 94%|█████████▍| 282/300 [01:37<00:05,  3.03it/s]

 94%|█████████▍| 283/300 [01:37<00:06,  2.81it/s]

 95%|█████████▍| 284/300 [01:37<00:05,  3.08it/s]

 95%|█████████▌| 285/300 [01:38<00:05,  2.67it/s]

 95%|█████████▌| 286/300 [01:38<00:05,  2.73it/s]

 96%|█████████▌| 287/300 [01:38<00:04,  2.61it/s]

 96%|█████████▌| 288/300 [01:39<00:03,  3.03it/s]

 96%|█████████▋| 289/300 [01:39<00:03,  2.79it/s]

 97%|█████████▋| 290/300 [01:39<00:03,  2.81it/s]

 97%|█████████▋| 291/300 [01:40<00:03,  2.61it/s]

 97%|█████████▋| 292/300 [01:40<00:02,  2.93it/s]

 98%|█████████▊| 293/300 [01:41<00:02,  2.53it/s]

 98%|█████████▊| 294/300 [01:41<00:02,  2.80it/s]

 98%|█████████▊| 295/300 [01:41<00:01,  2.79it/s]

 99%|█████████▊| 296/300 [01:42<00:01,  3.02it/s]

 99%|█████████▉| 297/300 [01:42<00:01,  2.83it/s]

 99%|█████████▉| 298/300 [01:42<00:00,  3.20it/s]

100%|█████████▉| 299/300 [01:43<00:00,  2.84it/s]

100%|██████████| 300/300 [01:43<00:00,  2.79it/s]
                                                       

                                                 

 36%|███▋      | 1000/2757 [2:17:23<3:55:40,  8.05s/it]

100%|██████████| 300/300 [01:43<00:00,  2.79it/s]

                                                 Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-1000/tokenizer_config.json.

 36%|███▋      | 1001/2757 [2:17:37<20:00:37, 41.02s/it]
 36%|███▋      | 1002/2757 [2:17:44<15:08:38, 31.06s/it]
 36%|███▋      | 1003/2757 [2:17:52<11:41:21, 23.99s/it]
 36%|███▋      | 1004/2757 [2:18:00<9:22:18, 19.25s/it] 
 36%|███▋      | 1005/2757 [2:18:08<7:42:17, 15.83s/it]
 36%|███▋      | 1006/2757 [2:18:16<6:34:13, 13.51s/it]
 37%|███▋      | 1007/2757 [2:18:24<5:44:52, 11.82s/it]
 37%|███▋      | 1008/2757 [2:18:32<5:11:16, 10.68s/it]
 37%|███▋      | 1009/2757 [2:18:40<4:49:29,  9.94s/it]
 37%|███▋      | 1010/2757 [2:18:48<4:33:40,  9.40s/it]
                                                       

 37%|███▋      | 1010/2757 [2:18:48<4:33:40,  9.40s/it]
 37%|███▋      | 1011/2757 [2:18:57<4:23:38,  9.06s/it]
 37%|███▋      | 1012/2757 [2:19:04<4:11:48,  8.66s/it]
 37%|███▋      | 1013/2757 [2:19:12<4:04:05,  8.40s/it]
 37%|███▋      | 1014/2757 [2:19:20<4:01:30,  8.31s/it]
 37%|███▋      | 1015/2757 [2:19:29<4:02:12,  8.34s/it]
 37%|███▋      | 1016/2757 [2:19:37<4:04:17,  8.42s/it]
 37%|███▋      | 1017/2757 [2:19:45<3:58:22,  8.22s/it]
 37%|███▋      | 1018/2757 [2:19:53<3:58:32,  8.23s/it]
 37%|███▋      | 1019/2757 [2:20:01<3:56:37,  8.17s/it]
 37%|███▋      | 1020/2757 [2:20:09<3:55:06,  8.12s/it]
                                                       

 37%|███▋      | 1020/2757 [2:20:09<3:55:06,  8.12s/it]
 37%|███▋      | 1021/2757 [2:20:18<3:57:07,  8.20s/it]
 37%|███▋      | 1022/2757 [2:20:25<3:53:43,  8.08s/it]
 37%|███▋      | 1023/2757 [2:20:33<3:52:17,  8.04s/it]
 37%|███▋      | 1024/2757 [2:20:41<3:51:47,  8.02s/it]
 37%|███▋      | 1025/2757 [2:20:50<3:56:14,  8.18s/it]
 37%|███▋      | 1026/2757 [2:20:58<3:57:40,  8.24s/it]
 37%|███▋      | 1027/2757 [2:21:07<4:00:06,  8.33s/it]
 37%|███▋      | 1028/2757 [2:21:15<3:55:40,  8.18s/it]
 37%|███▋      | 1029/2757 [2:21:22<3:50:48,  8.01s/it]
 37%|███▋      | 1030/2757 [2:21:30<3:51:07,  8.03s/it]
                                                       

 37%|███▋      | 1030/2757 [2:21:30<3:51:07,  8.03s/it]
 37%|███▋      | 1031/2757 [2:21:38<3:49:36,  7.98s/it]
 37%|███▋      | 1032/2757 [2:21:46<3:48:41,  7.95s/it]
 37%|███▋      | 1033/2757 [2:21:54<3:49:13,  7.98s/it]
 38%|███▊      | 1034/2757 [2:22:02<3:47:37,  7.93s/it]
 38%|███▊      | 1035/2757 [2:22:10<3:50:18,  8.02s/it]
 38%|███▊      | 1036/2757 [2:22:18<3:52:05,  8.09s/it]
 38%|███▊      | 1037/2757 [2:22:26<3:47:29,  7.94s/it]
 38%|███▊      | 1038/2757 [2:22:34<3:44:24,  7.83s/it]
 38%|███▊      | 1039/2757 [2:22:41<3:44:49,  7.85s/it]
 38%|███▊      | 1040/2757 [2:22:50<3:47:11,  7.94s/it]
                                                       

 38%|███▊      | 1040/2757 [2:22:50<3:47:11,  7.94s/it]
 38%|███▊      | 1041/2757 [2:22:58<3:49:38,  8.03s/it]
 38%|███▊      | 1042/2757 [2:23:06<3:46:26,  7.92s/it]
 38%|███▊      | 1043/2757 [2:23:14<3:47:21,  7.96s/it]
 38%|███▊      | 1044/2757 [2:23:21<3:45:36,  7.90s/it]
 38%|███▊      | 1045/2757 [2:23:29<3:45:19,  7.90s/it]
 38%|███▊      | 1046/2757 [2:23:37<3:47:21,  7.97s/it]
 38%|███▊      | 1047/2757 [2:23:45<3:45:12,  7.90s/it]
 38%|███▊      | 1048/2757 [2:23:53<3:44:51,  7.89s/it]
 38%|███▊      | 1049/2757 [2:24:01<3:48:04,  8.01s/it]
 38%|███▊      | 1050/2757 [2:24:09<3:46:46,  7.97s/it]
                                                       

 38%|███▊      | 1050/2757 [2:24:09<3:46:46,  7.97s/it]
 38%|███▊      | 1051/2757 [2:24:17<3:43:57,  7.88s/it]
 38%|███▊      | 1052/2757 [2:24:25<3:46:32,  7.97s/it]
 38%|███▊      | 1053/2757 [2:24:33<3:46:54,  7.99s/it]
 38%|███▊      | 1054/2757 [2:24:41<3:46:38,  7.99s/it]
 38%|███▊      | 1055/2757 [2:24:49<3:44:33,  7.92s/it]
 38%|███▊      | 1056/2757 [2:24:57<3:47:43,  8.03s/it]
 38%|███▊      | 1057/2757 [2:25:06<3:52:17,  8.20s/it]
 38%|███▊      | 1058/2757 [2:25:13<3:46:57,  8.02s/it]
 38%|███▊      | 1059/2757 [2:25:21<3:47:24,  8.04s/it]
 38%|███▊      | 1060/2757 [2:25:29<3:44:58,  7.95s/it]
                                                       

 38%|███▊      | 1060/2757 [2:25:29<3:44:58,  7.95s/it]
 38%|███▊      | 1061/2757 [2:25:37<3:45:14,  7.97s/it]
 39%|███▊      | 1062/2757 [2:25:45<3:47:24,  8.05s/it]
 39%|███▊      | 1063/2757 [2:25:53<3:43:27,  7.91s/it]
 39%|███▊      | 1064/2757 [2:26:01<3:43:26,  7.92s/it]
 39%|███▊      | 1065/2757 [2:26:09<3:42:52,  7.90s/it]
 39%|███▊      | 1066/2757 [2:26:16<3:39:21,  7.78s/it]
 39%|███▊      | 1067/2757 [2:26:24<3:42:07,  7.89s/it]
 39%|███▊      | 1068/2757 [2:26:32<3:41:29,  7.87s/it]
 39%|███▉      | 1069/2757 [2:26:39<3:35:54,  7.67s/it]
 39%|███▉      | 1070/2757 [2:26:47<3:36:09,  7.69s/it]
                                                       

 39%|███▉      | 1070/2757 [2:26:47<3:36:09,  7.69s/it]
 39%|███▉      | 1071/2757 [2:26:56<3:44:08,  7.98s/it]
 39%|███▉      | 1072/2757 [2:27:04<3:44:50,  8.01s/it]
 39%|███▉      | 1073/2757 [2:27:12<3:48:06,  8.13s/it]
 39%|███▉      | 1074/2757 [2:27:20<3:45:16,  8.03s/it]
 39%|███▉      | 1075/2757 [2:27:28<3:42:10,  7.93s/it]
 39%|███▉      | 1076/2757 [2:27:36<3:41:03,  7.89s/it]
 39%|███▉      | 1077/2757 [2:27:43<3:40:19,  7.87s/it]
 39%|███▉      | 1078/2757 [2:27:51<3:37:01,  7.76s/it]
 39%|███▉      | 1079/2757 [2:27:59<3:39:38,  7.85s/it]
 39%|███▉      | 1080/2757 [2:28:07<3:39:26,  7.85s/it]
                                                       

 39%|███▉      | 1080/2757 [2:28:07<3:39:26,  7.85s/it]
 39%|███▉      | 1081/2757 [2:28:15<3:41:14,  7.92s/it]
 39%|███▉      | 1082/2757 [2:28:23<3:43:29,  8.01s/it]
 39%|███▉      | 1083/2757 [2:28:31<3:44:33,  8.05s/it]
 39%|███▉      | 1084/2757 [2:28:39<3:44:05,  8.04s/it]
 39%|███▉      | 1085/2757 [2:28:47<3:43:14,  8.01s/it]
 39%|███▉      | 1086/2757 [2:28:55<3:41:24,  7.95s/it]
 39%|███▉      | 1087/2757 [2:29:03<3:44:07,  8.05s/it]
 39%|███▉      | 1088/2757 [2:29:12<3:49:19,  8.24s/it]
 39%|███▉      | 1089/2757 [2:29:20<3:45:48,  8.12s/it]
 40%|███▉      | 1090/2757 [2:29:28<3:46:57,  8.17s/it]
                                                       

 40%|███▉      | 1090/2757 [2:29:28<3:46:57,  8.17s/it]
 40%|███▉      | 1091/2757 [2:29:36<3:45:01,  8.10s/it]
 40%|███▉      | 1092/2757 [2:29:44<3:42:01,  8.00s/it]
 40%|███▉      | 1093/2757 [2:29:52<3:40:48,  7.96s/it]
 40%|███▉      | 1094/2757 [2:30:00<3:42:19,  8.02s/it]
 40%|███▉      | 1095/2757 [2:30:08<3:41:06,  7.98s/it]
 40%|███▉      | 1096/2757 [2:30:16<3:39:58,  7.95s/it]
 40%|███▉      | 1097/2757 [2:30:24<3:40:51,  7.98s/it]
 40%|███▉      | 1098/2757 [2:30:32<3:40:32,  7.98s/it]
 40%|███▉      | 1099/2757 [2:30:40<3:43:54,  8.10s/it]
 40%|███▉      | 1100/2757 [2:30:48<3:46:13,  8.19s/it]
                                                       

 40%|███▉      | 1100/2757 [2:30:48<3:46:13,  8.19s/it]
 40%|███▉      | 1101/2757 [2:30:56<3:41:36,  8.03s/it]
 40%|███▉      | 1102/2757 [2:31:04<3:41:38,  8.04s/it]
 40%|████      | 1103/2757 [2:31:12<3:41:56,  8.05s/it]
 40%|████      | 1104/2757 [2:31:20<3:40:57,  8.02s/it]
 40%|████      | 1105/2757 [2:31:28<3:40:23,  8.00s/it]
 40%|████      | 1106/2757 [2:31:36<3:40:05,  8.00s/it]
 40%|████      | 1107/2757 [2:31:44<3:41:23,  8.05s/it]
 40%|████      | 1108/2757 [2:31:52<3:40:40,  8.03s/it]
 40%|████      | 1109/2757 [2:32:00<3:38:25,  7.95s/it]
 40%|████      | 1110/2757 [2:32:09<3:44:33,  8.18s/it]
                                                       

 40%|████      | 1110/2757 [2:32:09<3:44:33,  8.18s/it]
 40%|████      | 1111/2757 [2:32:16<3:40:05,  8.02s/it]
 40%|████      | 1112/2757 [2:32:24<3:36:24,  7.89s/it]
 40%|████      | 1113/2757 [2:32:32<3:35:27,  7.86s/it]
 40%|████      | 1114/2757 [2:32:40<3:34:33,  7.84s/it]
 40%|████      | 1115/2757 [2:32:47<3:34:06,  7.82s/it]
 40%|████      | 1116/2757 [2:32:56<3:37:40,  7.96s/it]
 41%|████      | 1117/2757 [2:33:04<3:37:24,  7.95s/it]
 41%|████      | 1118/2757 [2:33:12<3:37:08,  7.95s/it]
 41%|████      | 1119/2757 [2:33:19<3:34:09,  7.84s/it]
 41%|████      | 1120/2757 [2:33:28<3:38:55,  8.02s/it]
                                                       

 41%|████      | 1120/2757 [2:33:28<3:38:55,  8.02s/it]
 41%|████      | 1121/2757 [2:33:35<3:37:24,  7.97s/it]
 41%|████      | 1122/2757 [2:33:44<3:42:50,  8.18s/it]
 41%|████      | 1123/2757 [2:33:52<3:42:00,  8.15s/it]
 41%|████      | 1124/2757 [2:34:00<3:39:24,  8.06s/it]
 41%|████      | 1125/2757 [2:34:08<3:36:24,  7.96s/it]
 41%|████      | 1126/2757 [2:34:15<3:32:23,  7.81s/it]
 41%|████      | 1127/2757 [2:34:23<3:32:42,  7.83s/it]
 41%|████      | 1128/2757 [2:34:31<3:33:16,  7.86s/it]
 41%|████      | 1129/2757 [2:34:39<3:31:42,  7.80s/it]
 41%|████      | 1130/2757 [2:34:47<3:34:22,  7.91s/it]
                                                       

 41%|████      | 1130/2757 [2:34:47<3:34:22,  7.91s/it]
 41%|████      | 1131/2757 [2:34:55<3:36:04,  7.97s/it]
 41%|████      | 1132/2757 [2:35:03<3:38:48,  8.08s/it]
 41%|████      | 1133/2757 [2:35:11<3:38:52,  8.09s/it]
 41%|████      | 1134/2757 [2:35:19<3:35:26,  7.96s/it]
 41%|████      | 1135/2757 [2:35:27<3:33:57,  7.91s/it]
 41%|████      | 1136/2757 [2:35:35<3:34:05,  7.92s/it]
 41%|████      | 1137/2757 [2:35:43<3:35:11,  7.97s/it]
 41%|████▏     | 1138/2757 [2:35:51<3:36:04,  8.01s/it]
 41%|████▏     | 1139/2757 [2:35:59<3:36:32,  8.03s/it]
 41%|████▏     | 1140/2757 [2:36:07<3:37:38,  8.08s/it]
                                                       

 41%|████▏     | 1140/2757 [2:36:07<3:37:38,  8.08s/it]
 41%|████▏     | 1141/2757 [2:36:15<3:38:28,  8.11s/it]
 41%|████▏     | 1142/2757 [2:36:23<3:36:06,  8.03s/it]
 41%|████▏     | 1143/2757 [2:36:31<3:36:47,  8.06s/it]
 41%|████▏     | 1144/2757 [2:36:40<3:37:03,  8.07s/it]
 42%|████▏     | 1145/2757 [2:36:47<3:34:09,  7.97s/it]
 42%|████▏     | 1146/2757 [2:36:55<3:34:12,  7.98s/it]
 42%|████▏     | 1147/2757 [2:37:03<3:31:59,  7.90s/it]
 42%|████▏     | 1148/2757 [2:37:11<3:30:10,  7.84s/it]
 42%|████▏     | 1149/2757 [2:37:19<3:32:15,  7.92s/it]
 42%|████▏     | 1150/2757 [2:37:27<3:34:28,  8.01s/it]
                                                       

 42%|████▏     | 1150/2757 [2:37:27<3:34:28,  8.01s/it]
 42%|████▏     | 1151/2757 [2:37:35<3:32:27,  7.94s/it]
 42%|████▏     | 1152/2757 [2:37:42<3:30:41,  7.88s/it]
 42%|████▏     | 1153/2757 [2:37:51<3:31:55,  7.93s/it]
 42%|████▏     | 1154/2757 [2:37:58<3:30:24,  7.88s/it]
 42%|████▏     | 1155/2757 [2:38:07<3:33:42,  8.00s/it]
 42%|████▏     | 1156/2757 [2:38:14<3:32:42,  7.97s/it]
 42%|████▏     | 1157/2757 [2:38:22<3:30:22,  7.89s/it]
 42%|████▏     | 1158/2757 [2:38:30<3:33:08,  8.00s/it]
 42%|████▏     | 1159/2757 [2:38:39<3:33:56,  8.03s/it]
 42%|████▏     | 1160/2757 [2:38:47<3:36:16,  8.13s/it]
                                                       

 42%|████▏     | 1160/2757 [2:38:47<3:36:16,  8.13s/it]
 42%|████▏     | 1161/2757 [2:38:55<3:35:30,  8.10s/it]
 42%|████▏     | 1162/2757 [2:39:03<3:37:23,  8.18s/it]
 42%|████▏     | 1163/2757 [2:39:11<3:34:36,  8.08s/it]
 42%|████▏     | 1164/2757 [2:39:19<3:32:32,  8.01s/it]
 42%|████▏     | 1165/2757 [2:39:27<3:32:39,  8.01s/it]
 42%|████▏     | 1166/2757 [2:39:36<3:36:55,  8.18s/it]
 42%|████▏     | 1167/2757 [2:39:44<3:39:19,  8.28s/it]
 42%|████▏     | 1168/2757 [2:39:52<3:35:20,  8.13s/it]
 42%|████▏     | 1169/2757 [2:40:00<3:36:03,  8.16s/it]
 42%|████▏     | 1170/2757 [2:40:08<3:33:27,  8.07s/it]
                                                       

 42%|████▏     | 1170/2757 [2:40:08<3:33:27,  8.07s/it]
 42%|████▏     | 1171/2757 [2:40:16<3:31:19,  7.99s/it]
 43%|████▎     | 1172/2757 [2:40:23<3:27:05,  7.84s/it]
 43%|████▎     | 1173/2757 [2:40:31<3:30:00,  7.95s/it]
 43%|████▎     | 1174/2757 [2:40:39<3:30:24,  7.98s/it]
 43%|████▎     | 1175/2757 [2:40:47<3:27:18,  7.86s/it]
 43%|████▎     | 1176/2757 [2:40:55<3:27:51,  7.89s/it]
 43%|████▎     | 1177/2757 [2:41:03<3:30:12,  7.98s/it]
 43%|████▎     | 1178/2757 [2:41:11<3:29:20,  7.95s/it]
 43%|████▎     | 1179/2757 [2:41:19<3:30:36,  8.01s/it]
 43%|████▎     | 1180/2757 [2:41:27<3:31:47,  8.06s/it]
                                                       

 43%|████▎     | 1180/2757 [2:41:27<3:31:47,  8.06s/it]
 43%|████▎     | 1181/2757 [2:41:35<3:29:14,  7.97s/it]
 43%|████▎     | 1182/2757 [2:41:43<3:30:10,  8.01s/it]
 43%|████▎     | 1183/2757 [2:41:51<3:29:26,  7.98s/it]
 43%|████▎     | 1184/2757 [2:41:59<3:27:46,  7.93s/it]
 43%|████▎     | 1185/2757 [2:42:07<3:25:36,  7.85s/it]
 43%|████▎     | 1186/2757 [2:42:15<3:28:24,  7.96s/it]
 43%|████▎     | 1187/2757 [2:42:23<3:25:33,  7.86s/it]
 43%|████▎     | 1188/2757 [2:42:30<3:21:06,  7.69s/it]
 43%|████▎     | 1189/2757 [2:42:38<3:21:58,  7.73s/it]
 43%|████▎     | 1190/2757 [2:42:46<3:24:12,  7.82s/it]
                                                       

 43%|████▎     | 1190/2757 [2:42:46<3:24:12,  7.82s/it]
 43%|████▎     | 1191/2757 [2:42:54<3:24:38,  7.84s/it]
 43%|████▎     | 1192/2757 [2:43:02<3:26:02,  7.90s/it]
 43%|████▎     | 1193/2757 [2:43:10<3:31:12,  8.10s/it]
 43%|████▎     | 1194/2757 [2:43:18<3:30:15,  8.07s/it]
 43%|████▎     | 1195/2757 [2:43:26<3:27:19,  7.96s/it]
 43%|████▎     | 1196/2757 [2:43:34<3:26:53,  7.95s/it]
 43%|████▎     | 1197/2757 [2:43:42<3:28:13,  8.01s/it]
 43%|████▎     | 1198/2757 [2:43:50<3:25:55,  7.93s/it]
 43%|████▎     | 1199/2757 [2:43:57<3:24:56,  7.89s/it]
 44%|████▎     | 1200/2757 [2:44:05<3:24:46,  7.89s/it]
                                                       

 44%|████▎     | 1200/2757 [2:44:05<3:24:46,  7.89s/it]
 44%|████▎     | 1201/2757 [2:44:13<3:19:18,  7.69s/it]
 44%|████▎     | 1202/2757 [2:44:20<3:20:08,  7.72s/it]
 44%|████▎     | 1203/2757 [2:44:29<3:25:54,  7.95s/it]
 44%|████▎     | 1204/2757 [2:44:36<3:21:35,  7.79s/it]
 44%|████▎     | 1205/2757 [2:44:44<3:23:11,  7.86s/it]
 44%|████▎     | 1206/2757 [2:44:52<3:21:44,  7.80s/it]
 44%|████▍     | 1207/2757 [2:45:01<3:27:20,  8.03s/it]
 44%|████▍     | 1208/2757 [2:45:09<3:27:17,  8.03s/it]
 44%|████▍     | 1209/2757 [2:45:17<3:27:02,  8.03s/it]
 44%|████▍     | 1210/2757 [2:45:25<3:27:06,  8.03s/it]
                                                       

 44%|████▍     | 1210/2757 [2:45:25<3:27:06,  8.03s/it]
 44%|████▍     | 1211/2757 [2:45:33<3:27:14,  8.04s/it]
 44%|████▍     | 1212/2757 [2:45:41<3:26:38,  8.03s/it]
 44%|████▍     | 1213/2757 [2:45:49<3:27:54,  8.08s/it]
 44%|████▍     | 1214/2757 [2:45:57<3:29:26,  8.14s/it]
 44%|████▍     | 1215/2757 [2:46:05<3:25:30,  8.00s/it]
 44%|████▍     | 1216/2757 [2:46:13<3:24:54,  7.98s/it]
 44%|████▍     | 1217/2757 [2:46:21<3:26:17,  8.04s/it]
 44%|████▍     | 1218/2757 [2:46:29<3:26:47,  8.06s/it]
 44%|████▍     | 1219/2757 [2:46:37<3:28:13,  8.12s/it]
 44%|████▍     | 1220/2757 [2:46:46<3:30:16,  8.21s/it]
                                                       

 44%|████▍     | 1220/2757 [2:46:46<3:30:16,  8.21s/it]
 44%|████▍     | 1221/2757 [2:46:53<3:26:35,  8.07s/it]
 44%|████▍     | 1222/2757 [2:47:01<3:25:39,  8.04s/it]
 44%|████▍     | 1223/2757 [2:47:09<3:21:08,  7.87s/it]
 44%|████▍     | 1224/2757 [2:47:17<3:20:22,  7.84s/it]
 44%|████▍     | 1225/2757 [2:47:24<3:19:43,  7.82s/it]
 44%|████▍     | 1226/2757 [2:47:32<3:20:51,  7.87s/it]
 45%|████▍     | 1227/2757 [2:47:40<3:20:02,  7.84s/it]
 45%|████▍     | 1228/2757 [2:47:48<3:22:20,  7.94s/it]
 45%|████▍     | 1229/2757 [2:47:56<3:22:31,  7.95s/it]
 45%|████▍     | 1230/2757 [2:48:05<3:24:41,  8.04s/it]
                                                       

 45%|████▍     | 1230/2757 [2:48:05<3:24:41,  8.04s/it]
 45%|████▍     | 1231/2757 [2:48:13<3:27:11,  8.15s/it]
 45%|████▍     | 1232/2757 [2:48:21<3:24:10,  8.03s/it]
 45%|████▍     | 1233/2757 [2:48:29<3:24:21,  8.05s/it]
 45%|████▍     | 1234/2757 [2:48:37<3:24:53,  8.07s/it]
 45%|████▍     | 1235/2757 [2:48:45<3:23:15,  8.01s/it]
 45%|████▍     | 1236/2757 [2:48:53<3:24:18,  8.06s/it]
 45%|████▍     | 1237/2757 [2:49:02<3:27:55,  8.21s/it]
 45%|████▍     | 1238/2757 [2:49:09<3:25:16,  8.11s/it]
 45%|████▍     | 1239/2757 [2:49:18<3:28:38,  8.25s/it]
 45%|████▍     | 1240/2757 [2:49:26<3:23:16,  8.04s/it]
                                                       

 45%|████▍     | 1240/2757 [2:49:26<3:23:16,  8.04s/it]
 45%|████▌     | 1241/2757 [2:49:33<3:21:53,  7.99s/it]
 45%|████▌     | 1242/2757 [2:49:41<3:20:26,  7.94s/it]
 45%|████▌     | 1243/2757 [2:49:49<3:19:45,  7.92s/it]
 45%|████▌     | 1244/2757 [2:49:57<3:21:52,  8.01s/it]
 45%|████▌     | 1245/2757 [2:50:06<3:23:45,  8.09s/it]
 45%|████▌     | 1246/2757 [2:50:14<3:22:41,  8.05s/it]
 45%|████▌     | 1247/2757 [2:50:22<3:25:53,  8.18s/it]
 45%|████▌     | 1248/2757 [2:50:30<3:24:47,  8.14s/it]
 45%|████▌     | 1249/2757 [2:50:38<3:21:29,  8.02s/it]
 45%|████▌     | 1250/2757 [2:50:46<3:25:23,  8.18s/it]
                                                       
{'eval_loss': '0.5218', 'eval_runtime': '104.5', 'eval_samples_per_second': '22.97', 'eval_steps_per_second': '2.871', 'eval_rewards/chosen': '-1.5', 'eval_rewards/rejected': '-2.382', 'eval_rewards/accuracies': '0.6712', 'eval_rewards/margins': '0.8821', 'eval_logps/chosen': '-94.61', 'eval_logps/rejected': '-109.7', 'eval_logits/chosen': '-4.036', 'eval_logits/rejected': '-3.984', 'epoch': '1.088'}
{'loss': '0.3826', 'grad_norm': '0.5595', 'learning_rate': '0.0001307', 'rewards/chosen': '-1.484', 'rewards/rejected': '-2.78', 'rewards/accuracies': '0.7914', 'rewards/margins': '1.296', 'logps/chosen': '-99.29', 'logps/rejected': '-118.4', 'logits/chosen': '-3.986', 'logits/rejected': '-3.91', 'epoch': '1.099'}
{'loss': '0.3769', 'grad_norm': '0.5151', 'learning_rate': '0.00013', 'rewards/chosen': '-1.544', 'rewards/rejected': '-2.896', 'rewards/accuracies': '0.7859', 'rewards/margins': '1.352', 'logps/chosen': '-95.98', 'logps/rejected': '-114.6', 'logits/chosen': '-3.936', 'logits/rejected': '-3.925', 'epoch': '1.11'}
{'loss': '0.3913', 'grad_norm': '0.5914', 'learning_rate': '0.0001292', 'rewards/chosen': '-1.531', 'rewards/rejected': '-2.838', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.307', 'logps/chosen': '-95.18', 'logps/rejected': '-113.3', 'logits/chosen': '-3.944', 'logits/rejected': '-3.977', 'epoch': '1.121'}
{'loss': '0.4064', 'grad_norm': '0.609', 'learning_rate': '0.0001285', 'rewards/chosen': '-1.362', 'rewards/rejected': '-2.581', 'rewards/accuracies': '0.7563', 'rewards/margins': '1.219', 'logps/chosen': '-91.94', 'logps/rejected': '-109.3', 'logits/chosen': '-3.923', 'logits/rejected': '-3.879', 'epoch': '1.132'}
{'loss': '0.3909', 'grad_norm': '0.558', 'learning_rate': '0.0001277', 'rewards/chosen': '-1.129', 'rewards/rejected': '-2.423', 'rewards/accuracies': '0.7781', 'rewards/margins': '1.294', 'logps/chosen': '-93.53', 'logps/rejected': '-110.6', 'logits/chosen': '-4.08', 'logits/rejected': '-4.015', 'epoch': '1.143'}
{'loss': '0.4011', 'grad_norm': '0.6687', 'learning_rate': '0.000127', 'rewards/chosen': '-1.27', 'rewards/rejected': '-2.525', 'rewards/accuracies': '0.7695', 'rewards/margins': '1.256', 'logps/chosen': '-92.7', 'logps/rejected': '-109', 'logits/chosen': '-3.949', 'logits/rejected': '-3.86', 'epoch': '1.153'}
{'loss': '0.3885', 'grad_norm': '0.5773', 'learning_rate': '0.0001263', 'rewards/chosen': '-1.37', 'rewards/rejected': '-2.672', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.302', 'logps/chosen': '-93.09', 'logps/rejected': '-112.1', 'logits/chosen': '-4.051', 'logits/rejected': '-3.921', 'epoch': '1.164'}
{'loss': '0.3903', 'grad_norm': '0.5501', 'learning_rate': '0.0001255', 'rewards/chosen': '-1.401', 'rewards/rejected': '-2.699', 'rewards/accuracies': '0.7797', 'rewards/margins': '1.299', 'logps/chosen': '-93.73', 'logps/rejected': '-112.1', 'logits/chosen': '-3.901', 'logits/rejected': '-3.798', 'epoch': '1.175'}
{'loss': '0.3811', 'grad_norm': '0.602', 'learning_rate': '0.0001248', 'rewards/chosen': '-1.542', 'rewards/rejected': '-2.905', 'rewards/accuracies': '0.7859', 'rewards/margins': '1.362', 'logps/chosen': '-97.4', 'logps/rejected': '-116.5', 'logits/chosen': '-3.874', 'logits/rejected': '-3.843', 'epoch': '1.186'}
{'loss': '0.3859', 'grad_norm': '0.5466', 'learning_rate': '0.000124', 'rewards/chosen': '-1.562', 'rewards/rejected': '-2.871', 'rewards/accuracies': '0.7867', 'rewards/margins': '1.309', 'logps/chosen': '-97.63', 'logps/rejected': '-116.3', 'logits/chosen': '-3.822', 'logits/rejected': '-3.789', 'epoch': '1.197'}
{'loss': '0.3874', 'grad_norm': '0.6201', 'learning_rate': '0.0001233', 'rewards/chosen': '-1.538', 'rewards/rejected': '-2.896', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.358', 'logps/chosen': '-94.8', 'logps/rejected': '-114.1', 'logits/chosen': '-3.869', 'logits/rejected': '-3.814', 'epoch': '1.208'}
{'loss': '0.3829', 'grad_norm': '0.6293', 'learning_rate': '0.0001225', 'rewards/chosen': '-1.702', 'rewards/rejected': '-3.064', 'rewards/accuracies': '0.782', 'rewards/margins': '1.362', 'logps/chosen': '-98.96', 'logps/rejected': '-118.2', 'logits/chosen': '-3.824', 'logits/rejected': '-3.681', 'epoch': '1.219'}
{'loss': '0.3882', 'grad_norm': '0.6542', 'learning_rate': '0.0001218', 'rewards/chosen': '-1.607', 'rewards/rejected': '-2.912', 'rewards/accuracies': '0.7711', 'rewards/margins': '1.305', 'logps/chosen': '-95.04', 'logps/rejected': '-113.7', 'logits/chosen': '-3.806', 'logits/rejected': '-3.804', 'epoch': '1.23'}
{'loss': '0.3934', 'grad_norm': '0.6825', 'learning_rate': '0.000121', 'rewards/chosen': '-1.642', 'rewards/rejected': '-2.956', 'rewards/accuracies': '0.7797', 'rewards/margins': '1.314', 'logps/chosen': '-97.03', 'logps/rejected': '-116', 'logits/chosen': '-3.831', 'logits/rejected': '-3.731', 'epoch': '1.241'}
{'loss': '0.3808', 'grad_norm': '0.5602', 'learning_rate': '0.0001203', 'rewards/chosen': '-1.608', 'rewards/rejected': '-2.998', 'rewards/accuracies': '0.7898', 'rewards/margins': '1.39', 'logps/chosen': '-96.62', 'logps/rejected': '-116.4', 'logits/chosen': '-3.864', 'logits/rejected': '-3.806', 'epoch': '1.251'}
{'loss': '0.3848', 'grad_norm': '0.5861', 'learning_rate': '0.0001195', 'rewards/chosen': '-1.649', 'rewards/rejected': '-3.021', 'rewards/accuracies': '0.775', 'rewards/margins': '1.372', 'logps/chosen': '-95.55', 'logps/rejected': '-114.5', 'logits/chosen': '-3.834', 'logits/rejected': '-3.734', 'epoch': '1.262'}
{'loss': '0.3743', 'grad_norm': '0.5589', 'learning_rate': '0.0001188', 'rewards/chosen': '-1.889', 'rewards/rejected': '-3.314', 'rewards/accuracies': '0.7883', 'rewards/margins': '1.426', 'logps/chosen': '-98.93', 'logps/rejected': '-120.1', 'logits/chosen': '-3.808', 'logits/rejected': '-3.658', 'epoch': '1.273'}
{'loss': '0.3917', 'grad_norm': '0.7734', 'learning_rate': '0.000118', 'rewards/chosen': '-2.133', 'rewards/rejected': '-3.482', 'rewards/accuracies': '0.7727', 'rewards/margins': '1.35', 'logps/chosen': '-100.4', 'logps/rejected': '-119.5', 'logits/chosen': '-3.791', 'logits/rejected': '-3.751', 'epoch': '1.284'}
{'loss': '0.3873', 'grad_norm': '0.6508', 'learning_rate': '0.0001173', 'rewards/chosen': '-1.986', 'rewards/rejected': '-3.376', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.39', 'logps/chosen': '-102.7', 'logps/rejected': '-119.9', 'logits/chosen': '-3.632', 'logits/rejected': '-3.56', 'epoch': '1.295'}
{'loss': '0.3865', 'grad_norm': '0.5762', 'learning_rate': '0.0001165', 'rewards/chosen': '-1.77', 'rewards/rejected': '-3.147', 'rewards/accuracies': '0.7789', 'rewards/margins': '1.377', 'logps/chosen': '-96.95', 'logps/rejected': '-118.6', 'logits/chosen': '-3.736', 'logits/rejected': '-3.719', 'epoch': '1.306'}
{'loss': '0.3787', 'grad_norm': '0.7514', 'learning_rate': '0.0001158', 'rewards/chosen': '-1.872', 'rewards/rejected': '-3.307', 'rewards/accuracies': '0.7898', 'rewards/margins': '1.435', 'logps/chosen': '-99.3', 'logps/rejected': '-118.7', 'logits/chosen': '-3.761', 'logits/rejected': '-3.714', 'epoch': '1.317'}
{'loss': '0.3696', 'grad_norm': '0.6535', 'learning_rate': '0.000115', 'rewards/chosen': '-1.846', 'rewards/rejected': '-3.268', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.422', 'logps/chosen': '-100.5', 'logps/rejected': '-121', 'logits/chosen': '-3.695', 'logits/rejected': '-3.616', 'epoch': '1.328'}
{'loss': '0.3887', 'grad_norm': '0.6603', 'learning_rate': '0.0001143', 'rewards/chosen': '-1.846', 'rewards/rejected': '-3.164', 'rewards/accuracies': '0.7828', 'rewards/margins': '1.318', 'logps/chosen': '-98.53', 'logps/rejected': '-115.1', 'logits/chosen': '-3.787', 'logits/rejected': '-3.732', 'epoch': '1.339'}
{'loss': '0.381', 'grad_norm': '0.7037', 'learning_rate': '0.0001135', 'rewards/chosen': '-1.664', 'rewards/rejected': '-3.028', 'rewards/accuracies': '0.7875', 'rewards/margins': '1.364', 'logps/chosen': '-96.31', 'logps/rejected': '-115.1', 'logits/chosen': '-3.723', 'logits/rejected': '-3.703', 'epoch': '1.349'}
{'loss': '0.3888', 'grad_norm': '0.638', 'learning_rate': '0.0001128', 'rewards/chosen': '-1.572', 'rewards/rejected': '-2.942', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.371', 'logps/chosen': '-95.81', 'logps/rejected': '-114.3', 'logits/chosen': '-3.744', 'logits/rejected': '-3.591', 'epoch': '1.36'}
 45%|████▌     | 1250/2757 [2:50:46<3:25:23,  8.18s/it]
 45%|████▌     | 1251/2757 [2:50:59<3:56:02,  9.40s/it]
 45%|████▌     | 1252/2757 [2:51:07<3:46:44,  9.04s/it]
 45%|████▌     | 1253/2757 [2:51:14<3:35:40,  8.60s/it]
 45%|████▌     | 1254/2757 [2:51:22<3:30:16,  8.39s/it]
 46%|████▌     | 1255/2757 [2:51:31<3:30:03,  8.39s/it]
 46%|████▌     | 1256/2757 [2:51:38<3:24:39,  8.18s/it]
 46%|████▌     | 1257/2757 [2:51:46<3:18:50,  7.95s/it]
 46%|████▌     | 1258/2757 [2:51:54<3:17:15,  7.90s/it]
 46%|████▌     | 1259/2757 [2:52:02<3:18:40,  7.96s/it]
 46%|████▌     | 1260/2757 [2:52:10<3:20:41,  8.04s/it]
                                                       

 46%|████▌     | 1260/2757 [2:52:10<3:20:41,  8.04s/it]
 46%|████▌     | 1261/2757 [2:52:18<3:19:17,  7.99s/it]
 46%|████▌     | 1262/2757 [2:52:26<3:16:47,  7.90s/it]
 46%|████▌     | 1263/2757 [2:52:34<3:20:51,  8.07s/it]
 46%|████▌     | 1264/2757 [2:52:42<3:19:12,  8.01s/it]
 46%|████▌     | 1265/2757 [2:52:50<3:18:42,  7.99s/it]
 46%|████▌     | 1266/2757 [2:52:58<3:22:26,  8.15s/it]
 46%|████▌     | 1267/2757 [2:53:07<3:23:52,  8.21s/it]
 46%|████▌     | 1268/2757 [2:53:15<3:22:54,  8.18s/it]
 46%|████▌     | 1269/2757 [2:53:23<3:19:56,  8.06s/it]
 46%|████▌     | 1270/2757 [2:53:31<3:20:16,  8.08s/it]
                                                       

 46%|████▌     | 1270/2757 [2:53:31<3:20:16,  8.08s/it]
 46%|████▌     | 1271/2757 [2:53:39<3:20:24,  8.09s/it]
 46%|████▌     | 1272/2757 [2:53:47<3:21:51,  8.16s/it]
 46%|████▌     | 1273/2757 [2:53:55<3:18:37,  8.03s/it]
 46%|████▌     | 1274/2757 [2:54:03<3:20:09,  8.10s/it]
 46%|████▌     | 1275/2757 [2:54:11<3:18:43,  8.05s/it]
 46%|████▋     | 1276/2757 [2:54:19<3:18:58,  8.06s/it]
 46%|████▋     | 1277/2757 [2:54:27<3:19:54,  8.10s/it]
 46%|████▋     | 1278/2757 [2:54:35<3:18:47,  8.06s/it]
 46%|████▋     | 1279/2757 [2:54:43<3:18:24,  8.05s/it]
 46%|████▋     | 1280/2757 [2:54:51<3:16:33,  7.98s/it]
                                                       

 46%|████▋     | 1280/2757 [2:54:51<3:16:33,  7.98s/it]
 46%|████▋     | 1281/2757 [2:54:59<3:14:45,  7.92s/it]
 46%|████▋     | 1282/2757 [2:55:07<3:13:50,  7.89s/it]
 47%|████▋     | 1283/2757 [2:55:15<3:14:51,  7.93s/it]
 47%|████▋     | 1284/2757 [2:55:23<3:15:39,  7.97s/it]
 47%|████▋     | 1285/2757 [2:55:31<3:17:58,  8.07s/it]
 47%|████▋     | 1286/2757 [2:55:39<3:17:15,  8.05s/it]
 47%|████▋     | 1287/2757 [2:55:47<3:18:06,  8.09s/it]
 47%|████▋     | 1288/2757 [2:55:55<3:16:46,  8.04s/it]
 47%|████▋     | 1289/2757 [2:56:03<3:14:41,  7.96s/it]
 47%|████▋     | 1290/2757 [2:56:11<3:17:04,  8.06s/it]
                                                       

 47%|████▋     | 1290/2757 [2:56:11<3:17:04,  8.06s/it]
 47%|████▋     | 1291/2757 [2:56:20<3:20:07,  8.19s/it]
 47%|████▋     | 1292/2757 [2:56:28<3:20:44,  8.22s/it]
 47%|████▋     | 1293/2757 [2:56:36<3:20:21,  8.21s/it]
 47%|████▋     | 1294/2757 [2:56:45<3:22:26,  8.30s/it]
 47%|████▋     | 1295/2757 [2:56:53<3:18:58,  8.17s/it]
 47%|████▋     | 1296/2757 [2:57:01<3:20:48,  8.25s/it]
 47%|████▋     | 1297/2757 [2:57:09<3:19:47,  8.21s/it]
 47%|████▋     | 1298/2757 [2:57:17<3:17:15,  8.11s/it]
 47%|████▋     | 1299/2757 [2:57:25<3:15:30,  8.05s/it]
 47%|████▋     | 1300/2757 [2:57:33<3:14:02,  7.99s/it]
                                                       

 47%|████▋     | 1300/2757 [2:57:33<3:14:02,  7.99s/it]
 47%|████▋     | 1301/2757 [2:57:41<3:12:54,  7.95s/it]
 47%|████▋     | 1302/2757 [2:57:49<3:13:23,  7.98s/it]
 47%|████▋     | 1303/2757 [2:57:57<3:15:53,  8.08s/it]
 47%|████▋     | 1304/2757 [2:58:05<3:15:15,  8.06s/it]
 47%|████▋     | 1305/2757 [2:58:13<3:15:58,  8.10s/it]
 47%|████▋     | 1306/2757 [2:58:21<3:12:35,  7.96s/it]
 47%|████▋     | 1307/2757 [2:58:29<3:13:35,  8.01s/it]
 47%|████▋     | 1308/2757 [2:58:37<3:11:35,  7.93s/it]
 47%|████▋     | 1309/2757 [2:58:45<3:13:05,  8.00s/it]
 48%|████▊     | 1310/2757 [2:58:53<3:12:11,  7.97s/it]
                                                       

 48%|████▊     | 1310/2757 [2:58:53<3:12:11,  7.97s/it]
 48%|████▊     | 1311/2757 [2:59:01<3:12:06,  7.97s/it]
 48%|████▊     | 1312/2757 [2:59:09<3:10:03,  7.89s/it]
 48%|████▊     | 1313/2757 [2:59:16<3:09:00,  7.85s/it]
 48%|████▊     | 1314/2757 [2:59:24<3:08:56,  7.86s/it]
 48%|████▊     | 1315/2757 [2:59:32<3:08:55,  7.86s/it]
 48%|████▊     | 1316/2757 [2:59:41<3:13:52,  8.07s/it]
 48%|████▊     | 1317/2757 [2:59:48<3:11:52,  7.99s/it]
 48%|████▊     | 1318/2757 [2:59:57<3:13:06,  8.05s/it]
 48%|████▊     | 1319/2757 [3:00:05<3:12:08,  8.02s/it]
 48%|████▊     | 1320/2757 [3:00:13<3:13:20,  8.07s/it]
                                                       

 48%|████▊     | 1320/2757 [3:00:13<3:13:20,  8.07s/it]
 48%|████▊     | 1321/2757 [3:00:21<3:12:11,  8.03s/it]
 48%|████▊     | 1322/2757 [3:00:29<3:11:50,  8.02s/it]
 48%|████▊     | 1323/2757 [3:00:37<3:14:13,  8.13s/it]
 48%|████▊     | 1324/2757 [3:00:45<3:09:25,  7.93s/it]
 48%|████▊     | 1325/2757 [3:00:53<3:11:57,  8.04s/it]
 48%|████▊     | 1326/2757 [3:01:00<3:08:20,  7.90s/it]
 48%|████▊     | 1327/2757 [3:01:08<3:05:27,  7.78s/it]
 48%|████▊     | 1328/2757 [3:01:16<3:06:54,  7.85s/it]
 48%|████▊     | 1329/2757 [3:01:24<3:06:53,  7.85s/it]
 48%|████▊     | 1330/2757 [3:01:32<3:06:35,  7.85s/it]
                                                       

 48%|████▊     | 1330/2757 [3:01:32<3:06:35,  7.85s/it]
 48%|████▊     | 1331/2757 [3:01:40<3:07:09,  7.88s/it]
 48%|████▊     | 1332/2757 [3:01:48<3:11:17,  8.05s/it]
 48%|████▊     | 1333/2757 [3:01:56<3:11:39,  8.08s/it]
 48%|████▊     | 1334/2757 [3:02:04<3:11:44,  8.08s/it]
 48%|████▊     | 1335/2757 [3:02:12<3:09:34,  8.00s/it]
 48%|████▊     | 1336/2757 [3:02:20<3:11:21,  8.08s/it]
 48%|████▊     | 1337/2757 [3:02:28<3:11:00,  8.07s/it]
 49%|████▊     | 1338/2757 [3:02:36<3:11:16,  8.09s/it]
 49%|████▊     | 1339/2757 [3:02:44<3:08:38,  7.98s/it]
 49%|████▊     | 1340/2757 [3:02:52<3:08:37,  7.99s/it]
                                                       

 49%|████▊     | 1340/2757 [3:02:52<3:08:37,  7.99s/it]
 49%|████▊     | 1341/2757 [3:03:00<3:07:30,  7.95s/it]
 49%|████▊     | 1342/2757 [3:03:08<3:10:27,  8.08s/it]
 49%|████▊     | 1343/2757 [3:03:17<3:10:31,  8.08s/it]
 49%|████▊     | 1344/2757 [3:03:24<3:07:43,  7.97s/it]
 49%|████▉     | 1345/2757 [3:03:32<3:06:41,  7.93s/it]
 49%|████▉     | 1346/2757 [3:03:40<3:07:15,  7.96s/it]
 49%|████▉     | 1347/2757 [3:03:48<3:08:46,  8.03s/it]
 49%|████▉     | 1348/2757 [3:03:57<3:11:35,  8.16s/it]
 49%|████▉     | 1349/2757 [3:04:05<3:10:40,  8.13s/it]
 49%|████▉     | 1350/2757 [3:04:14<3:16:22,  8.37s/it]
                                                       

 49%|████▉     | 1350/2757 [3:04:14<3:16:22,  8.37s/it]
 49%|████▉     | 1351/2757 [3:04:22<3:13:11,  8.24s/it]
 49%|████▉     | 1352/2757 [3:04:30<3:13:27,  8.26s/it]
 49%|████▉     | 1353/2757 [3:04:38<3:09:31,  8.10s/it]
 49%|████▉     | 1354/2757 [3:04:46<3:09:28,  8.10s/it]
 49%|████▉     | 1355/2757 [3:04:54<3:06:19,  7.97s/it]
 49%|████▉     | 1356/2757 [3:05:02<3:10:52,  8.17s/it]
 49%|████▉     | 1357/2757 [3:05:10<3:11:18,  8.20s/it]
 49%|████▉     | 1358/2757 [3:05:18<3:07:08,  8.03s/it]
 49%|████▉     | 1359/2757 [3:05:26<3:06:58,  8.02s/it]
 49%|████▉     | 1360/2757 [3:05:34<3:07:45,  8.06s/it]
                                                       

 49%|████▉     | 1360/2757 [3:05:34<3:07:45,  8.06s/it]
 49%|████▉     | 1361/2757 [3:05:43<3:11:06,  8.21s/it]
 49%|████▉     | 1362/2757 [3:05:51<3:09:22,  8.15s/it]
 49%|████▉     | 1363/2757 [3:05:58<3:06:20,  8.02s/it]
 49%|████▉     | 1364/2757 [3:06:07<3:07:24,  8.07s/it]
 50%|████▉     | 1365/2757 [3:06:14<3:03:48,  7.92s/it]
 50%|████▉     | 1366/2757 [3:06:22<3:02:14,  7.86s/it]
 50%|████▉     | 1367/2757 [3:06:30<3:02:49,  7.89s/it]
 50%|████▉     | 1368/2757 [3:06:38<3:01:50,  7.86s/it]
 50%|████▉     | 1369/2757 [3:06:46<3:03:11,  7.92s/it]
 50%|████▉     | 1370/2757 [3:06:54<3:03:10,  7.92s/it]
                                                       

 50%|████▉     | 1370/2757 [3:06:54<3:03:10,  7.92s/it]
 50%|████▉     | 1371/2757 [3:07:02<3:07:24,  8.11s/it]
 50%|████▉     | 1372/2757 [3:07:11<3:08:26,  8.16s/it]
 50%|████▉     | 1373/2757 [3:07:19<3:10:48,  8.27s/it]
 50%|████▉     | 1374/2757 [3:07:27<3:09:40,  8.23s/it]
 50%|████▉     | 1375/2757 [3:07:35<3:08:18,  8.18s/it]
 50%|████▉     | 1376/2757 [3:07:43<3:06:53,  8.12s/it]
 50%|████▉     | 1377/2757 [3:07:51<3:06:19,  8.10s/it]
 50%|████▉     | 1378/2757 [3:07:59<3:02:09,  7.93s/it]
 50%|█████     | 1379/2757 [3:08:07<3:03:40,  8.00s/it]
 50%|█████     | 1380/2757 [3:08:15<3:03:55,  8.01s/it]
                                                       

 50%|█████     | 1380/2757 [3:08:15<3:03:55,  8.01s/it]
 50%|█████     | 1381/2757 [3:08:23<3:03:27,  8.00s/it]
 50%|█████     | 1382/2757 [3:08:31<3:04:35,  8.05s/it]
 50%|█████     | 1383/2757 [3:08:40<3:09:10,  8.26s/it]
 50%|█████     | 1384/2757 [3:08:48<3:09:37,  8.29s/it]
 50%|█████     | 1385/2757 [3:08:56<3:08:25,  8.24s/it]
 50%|█████     | 1386/2757 [3:09:04<3:04:38,  8.08s/it]
 50%|█████     | 1387/2757 [3:09:12<3:01:43,  7.96s/it]
 50%|█████     | 1388/2757 [3:09:20<3:04:36,  8.09s/it]
 50%|█████     | 1389/2757 [3:09:28<3:03:47,  8.06s/it]
 50%|█████     | 1390/2757 [3:09:36<3:05:02,  8.12s/it]
                                                       

 50%|█████     | 1390/2757 [3:09:36<3:05:02,  8.12s/it]
 50%|█████     | 1391/2757 [3:09:44<3:02:06,  8.00s/it]
 50%|█████     | 1392/2757 [3:09:52<3:03:50,  8.08s/it]
 51%|█████     | 1393/2757 [3:10:01<3:05:17,  8.15s/it]
 51%|█████     | 1394/2757 [3:10:08<3:01:30,  7.99s/it]
 51%|█████     | 1395/2757 [3:10:16<3:00:49,  7.97s/it]
 51%|█████     | 1396/2757 [3:10:24<2:59:38,  7.92s/it]
 51%|█████     | 1397/2757 [3:10:32<3:00:00,  7.94s/it]
 51%|█████     | 1398/2757 [3:10:40<3:01:48,  8.03s/it]
 51%|█████     | 1399/2757 [3:10:48<3:00:37,  7.98s/it]
 51%|█████     | 1400/2757 [3:10:56<3:01:22,  8.02s/it]
                                                       

 51%|█████     | 1400/2757 [3:10:56<3:01:22,  8.02s/it]
 51%|█████     | 1401/2757 [3:11:05<3:02:56,  8.09s/it]
 51%|█████     | 1402/2757 [3:11:12<3:01:03,  8.02s/it]
 51%|█████     | 1403/2757 [3:11:20<2:57:20,  7.86s/it]
 51%|█████     | 1404/2757 [3:11:28<2:59:22,  7.95s/it]
 51%|█████     | 1405/2757 [3:11:36<2:56:15,  7.82s/it]
 51%|█████     | 1406/2757 [3:11:44<2:58:16,  7.92s/it]
 51%|█████     | 1407/2757 [3:11:52<2:58:30,  7.93s/it]
 51%|█████     | 1408/2757 [3:12:00<2:59:55,  8.00s/it]
 51%|█████     | 1409/2757 [3:12:08<3:00:00,  8.01s/it]
 51%|█████     | 1410/2757 [3:12:16<3:00:09,  8.02s/it]
                                                       

 51%|█████     | 1410/2757 [3:12:16<3:00:09,  8.02s/it]
 51%|█████     | 1411/2757 [3:12:25<3:03:58,  8.20s/it]
 51%|█████     | 1412/2757 [3:12:32<3:00:56,  8.07s/it]
 51%|█████▏    | 1413/2757 [3:12:40<3:00:47,  8.07s/it]
 51%|█████▏    | 1414/2757 [3:12:48<2:58:51,  7.99s/it]
 51%|█████▏    | 1415/2757 [3:12:56<2:59:31,  8.03s/it]
 51%|█████▏    | 1416/2757 [3:13:04<2:58:24,  7.98s/it]
 51%|█████▏    | 1417/2757 [3:13:12<2:57:24,  7.94s/it]
 51%|█████▏    | 1418/2757 [3:13:20<2:55:21,  7.86s/it]
 51%|█████▏    | 1419/2757 [3:13:28<2:57:04,  7.94s/it]
 52%|█████▏    | 1420/2757 [3:13:36<2:56:02,  7.90s/it]
                                                       

 52%|█████▏    | 1420/2757 [3:13:36<2:56:02,  7.90s/it]
 52%|█████▏    | 1421/2757 [3:13:44<2:57:16,  7.96s/it]
 52%|█████▏    | 1422/2757 [3:13:52<2:59:59,  8.09s/it]
 52%|█████▏    | 1423/2757 [3:14:00<2:58:51,  8.04s/it]
 52%|█████▏    | 1424/2757 [3:14:08<2:57:54,  8.01s/it]
 52%|█████▏    | 1425/2757 [3:14:16<2:57:20,  7.99s/it]
 52%|█████▏    | 1426/2757 [3:14:24<2:59:31,  8.09s/it]
 52%|█████▏    | 1427/2757 [3:14:33<3:01:16,  8.18s/it]
 52%|█████▏    | 1428/2757 [3:14:41<2:59:45,  8.12s/it]
 52%|█████▏    | 1429/2757 [3:14:49<3:00:33,  8.16s/it]
 52%|█████▏    | 1430/2757 [3:14:57<2:57:46,  8.04s/it]
                                                       

 52%|█████▏    | 1430/2757 [3:14:57<2:57:46,  8.04s/it]
 52%|█████▏    | 1431/2757 [3:15:05<2:59:10,  8.11s/it]
 52%|█████▏    | 1432/2757 [3:15:13<2:56:01,  7.97s/it]
 52%|█████▏    | 1433/2757 [3:15:20<2:55:31,  7.95s/it]
 52%|█████▏    | 1434/2757 [3:15:28<2:52:28,  7.82s/it]
 52%|█████▏    | 1435/2757 [3:15:36<2:54:57,  7.94s/it]
 52%|█████▏    | 1436/2757 [3:15:44<2:56:56,  8.04s/it]
 52%|█████▏    | 1437/2757 [3:15:53<2:57:01,  8.05s/it]
 52%|█████▏    | 1438/2757 [3:16:01<2:56:35,  8.03s/it]
 52%|█████▏    | 1439/2757 [3:16:08<2:52:47,  7.87s/it]
 52%|█████▏    | 1440/2757 [3:16:16<2:54:25,  7.95s/it]
                                                       

 52%|█████▏    | 1440/2757 [3:16:16<2:54:25,  7.95s/it]
 52%|█████▏    | 1441/2757 [3:16:24<2:53:38,  7.92s/it]
 52%|█████▏    | 1442/2757 [3:16:32<2:56:11,  8.04s/it]
 52%|█████▏    | 1443/2757 [3:16:41<2:59:12,  8.18s/it]
 52%|█████▏    | 1444/2757 [3:16:49<2:56:38,  8.07s/it]
 52%|█████▏    | 1445/2757 [3:16:57<2:55:19,  8.02s/it]
 52%|█████▏    | 1446/2757 [3:17:05<2:56:00,  8.06s/it]
 52%|█████▏    | 1447/2757 [3:17:12<2:54:06,  7.97s/it]
 53%|█████▎    | 1448/2757 [3:17:20<2:53:12,  7.94s/it]
 53%|█████▎    | 1449/2757 [3:17:29<2:55:14,  8.04s/it]
 53%|█████▎    | 1450/2757 [3:17:37<2:56:59,  8.13s/it]
                                                       

 53%|█████▎    | 1450/2757 [3:17:37<2:56:59,  8.13s/it]
 53%|█████▎    | 1451/2757 [3:17:45<2:59:53,  8.26s/it]
 53%|█████▎    | 1452/2757 [3:17:54<3:00:28,  8.30s/it]
 53%|█████▎    | 1453/2757 [3:18:02<2:59:46,  8.27s/it]
 53%|█████▎    | 1454/2757 [3:18:10<2:58:12,  8.21s/it]
 53%|█████▎    | 1455/2757 [3:18:18<2:55:35,  8.09s/it]
 53%|█████▎    | 1456/2757 [3:18:26<2:56:48,  8.15s/it]
 53%|█████▎    | 1457/2757 [3:18:34<2:56:19,  8.14s/it]
 53%|█████▎    | 1458/2757 [3:18:42<2:53:41,  8.02s/it]
 53%|█████▎    | 1459/2757 [3:18:50<2:55:00,  8.09s/it]
 53%|█████▎    | 1460/2757 [3:18:58<2:52:27,  7.98s/it]
                                                       

 53%|█████▎    | 1460/2757 [3:18:58<2:52:27,  7.98s/it]
 53%|█████▎    | 1461/2757 [3:19:06<2:54:08,  8.06s/it]
 53%|█████▎    | 1462/2757 [3:19:14<2:52:08,  7.98s/it]
 53%|█████▎    | 1463/2757 [3:19:22<2:51:51,  7.97s/it]
 53%|█████▎    | 1464/2757 [3:19:30<2:53:01,  8.03s/it]
 53%|█████▎    | 1465/2757 [3:19:38<2:54:25,  8.10s/it]
 53%|█████▎    | 1466/2757 [3:19:47<2:54:37,  8.12s/it]
 53%|█████▎    | 1467/2757 [3:19:55<2:56:10,  8.19s/it]
 53%|█████▎    | 1468/2757 [3:20:03<2:57:44,  8.27s/it]
 53%|█████▎    | 1469/2757 [3:20:12<2:56:20,  8.21s/it]
 53%|█████▎    | 1470/2757 [3:20:20<2:57:17,  8.26s/it]
                                                       

 53%|█████▎    | 1470/2757 [3:20:20<2:57:17,  8.26s/it]
 53%|█████▎    | 1471/2757 [3:20:28<2:55:03,  8.17s/it]
 53%|█████▎    | 1472/2757 [3:20:36<2:54:08,  8.13s/it]
 53%|█████▎    | 1473/2757 [3:20:43<2:49:53,  7.94s/it]
 53%|█████▎    | 1474/2757 [3:20:51<2:49:18,  7.92s/it]
 54%|█████▎    | 1475/2757 [3:20:59<2:46:52,  7.81s/it]
 54%|█████▎    | 1476/2757 [3:21:07<2:48:25,  7.89s/it]
 54%|█████▎    | 1477/2757 [3:21:15<2:49:12,  7.93s/it]
 54%|█████▎    | 1478/2757 [3:21:23<2:48:49,  7.92s/it]
 54%|█████▎    | 1479/2757 [3:21:30<2:45:53,  7.79s/it]
 54%|█████▎    | 1480/2757 [3:21:38<2:45:55,  7.80s/it]
                                                       

 54%|█████▎    | 1480/2757 [3:21:38<2:45:55,  7.80s/it]
 54%|█████▎    | 1481/2757 [3:21:46<2:45:43,  7.79s/it]
 54%|█████▍    | 1482/2757 [3:21:54<2:46:29,  7.83s/it]
 54%|█████▍    | 1483/2757 [3:22:02<2:47:57,  7.91s/it]
 54%|█████▍    | 1484/2757 [3:22:10<2:48:01,  7.92s/it]
 54%|█████▍    | 1485/2757 [3:22:18<2:47:51,  7.92s/it]
 54%|█████▍    | 1486/2757 [3:22:26<2:46:56,  7.88s/it]
 54%|█████▍    | 1487/2757 [3:22:33<2:45:08,  7.80s/it]
 54%|█████▍    | 1488/2757 [3:22:41<2:47:17,  7.91s/it]
 54%|█████▍    | 1489/2757 [3:22:49<2:47:41,  7.93s/it]
 54%|█████▍    | 1490/2757 [3:22:57<2:46:24,  7.88s/it]
                                                       

 54%|█████▍    | 1490/2757 [3:22:57<2:46:24,  7.88s/it]
 54%|█████▍    | 1491/2757 [3:23:05<2:44:54,  7.82s/it]
 54%|█████▍    | 1492/2757 [3:23:12<2:43:34,  7.76s/it]
 54%|█████▍    | 1493/2757 [3:23:21<2:47:17,  7.94s/it]
 54%|█████▍    | 1494/2757 [3:23:29<2:47:01,  7.93s/it]
 54%|█████▍    | 1495/2757 [3:23:37<2:47:16,  7.95s/it]
 54%|█████▍    | 1496/2757 [3:23:44<2:45:00,  7.85s/it]
 54%|█████▍    | 1497/2757 [3:23:53<2:48:12,  8.01s/it]
 54%|█████▍    | 1498/2757 [3:24:01<2:47:40,  7.99s/it]
 54%|█████▍    | 1499/2757 [3:24:09<2:47:46,  8.00s/it]
 54%|█████▍    | 1500/2757 [3:24:17<2:49:38,  8.10s/it]
                                                       

 54%|█████▍    | 1500/2757 [3:24:17<2:49:38,  8.10s/it]
{'loss': '0.3822', 'grad_norm': '0.6954', 'learning_rate': '0.000112', 'rewards/chosen': '-1.646', 'rewards/rejected': '-3.055', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.409', 'logps/chosen': '-94.91', 'logps/rejected': '-116.6', 'logits/chosen': '-3.762', 'logits/rejected': '-3.743', 'epoch': '1.371'}
{'loss': '0.3603', 'grad_norm': '0.5787', 'learning_rate': '0.0001113', 'rewards/chosen': '-1.818', 'rewards/rejected': '-3.225', 'rewards/accuracies': '0.8047', 'rewards/margins': '1.406', 'logps/chosen': '-98.85', 'logps/rejected': '-118.1', 'logits/chosen': '-3.881', 'logits/rejected': '-3.811', 'epoch': '1.382'}
{'loss': '0.3662', 'grad_norm': '0.668', 'learning_rate': '0.0001105', 'rewards/chosen': '-1.788', 'rewards/rejected': '-3.26', 'rewards/accuracies': '0.8039', 'rewards/margins': '1.471', 'logps/chosen': '-100.2', 'logps/rejected': '-120.3', 'logits/chosen': '-3.82', 'logits/rejected': '-3.791', 'epoch': '1.393'}
{'loss': '0.3878', 'grad_norm': '0.6281', 'learning_rate': '0.0001098', 'rewards/chosen': '-1.594', 'rewards/rejected': '-3.012', 'rewards/accuracies': '0.7703', 'rewards/margins': '1.418', 'logps/chosen': '-95.59', 'logps/rejected': '-115.6', 'logits/chosen': '-3.73', 'logits/rejected': '-3.709', 'epoch': '1.404'}
{'loss': '0.3713', 'grad_norm': '0.6394', 'learning_rate': '0.0001091', 'rewards/chosen': '-1.553', 'rewards/rejected': '-2.992', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.439', 'logps/chosen': '-97.09', 'logps/rejected': '-116', 'logits/chosen': '-3.678', 'logits/rejected': '-3.603', 'epoch': '1.415'}
{'loss': '0.3837', 'grad_norm': '0.5887', 'learning_rate': '0.0001083', 'rewards/chosen': '-1.771', 'rewards/rejected': '-3.158', 'rewards/accuracies': '0.7773', 'rewards/margins': '1.388', 'logps/chosen': '-97.09', 'logps/rejected': '-115', 'logits/chosen': '-3.858', 'logits/rejected': '-3.729', 'epoch': '1.426'}
{'loss': '0.3637', 'grad_norm': '0.6475', 'learning_rate': '0.0001076', 'rewards/chosen': '-1.754', 'rewards/rejected': '-3.263', 'rewards/accuracies': '0.7883', 'rewards/margins': '1.509', 'logps/chosen': '-97.05', 'logps/rejected': '-119.6', 'logits/chosen': '-3.94', 'logits/rejected': '-3.782', 'epoch': '1.436'}
{'loss': '0.3758', 'grad_norm': '0.6646', 'learning_rate': '0.0001068', 'rewards/chosen': '-1.683', 'rewards/rejected': '-3.217', 'rewards/accuracies': '0.7719', 'rewards/margins': '1.534', 'logps/chosen': '-96.41', 'logps/rejected': '-117.3', 'logits/chosen': '-3.977', 'logits/rejected': '-3.893', 'epoch': '1.447'}
{'loss': '0.3911', 'grad_norm': '0.7959', 'learning_rate': '0.0001061', 'rewards/chosen': '-1.504', 'rewards/rejected': '-2.96', 'rewards/accuracies': '0.7672', 'rewards/margins': '1.456', 'logps/chosen': '-92.38', 'logps/rejected': '-112.3', 'logits/chosen': '-4.019', 'logits/rejected': '-3.9', 'epoch': '1.458'}
{'loss': '0.3842', 'grad_norm': '0.6806', 'learning_rate': '0.0001053', 'rewards/chosen': '-1.673', 'rewards/rejected': '-3.111', 'rewards/accuracies': '0.7758', 'rewards/margins': '1.438', 'logps/chosen': '-101.7', 'logps/rejected': '-121', 'logits/chosen': '-3.86', 'logits/rejected': '-3.82', 'epoch': '1.469'}
{'loss': '0.3684', 'grad_norm': '0.6531', 'learning_rate': '0.0001046', 'rewards/chosen': '-1.91', 'rewards/rejected': '-3.38', 'rewards/accuracies': '0.7852', 'rewards/margins': '1.47', 'logps/chosen': '-100.1', 'logps/rejected': '-119.4', 'logits/chosen': '-3.826', 'logits/rejected': '-3.773', 'epoch': '1.48'}
{'loss': '0.3801', 'grad_norm': '0.6556', 'learning_rate': '0.0001038', 'rewards/chosen': '-1.97', 'rewards/rejected': '-3.358', 'rewards/accuracies': '0.7648', 'rewards/margins': '1.388', 'logps/chosen': '-101.1', 'logps/rejected': '-120.3', 'logits/chosen': '-3.857', 'logits/rejected': '-3.755', 'epoch': '1.491'}
{'loss': '0.37', 'grad_norm': '0.5829', 'learning_rate': '0.0001031', 'rewards/chosen': '-1.854', 'rewards/rejected': '-3.286', 'rewards/accuracies': '0.8031', 'rewards/margins': '1.432', 'logps/chosen': '-99.89', 'logps/rejected': '-118.7', 'logits/chosen': '-3.933', 'logits/rejected': '-3.822', 'epoch': '1.502'}
{'loss': '0.3734', 'grad_norm': '0.7183', 'learning_rate': '0.0001023', 'rewards/chosen': '-1.893', 'rewards/rejected': '-3.374', 'rewards/accuracies': '0.7828', 'rewards/margins': '1.481', 'logps/chosen': '-97.94', 'logps/rejected': '-117.8', 'logits/chosen': '-4.136', 'logits/rejected': '-3.982', 'epoch': '1.513'}
{'loss': '0.3624', 'grad_norm': '0.6229', 'learning_rate': '0.0001016', 'rewards/chosen': '-1.698', 'rewards/rejected': '-3.211', 'rewards/accuracies': '0.7875', 'rewards/margins': '1.513', 'logps/chosen': '-101.3', 'logps/rejected': '-122.4', 'logits/chosen': '-3.943', 'logits/rejected': '-3.885', 'epoch': '1.524'}
{'loss': '0.3612', 'grad_norm': '0.7118', 'learning_rate': '0.0001008', 'rewards/chosen': '-1.797', 'rewards/rejected': '-3.256', 'rewards/accuracies': '0.7961', 'rewards/margins': '1.459', 'logps/chosen': '-96.92', 'logps/rejected': '-115.9', 'logits/chosen': '-4.146', 'logits/rejected': '-4.064', 'epoch': '1.534'}
{'loss': '0.3773', 'grad_norm': '0.4945', 'learning_rate': '0.0001001', 'rewards/chosen': '-1.945', 'rewards/rejected': '-3.407', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.461', 'logps/chosen': '-100.1', 'logps/rejected': '-120.7', 'logits/chosen': '-4.004', 'logits/rejected': '-3.955', 'epoch': '1.545'}
{'loss': '0.3719', 'grad_norm': '0.5279', 'learning_rate': '9.933e-05', 'rewards/chosen': '-1.995', 'rewards/rejected': '-3.502', 'rewards/accuracies': '0.7797', 'rewards/margins': '1.507', 'logps/chosen': '-102.5', 'logps/rejected': '-122.1', 'logits/chosen': '-3.986', 'logits/rejected': '-3.933', 'epoch': '1.556'}
{'loss': '0.3743', 'grad_norm': '0.5255', 'learning_rate': '9.858e-05', 'rewards/chosen': '-1.921', 'rewards/rejected': '-3.381', 'rewards/accuracies': '0.7891', 'rewards/margins': '1.461', 'logps/chosen': '-100.2', 'logps/rejected': '-119.8', 'logits/chosen': '-3.869', 'logits/rejected': '-3.812', 'epoch': '1.567'}
{'loss': '0.3566', 'grad_norm': '0.6176', 'learning_rate': '9.783e-05', 'rewards/chosen': '-1.973', 'rewards/rejected': '-3.487', 'rewards/accuracies': '0.8', 'rewards/margins': '1.513', 'logps/chosen': '-101.5', 'logps/rejected': '-121.8', 'logits/chosen': '-3.869', 'logits/rejected': '-3.742', 'epoch': '1.578'}
{'loss': '0.3684', 'grad_norm': '0.6701', 'learning_rate': '9.708e-05', 'rewards/chosen': '-2.017', 'rewards/rejected': '-3.525', 'rewards/accuracies': '0.7781', 'rewards/margins': '1.508', 'logps/chosen': '-97.51', 'logps/rejected': '-118.8', 'logits/chosen': '-3.919', 'logits/rejected': '-3.9', 'epoch': '1.589'}
{'loss': '0.369', 'grad_norm': '0.8422', 'learning_rate': '9.634e-05', 'rewards/chosen': '-1.953', 'rewards/rejected': '-3.507', 'rewards/accuracies': '0.7836', 'rewards/margins': '1.554', 'logps/chosen': '-103.5', 'logps/rejected': '-125.3', 'logits/chosen': '-3.815', 'logits/rejected': '-3.763', 'epoch': '1.6'}
{'loss': '0.3787', 'grad_norm': '0.626', 'learning_rate': '9.559e-05', 'rewards/chosen': '-1.782', 'rewards/rejected': '-3.268', 'rewards/accuracies': '0.7883', 'rewards/margins': '1.486', 'logps/chosen': '-98.55', 'logps/rejected': '-118.6', 'logits/chosen': '-3.939', 'logits/rejected': '-3.889', 'epoch': '1.611'}
{'loss': '0.3668', 'grad_norm': '0.6023', 'learning_rate': '9.484e-05', 'rewards/chosen': '-1.582', 'rewards/rejected': '-3.109', 'rewards/accuracies': '0.7812', 'rewards/margins': '1.527', 'logps/chosen': '-96.4', 'logps/rejected': '-115.9', 'logits/chosen': '-4.026', 'logits/rejected': '-3.953', 'epoch': '1.621'}
{'loss': '0.3673', 'grad_norm': '0.5291', 'learning_rate': '9.409e-05', 'rewards/chosen': '-1.715', 'rewards/rejected': '-3.256', 'rewards/accuracies': '0.7898', 'rewards/margins': '1.542', 'logps/chosen': '-96.86', 'logps/rejected': '-118.7', 'logits/chosen': '-3.99', 'logits/rejected': '-3.968', 'epoch': '1.632'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)


  0%|          | 0/300 [00:00<?, ?it/s]

  1%|          | 2/300 [00:00<00:52,  5.73it/s]

  1%|          | 3/300 [00:00<01:47,  2.76it/s]

  1%|▏         | 4/300 [00:01<01:35,  3.09it/s]

  2%|▏         | 5/300 [00:01<01:45,  2.80it/s]

  2%|▏         | 6/300 [00:01<01:40,  2.91it/s]

  2%|▏         | 7/300 [00:02<01:55,  2.54it/s]

  3%|▎         | 8/300 [00:02<01:51,  2.62it/s]

  3%|▎         | 9/300 [00:03<01:52,  2.58it/s]

  3%|▎         | 10/300 [00:03<01:51,  2.60it/s]

  4%|▎         | 11/300 [00:04<01:53,  2.56it/s]

  4%|▍         | 12/300 [00:04<01:39,  2.88it/s]

  4%|▍         | 13/300 [00:04<01:45,  2.73it/s]

  5%|▍         | 14/300 [00:04<01:37,  2.94it/s]

  5%|▌         | 15/300 [00:05<01:47,  2.66it/s]

  5%|▌         | 16/300 [00:05<01:35,  2.99it/s]

  6%|▌         | 17/300 [00:06<01:38,  2.86it/s]

  6%|▌         | 18/300 [00:06<01:29,  3.17it/s]

  6%|▋         | 19/300 [00:06<01:44,  2.69it/s]

  7%|▋         | 20/300 [00:07<01:35,  2.92it/s]

  7%|▋         | 21/300 [00:07<01:47,  2.59it/s]

  7%|▋         | 22/300 [00:07<01:35,  2.92it/s]

  8%|▊         | 23/300 [00:08<01:35,  2.90it/s]

  8%|▊         | 24/300 [00:08<01:25,  3.22it/s]

  8%|▊         | 25/300 [00:08<01:37,  2.82it/s]

  9%|▊         | 26/300 [00:09<01:30,  3.03it/s]

  9%|▉         | 27/300 [00:10<02:19,  1.95it/s]

  9%|▉         | 28/300 [00:10<01:57,  2.32it/s]

 10%|▉         | 29/300 [00:10<02:04,  2.17it/s]

 10%|█         | 30/300 [00:11<01:47,  2.52it/s]

 10%|█         | 31/300 [00:11<01:55,  2.34it/s]

 11%|█         | 32/300 [00:11<01:55,  2.31it/s]

 11%|█         | 33/300 [00:12<01:46,  2.50it/s]

 11%|█▏        | 34/300 [00:12<01:34,  2.80it/s]

 12%|█▏        | 35/300 [00:12<01:37,  2.72it/s]

 12%|█▏        | 36/300 [00:13<01:28,  2.97it/s]

 12%|█▏        | 37/300 [00:13<01:39,  2.63it/s]

 13%|█▎        | 38/300 [00:13<01:27,  3.00it/s]

 13%|█▎        | 39/300 [00:14<01:36,  2.70it/s]

 13%|█▎        | 40/300 [00:14<01:25,  3.04it/s]

 14%|█▎        | 41/300 [00:15<01:35,  2.71it/s]

 14%|█▍        | 42/300 [00:15<01:24,  3.05it/s]

 14%|█▍        | 43/300 [00:15<01:27,  2.95it/s]

 15%|█▍        | 44/300 [00:15<01:19,  3.24it/s]

 15%|█▌        | 45/300 [00:16<01:25,  2.99it/s]

 15%|█▌        | 46/300 [00:16<01:25,  2.97it/s]

 16%|█▌        | 47/300 [00:17<01:30,  2.78it/s]

 16%|█▌        | 48/300 [00:17<01:23,  3.02it/s]

 16%|█▋        | 49/300 [00:17<01:23,  3.00it/s]

 17%|█▋        | 50/300 [00:17<01:15,  3.33it/s]

 17%|█▋        | 51/300 [00:18<01:18,  3.18it/s]

 17%|█▋        | 52/300 [00:18<01:15,  3.29it/s]

 18%|█▊        | 53/300 [00:18<01:28,  2.79it/s]

 18%|█▊        | 54/300 [00:19<01:19,  3.09it/s]

 18%|█▊        | 55/300 [00:19<01:27,  2.80it/s]

 19%|█▊        | 56/300 [00:19<01:15,  3.21it/s]

 19%|█▉        | 57/300 [00:20<01:29,  2.72it/s]

 19%|█▉        | 58/300 [00:20<01:28,  2.73it/s]

 20%|█▉        | 59/300 [00:21<01:31,  2.62it/s]

 20%|██        | 60/300 [00:21<01:21,  2.95it/s]

 20%|██        | 61/300 [00:21<01:23,  2.85it/s]

 21%|██        | 62/300 [00:22<01:22,  2.88it/s]

 21%|██        | 63/300 [00:22<01:27,  2.70it/s]

 21%|██▏       | 64/300 [00:22<01:25,  2.76it/s]

 22%|██▏       | 65/300 [00:23<01:28,  2.66it/s]

 22%|██▏       | 66/300 [00:23<01:20,  2.92it/s]

 22%|██▏       | 67/300 [00:24<01:27,  2.66it/s]

 23%|██▎       | 68/300 [00:24<01:18,  2.95it/s]

 23%|██▎       | 69/300 [00:24<01:21,  2.84it/s]

 23%|██▎       | 70/300 [00:24<01:13,  3.14it/s]

 24%|██▎       | 71/300 [00:25<01:15,  3.02it/s]

 24%|██▍       | 72/300 [00:25<01:09,  3.28it/s]

 24%|██▍       | 73/300 [00:25<01:12,  3.14it/s]

 25%|██▍       | 74/300 [00:26<01:08,  3.32it/s]

 25%|██▌       | 75/300 [00:26<01:13,  3.08it/s]

 25%|██▌       | 76/300 [00:26<01:14,  3.01it/s]

 26%|██▌       | 77/300 [00:27<01:19,  2.81it/s]

 26%|██▌       | 78/300 [00:27<01:12,  3.05it/s]

 26%|██▋       | 79/300 [00:27<01:15,  2.94it/s]

 27%|██▋       | 80/300 [00:28<01:11,  3.06it/s]

 27%|██▋       | 81/300 [00:28<01:20,  2.74it/s]

 27%|██▋       | 82/300 [00:28<01:11,  3.04it/s]

 28%|██▊       | 83/300 [00:29<01:15,  2.88it/s]

 28%|██▊       | 84/300 [00:29<01:15,  2.86it/s]

 28%|██▊       | 85/300 [00:30<01:19,  2.70it/s]

 29%|██▊       | 86/300 [00:30<01:11,  3.00it/s]

 29%|██▉       | 87/300 [00:30<01:13,  2.91it/s]

 29%|██▉       | 88/300 [00:30<01:07,  3.16it/s]

 30%|██▉       | 89/300 [00:31<01:11,  2.96it/s]

 30%|███       | 90/300 [00:31<01:05,  3.19it/s]

 30%|███       | 91/300 [00:31<01:09,  3.00it/s]

 31%|███       | 92/300 [00:32<01:03,  3.27it/s]

 31%|███       | 93/300 [00:32<01:09,  2.98it/s]

 31%|███▏      | 94/300 [00:32<01:02,  3.28it/s]

 32%|███▏      | 95/300 [00:33<01:08,  3.01it/s]

 32%|███▏      | 96/300 [00:33<01:03,  3.20it/s]

 32%|███▏      | 97/300 [00:33<01:15,  2.67it/s]

 33%|███▎      | 98/300 [00:34<01:09,  2.92it/s]

 33%|███▎      | 99/300 [00:34<01:12,  2.78it/s]

 33%|███▎      | 100/300 [00:34<01:06,  2.99it/s]

 34%|███▎      | 101/300 [00:35<01:09,  2.86it/s]

 34%|███▍      | 102/300 [00:35<01:07,  2.94it/s]

 34%|███▍      | 103/300 [00:36<01:10,  2.79it/s]

 35%|███▍      | 104/300 [00:36<01:02,  3.11it/s]

 35%|███▌      | 105/300 [00:36<01:05,  2.96it/s]

 35%|███▌      | 106/300 [00:36<01:01,  3.17it/s]

 36%|███▌      | 107/300 [00:37<01:05,  2.95it/s]

 36%|███▌      | 108/300 [00:37<01:03,  3.05it/s]

 36%|███▋      | 109/300 [00:38<01:06,  2.86it/s]

 37%|███▋      | 110/300 [00:38<01:02,  3.03it/s]

 37%|███▋      | 111/300 [00:38<01:11,  2.65it/s]

 37%|███▋      | 112/300 [00:39<01:06,  2.84it/s]

 38%|███▊      | 113/300 [00:39<01:09,  2.70it/s]

 38%|███▊      | 114/300 [00:39<01:07,  2.74it/s]

 38%|███▊      | 115/300 [00:40<01:10,  2.62it/s]

 39%|███▊      | 116/300 [00:40<01:02,  2.95it/s]

 39%|███▉      | 117/300 [00:40<01:03,  2.88it/s]

 39%|███▉      | 118/300 [00:41<00:58,  3.10it/s]

 40%|███▉      | 119/300 [00:41<01:00,  2.98it/s]

 40%|████      | 120/300 [00:41<00:59,  3.01it/s]

 40%|████      | 121/300 [00:42<01:02,  2.89it/s]

 41%|████      | 122/300 [00:42<00:57,  3.11it/s]

 41%|████      | 123/300 [00:42<01:02,  2.84it/s]

 41%|████▏     | 124/300 [00:43<00:58,  3.02it/s]

 42%|████▏     | 125/300 [00:43<00:59,  2.92it/s]

 42%|████▏     | 126/300 [00:43<01:00,  2.90it/s]

 42%|████▏     | 127/300 [00:44<01:02,  2.78it/s]

 43%|████▎     | 128/300 [00:44<00:57,  2.98it/s]

 43%|████▎     | 129/300 [00:44<00:59,  2.86it/s]

 43%|████▎     | 130/300 [00:45<00:54,  3.14it/s]

 44%|████▎     | 131/300 [00:45<01:00,  2.78it/s]

 44%|████▍     | 132/300 [00:45<00:55,  3.02it/s]

 44%|████▍     | 133/300 [00:46<01:02,  2.67it/s]

 45%|████▍     | 134/300 [00:46<00:56,  2.92it/s]

 45%|████▌     | 135/300 [00:47<00:58,  2.80it/s]

 45%|████▌     | 136/300 [00:47<00:53,  3.09it/s]

 46%|████▌     | 137/300 [00:47<00:53,  3.02it/s]

 46%|████▌     | 138/300 [00:47<00:50,  3.23it/s]

 46%|████▋     | 139/300 [00:48<00:51,  3.16it/s]

 47%|████▋     | 140/300 [00:48<00:46,  3.42it/s]

 47%|████▋     | 141/300 [00:48<00:48,  3.26it/s]

 47%|████▋     | 142/300 [00:49<00:50,  3.13it/s]

 48%|████▊     | 143/300 [00:49<00:54,  2.88it/s]

 48%|████▊     | 144/300 [00:49<00:48,  3.19it/s]

 48%|████▊     | 145/300 [00:50<00:49,  3.14it/s]

 49%|████▊     | 146/300 [00:50<00:46,  3.29it/s]

 49%|████▉     | 147/300 [00:50<00:53,  2.86it/s]

 49%|████▉     | 148/300 [00:51<00:50,  3.03it/s]

 50%|████▉     | 149/300 [00:51<00:53,  2.80it/s]

 50%|█████     | 150/300 [00:51<00:49,  3.00it/s]

 50%|█████     | 151/300 [00:52<00:51,  2.88it/s]

 51%|█████     | 152/300 [00:52<00:46,  3.19it/s]

 51%|█████     | 153/300 [00:52<00:47,  3.08it/s]

 51%|█████▏    | 154/300 [00:53<00:43,  3.36it/s]

 52%|█████▏    | 155/300 [00:53<00:45,  3.17it/s]

 52%|█████▏    | 156/300 [00:53<00:41,  3.49it/s]

 52%|█████▏    | 157/300 [00:54<00:45,  3.13it/s]

 53%|█████▎    | 158/300 [00:54<00:43,  3.27it/s]

 53%|█████▎    | 159/300 [00:54<00:53,  2.66it/s]

 53%|█████▎    | 160/300 [00:55<00:48,  2.91it/s]

 54%|█████▎    | 161/300 [00:55<00:51,  2.71it/s]

 54%|█████▍    | 162/300 [00:55<00:50,  2.75it/s]

 54%|█████▍    | 163/300 [00:56<00:56,  2.41it/s]

 55%|█████▍    | 164/300 [00:56<00:53,  2.53it/s]

 55%|█████▌    | 165/300 [00:57<00:53,  2.54it/s]

 55%|█████▌    | 166/300 [00:57<00:50,  2.65it/s]

 56%|█████▌    | 167/300 [00:57<00:54,  2.43it/s]

 56%|█████▌    | 168/300 [00:58<00:48,  2.74it/s]

 56%|█████▋    | 169/300 [00:58<00:52,  2.48it/s]

 57%|█████▋    | 170/300 [00:59<00:50,  2.58it/s]

 57%|█████▋    | 171/300 [00:59<00:50,  2.54it/s]

 57%|█████▋    | 172/300 [00:59<00:44,  2.88it/s]

 58%|█████▊    | 173/300 [01:00<00:46,  2.72it/s]

 58%|█████▊    | 174/300 [01:00<00:41,  3.05it/s]

 58%|█████▊    | 175/300 [01:00<00:44,  2.79it/s]

 59%|█████▊    | 176/300 [01:01<00:39,  3.11it/s]

 59%|█████▉    | 177/300 [01:01<00:46,  2.65it/s]

 59%|█████▉    | 178/300 [01:01<00:39,  3.08it/s]

 60%|█████▉    | 179/300 [01:02<00:42,  2.88it/s]

 60%|██████    | 180/300 [01:02<00:38,  3.16it/s]

 60%|██████    | 181/300 [01:02<00:41,  2.85it/s]

 61%|██████    | 182/300 [01:03<00:37,  3.15it/s]

 61%|██████    | 183/300 [01:03<00:40,  2.89it/s]

 61%|██████▏   | 184/300 [01:03<00:40,  2.88it/s]

 62%|██████▏   | 185/300 [01:04<00:48,  2.38it/s]

 62%|██████▏   | 186/300 [01:04<00:43,  2.63it/s]

 62%|██████▏   | 187/300 [01:05<00:42,  2.68it/s]

 63%|██████▎   | 188/300 [01:05<00:37,  2.99it/s]

 63%|██████▎   | 189/300 [01:05<00:40,  2.76it/s]

 63%|██████▎   | 190/300 [01:05<00:36,  3.02it/s]

 64%|██████▎   | 191/300 [01:06<00:39,  2.77it/s]

 64%|██████▍   | 192/300 [01:06<00:38,  2.84it/s]

 64%|██████▍   | 193/300 [01:07<00:42,  2.51it/s]

 65%|██████▍   | 194/300 [01:07<00:40,  2.61it/s]

 65%|██████▌   | 195/300 [01:08<00:43,  2.42it/s]

 65%|██████▌   | 196/300 [01:08<00:38,  2.67it/s]

 66%|██████▌   | 197/300 [01:08<00:38,  2.67it/s]

 66%|██████▌   | 198/300 [01:09<00:35,  2.91it/s]

 66%|██████▋   | 199/300 [01:09<00:36,  2.81it/s]

 67%|██████▋   | 200/300 [01:09<00:33,  2.99it/s]

 67%|██████▋   | 201/300 [01:10<00:35,  2.79it/s]

 67%|██████▋   | 202/300 [01:10<00:31,  3.08it/s]

 68%|██████▊   | 203/300 [01:10<00:33,  2.94it/s]

 68%|██████▊   | 204/300 [01:10<00:29,  3.23it/s]

 68%|██████▊   | 205/300 [01:11<00:31,  3.01it/s]

 69%|██████▊   | 206/300 [01:11<00:27,  3.40it/s]

 69%|██████▉   | 207/300 [01:12<00:31,  2.93it/s]

 69%|██████▉   | 208/300 [01:12<00:31,  2.94it/s]

 70%|██████▉   | 209/300 [01:12<00:32,  2.80it/s]

 70%|███████   | 210/300 [01:13<00:29,  3.01it/s]

 70%|███████   | 211/300 [01:13<00:33,  2.66it/s]

 71%|███████   | 212/300 [01:13<00:29,  2.98it/s]

 71%|███████   | 213/300 [01:14<00:31,  2.74it/s]

 71%|███████▏  | 214/300 [01:14<00:28,  3.00it/s]

 72%|███████▏  | 215/300 [01:14<00:30,  2.77it/s]

 72%|███████▏  | 216/300 [01:15<00:27,  3.07it/s]

 72%|███████▏  | 217/300 [01:15<00:30,  2.75it/s]

 73%|███████▎  | 218/300 [01:15<00:25,  3.17it/s]

 73%|███████▎  | 219/300 [01:16<00:28,  2.87it/s]

 73%|███████▎  | 220/300 [01:16<00:25,  3.15it/s]

 74%|███████▎  | 221/300 [01:16<00:28,  2.79it/s]

 74%|███████▍  | 222/300 [01:17<00:25,  3.09it/s]

 74%|███████▍  | 223/300 [01:17<00:28,  2.75it/s]

 75%|███████▍  | 224/300 [01:17<00:25,  2.95it/s]

 75%|███████▌  | 225/300 [01:18<00:26,  2.82it/s]

 75%|███████▌  | 226/300 [01:18<00:24,  3.05it/s]

 76%|███████▌  | 227/300 [01:18<00:26,  2.79it/s]

 76%|███████▌  | 228/300 [01:19<00:23,  3.12it/s]

 76%|███████▋  | 229/300 [01:19<00:25,  2.76it/s]

 77%|███████▋  | 230/300 [01:19<00:24,  2.86it/s]

 77%|███████▋  | 231/300 [01:20<00:26,  2.59it/s]

 77%|███████▋  | 232/300 [01:20<00:25,  2.69it/s]

 78%|███████▊  | 233/300 [01:21<00:27,  2.42it/s]

 78%|███████▊  | 234/300 [01:21<00:23,  2.78it/s]

 78%|███████▊  | 235/300 [01:21<00:25,  2.55it/s]

 79%|███████▊  | 236/300 [01:22<00:24,  2.64it/s]

 79%|███████▉  | 237/300 [01:22<00:24,  2.56it/s]

 79%|███████▉  | 238/300 [01:22<00:21,  2.88it/s]

 80%|███████▉  | 239/300 [01:23<00:21,  2.87it/s]

 80%|████████  | 240/300 [01:23<00:18,  3.18it/s]

 80%|████████  | 241/300 [01:23<00:19,  3.07it/s]

 81%|████████  | 242/300 [01:24<00:18,  3.10it/s]

 81%|████████  | 243/300 [01:24<00:20,  2.74it/s]

 81%|████████▏ | 244/300 [01:25<00:19,  2.84it/s]

 82%|████████▏ | 245/300 [01:25<00:20,  2.67it/s]

 82%|████████▏ | 246/300 [01:25<00:19,  2.76it/s]

 82%|████████▏ | 247/300 [01:26<00:20,  2.59it/s]

 83%|████████▎ | 248/300 [01:26<00:17,  2.92it/s]

 83%|████████▎ | 249/300 [01:26<00:18,  2.78it/s]

 83%|████████▎ | 250/300 [01:27<00:17,  2.90it/s]

 84%|████████▎ | 251/300 [01:27<00:17,  2.80it/s]

 84%|████████▍ | 252/300 [01:27<00:16,  2.99it/s]

 84%|████████▍ | 253/300 [01:28<00:16,  2.83it/s]

 85%|████████▍ | 254/300 [01:28<00:14,  3.20it/s]

 85%|████████▌ | 255/300 [01:28<00:15,  2.86it/s]

 85%|████████▌ | 256/300 [01:29<00:13,  3.18it/s]

 86%|████████▌ | 257/300 [01:29<00:14,  2.97it/s]

 86%|████████▌ | 258/300 [01:29<00:12,  3.24it/s]

 86%|████████▋ | 259/300 [01:30<00:13,  2.99it/s]

 87%|████████▋ | 260/300 [01:30<00:12,  3.17it/s]

 87%|████████▋ | 261/300 [01:30<00:13,  3.00it/s]

 87%|████████▋ | 262/300 [01:31<00:11,  3.29it/s]

 88%|████████▊ | 263/300 [01:31<00:12,  3.02it/s]

 88%|████████▊ | 264/300 [01:31<00:12,  2.99it/s]

 88%|████████▊ | 265/300 [01:32<00:13,  2.58it/s]

 89%|████████▊ | 266/300 [01:32<00:12,  2.68it/s]

 89%|████████▉ | 267/300 [01:33<00:13,  2.42it/s]

 89%|████████▉ | 268/300 [01:33<00:11,  2.78it/s]

 90%|████████▉ | 269/300 [01:33<00:12,  2.46it/s]

 90%|█████████ | 270/300 [01:34<00:11,  2.63it/s]

 90%|█████████ | 271/300 [01:34<00:12,  2.37it/s]

 91%|█████████ | 272/300 [01:35<00:10,  2.64it/s]

 91%|█████████ | 273/300 [01:35<00:11,  2.29it/s]

 91%|█████████▏| 274/300 [01:35<00:09,  2.63it/s]

 92%|█████████▏| 275/300 [01:36<00:10,  2.40it/s]

 92%|█████████▏| 276/300 [01:36<00:09,  2.59it/s]

 92%|█████████▏| 277/300 [01:37<00:09,  2.48it/s]

 93%|█████████▎| 278/300 [01:37<00:07,  2.81it/s]

 93%|█████████▎| 279/300 [01:37<00:07,  2.70it/s]

 93%|█████████▎| 280/300 [01:38<00:07,  2.82it/s]

 94%|█████████▎| 281/300 [01:38<00:07,  2.67it/s]

 94%|█████████▍| 282/300 [01:38<00:06,  2.99it/s]

 94%|█████████▍| 283/300 [01:39<00:06,  2.78it/s]

 95%|█████████▍| 284/300 [01:39<00:05,  3.06it/s]

 95%|█████████▌| 285/300 [01:39<00:05,  2.65it/s]

 95%|█████████▌| 286/300 [01:40<00:05,  2.71it/s]

 96%|█████████▌| 287/300 [01:40<00:04,  2.64it/s]

 96%|█████████▌| 288/300 [01:40<00:03,  3.06it/s]

 96%|█████████▋| 289/300 [01:41<00:03,  2.80it/s]

 97%|█████████▋| 290/300 [01:41<00:03,  2.82it/s]

 97%|█████████▋| 291/300 [01:42<00:03,  2.56it/s]

 97%|█████████▋| 292/300 [01:42<00:02,  2.89it/s]

 98%|█████████▊| 293/300 [01:42<00:02,  2.75it/s]

 98%|█████████▊| 294/300 [01:42<00:02,  2.99it/s]

 98%|█████████▊| 295/300 [01:43<00:01,  2.94it/s]

 99%|█████████▊| 296/300 [01:43<00:01,  3.14it/s]

 99%|█████████▉| 297/300 [01:44<00:01,  2.90it/s]

 99%|█████████▉| 298/300 [01:44<00:00,  3.24it/s]

100%|█████████▉| 299/300 [01:44<00:00,  2.85it/s]

100%|██████████| 300/300 [01:45<00:00,  2.89it/s]
                                                       

                                                 

 54%|█████▍    | 1500/2757 [3:26:03<2:49:38,  8.10s/it]

100%|██████████| 300/300 [01:45<00:00,  2.89it/s]

                                                 Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-1500/tokenizer_config.json.

 54%|█████▍    | 1501/2757 [3:26:15<14:22:18, 41.19s/it]
 54%|█████▍    | 1502/2757 [3:26:23<10:53:54, 31.26s/it]
 55%|█████▍    | 1503/2757 [3:26:32<8:28:57, 24.35s/it] 
 55%|█████▍    | 1504/2757 [3:26:40<6:45:34, 19.42s/it]
 55%|█████▍    | 1505/2757 [3:26:48<5:34:40, 16.04s/it]
 55%|█████▍    | 1506/2757 [3:26:56<4:45:26, 13.69s/it]
 55%|█████▍    | 1507/2757 [3:27:04<4:08:18, 11.92s/it]
 55%|█████▍    | 1508/2757 [3:27:12<3:42:04, 10.67s/it]
 55%|█████▍    | 1509/2757 [3:27:20<3:29:03, 10.05s/it]
 55%|█████▍    | 1510/2757 [3:27:28<3:15:28,  9.41s/it]
                                                       

 55%|█████▍    | 1510/2757 [3:27:28<3:15:28,  9.41s/it]
 55%|█████▍    | 1511/2757 [3:27:36<3:06:48,  9.00s/it]
 55%|█████▍    | 1512/2757 [3:27:44<3:01:00,  8.72s/it]
 55%|█████▍    | 1513/2757 [3:27:52<2:56:43,  8.52s/it]
 55%|█████▍    | 1514/2757 [3:28:00<2:53:37,  8.38s/it]
 55%|█████▍    | 1515/2757 [3:28:09<2:52:54,  8.35s/it]
 55%|█████▍    | 1516/2757 [3:28:16<2:47:34,  8.10s/it]
 55%|█████▌    | 1517/2757 [3:28:24<2:45:43,  8.02s/it]
 55%|█████▌    | 1518/2757 [3:28:32<2:45:29,  8.01s/it]
 55%|█████▌    | 1519/2757 [3:28:40<2:46:13,  8.06s/it]
 55%|█████▌    | 1520/2757 [3:28:48<2:45:21,  8.02s/it]
                                                       

 55%|█████▌    | 1520/2757 [3:28:48<2:45:21,  8.02s/it]
 55%|█████▌    | 1521/2757 [3:28:56<2:44:21,  7.98s/it]
 55%|█████▌    | 1522/2757 [3:29:04<2:42:17,  7.88s/it]
 55%|█████▌    | 1523/2757 [3:29:11<2:42:34,  7.90s/it]
 55%|█████▌    | 1524/2757 [3:29:20<2:43:54,  7.98s/it]
 55%|█████▌    | 1525/2757 [3:29:27<2:42:59,  7.94s/it]
 55%|█████▌    | 1526/2757 [3:29:35<2:41:58,  7.89s/it]
 55%|█████▌    | 1527/2757 [3:29:43<2:43:24,  7.97s/it]
 55%|█████▌    | 1528/2757 [3:29:51<2:42:23,  7.93s/it]
 55%|█████▌    | 1529/2757 [3:29:59<2:42:14,  7.93s/it]
 55%|█████▌    | 1530/2757 [3:30:07<2:42:41,  7.96s/it]
                                                       

 55%|█████▌    | 1530/2757 [3:30:07<2:42:41,  7.96s/it]
 56%|█████▌    | 1531/2757 [3:30:15<2:43:13,  7.99s/it]
 56%|█████▌    | 1532/2757 [3:30:23<2:43:22,  8.00s/it]
 56%|█████▌    | 1533/2757 [3:30:31<2:44:05,  8.04s/it]
 56%|█████▌    | 1534/2757 [3:30:40<2:44:21,  8.06s/it]
 56%|█████▌    | 1535/2757 [3:30:47<2:40:01,  7.86s/it]
 56%|█████▌    | 1536/2757 [3:30:55<2:40:58,  7.91s/it]
 56%|█████▌    | 1537/2757 [3:31:02<2:38:28,  7.79s/it]
 56%|█████▌    | 1538/2757 [3:31:11<2:40:21,  7.89s/it]
 56%|█████▌    | 1539/2757 [3:31:19<2:41:45,  7.97s/it]
 56%|█████▌    | 1540/2757 [3:31:27<2:41:57,  7.98s/it]
                                                       

 56%|█████▌    | 1540/2757 [3:31:27<2:41:57,  7.98s/it]
 56%|█████▌    | 1541/2757 [3:31:34<2:40:00,  7.90s/it]
 56%|█████▌    | 1542/2757 [3:31:43<2:41:37,  7.98s/it]
 56%|█████▌    | 1543/2757 [3:31:51<2:44:50,  8.15s/it]
 56%|█████▌    | 1544/2757 [3:31:59<2:45:32,  8.19s/it]
 56%|█████▌    | 1545/2757 [3:32:07<2:43:52,  8.11s/it]
 56%|█████▌    | 1546/2757 [3:32:15<2:40:52,  7.97s/it]
 56%|█████▌    | 1547/2757 [3:32:23<2:40:24,  7.95s/it]
 56%|█████▌    | 1548/2757 [3:32:31<2:40:04,  7.94s/it]
 56%|█████▌    | 1549/2757 [3:32:38<2:34:09,  7.66s/it]
 56%|█████▌    | 1550/2757 [3:32:46<2:39:44,  7.94s/it]
                                                       

 56%|█████▌    | 1550/2757 [3:32:46<2:39:44,  7.94s/it]
 56%|█████▋    | 1551/2757 [3:32:55<2:43:11,  8.12s/it]
 56%|█████▋    | 1552/2757 [3:33:03<2:44:55,  8.21s/it]
 56%|█████▋    | 1553/2757 [3:33:11<2:42:47,  8.11s/it]
 56%|█████▋    | 1554/2757 [3:33:20<2:43:39,  8.16s/it]
 56%|█████▋    | 1555/2757 [3:33:27<2:40:03,  7.99s/it]
 56%|█████▋    | 1556/2757 [3:33:35<2:41:31,  8.07s/it]
 56%|█████▋    | 1557/2757 [3:33:44<2:42:18,  8.12s/it]
 57%|█████▋    | 1558/2757 [3:33:52<2:43:52,  8.20s/it]
 57%|█████▋    | 1559/2757 [3:34:00<2:41:43,  8.10s/it]
 57%|█████▋    | 1560/2757 [3:34:08<2:38:52,  7.96s/it]
                                                       

 57%|█████▋    | 1560/2757 [3:34:08<2:38:52,  7.96s/it]
 57%|█████▋    | 1561/2757 [3:34:15<2:37:05,  7.88s/it]
 57%|█████▋    | 1562/2757 [3:34:23<2:37:48,  7.92s/it]
 57%|█████▋    | 1563/2757 [3:34:31<2:34:50,  7.78s/it]
 57%|█████▋    | 1564/2757 [3:34:39<2:36:01,  7.85s/it]
 57%|█████▋    | 1565/2757 [3:34:47<2:35:44,  7.84s/it]
 57%|█████▋    | 1566/2757 [3:34:54<2:35:34,  7.84s/it]
 57%|█████▋    | 1567/2757 [3:35:03<2:37:16,  7.93s/it]
 57%|█████▋    | 1568/2757 [3:35:10<2:36:32,  7.90s/it]
 57%|█████▋    | 1569/2757 [3:35:18<2:35:05,  7.83s/it]
 57%|█████▋    | 1570/2757 [3:35:26<2:37:06,  7.94s/it]
                                                       

 57%|█████▋    | 1570/2757 [3:35:26<2:37:06,  7.94s/it]
 57%|█████▋    | 1571/2757 [3:35:34<2:36:30,  7.92s/it]
 57%|█████▋    | 1572/2757 [3:35:42<2:33:50,  7.79s/it]
 57%|█████▋    | 1573/2757 [3:35:50<2:34:37,  7.84s/it]
 57%|█████▋    | 1574/2757 [3:35:58<2:37:04,  7.97s/it]
 57%|█████▋    | 1575/2757 [3:36:06<2:37:12,  7.98s/it]
 57%|█████▋    | 1576/2757 [3:36:14<2:36:36,  7.96s/it]
 57%|█████▋    | 1577/2757 [3:36:22<2:36:21,  7.95s/it]
 57%|█████▋    | 1578/2757 [3:36:29<2:35:30,  7.91s/it]
 57%|█████▋    | 1579/2757 [3:36:37<2:34:24,  7.86s/it]
 57%|█████▋    | 1580/2757 [3:36:45<2:33:15,  7.81s/it]
                                                       

 57%|█████▋    | 1580/2757 [3:36:45<2:33:15,  7.81s/it]
 57%|█████▋    | 1581/2757 [3:36:53<2:36:48,  8.00s/it]
 57%|█████▋    | 1582/2757 [3:37:01<2:35:05,  7.92s/it]
 57%|█████▋    | 1583/2757 [3:37:09<2:35:15,  7.93s/it]
 57%|█████▋    | 1584/2757 [3:37:17<2:37:33,  8.06s/it]
 57%|█████▋    | 1585/2757 [3:37:25<2:36:44,  8.02s/it]
 58%|█████▊    | 1586/2757 [3:37:33<2:35:44,  7.98s/it]
 58%|█████▊    | 1587/2757 [3:37:41<2:36:08,  8.01s/it]
 58%|█████▊    | 1588/2757 [3:37:49<2:35:20,  7.97s/it]
 58%|█████▊    | 1589/2757 [3:37:57<2:33:21,  7.88s/it]
 58%|█████▊    | 1590/2757 [3:38:04<2:31:41,  7.80s/it]
                                                       

 58%|█████▊    | 1590/2757 [3:38:04<2:31:41,  7.80s/it]
 58%|█████▊    | 1591/2757 [3:38:13<2:33:55,  7.92s/it]
 58%|█████▊    | 1592/2757 [3:38:21<2:35:36,  8.01s/it]
 58%|█████▊    | 1593/2757 [3:38:29<2:34:55,  7.99s/it]
 58%|█████▊    | 1594/2757 [3:38:37<2:36:52,  8.09s/it]
 58%|█████▊    | 1595/2757 [3:38:45<2:37:45,  8.15s/it]
 58%|█████▊    | 1596/2757 [3:38:53<2:35:29,  8.04s/it]
 58%|█████▊    | 1597/2757 [3:39:01<2:33:29,  7.94s/it]
 58%|█████▊    | 1598/2757 [3:39:09<2:33:27,  7.94s/it]
 58%|█████▊    | 1599/2757 [3:39:17<2:33:29,  7.95s/it]
 58%|█████▊    | 1600/2757 [3:39:25<2:35:19,  8.05s/it]
                                                       

 58%|█████▊    | 1600/2757 [3:39:25<2:35:19,  8.05s/it]
 58%|█████▊    | 1601/2757 [3:39:33<2:33:13,  7.95s/it]
 58%|█████▊    | 1602/2757 [3:39:41<2:34:49,  8.04s/it]
 58%|█████▊    | 1603/2757 [3:39:49<2:32:06,  7.91s/it]
 58%|█████▊    | 1604/2757 [3:39:56<2:28:41,  7.74s/it]
 58%|█████▊    | 1605/2757 [3:40:04<2:30:23,  7.83s/it]
 58%|█████▊    | 1606/2757 [3:40:12<2:29:21,  7.79s/it]
 58%|█████▊    | 1607/2757 [3:40:20<2:30:20,  7.84s/it]
 58%|█████▊    | 1608/2757 [3:40:28<2:33:01,  7.99s/it]
 58%|█████▊    | 1609/2757 [3:40:36<2:32:22,  7.96s/it]
 58%|█████▊    | 1610/2757 [3:40:44<2:31:20,  7.92s/it]
                                                       

 58%|█████▊    | 1610/2757 [3:40:44<2:31:20,  7.92s/it]
 58%|█████▊    | 1611/2757 [3:40:52<2:32:06,  7.96s/it]
 58%|█████▊    | 1612/2757 [3:41:00<2:30:30,  7.89s/it]
 59%|█████▊    | 1613/2757 [3:41:08<2:32:52,  8.02s/it]
 59%|█████▊    | 1614/2757 [3:41:16<2:35:14,  8.15s/it]
 59%|█████▊    | 1615/2757 [3:41:25<2:36:37,  8.23s/it]
 59%|█████▊    | 1616/2757 [3:41:33<2:34:48,  8.14s/it]
 59%|█████▊    | 1617/2757 [3:41:41<2:33:45,  8.09s/it]
 59%|█████▊    | 1618/2757 [3:41:49<2:32:13,  8.02s/it]
 59%|█████▊    | 1619/2757 [3:41:57<2:33:27,  8.09s/it]
 59%|█████▉    | 1620/2757 [3:42:04<2:30:09,  7.92s/it]
                                                       

 59%|█████▉    | 1620/2757 [3:42:04<2:30:09,  7.92s/it]
 59%|█████▉    | 1621/2757 [3:42:12<2:28:50,  7.86s/it]
 59%|█████▉    | 1622/2757 [3:42:20<2:28:14,  7.84s/it]
 59%|█████▉    | 1623/2757 [3:42:28<2:29:12,  7.89s/it]
 59%|█████▉    | 1624/2757 [3:42:36<2:29:59,  7.94s/it]
 59%|█████▉    | 1625/2757 [3:42:44<2:29:35,  7.93s/it]
 59%|█████▉    | 1626/2757 [3:42:52<2:32:18,  8.08s/it]
 59%|█████▉    | 1627/2757 [3:43:00<2:28:16,  7.87s/it]
 59%|█████▉    | 1628/2757 [3:43:07<2:26:55,  7.81s/it]
 59%|█████▉    | 1629/2757 [3:43:15<2:27:58,  7.87s/it]
 59%|█████▉    | 1630/2757 [3:43:23<2:29:21,  7.95s/it]
                                                       

 59%|█████▉    | 1630/2757 [3:43:23<2:29:21,  7.95s/it]
 59%|█████▉    | 1631/2757 [3:43:32<2:32:20,  8.12s/it]
 59%|█████▉    | 1632/2757 [3:43:40<2:33:26,  8.18s/it]
 59%|█████▉    | 1633/2757 [3:43:48<2:31:39,  8.10s/it]
 59%|█████▉    | 1634/2757 [3:43:56<2:29:50,  8.01s/it]
 59%|█████▉    | 1635/2757 [3:44:04<2:28:14,  7.93s/it]
 59%|█████▉    | 1636/2757 [3:44:12<2:29:27,  8.00s/it]
 59%|█████▉    | 1637/2757 [3:44:20<2:27:23,  7.90s/it]
 59%|█████▉    | 1638/2757 [3:44:28<2:28:46,  7.98s/it]
 59%|█████▉    | 1639/2757 [3:44:36<2:29:47,  8.04s/it]
 59%|█████▉    | 1640/2757 [3:44:44<2:29:50,  8.05s/it]
                                                       

 59%|█████▉    | 1640/2757 [3:44:44<2:29:50,  8.05s/it]
 60%|█████▉    | 1641/2757 [3:44:52<2:30:20,  8.08s/it]
 60%|█████▉    | 1642/2757 [3:45:00<2:30:49,  8.12s/it]
 60%|█████▉    | 1643/2757 [3:45:08<2:30:50,  8.12s/it]
 60%|█████▉    | 1644/2757 [3:45:16<2:29:15,  8.05s/it]
 60%|█████▉    | 1645/2757 [3:45:24<2:29:03,  8.04s/it]
 60%|█████▉    | 1646/2757 [3:45:32<2:29:28,  8.07s/it]
 60%|█████▉    | 1647/2757 [3:45:40<2:28:48,  8.04s/it]
 60%|█████▉    | 1648/2757 [3:45:48<2:26:24,  7.92s/it]
 60%|█████▉    | 1649/2757 [3:45:56<2:26:27,  7.93s/it]
 60%|█████▉    | 1650/2757 [3:46:04<2:26:23,  7.93s/it]
                                                       

 60%|█████▉    | 1650/2757 [3:46:04<2:26:23,  7.93s/it]
 60%|█████▉    | 1651/2757 [3:46:12<2:27:02,  7.98s/it]
 60%|█████▉    | 1652/2757 [3:46:20<2:28:14,  8.05s/it]
 60%|█████▉    | 1653/2757 [3:46:28<2:27:35,  8.02s/it]
 60%|█████▉    | 1654/2757 [3:46:36<2:28:01,  8.05s/it]
 60%|██████    | 1655/2757 [3:46:44<2:26:05,  7.95s/it]
 60%|██████    | 1656/2757 [3:46:52<2:27:05,  8.02s/it]
 60%|██████    | 1657/2757 [3:47:00<2:27:28,  8.04s/it]
 60%|██████    | 1658/2757 [3:47:08<2:25:39,  7.95s/it]
 60%|██████    | 1659/2757 [3:47:16<2:26:34,  8.01s/it]
 60%|██████    | 1660/2757 [3:47:24<2:27:34,  8.07s/it]
                                                       

 60%|██████    | 1660/2757 [3:47:24<2:27:34,  8.07s/it]
 60%|██████    | 1661/2757 [3:47:33<2:28:13,  8.11s/it]
 60%|██████    | 1662/2757 [3:47:41<2:28:19,  8.13s/it]
 60%|██████    | 1663/2757 [3:47:49<2:26:46,  8.05s/it]
 60%|██████    | 1664/2757 [3:47:56<2:23:03,  7.85s/it]
 60%|██████    | 1665/2757 [3:48:04<2:24:51,  7.96s/it]
 60%|██████    | 1666/2757 [3:48:12<2:25:12,  7.99s/it]
 60%|██████    | 1667/2757 [3:48:21<2:28:03,  8.15s/it]
 61%|██████    | 1668/2757 [3:48:28<2:24:42,  7.97s/it]
 61%|██████    | 1669/2757 [3:48:37<2:26:02,  8.05s/it]
 61%|██████    | 1670/2757 [3:48:45<2:24:51,  8.00s/it]
                                                       

 61%|██████    | 1670/2757 [3:48:45<2:24:51,  8.00s/it]
 61%|██████    | 1671/2757 [3:48:52<2:23:56,  7.95s/it]
 61%|██████    | 1672/2757 [3:49:00<2:24:08,  7.97s/it]
 61%|██████    | 1673/2757 [3:49:08<2:22:48,  7.90s/it]
 61%|██████    | 1674/2757 [3:49:16<2:23:38,  7.96s/it]
 61%|██████    | 1675/2757 [3:49:24<2:24:13,  8.00s/it]
 61%|██████    | 1676/2757 [3:49:32<2:23:45,  7.98s/it]
 61%|██████    | 1677/2757 [3:49:40<2:21:42,  7.87s/it]
 61%|██████    | 1678/2757 [3:49:48<2:22:07,  7.90s/it]
 61%|██████    | 1679/2757 [3:49:56<2:21:31,  7.88s/it]
 61%|██████    | 1680/2757 [3:50:04<2:22:24,  7.93s/it]
                                                       

 61%|██████    | 1680/2757 [3:50:04<2:22:24,  7.93s/it]
 61%|██████    | 1681/2757 [3:50:11<2:19:00,  7.75s/it]
 61%|██████    | 1682/2757 [3:50:20<2:22:43,  7.97s/it]
 61%|██████    | 1683/2757 [3:50:27<2:21:00,  7.88s/it]
 61%|██████    | 1684/2757 [3:50:35<2:21:08,  7.89s/it]
 61%|██████    | 1685/2757 [3:50:43<2:21:24,  7.91s/it]
 61%|██████    | 1686/2757 [3:50:51<2:21:32,  7.93s/it]
 61%|██████    | 1687/2757 [3:50:59<2:23:33,  8.05s/it]
 61%|██████    | 1688/2757 [3:51:08<2:24:46,  8.13s/it]
 61%|██████▏   | 1689/2757 [3:51:16<2:22:52,  8.03s/it]
 61%|██████▏   | 1690/2757 [3:51:24<2:23:29,  8.07s/it]
                                                       

 61%|██████▏   | 1690/2757 [3:51:24<2:23:29,  8.07s/it]
 61%|██████▏   | 1691/2757 [3:51:32<2:25:51,  8.21s/it]
 61%|██████▏   | 1692/2757 [3:51:40<2:21:26,  7.97s/it]
 61%|██████▏   | 1693/2757 [3:51:47<2:19:42,  7.88s/it]
 61%|██████▏   | 1694/2757 [3:51:55<2:20:31,  7.93s/it]
 61%|██████▏   | 1695/2757 [3:52:04<2:22:27,  8.05s/it]
 62%|██████▏   | 1696/2757 [3:52:11<2:20:59,  7.97s/it]
 62%|██████▏   | 1697/2757 [3:52:19<2:19:02,  7.87s/it]
 62%|██████▏   | 1698/2757 [3:52:27<2:20:44,  7.97s/it]
 62%|██████▏   | 1699/2757 [3:52:35<2:21:28,  8.02s/it]
 62%|██████▏   | 1700/2757 [3:52:43<2:19:49,  7.94s/it]
                                                       

 62%|██████▏   | 1700/2757 [3:52:43<2:19:49,  7.94s/it]
 62%|██████▏   | 1701/2757 [3:52:51<2:19:51,  7.95s/it]
 62%|██████▏   | 1702/2757 [3:52:59<2:20:10,  7.97s/it]
 62%|██████▏   | 1703/2757 [3:53:07<2:19:24,  7.94s/it]
 62%|██████▏   | 1704/2757 [3:53:15<2:17:59,  7.86s/it]
 62%|██████▏   | 1705/2757 [3:53:23<2:21:14,  8.06s/it]
 62%|██████▏   | 1706/2757 [3:53:31<2:21:03,  8.05s/it]
 62%|██████▏   | 1707/2757 [3:53:39<2:20:00,  8.00s/it]
 62%|██████▏   | 1708/2757 [3:53:47<2:19:10,  7.96s/it]
 62%|██████▏   | 1709/2757 [3:53:55<2:18:14,  7.91s/it]
 62%|██████▏   | 1710/2757 [3:54:03<2:19:38,  8.00s/it]
                                                       

 62%|██████▏   | 1710/2757 [3:54:03<2:19:38,  8.00s/it]
 62%|██████▏   | 1711/2757 [3:54:11<2:20:31,  8.06s/it]
 62%|██████▏   | 1712/2757 [3:54:19<2:18:08,  7.93s/it]
 62%|██████▏   | 1713/2757 [3:54:27<2:17:38,  7.91s/it]
 62%|██████▏   | 1714/2757 [3:54:35<2:18:39,  7.98s/it]
 62%|██████▏   | 1715/2757 [3:54:43<2:19:08,  8.01s/it]
 62%|██████▏   | 1716/2757 [3:54:51<2:18:06,  7.96s/it]
 62%|██████▏   | 1717/2757 [3:54:58<2:16:32,  7.88s/it]
 62%|██████▏   | 1718/2757 [3:55:06<2:15:58,  7.85s/it]
 62%|██████▏   | 1719/2757 [3:55:14<2:13:35,  7.72s/it]
 62%|██████▏   | 1720/2757 [3:55:21<2:12:15,  7.65s/it]
                                                       

 62%|██████▏   | 1720/2757 [3:55:21<2:12:15,  7.65s/it]
 62%|██████▏   | 1721/2757 [3:55:29<2:12:56,  7.70s/it]
 62%|██████▏   | 1722/2757 [3:55:37<2:15:56,  7.88s/it]
 62%|██████▏   | 1723/2757 [3:55:45<2:15:04,  7.84s/it]
 63%|██████▎   | 1724/2757 [3:55:53<2:16:23,  7.92s/it]
 63%|██████▎   | 1725/2757 [3:56:01<2:18:17,  8.04s/it]
 63%|██████▎   | 1726/2757 [3:56:09<2:16:00,  7.92s/it]
 63%|██████▎   | 1727/2757 [3:56:17<2:16:01,  7.92s/it]
 63%|██████▎   | 1728/2757 [3:56:25<2:14:28,  7.84s/it]
 63%|██████▎   | 1729/2757 [3:56:33<2:16:04,  7.94s/it]
 63%|██████▎   | 1730/2757 [3:56:41<2:15:57,  7.94s/it]
                                                       

 63%|██████▎   | 1730/2757 [3:56:41<2:15:57,  7.94s/it]
 63%|██████▎   | 1731/2757 [3:56:49<2:15:07,  7.90s/it]
 63%|██████▎   | 1732/2757 [3:56:57<2:15:15,  7.92s/it]
 63%|██████▎   | 1733/2757 [3:57:04<2:14:42,  7.89s/it]
 63%|██████▎   | 1734/2757 [3:57:12<2:13:17,  7.82s/it]
 63%|██████▎   | 1735/2757 [3:57:20<2:15:50,  7.98s/it]
 63%|██████▎   | 1736/2757 [3:57:29<2:16:56,  8.05s/it]
 63%|██████▎   | 1737/2757 [3:57:37<2:16:39,  8.04s/it]
 63%|██████▎   | 1738/2757 [3:57:45<2:17:39,  8.11s/it]
 63%|██████▎   | 1739/2757 [3:57:53<2:16:36,  8.05s/it]
 63%|██████▎   | 1740/2757 [3:58:01<2:16:50,  8.07s/it]
                                                       

 63%|██████▎   | 1740/2757 [3:58:01<2:16:50,  8.07s/it]
 63%|██████▎   | 1741/2757 [3:58:09<2:15:32,  8.00s/it]
 63%|██████▎   | 1742/2757 [3:58:17<2:15:19,  8.00s/it]
 63%|██████▎   | 1743/2757 [3:58:25<2:17:43,  8.15s/it]
 63%|██████▎   | 1744/2757 [3:58:33<2:17:05,  8.12s/it]
 63%|██████▎   | 1745/2757 [3:58:42<2:17:22,  8.14s/it]
 63%|██████▎   | 1746/2757 [3:58:49<2:16:07,  8.08s/it]
 63%|██████▎   | 1747/2757 [3:58:58<2:16:30,  8.11s/it]
 63%|██████▎   | 1748/2757 [3:59:06<2:15:49,  8.08s/it]
 63%|██████▎   | 1749/2757 [3:59:14<2:16:40,  8.14s/it]
 63%|██████▎   | 1750/2757 [3:59:23<2:20:57,  8.40s/it]
                                                       
{'eval_loss': '0.4785', 'eval_runtime': '106', 'eval_samples_per_second': '22.64', 'eval_steps_per_second': '2.83', 'eval_rewards/chosen': '-1.847', 'eval_rewards/rejected': '-2.994', 'eval_rewards/accuracies': '0.7079', 'eval_rewards/margins': '1.147', 'eval_logps/chosen': '-98.07', 'eval_logps/rejected': '-115.8', 'eval_logits/chosen': '-4.041', 'eval_logits/rejected': '-3.988', 'epoch': '1.632'}
{'loss': '0.3725', 'grad_norm': '0.6329', 'learning_rate': '9.334e-05', 'rewards/chosen': '-1.795', 'rewards/rejected': '-3.295', 'rewards/accuracies': '0.775', 'rewards/margins': '1.5', 'logps/chosen': '-97.45', 'logps/rejected': '-118.5', 'logits/chosen': '-4.012', 'logits/rejected': '-3.945', 'epoch': '1.643'}
{'loss': '0.3885', 'grad_norm': '0.8267', 'learning_rate': '9.26e-05', 'rewards/chosen': '-2.057', 'rewards/rejected': '-3.51', 'rewards/accuracies': '0.7766', 'rewards/margins': '1.453', 'logps/chosen': '-99.85', 'logps/rejected': '-118.9', 'logits/chosen': '-4.003', 'logits/rejected': '-3.953', 'epoch': '1.654'}
{'loss': '0.3619', 'grad_norm': '0.6179', 'learning_rate': '9.185e-05', 'rewards/chosen': '-1.993', 'rewards/rejected': '-3.517', 'rewards/accuracies': '0.7969', 'rewards/margins': '1.524', 'logps/chosen': '-100.3', 'logps/rejected': '-121', 'logits/chosen': '-3.809', 'logits/rejected': '-3.735', 'epoch': '1.665'}
{'loss': '0.3576', 'grad_norm': '0.6443', 'learning_rate': '9.11e-05', 'rewards/chosen': '-2.11', 'rewards/rejected': '-3.644', 'rewards/accuracies': '0.7937', 'rewards/margins': '1.534', 'logps/chosen': '-103.8', 'logps/rejected': '-123.3', 'logits/chosen': '-3.819', 'logits/rejected': '-3.749', 'epoch': '1.676'}
{'loss': '0.374', 'grad_norm': '0.7384', 'learning_rate': '9.035e-05', 'rewards/chosen': '-2.21', 'rewards/rejected': '-3.729', 'rewards/accuracies': '0.768', 'rewards/margins': '1.519', 'logps/chosen': '-103.8', 'logps/rejected': '-122.9', 'logits/chosen': '-3.885', 'logits/rejected': '-3.787', 'epoch': '1.687'}
{'loss': '0.3584', 'grad_norm': '0.5654', 'learning_rate': '8.96e-05', 'rewards/chosen': '-2.004', 'rewards/rejected': '-3.579', 'rewards/accuracies': '0.7852', 'rewards/margins': '1.574', 'logps/chosen': '-100.5', 'logps/rejected': '-120.8', 'logits/chosen': '-3.935', 'logits/rejected': '-3.844', 'epoch': '1.698'}
{'loss': '0.3536', 'grad_norm': '0.8251', 'learning_rate': '8.886e-05', 'rewards/chosen': '-2.175', 'rewards/rejected': '-3.769', 'rewards/accuracies': '0.8062', 'rewards/margins': '1.595', 'logps/chosen': '-102.4', 'logps/rejected': '-124.2', 'logits/chosen': '-4.114', 'logits/rejected': '-4.016', 'epoch': '1.709'}
{'loss': '0.3635', 'grad_norm': '0.7069', 'learning_rate': '8.811e-05', 'rewards/chosen': '-2.133', 'rewards/rejected': '-3.733', 'rewards/accuracies': '0.7836', 'rewards/margins': '1.6', 'logps/chosen': '-100.6', 'logps/rejected': '-121.4', 'logits/chosen': '-4.129', 'logits/rejected': '-4.013', 'epoch': '1.719'}
{'loss': '0.3753', 'grad_norm': '0.8969', 'learning_rate': '8.736e-05', 'rewards/chosen': '-2.151', 'rewards/rejected': '-3.742', 'rewards/accuracies': '0.7781', 'rewards/margins': '1.591', 'logps/chosen': '-101.7', 'logps/rejected': '-124.9', 'logits/chosen': '-4.027', 'logits/rejected': '-4.019', 'epoch': '1.73'}
{'loss': '0.3766', 'grad_norm': '0.7352', 'learning_rate': '8.661e-05', 'rewards/chosen': '-2.379', 'rewards/rejected': '-3.885', 'rewards/accuracies': '0.7742', 'rewards/margins': '1.506', 'logps/chosen': '-103.6', 'logps/rejected': '-124.1', 'logits/chosen': '-3.981', 'logits/rejected': '-3.917', 'epoch': '1.741'}
{'loss': '0.3653', 'grad_norm': '0.5647', 'learning_rate': '8.586e-05', 'rewards/chosen': '-2.353', 'rewards/rejected': '-3.948', 'rewards/accuracies': '0.7844', 'rewards/margins': '1.595', 'logps/chosen': '-103.6', 'logps/rejected': '-125.9', 'logits/chosen': '-4.083', 'logits/rejected': '-4.028', 'epoch': '1.752'}
{'loss': '0.3691', 'grad_norm': '0.7188', 'learning_rate': '8.512e-05', 'rewards/chosen': '-2.336', 'rewards/rejected': '-3.85', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.515', 'logps/chosen': '-104.9', 'logps/rejected': '-125', 'logits/chosen': '-3.872', 'logits/rejected': '-3.833', 'epoch': '1.763'}
{'loss': '0.3572', 'grad_norm': '0.6679', 'learning_rate': '8.437e-05', 'rewards/chosen': '-2.258', 'rewards/rejected': '-3.829', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.571', 'logps/chosen': '-103.7', 'logps/rejected': '-125', 'logits/chosen': '-4.03', 'logits/rejected': '-3.973', 'epoch': '1.774'}
{'loss': '0.3459', 'grad_norm': '0.6013', 'learning_rate': '8.362e-05', 'rewards/chosen': '-2.236', 'rewards/rejected': '-3.897', 'rewards/accuracies': '0.8023', 'rewards/margins': '1.661', 'logps/chosen': '-100.5', 'logps/rejected': '-123.6', 'logits/chosen': '-3.995', 'logits/rejected': '-3.911', 'epoch': '1.785'}
{'loss': '0.3556', 'grad_norm': '0.6696', 'learning_rate': '8.287e-05', 'rewards/chosen': '-2.186', 'rewards/rejected': '-3.82', 'rewards/accuracies': '0.793', 'rewards/margins': '1.634', 'logps/chosen': '-100.8', 'logps/rejected': '-123.3', 'logits/chosen': '-3.975', 'logits/rejected': '-3.9', 'epoch': '1.796'}
{'loss': '0.3668', 'grad_norm': '0.5696', 'learning_rate': '8.212e-05', 'rewards/chosen': '-2.07', 'rewards/rejected': '-3.639', 'rewards/accuracies': '0.7891', 'rewards/margins': '1.569', 'logps/chosen': '-99.68', 'logps/rejected': '-120.6', 'logits/chosen': '-4.077', 'logits/rejected': '-3.98', 'epoch': '1.807'}
{'loss': '0.377', 'grad_norm': '0.6719', 'learning_rate': '8.138e-05', 'rewards/chosen': '-2.11', 'rewards/rejected': '-3.654', 'rewards/accuracies': '0.7898', 'rewards/margins': '1.544', 'logps/chosen': '-103.2', 'logps/rejected': '-124.1', 'logits/chosen': '-4.046', 'logits/rejected': '-3.934', 'epoch': '1.817'}
{'loss': '0.3561', 'grad_norm': '0.6737', 'learning_rate': '8.063e-05', 'rewards/chosen': '-2.13', 'rewards/rejected': '-3.775', 'rewards/accuracies': '0.7891', 'rewards/margins': '1.645', 'logps/chosen': '-99.72', 'logps/rejected': '-121.9', 'logits/chosen': '-4.077', 'logits/rejected': '-4.007', 'epoch': '1.828'}
{'loss': '0.3536', 'grad_norm': '0.7005', 'learning_rate': '7.988e-05', 'rewards/chosen': '-2.293', 'rewards/rejected': '-3.948', 'rewards/accuracies': '0.8031', 'rewards/margins': '1.655', 'logps/chosen': '-105.8', 'logps/rejected': '-128', 'logits/chosen': '-3.916', 'logits/rejected': '-3.911', 'epoch': '1.839'}
{'loss': '0.3671', 'grad_norm': '0.5833', 'learning_rate': '7.913e-05', 'rewards/chosen': '-2.238', 'rewards/rejected': '-3.842', 'rewards/accuracies': '0.7609', 'rewards/margins': '1.604', 'logps/chosen': '-98.68', 'logps/rejected': '-118.4', 'logits/chosen': '-4.208', 'logits/rejected': '-4.131', 'epoch': '1.85'}
{'loss': '0.3505', 'grad_norm': '0.5559', 'learning_rate': '7.838e-05', 'rewards/chosen': '-2.235', 'rewards/rejected': '-3.849', 'rewards/accuracies': '0.8094', 'rewards/margins': '1.614', 'logps/chosen': '-99.44', 'logps/rejected': '-122.8', 'logits/chosen': '-4.181', 'logits/rejected': '-4.108', 'epoch': '1.861'}
{'loss': '0.3608', 'grad_norm': '0.5495', 'learning_rate': '7.764e-05', 'rewards/chosen': '-2.364', 'rewards/rejected': '-3.972', 'rewards/accuracies': '0.7828', 'rewards/margins': '1.608', 'logps/chosen': '-104.2', 'logps/rejected': '-125.3', 'logits/chosen': '-4.158', 'logits/rejected': '-4.096', 'epoch': '1.872'}
{'loss': '0.355', 'grad_norm': '0.7562', 'learning_rate': '7.689e-05', 'rewards/chosen': '-2.264', 'rewards/rejected': '-3.919', 'rewards/accuracies': '0.7906', 'rewards/margins': '1.655', 'logps/chosen': '-102.7', 'logps/rejected': '-123.5', 'logits/chosen': '-4.054', 'logits/rejected': '-3.934', 'epoch': '1.883'}
{'loss': '0.3495', 'grad_norm': '0.6611', 'learning_rate': '7.614e-05', 'rewards/chosen': '-2.171', 'rewards/rejected': '-3.777', 'rewards/accuracies': '0.7953', 'rewards/margins': '1.607', 'logps/chosen': '-104.4', 'logps/rejected': '-125.9', 'logits/chosen': '-3.961', 'logits/rejected': '-3.83', 'epoch': '1.894'}
{'loss': '0.3612', 'grad_norm': '0.6473', 'learning_rate': '7.539e-05', 'rewards/chosen': '-2.068', 'rewards/rejected': '-3.683', 'rewards/accuracies': '0.7836', 'rewards/margins': '1.615', 'logps/chosen': '-106', 'logps/rejected': '-125.6', 'logits/chosen': '-3.913', 'logits/rejected': '-3.869', 'epoch': '1.904'}
 63%|██████▎   | 1750/2757 [3:59:23<2:20:57,  8.40s/it]
 64%|██████▎   | 1751/2757 [3:59:37<2:48:24, 10.04s/it]
 64%|██████▎   | 1752/2757 [3:59:45<2:37:55,  9.43s/it]
 64%|██████▎   | 1753/2757 [3:59:53<2:33:59,  9.20s/it]
 64%|██████▎   | 1754/2757 [4:00:02<2:28:18,  8.87s/it]
 64%|██████▎   | 1755/2757 [4:00:09<2:21:56,  8.50s/it]
 64%|██████▎   | 1756/2757 [4:00:17<2:18:21,  8.29s/it]
 64%|██████▎   | 1757/2757 [4:00:26<2:20:33,  8.43s/it]
 64%|██████▍   | 1758/2757 [4:00:34<2:17:39,  8.27s/it]
 64%|██████▍   | 1759/2757 [4:00:42<2:17:16,  8.25s/it]
 64%|██████▍   | 1760/2757 [4:00:50<2:17:25,  8.27s/it]
                                                       

 64%|██████▍   | 1760/2757 [4:00:50<2:17:25,  8.27s/it]
 64%|██████▍   | 1761/2757 [4:00:58<2:16:03,  8.20s/it]
 64%|██████▍   | 1762/2757 [4:01:06<2:13:52,  8.07s/it]
 64%|██████▍   | 1763/2757 [4:01:14<2:12:44,  8.01s/it]
 64%|██████▍   | 1764/2757 [4:01:21<2:10:42,  7.90s/it]
 64%|██████▍   | 1765/2757 [4:01:29<2:09:45,  7.85s/it]
 64%|██████▍   | 1766/2757 [4:01:37<2:10:22,  7.89s/it]
 64%|██████▍   | 1767/2757 [4:01:45<2:11:38,  7.98s/it]
 64%|██████▍   | 1768/2757 [4:01:54<2:12:42,  8.05s/it]
 64%|██████▍   | 1769/2757 [4:02:01<2:11:29,  7.99s/it]
 64%|██████▍   | 1770/2757 [4:02:10<2:11:45,  8.01s/it]
                                                       

 64%|██████▍   | 1770/2757 [4:02:10<2:11:45,  8.01s/it]
 64%|██████▍   | 1771/2757 [4:02:17<2:11:17,  7.99s/it]
 64%|██████▍   | 1772/2757 [4:02:25<2:11:17,  8.00s/it]
 64%|██████▍   | 1773/2757 [4:02:33<2:10:10,  7.94s/it]
 64%|██████▍   | 1774/2757 [4:02:41<2:09:21,  7.90s/it]
 64%|██████▍   | 1775/2757 [4:02:49<2:10:24,  7.97s/it]
 64%|██████▍   | 1776/2757 [4:02:57<2:08:37,  7.87s/it]
 64%|██████▍   | 1777/2757 [4:03:05<2:08:46,  7.88s/it]
 64%|██████▍   | 1778/2757 [4:03:13<2:08:29,  7.88s/it]
 65%|██████▍   | 1779/2757 [4:03:21<2:08:30,  7.88s/it]
 65%|██████▍   | 1780/2757 [4:03:29<2:08:59,  7.92s/it]
                                                       

 65%|██████▍   | 1780/2757 [4:03:29<2:08:59,  7.92s/it]
 65%|██████▍   | 1781/2757 [4:03:37<2:09:10,  7.94s/it]
 65%|██████▍   | 1782/2757 [4:03:44<2:07:23,  7.84s/it]
 65%|██████▍   | 1783/2757 [4:03:52<2:07:50,  7.88s/it]
 65%|██████▍   | 1784/2757 [4:04:00<2:07:14,  7.85s/it]
 65%|██████▍   | 1785/2757 [4:04:08<2:08:56,  7.96s/it]
 65%|██████▍   | 1786/2757 [4:04:16<2:11:05,  8.10s/it]
 65%|██████▍   | 1787/2757 [4:04:25<2:10:45,  8.09s/it]
 65%|██████▍   | 1788/2757 [4:04:33<2:10:11,  8.06s/it]
 65%|██████▍   | 1789/2757 [4:04:40<2:08:11,  7.95s/it]
 65%|██████▍   | 1790/2757 [4:04:48<2:05:59,  7.82s/it]
                                                       

 65%|██████▍   | 1790/2757 [4:04:48<2:05:59,  7.82s/it]
 65%|██████▍   | 1791/2757 [4:04:56<2:07:23,  7.91s/it]
 65%|██████▍   | 1792/2757 [4:05:04<2:09:32,  8.05s/it]
 65%|██████▌   | 1793/2757 [4:05:13<2:11:49,  8.21s/it]
 65%|██████▌   | 1794/2757 [4:05:21<2:11:10,  8.17s/it]
 65%|██████▌   | 1795/2757 [4:05:29<2:09:08,  8.05s/it]
 65%|██████▌   | 1796/2757 [4:05:36<2:06:32,  7.90s/it]
 65%|██████▌   | 1797/2757 [4:05:44<2:07:22,  7.96s/it]
 65%|██████▌   | 1798/2757 [4:05:52<2:07:04,  7.95s/it]
 65%|██████▌   | 1799/2757 [4:06:00<2:05:05,  7.83s/it]
 65%|██████▌   | 1800/2757 [4:06:08<2:04:31,  7.81s/it]
                                                       

 65%|██████▌   | 1800/2757 [4:06:08<2:04:31,  7.81s/it]
 65%|██████▌   | 1801/2757 [4:06:15<2:04:46,  7.83s/it]
 65%|██████▌   | 1802/2757 [4:06:24<2:06:22,  7.94s/it]
 65%|██████▌   | 1803/2757 [4:06:32<2:07:22,  8.01s/it]
 65%|██████▌   | 1804/2757 [4:06:40<2:06:41,  7.98s/it]
 65%|██████▌   | 1805/2757 [4:06:47<2:04:06,  7.82s/it]
 66%|██████▌   | 1806/2757 [4:06:55<2:06:11,  7.96s/it]
 66%|██████▌   | 1807/2757 [4:07:04<2:07:15,  8.04s/it]
 66%|██████▌   | 1808/2757 [4:07:12<2:07:48,  8.08s/it]
 66%|██████▌   | 1809/2757 [4:07:20<2:07:06,  8.04s/it]
 66%|██████▌   | 1810/2757 [4:07:28<2:07:38,  8.09s/it]
                                                       

 66%|██████▌   | 1810/2757 [4:07:28<2:07:38,  8.09s/it]
 66%|██████▌   | 1811/2757 [4:07:36<2:06:59,  8.05s/it]
 66%|██████▌   | 1812/2757 [4:07:44<2:05:10,  7.95s/it]
 66%|██████▌   | 1813/2757 [4:07:52<2:04:27,  7.91s/it]
 66%|██████▌   | 1814/2757 [4:07:59<2:03:28,  7.86s/it]
 66%|██████▌   | 1815/2757 [4:08:07<2:03:01,  7.84s/it]
 66%|██████▌   | 1816/2757 [4:08:15<2:03:38,  7.88s/it]
 66%|██████▌   | 1817/2757 [4:08:23<2:02:31,  7.82s/it]
 66%|██████▌   | 1818/2757 [4:08:31<2:04:24,  7.95s/it]
 66%|██████▌   | 1819/2757 [4:08:39<2:06:11,  8.07s/it]
 66%|██████▌   | 1820/2757 [4:08:47<2:05:42,  8.05s/it]
                                                       

 66%|██████▌   | 1820/2757 [4:08:47<2:05:42,  8.05s/it]
 66%|██████▌   | 1821/2757 [4:08:55<2:05:30,  8.05s/it]
 66%|██████▌   | 1822/2757 [4:09:03<2:04:18,  7.98s/it]
 66%|██████▌   | 1823/2757 [4:09:11<2:02:28,  7.87s/it]
 66%|██████▌   | 1824/2757 [4:09:19<2:04:37,  8.01s/it]
 66%|██████▌   | 1825/2757 [4:09:27<2:04:53,  8.04s/it]
 66%|██████▌   | 1826/2757 [4:09:35<2:05:38,  8.10s/it]
 66%|██████▋   | 1827/2757 [4:09:43<2:03:48,  7.99s/it]
 66%|██████▋   | 1828/2757 [4:09:51<2:03:12,  7.96s/it]
 66%|██████▋   | 1829/2757 [4:09:59<2:02:39,  7.93s/it]
 66%|██████▋   | 1830/2757 [4:10:07<2:03:18,  7.98s/it]
                                                       

 66%|██████▋   | 1830/2757 [4:10:07<2:03:18,  7.98s/it]
 66%|██████▋   | 1831/2757 [4:10:16<2:06:03,  8.17s/it]
 66%|██████▋   | 1832/2757 [4:10:24<2:05:08,  8.12s/it]
 66%|██████▋   | 1833/2757 [4:10:32<2:04:45,  8.10s/it]
 67%|██████▋   | 1834/2757 [4:10:40<2:05:18,  8.15s/it]
 67%|██████▋   | 1835/2757 [4:10:48<2:04:47,  8.12s/it]
 67%|██████▋   | 1836/2757 [4:10:56<2:04:35,  8.12s/it]
 67%|██████▋   | 1837/2757 [4:11:04<2:05:03,  8.16s/it]
 67%|██████▋   | 1838/2757 [4:11:10<1:55:31,  7.54s/it]
{'loss': '0.3603', 'grad_norm': '0.7833', 'learning_rate': '7.464e-05', 'rewards/chosen': '-1.853', 'rewards/rejected': '-3.423', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.57', 'logps/chosen': '-97.6', 'logps/rejected': '-119.1', 'logits/chosen': '-3.939', 'logits/rejected': '-3.906', 'epoch': '1.915'}
{'loss': '0.3666', 'grad_norm': '0.72', 'learning_rate': '7.39e-05', 'rewards/chosen': '-2.322', 'rewards/rejected': '-3.87', 'rewards/accuracies': '0.7812', 'rewards/margins': '1.548', 'logps/chosen': '-104.4', 'logps/rejected': '-122.6', 'logits/chosen': '-3.854', 'logits/rejected': '-3.82', 'epoch': '1.926'}
{'loss': '0.3637', 'grad_norm': '0.6587', 'learning_rate': '7.315e-05', 'rewards/chosen': '-2.446', 'rewards/rejected': '-4.051', 'rewards/accuracies': '0.7805', 'rewards/margins': '1.605', 'logps/chosen': '-106', 'logps/rejected': '-127.8', 'logits/chosen': '-3.952', 'logits/rejected': '-4.003', 'epoch': '1.937'}
{'loss': '0.3613', 'grad_norm': '0.5735', 'learning_rate': '7.24e-05', 'rewards/chosen': '-2.576', 'rewards/rejected': '-4.142', 'rewards/accuracies': '0.7812', 'rewards/margins': '1.566', 'logps/chosen': '-100.7', 'logps/rejected': '-121.9', 'logits/chosen': '-4', 'logits/rejected': '-3.923', 'epoch': '1.948'}
{'loss': '0.3469', 'grad_norm': '0.6537', 'learning_rate': '7.165e-05', 'rewards/chosen': '-2.413', 'rewards/rejected': '-4.107', 'rewards/accuracies': '0.8195', 'rewards/margins': '1.694', 'logps/chosen': '-103.4', 'logps/rejected': '-125.7', 'logits/chosen': '-3.983', 'logits/rejected': '-3.873', 'epoch': '1.959'}
{'loss': '0.3626', 'grad_norm': '0.6989', 'learning_rate': '7.091e-05', 'rewards/chosen': '-2.356', 'rewards/rejected': '-3.993', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.637', 'logps/chosen': '-106.2', 'logps/rejected': '-126.7', 'logits/chosen': '-3.916', 'logits/rejected': '-3.87', 'epoch': '1.97'}
{'loss': '0.3597', 'grad_norm': '0.7157', 'learning_rate': '7.016e-05', 'rewards/chosen': '-2.092', 'rewards/rejected': '-3.722', 'rewards/accuracies': '0.7781', 'rewards/margins': '1.63', 'logps/chosen': '-100.3', 'logps/rejected': '-121.5', 'logits/chosen': '-3.931', 'logits/rejected': '-3.873', 'epoch': '1.981'}
{'loss': '0.3599', 'grad_norm': '0.6526', 'learning_rate': '6.941e-05', 'rewards/chosen': '-1.991', 'rewards/rejected': '-3.627', 'rewards/accuracies': '0.7945', 'rewards/margins': '1.635', 'logps/chosen': '-100', 'logps/rejected': '-122', 'logits/chosen': '-3.948', 'logits/rejected': '-3.869', 'epoch': '1.992'}

 67%|██████▋   | 1839/2757 [4:11:19<2:00:17,  7.86s/it]
 67%|██████▋   | 1840/2757 [4:11:27<2:00:20,  7.87s/it]
                                                       

 67%|██████▋   | 1840/2757 [4:11:27<2:00:20,  7.87s/it]
 67%|██████▋   | 1841/2757 [4:11:35<2:00:20,  7.88s/it]
 67%|██████▋   | 1842/2757 [4:11:43<2:02:53,  8.06s/it]
 67%|██████▋   | 1843/2757 [4:11:51<2:03:00,  8.07s/it]
 67%|██████▋   | 1844/2757 [4:11:59<2:02:30,  8.05s/it]
 67%|██████▋   | 1845/2757 [4:12:08<2:03:20,  8.11s/it]
 67%|██████▋   | 1846/2757 [4:12:16<2:04:00,  8.17s/it]
 67%|██████▋   | 1847/2757 [4:12:24<2:03:45,  8.16s/it]
 67%|██████▋   | 1848/2757 [4:12:32<2:03:20,  8.14s/it]
 67%|██████▋   | 1849/2757 [4:12:40<2:02:34,  8.10s/it]
 67%|██████▋   | 1850/2757 [4:12:49<2:03:26,  8.17s/it]
                                                       

 67%|██████▋   | 1850/2757 [4:12:49<2:03:26,  8.17s/it]
 67%|██████▋   | 1851/2757 [4:12:57<2:02:05,  8.09s/it]
 67%|██████▋   | 1852/2757 [4:13:05<2:02:39,  8.13s/it]
 67%|██████▋   | 1853/2757 [4:13:13<2:01:21,  8.05s/it]
 67%|██████▋   | 1854/2757 [4:13:20<1:59:41,  7.95s/it]
 67%|██████▋   | 1855/2757 [4:13:28<1:59:20,  7.94s/it]
 67%|██████▋   | 1856/2757 [4:13:36<1:58:53,  7.92s/it]
 67%|██████▋   | 1857/2757 [4:13:44<1:59:03,  7.94s/it]
 67%|██████▋   | 1858/2757 [4:13:53<2:01:11,  8.09s/it]
 67%|██████▋   | 1859/2757 [4:14:01<2:02:45,  8.20s/it]
 67%|██████▋   | 1860/2757 [4:14:09<2:01:36,  8.13s/it]
                                                       

 67%|██████▋   | 1860/2757 [4:14:09<2:01:36,  8.13s/it]
 68%|██████▊   | 1861/2757 [4:14:17<1:59:31,  8.00s/it]
 68%|██████▊   | 1862/2757 [4:14:25<2:01:12,  8.13s/it]
 68%|██████▊   | 1863/2757 [4:14:33<2:00:43,  8.10s/it]
 68%|██████▊   | 1864/2757 [4:14:41<1:59:42,  8.04s/it]
 68%|██████▊   | 1865/2757 [4:14:49<1:57:59,  7.94s/it]
 68%|██████▊   | 1866/2757 [4:14:57<1:57:20,  7.90s/it]
 68%|██████▊   | 1867/2757 [4:15:04<1:55:58,  7.82s/it]
 68%|██████▊   | 1868/2757 [4:15:12<1:55:11,  7.77s/it]
 68%|██████▊   | 1869/2757 [4:15:20<1:55:19,  7.79s/it]
 68%|██████▊   | 1870/2757 [4:15:28<1:57:58,  7.98s/it]
                                                       

 68%|██████▊   | 1870/2757 [4:15:28<1:57:58,  7.98s/it]
 68%|██████▊   | 1871/2757 [4:15:36<1:58:00,  7.99s/it]
 68%|██████▊   | 1872/2757 [4:15:44<1:58:45,  8.05s/it]
 68%|██████▊   | 1873/2757 [4:15:52<1:57:26,  7.97s/it]
 68%|██████▊   | 1874/2757 [4:16:00<1:56:33,  7.92s/it]
 68%|██████▊   | 1875/2757 [4:16:08<1:57:30,  7.99s/it]
 68%|██████▊   | 1876/2757 [4:16:16<1:56:10,  7.91s/it]
 68%|██████▊   | 1877/2757 [4:16:24<1:55:26,  7.87s/it]
 68%|██████▊   | 1878/2757 [4:16:32<1:57:22,  8.01s/it]
 68%|██████▊   | 1879/2757 [4:16:40<1:57:48,  8.05s/it]
 68%|██████▊   | 1880/2757 [4:16:48<1:58:01,  8.07s/it]
                                                       

 68%|██████▊   | 1880/2757 [4:16:48<1:58:01,  8.07s/it]
 68%|██████▊   | 1881/2757 [4:16:56<1:57:07,  8.02s/it]
 68%|██████▊   | 1882/2757 [4:17:05<1:59:05,  8.17s/it]
 68%|██████▊   | 1883/2757 [4:17:13<1:58:48,  8.16s/it]
 68%|██████▊   | 1884/2757 [4:17:21<1:57:17,  8.06s/it]
 68%|██████▊   | 1885/2757 [4:17:28<1:56:14,  8.00s/it]
 68%|██████▊   | 1886/2757 [4:17:36<1:54:23,  7.88s/it]
 68%|██████▊   | 1887/2757 [4:17:44<1:52:45,  7.78s/it]
 68%|██████▊   | 1888/2757 [4:17:52<1:55:10,  7.95s/it]
 69%|██████▊   | 1889/2757 [4:18:00<1:55:26,  7.98s/it]
 69%|██████▊   | 1890/2757 [4:18:08<1:54:32,  7.93s/it]
                                                       

 69%|██████▊   | 1890/2757 [4:18:08<1:54:32,  7.93s/it]
 69%|██████▊   | 1891/2757 [4:18:16<1:56:40,  8.08s/it]
 69%|██████▊   | 1892/2757 [4:18:24<1:55:58,  8.04s/it]
 69%|██████▊   | 1893/2757 [4:18:33<1:58:03,  8.20s/it]
 69%|██████▊   | 1894/2757 [4:18:41<1:57:52,  8.19s/it]
 69%|██████▊   | 1895/2757 [4:18:49<1:56:48,  8.13s/it]
 69%|██████▉   | 1896/2757 [4:18:57<1:55:24,  8.04s/it]
 69%|██████▉   | 1897/2757 [4:19:05<1:55:36,  8.07s/it]
 69%|██████▉   | 1898/2757 [4:19:13<1:54:16,  7.98s/it]
 69%|██████▉   | 1899/2757 [4:19:20<1:52:46,  7.89s/it]
 69%|██████▉   | 1900/2757 [4:19:28<1:52:51,  7.90s/it]
                                                       

 69%|██████▉   | 1900/2757 [4:19:28<1:52:51,  7.90s/it]
 69%|██████▉   | 1901/2757 [4:19:36<1:53:16,  7.94s/it]
 69%|██████▉   | 1902/2757 [4:19:44<1:53:22,  7.96s/it]
 69%|██████▉   | 1903/2757 [4:19:53<1:55:36,  8.12s/it]
 69%|██████▉   | 1904/2757 [4:20:01<1:56:44,  8.21s/it]
 69%|██████▉   | 1905/2757 [4:20:09<1:53:19,  7.98s/it]
 69%|██████▉   | 1906/2757 [4:20:16<1:52:27,  7.93s/it]
 69%|██████▉   | 1907/2757 [4:20:24<1:52:39,  7.95s/it]
 69%|██████▉   | 1908/2757 [4:20:32<1:50:43,  7.82s/it]
 69%|██████▉   | 1909/2757 [4:20:40<1:53:00,  8.00s/it]
 69%|██████▉   | 1910/2757 [4:20:48<1:50:36,  7.83s/it]
                                                       

 69%|██████▉   | 1910/2757 [4:20:48<1:50:36,  7.83s/it]
 69%|██████▉   | 1911/2757 [4:20:56<1:50:34,  7.84s/it]
 69%|██████▉   | 1912/2757 [4:21:04<1:53:38,  8.07s/it]
 69%|██████▉   | 1913/2757 [4:21:12<1:52:51,  8.02s/it]
 69%|██████▉   | 1914/2757 [4:21:20<1:51:46,  7.96s/it]
 69%|██████▉   | 1915/2757 [4:21:28<1:52:38,  8.03s/it]
 69%|██████▉   | 1916/2757 [4:21:36<1:53:39,  8.11s/it]
 70%|██████▉   | 1917/2757 [4:21:44<1:51:56,  8.00s/it]
 70%|██████▉   | 1918/2757 [4:21:52<1:52:33,  8.05s/it]
 70%|██████▉   | 1919/2757 [4:22:00<1:50:30,  7.91s/it]
 70%|██████▉   | 1920/2757 [4:22:08<1:49:20,  7.84s/it]
                                                       

 70%|██████▉   | 1920/2757 [4:22:08<1:49:20,  7.84s/it]
 70%|██████▉   | 1921/2757 [4:22:16<1:49:55,  7.89s/it]
 70%|██████▉   | 1922/2757 [4:22:24<1:50:57,  7.97s/it]
 70%|██████▉   | 1923/2757 [4:22:32<1:52:03,  8.06s/it]
 70%|██████▉   | 1924/2757 [4:22:40<1:51:21,  8.02s/it]
 70%|██████▉   | 1925/2757 [4:22:48<1:51:08,  8.01s/it]
 70%|██████▉   | 1926/2757 [4:22:56<1:50:32,  7.98s/it]
 70%|██████▉   | 1927/2757 [4:23:03<1:48:46,  7.86s/it]
 70%|██████▉   | 1928/2757 [4:23:12<1:49:20,  7.91s/it]
 70%|██████▉   | 1929/2757 [4:23:19<1:49:17,  7.92s/it]
 70%|███████   | 1930/2757 [4:23:27<1:48:58,  7.91s/it]
                                                       

 70%|███████   | 1930/2757 [4:23:27<1:48:58,  7.91s/it]
 70%|███████   | 1931/2757 [4:23:35<1:48:09,  7.86s/it]
 70%|███████   | 1932/2757 [4:23:43<1:49:22,  7.95s/it]
 70%|███████   | 1933/2757 [4:23:51<1:48:10,  7.88s/it]
 70%|███████   | 1934/2757 [4:23:59<1:47:18,  7.82s/it]
 70%|███████   | 1935/2757 [4:24:06<1:46:53,  7.80s/it]
 70%|███████   | 1936/2757 [4:24:15<1:48:23,  7.92s/it]
 70%|███████   | 1937/2757 [4:24:23<1:51:09,  8.13s/it]
 70%|███████   | 1938/2757 [4:24:31<1:50:27,  8.09s/it]
 70%|███████   | 1939/2757 [4:24:39<1:49:31,  8.03s/it]
 70%|███████   | 1940/2757 [4:24:47<1:49:07,  8.01s/it]
                                                       

 70%|███████   | 1940/2757 [4:24:47<1:49:07,  8.01s/it]
 70%|███████   | 1941/2757 [4:24:56<1:50:50,  8.15s/it]
 70%|███████   | 1942/2757 [4:25:03<1:49:34,  8.07s/it]
 70%|███████   | 1943/2757 [4:25:11<1:48:53,  8.03s/it]
 71%|███████   | 1944/2757 [4:25:19<1:47:29,  7.93s/it]
 71%|███████   | 1945/2757 [4:25:27<1:48:19,  8.00s/it]
 71%|███████   | 1946/2757 [4:25:35<1:48:14,  8.01s/it]
 71%|███████   | 1947/2757 [4:25:43<1:48:05,  8.01s/it]
 71%|███████   | 1948/2757 [4:25:51<1:46:40,  7.91s/it]
 71%|███████   | 1949/2757 [4:25:59<1:46:48,  7.93s/it]
 71%|███████   | 1950/2757 [4:26:06<1:44:57,  7.80s/it]
                                                       

 71%|███████   | 1950/2757 [4:26:06<1:44:57,  7.80s/it]
 71%|███████   | 1951/2757 [4:26:15<1:48:40,  8.09s/it]
 71%|███████   | 1952/2757 [4:26:23<1:47:47,  8.03s/it]
 71%|███████   | 1953/2757 [4:26:31<1:46:20,  7.94s/it]
 71%|███████   | 1954/2757 [4:26:39<1:46:59,  7.99s/it]
 71%|███████   | 1955/2757 [4:26:47<1:45:09,  7.87s/it]
 71%|███████   | 1956/2757 [4:26:55<1:46:05,  7.95s/it]
 71%|███████   | 1957/2757 [4:27:02<1:44:34,  7.84s/it]
 71%|███████   | 1958/2757 [4:27:09<1:41:23,  7.61s/it]
 71%|███████   | 1959/2757 [4:27:18<1:44:21,  7.85s/it]
 71%|███████   | 1960/2757 [4:27:26<1:46:44,  8.04s/it]
                                                       

 71%|███████   | 1960/2757 [4:27:26<1:46:44,  8.04s/it]
 71%|███████   | 1961/2757 [4:27:34<1:46:18,  8.01s/it]
 71%|███████   | 1962/2757 [4:27:42<1:46:09,  8.01s/it]
 71%|███████   | 1963/2757 [4:27:50<1:45:45,  7.99s/it]
 71%|███████   | 1964/2757 [4:27:58<1:45:40,  8.00s/it]
 71%|███████▏  | 1965/2757 [4:28:06<1:44:06,  7.89s/it]
 71%|███████▏  | 1966/2757 [4:28:14<1:45:06,  7.97s/it]
 71%|███████▏  | 1967/2757 [4:28:22<1:46:10,  8.06s/it]
 71%|███████▏  | 1968/2757 [4:28:30<1:43:44,  7.89s/it]
 71%|███████▏  | 1969/2757 [4:28:38<1:44:43,  7.97s/it]
 71%|███████▏  | 1970/2757 [4:28:46<1:45:39,  8.06s/it]
                                                       

 71%|███████▏  | 1970/2757 [4:28:46<1:45:39,  8.06s/it]
 71%|███████▏  | 1971/2757 [4:28:54<1:44:57,  8.01s/it]
 72%|███████▏  | 1972/2757 [4:29:01<1:42:40,  7.85s/it]
 72%|███████▏  | 1973/2757 [4:29:10<1:43:27,  7.92s/it]
 72%|███████▏  | 1974/2757 [4:29:18<1:45:16,  8.07s/it]
 72%|███████▏  | 1975/2757 [4:29:26<1:45:11,  8.07s/it]
 72%|███████▏  | 1976/2757 [4:29:34<1:44:24,  8.02s/it]
 72%|███████▏  | 1977/2757 [4:29:42<1:43:11,  7.94s/it]
 72%|███████▏  | 1978/2757 [4:29:50<1:42:34,  7.90s/it]
 72%|███████▏  | 1979/2757 [4:29:58<1:44:23,  8.05s/it]
 72%|███████▏  | 1980/2757 [4:30:06<1:46:02,  8.19s/it]
                                                       

 72%|███████▏  | 1980/2757 [4:30:06<1:46:02,  8.19s/it]
 72%|███████▏  | 1981/2757 [4:30:14<1:44:37,  8.09s/it]
 72%|███████▏  | 1982/2757 [4:30:22<1:44:49,  8.12s/it]
 72%|███████▏  | 1983/2757 [4:30:31<1:45:02,  8.14s/it]
 72%|███████▏  | 1984/2757 [4:30:39<1:44:00,  8.07s/it]
 72%|███████▏  | 1985/2757 [4:30:47<1:43:56,  8.08s/it]
 72%|███████▏  | 1986/2757 [4:30:55<1:43:54,  8.09s/it]
 72%|███████▏  | 1987/2757 [4:31:03<1:42:39,  8.00s/it]
 72%|███████▏  | 1988/2757 [4:31:11<1:42:57,  8.03s/it]
 72%|███████▏  | 1989/2757 [4:31:19<1:43:49,  8.11s/it]
 72%|███████▏  | 1990/2757 [4:31:27<1:42:50,  8.05s/it]
                                                       

 72%|███████▏  | 1990/2757 [4:31:27<1:42:50,  8.05s/it]
 72%|███████▏  | 1991/2757 [4:31:35<1:42:40,  8.04s/it]
 72%|███████▏  | 1992/2757 [4:31:43<1:43:28,  8.12s/it]
 72%|███████▏  | 1993/2757 [4:31:51<1:42:57,  8.09s/it]
 72%|███████▏  | 1994/2757 [4:31:59<1:41:24,  7.97s/it]
 72%|███████▏  | 1995/2757 [4:32:07<1:41:30,  7.99s/it]
 72%|███████▏  | 1996/2757 [4:32:15<1:41:40,  8.02s/it]
 72%|███████▏  | 1997/2757 [4:32:24<1:43:45,  8.19s/it]
 72%|███████▏  | 1998/2757 [4:32:32<1:42:34,  8.11s/it]
 73%|███████▎  | 1999/2757 [4:32:39<1:41:01,  8.00s/it]
 73%|███████▎  | 2000/2757 [4:32:47<1:39:59,  7.93s/it]
                                                       

 73%|███████▎  | 2000/2757 [4:32:47<1:39:59,  7.93s/it]{'loss': '0.3285', 'grad_norm': '0.4907', 'learning_rate': '6.866e-05', 'rewards/chosen': '-2.08', 'rewards/rejected': '-3.827', 'rewards/accuracies': '0.8221', 'rewards/margins': '1.747', 'logps/chosen': '-103.3', 'logps/rejected': '-127.7', 'logits/chosen': '-3.946', 'logits/rejected': '-3.856', 'epoch': '2.002'}
{'loss': '0.2799', 'grad_norm': '0.5088', 'learning_rate': '6.791e-05', 'rewards/chosen': '-2.049', 'rewards/rejected': '-3.981', 'rewards/accuracies': '0.8391', 'rewards/margins': '1.933', 'logps/chosen': '-101.1', 'logps/rejected': '-126.3', 'logits/chosen': '-3.941', 'logits/rejected': '-3.925', 'epoch': '2.013'}
{'loss': '0.2695', 'grad_norm': '0.5052', 'learning_rate': '6.717e-05', 'rewards/chosen': '-2.205', 'rewards/rejected': '-4.158', 'rewards/accuracies': '0.8586', 'rewards/margins': '1.953', 'logps/chosen': '-104.2', 'logps/rejected': '-128.5', 'logits/chosen': '-4.173', 'logits/rejected': '-4.081', 'epoch': '2.024'}
{'loss': '0.271', 'grad_norm': '0.4783', 'learning_rate': '6.642e-05', 'rewards/chosen': '-2.17', 'rewards/rejected': '-4.234', 'rewards/accuracies': '0.8555', 'rewards/margins': '2.064', 'logps/chosen': '-104.7', 'logps/rejected': '-129.3', 'logits/chosen': '-4.187', 'logits/rejected': '-4.043', 'epoch': '2.035'}
{'loss': '0.2763', 'grad_norm': '0.543', 'learning_rate': '6.567e-05', 'rewards/chosen': '-2.28', 'rewards/rejected': '-4.33', 'rewards/accuracies': '0.8445', 'rewards/margins': '2.05', 'logps/chosen': '-102.9', 'logps/rejected': '-129.7', 'logits/chosen': '-4.226', 'logits/rejected': '-4.173', 'epoch': '2.046'}
{'loss': '0.2582', 'grad_norm': '0.5694', 'learning_rate': '6.492e-05', 'rewards/chosen': '-2.344', 'rewards/rejected': '-4.545', 'rewards/accuracies': '0.8687', 'rewards/margins': '2.201', 'logps/chosen': '-103.5', 'logps/rejected': '-131.2', 'logits/chosen': '-4.404', 'logits/rejected': '-4.277', 'epoch': '2.057'}
{'loss': '0.2655', 'grad_norm': '0.4885', 'learning_rate': '6.417e-05', 'rewards/chosen': '-2.716', 'rewards/rejected': '-4.911', 'rewards/accuracies': '0.8445', 'rewards/margins': '2.195', 'logps/chosen': '-108.2', 'logps/rejected': '-136', 'logits/chosen': '-4.375', 'logits/rejected': '-4.236', 'epoch': '2.067'}
{'loss': '0.2693', 'grad_norm': '0.5405', 'learning_rate': '6.343e-05', 'rewards/chosen': '-2.618', 'rewards/rejected': '-4.733', 'rewards/accuracies': '0.8461', 'rewards/margins': '2.115', 'logps/chosen': '-108.1', 'logps/rejected': '-135.2', 'logits/chosen': '-4.263', 'logits/rejected': '-4.204', 'epoch': '2.078'}
{'loss': '0.2746', 'grad_norm': '0.5154', 'learning_rate': '6.268e-05', 'rewards/chosen': '-2.565', 'rewards/rejected': '-4.722', 'rewards/accuracies': '0.8398', 'rewards/margins': '2.157', 'logps/chosen': '-102.6', 'logps/rejected': '-129.8', 'logits/chosen': '-4.375', 'logits/rejected': '-4.328', 'epoch': '2.089'}
{'loss': '0.2775', 'grad_norm': '0.5592', 'learning_rate': '6.193e-05', 'rewards/chosen': '-2.541', 'rewards/rejected': '-4.662', 'rewards/accuracies': '0.8219', 'rewards/margins': '2.121', 'logps/chosen': '-107.2', 'logps/rejected': '-131.6', 'logits/chosen': '-4.222', 'logits/rejected': '-4.156', 'epoch': '2.1'}
{'loss': '0.2666', 'grad_norm': '0.4701', 'learning_rate': '6.118e-05', 'rewards/chosen': '-2.732', 'rewards/rejected': '-4.826', 'rewards/accuracies': '0.8531', 'rewards/margins': '2.093', 'logps/chosen': '-109.3', 'logps/rejected': '-134.7', 'logits/chosen': '-4.261', 'logits/rejected': '-4.192', 'epoch': '2.111'}
{'loss': '0.2717', 'grad_norm': '0.5637', 'learning_rate': '6.043e-05', 'rewards/chosen': '-2.671', 'rewards/rejected': '-4.832', 'rewards/accuracies': '0.8391', 'rewards/margins': '2.162', 'logps/chosen': '-107.6', 'logps/rejected': '-132.3', 'logits/chosen': '-4.387', 'logits/rejected': '-4.331', 'epoch': '2.122'}
{'loss': '0.2677', 'grad_norm': '0.574', 'learning_rate': '5.969e-05', 'rewards/chosen': '-2.751', 'rewards/rejected': '-4.891', 'rewards/accuracies': '0.8555', 'rewards/margins': '2.14', 'logps/chosen': '-110.6', 'logps/rejected': '-136.7', 'logits/chosen': '-4.307', 'logits/rejected': '-4.232', 'epoch': '2.133'}
{'loss': '0.2648', 'grad_norm': '0.501', 'learning_rate': '5.894e-05', 'rewards/chosen': '-2.646', 'rewards/rejected': '-4.777', 'rewards/accuracies': '0.8492', 'rewards/margins': '2.131', 'logps/chosen': '-108.6', 'logps/rejected': '-132.8', 'logits/chosen': '-4.332', 'logits/rejected': '-4.311', 'epoch': '2.144'}
{'loss': '0.2541', 'grad_norm': '0.6321', 'learning_rate': '5.819e-05', 'rewards/chosen': '-2.826', 'rewards/rejected': '-5.047', 'rewards/accuracies': '0.857', 'rewards/margins': '2.221', 'logps/chosen': '-107.1', 'logps/rejected': '-135', 'logits/chosen': '-4.377', 'logits/rejected': '-4.221', 'epoch': '2.155'}
{'loss': '0.2612', 'grad_norm': '0.5602', 'learning_rate': '5.744e-05', 'rewards/chosen': '-3.04', 'rewards/rejected': '-5.265', 'rewards/accuracies': '0.8609', 'rewards/margins': '2.225', 'logps/chosen': '-110.8', 'logps/rejected': '-138.8', 'logits/chosen': '-4.365', 'logits/rejected': '-4.279', 'epoch': '2.165'}
{'loss': '0.2652', 'grad_norm': '0.5245', 'learning_rate': '5.669e-05', 'rewards/chosen': '-3.134', 'rewards/rejected': '-5.365', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.231', 'logps/chosen': '-113.5', 'logps/rejected': '-140.8', 'logits/chosen': '-4.356', 'logits/rejected': '-4.298', 'epoch': '2.176'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)


  0%|          | 0/300 [00:00<?, ?it/s]

  1%|          | 2/300 [00:00<00:52,  5.68it/s]

  1%|          | 3/300 [00:00<01:48,  2.73it/s]

  1%|▏         | 4/300 [00:01<01:35,  3.09it/s]

  2%|▏         | 5/300 [00:01<01:45,  2.79it/s]

  2%|▏         | 6/300 [00:01<01:40,  2.91it/s]

  2%|▏         | 7/300 [00:02<01:54,  2.55it/s]

  3%|▎         | 8/300 [00:02<01:50,  2.63it/s]

  3%|▎         | 9/300 [00:03<01:52,  2.59it/s]

  3%|▎         | 10/300 [00:03<01:48,  2.67it/s]

  4%|▎         | 11/300 [00:04<02:06,  2.29it/s]

  4%|▍         | 12/300 [00:04<01:48,  2.65it/s]

  4%|▍         | 13/300 [00:04<01:49,  2.61it/s]

  5%|▍         | 14/300 [00:05<01:40,  2.85it/s]

  5%|▌         | 15/300 [00:05<01:49,  2.61it/s]

  5%|▌         | 16/300 [00:05<01:36,  2.95it/s]

  6%|▌         | 17/300 [00:06<01:39,  2.83it/s]

  6%|▌         | 18/300 [00:06<01:29,  3.14it/s]

  6%|▋         | 19/300 [00:06<01:40,  2.81it/s]

  7%|▋         | 20/300 [00:07<01:32,  3.04it/s]

  7%|▋         | 21/300 [00:07<01:41,  2.74it/s]

  7%|▋         | 22/300 [00:07<01:30,  3.07it/s]

  8%|▊         | 23/300 [00:08<01:35,  2.91it/s]

  8%|▊         | 24/300 [00:08<01:25,  3.23it/s]

  8%|▊         | 25/300 [00:08<01:37,  2.83it/s]

  9%|▊         | 26/300 [00:09<01:29,  3.05it/s]

  9%|▉         | 27/300 [00:09<01:38,  2.78it/s]

  9%|▉         | 28/300 [00:09<01:27,  3.12it/s]

 10%|▉         | 29/300 [00:10<01:29,  3.01it/s]

 10%|█         | 30/300 [00:10<01:21,  3.31it/s]

 10%|█         | 31/300 [00:10<01:24,  3.19it/s]

 11%|█         | 32/300 [00:10<01:17,  3.45it/s]

 11%|█         | 33/300 [00:11<01:20,  3.33it/s]

 11%|█▏        | 34/300 [00:11<01:15,  3.50it/s]

 12%|█▏        | 35/300 [00:11<01:22,  3.23it/s]

 12%|█▏        | 36/300 [00:12<01:17,  3.39it/s]

 12%|█▏        | 37/300 [00:12<01:27,  2.99it/s]

 13%|█▎        | 38/300 [00:12<01:18,  3.33it/s]

 13%|█▎        | 39/300 [00:13<01:26,  3.03it/s]

 13%|█▎        | 40/300 [00:13<01:18,  3.32it/s]

 14%|█▎        | 41/300 [00:13<01:25,  3.03it/s]

 14%|█▍        | 42/300 [00:14<01:17,  3.33it/s]

 14%|█▍        | 43/300 [00:14<01:23,  3.08it/s]

 15%|█▍        | 44/300 [00:14<01:16,  3.35it/s]

 15%|█▌        | 45/300 [00:15<01:22,  3.08it/s]

 15%|█▌        | 46/300 [00:15<01:24,  3.02it/s]

 16%|█▌        | 47/300 [00:15<01:29,  2.81it/s]

 16%|█▌        | 48/300 [00:16<01:22,  3.05it/s]

 16%|█▋        | 49/300 [00:16<01:23,  3.02it/s]

 17%|█▋        | 50/300 [00:16<01:14,  3.36it/s]

 17%|█▋        | 51/300 [00:16<01:17,  3.19it/s]

 17%|█▋        | 52/300 [00:17<01:14,  3.31it/s]

 18%|█▊        | 53/300 [00:17<01:27,  2.81it/s]

 18%|█▊        | 54/300 [00:17<01:19,  3.10it/s]

 18%|█▊        | 55/300 [00:18<01:26,  2.82it/s]

 19%|█▊        | 56/300 [00:18<01:15,  3.25it/s]

 19%|█▉        | 57/300 [00:18<01:19,  3.07it/s]

 19%|█▉        | 58/300 [00:19<01:20,  3.00it/s]

 20%|█▉        | 59/300 [00:19<01:25,  2.81it/s]

 20%|██        | 60/300 [00:19<01:17,  3.11it/s]

 20%|██        | 61/300 [00:20<01:20,  2.96it/s]

 21%|██        | 62/300 [00:20<01:20,  2.96it/s]

 21%|██        | 63/300 [00:21<01:25,  2.76it/s]

 21%|██▏       | 64/300 [00:21<01:24,  2.80it/s]

 22%|██▏       | 65/300 [00:21<01:27,  2.69it/s]

 22%|██▏       | 66/300 [00:22<01:19,  2.95it/s]

 22%|██▏       | 67/300 [00:22<01:26,  2.68it/s]

 23%|██▎       | 68/300 [00:22<01:18,  2.97it/s]

 23%|██▎       | 69/300 [00:23<01:21,  2.85it/s]

 23%|██▎       | 70/300 [00:23<01:12,  3.16it/s]

 24%|██▎       | 71/300 [00:23<01:15,  3.02it/s]

 24%|██▍       | 72/300 [00:24<01:09,  3.27it/s]

 24%|██▍       | 73/300 [00:24<01:12,  3.13it/s]

 25%|██▍       | 74/300 [00:24<01:08,  3.31it/s]

 25%|██▌       | 75/300 [00:25<01:13,  3.07it/s]

 25%|██▌       | 76/300 [00:25<01:14,  3.00it/s]

 26%|██▌       | 77/300 [00:25<01:19,  2.81it/s]

 26%|██▌       | 78/300 [00:26<01:13,  3.04it/s]

 26%|██▋       | 79/300 [00:26<01:15,  2.93it/s]

 27%|██▋       | 80/300 [00:26<01:11,  3.09it/s]

 27%|██▋       | 81/300 [00:27<01:19,  2.76it/s]

 27%|██▋       | 82/300 [00:27<01:11,  3.07it/s]

 28%|██▊       | 83/300 [00:27<01:13,  2.97it/s]

 28%|██▊       | 84/300 [00:28<01:13,  2.93it/s]

 28%|██▊       | 85/300 [00:28<01:18,  2.74it/s]

 29%|██▊       | 86/300 [00:28<01:10,  3.04it/s]

 29%|██▉       | 87/300 [00:29<01:12,  2.94it/s]

 29%|██▉       | 88/300 [00:29<01:06,  3.19it/s]

 30%|██▉       | 89/300 [00:29<01:10,  2.98it/s]

 30%|███       | 90/300 [00:30<01:05,  3.21it/s]

 30%|███       | 91/300 [00:30<01:09,  3.02it/s]

 31%|███       | 92/300 [00:30<01:03,  3.28it/s]

 31%|███       | 93/300 [00:31<01:09,  3.00it/s]

 31%|███▏      | 94/300 [00:31<01:02,  3.30it/s]

 32%|███▏      | 95/300 [00:31<01:07,  3.03it/s]

 32%|███▏      | 96/300 [00:31<01:03,  3.22it/s]

 32%|███▏      | 97/300 [00:32<01:05,  3.11it/s]

 33%|███▎      | 98/300 [00:32<01:01,  3.27it/s]

 33%|███▎      | 99/300 [00:33<01:08,  2.92it/s]

 33%|███▎      | 100/300 [00:33<01:04,  3.10it/s]

 34%|███▎      | 101/300 [00:33<01:07,  2.93it/s]

 34%|███▍      | 102/300 [00:34<01:06,  3.00it/s]

 34%|███▍      | 103/300 [00:34<01:18,  2.51it/s]

 35%|███▍      | 104/300 [00:34<01:08,  2.86it/s]

 35%|███▌      | 105/300 [00:35<01:09,  2.81it/s]

 35%|███▌      | 106/300 [00:35<01:03,  3.05it/s]

 36%|███▌      | 107/300 [00:35<01:09,  2.77it/s]

 36%|███▌      | 108/300 [00:36<01:05,  2.91it/s]

 36%|███▋      | 109/300 [00:36<01:08,  2.80it/s]

 37%|███▋      | 110/300 [00:36<01:03,  2.98it/s]

 37%|███▋      | 111/300 [00:37<01:12,  2.62it/s]

 37%|███▋      | 112/300 [00:37<01:06,  2.83it/s]

 38%|███▊      | 113/300 [00:38<01:08,  2.75it/s]

 38%|███▊      | 114/300 [00:38<01:07,  2.77it/s]

 38%|███▊      | 115/300 [00:38<01:09,  2.65it/s]

 39%|███▊      | 116/300 [00:39<01:01,  2.97it/s]

 39%|███▉      | 117/300 [00:39<01:03,  2.89it/s]

 39%|███▉      | 118/300 [00:39<00:58,  3.12it/s]

 40%|███▉      | 119/300 [00:40<01:01,  2.95it/s]

 40%|████      | 120/300 [00:40<01:00,  2.99it/s]

 40%|████      | 121/300 [00:40<01:02,  2.87it/s]

 41%|████      | 122/300 [00:40<00:57,  3.10it/s]

 41%|████      | 123/300 [00:41<01:01,  2.89it/s]

 41%|████▏     | 124/300 [00:41<00:57,  3.06it/s]

 42%|████▏     | 125/300 [00:42<00:59,  2.94it/s]

 42%|████▏     | 126/300 [00:42<00:59,  2.92it/s]

 42%|████▏     | 127/300 [00:42<01:01,  2.79it/s]

 43%|████▎     | 128/300 [00:43<00:57,  2.99it/s]

 43%|████▎     | 129/300 [00:43<01:00,  2.84it/s]

 43%|████▎     | 130/300 [00:43<00:54,  3.13it/s]

 44%|████▎     | 131/300 [00:44<01:00,  2.78it/s]

 44%|████▍     | 132/300 [00:44<00:55,  3.02it/s]

 44%|████▍     | 133/300 [00:44<01:02,  2.67it/s]

 45%|████▍     | 134/300 [00:45<00:56,  2.93it/s]

 45%|████▌     | 135/300 [00:45<00:58,  2.81it/s]

 45%|████▌     | 136/300 [00:45<00:52,  3.10it/s]

 46%|████▌     | 137/300 [00:46<00:53,  3.04it/s]

 46%|████▌     | 138/300 [00:46<00:55,  2.92it/s]

 46%|████▋     | 139/300 [00:46<00:54,  2.96it/s]

 47%|████▋     | 140/300 [00:47<00:49,  3.25it/s]

 47%|████▋     | 141/300 [00:47<00:50,  3.15it/s]

 47%|████▋     | 142/300 [00:47<00:51,  3.05it/s]

 48%|████▊     | 143/300 [00:48<00:54,  2.89it/s]

 48%|████▊     | 144/300 [00:48<00:48,  3.19it/s]

 48%|████▊     | 145/300 [00:48<00:49,  3.14it/s]

 49%|████▊     | 146/300 [00:48<00:46,  3.30it/s]

 49%|████▉     | 147/300 [00:49<00:53,  2.87it/s]

 49%|████▉     | 148/300 [00:49<00:50,  3.03it/s]

 50%|████▉     | 149/300 [00:50<00:59,  2.56it/s]

 50%|█████     | 150/300 [00:50<00:53,  2.80it/s]

 50%|█████     | 151/300 [00:50<00:54,  2.75it/s]

 51%|█████     | 152/300 [00:51<00:48,  3.08it/s]

 51%|█████     | 153/300 [00:51<00:48,  3.01it/s]

 51%|█████▏    | 154/300 [00:51<00:44,  3.30it/s]

 52%|█████▏    | 155/300 [00:52<00:46,  3.14it/s]

 52%|█████▏    | 156/300 [00:52<00:41,  3.47it/s]

 52%|█████▏    | 157/300 [00:52<00:45,  3.12it/s]

 53%|█████▎    | 158/300 [00:52<00:43,  3.26it/s]

 53%|█████▎    | 159/300 [00:53<00:51,  2.72it/s]

 53%|█████▎    | 160/300 [00:53<00:47,  2.96it/s]

 54%|█████▎    | 161/300 [00:54<00:50,  2.73it/s]

 54%|█████▍    | 162/300 [00:54<00:49,  2.77it/s]

 54%|█████▍    | 163/300 [00:55<00:55,  2.45it/s]

 55%|█████▍    | 164/300 [00:55<00:53,  2.56it/s]

 55%|█████▌    | 165/300 [00:55<00:52,  2.55it/s]

 55%|█████▌    | 166/300 [00:56<00:50,  2.66it/s]

 56%|█████▌    | 167/300 [00:56<00:54,  2.44it/s]

 56%|█████▌    | 168/300 [00:56<00:47,  2.76it/s]

 56%|█████▋    | 169/300 [00:57<00:52,  2.49it/s]

 57%|█████▋    | 170/300 [00:57<00:50,  2.59it/s]

 57%|█████▋    | 171/300 [00:58<00:50,  2.54it/s]

 57%|█████▋    | 172/300 [00:58<00:44,  2.88it/s]

 58%|█████▊    | 173/300 [00:58<00:46,  2.73it/s]

 58%|█████▊    | 174/300 [00:59<00:41,  3.06it/s]

 58%|█████▊    | 175/300 [00:59<00:44,  2.79it/s]

 59%|█████▊    | 176/300 [00:59<00:39,  3.12it/s]

 59%|█████▉    | 177/300 [01:00<00:44,  2.78it/s]

 59%|█████▉    | 178/300 [01:00<00:38,  3.20it/s]

 60%|█████▉    | 179/300 [01:00<00:41,  2.94it/s]

 60%|██████    | 180/300 [01:00<00:37,  3.22it/s]

 60%|██████    | 181/300 [01:01<00:41,  2.89it/s]

 61%|██████    | 182/300 [01:01<00:36,  3.20it/s]

 61%|██████    | 183/300 [01:02<00:40,  2.87it/s]

 61%|██████▏   | 184/300 [01:02<00:40,  2.86it/s]

 62%|██████▏   | 185/300 [01:02<00:42,  2.68it/s]

 62%|██████▏   | 186/300 [01:03<00:39,  2.88it/s]

 62%|██████▏   | 187/300 [01:03<00:39,  2.87it/s]

 63%|██████▎   | 188/300 [01:03<00:35,  3.15it/s]

 63%|██████▎   | 189/300 [01:04<00:38,  2.85it/s]

 63%|██████▎   | 190/300 [01:04<00:35,  3.09it/s]

 64%|██████▎   | 191/300 [01:04<00:41,  2.64it/s]

 64%|██████▍   | 192/300 [01:05<00:39,  2.75it/s]

 64%|██████▍   | 193/300 [01:05<00:43,  2.46it/s]

 65%|██████▍   | 194/300 [01:06<00:41,  2.57it/s]

 65%|██████▌   | 195/300 [01:06<00:42,  2.49it/s]

 65%|██████▌   | 196/300 [01:06<00:38,  2.73it/s]

 66%|██████▌   | 197/300 [01:07<00:38,  2.65it/s]

 66%|██████▌   | 198/300 [01:07<00:35,  2.90it/s]

 66%|██████▋   | 199/300 [01:07<00:36,  2.80it/s]

 67%|██████▋   | 200/300 [01:08<00:33,  2.99it/s]

 67%|██████▋   | 201/300 [01:08<00:35,  2.80it/s]

 67%|██████▋   | 202/300 [01:08<00:31,  3.09it/s]

 68%|██████▊   | 203/300 [01:09<00:33,  2.94it/s]

 68%|██████▊   | 204/300 [01:09<00:29,  3.23it/s]

 68%|██████▊   | 205/300 [01:09<00:31,  3.02it/s]

 69%|██████▊   | 206/300 [01:10<00:27,  3.41it/s]

 69%|██████▉   | 207/300 [01:10<00:31,  2.94it/s]

 69%|██████▉   | 208/300 [01:10<00:31,  2.95it/s]

 70%|██████▉   | 209/300 [01:11<00:32,  2.80it/s]

 70%|███████   | 210/300 [01:11<00:29,  3.02it/s]

 70%|███████   | 211/300 [01:11<00:33,  2.67it/s]

 71%|███████   | 212/300 [01:12<00:29,  2.98it/s]

 71%|███████   | 213/300 [01:12<00:31,  2.74it/s]

 71%|███████▏  | 214/300 [01:12<00:28,  3.00it/s]

 72%|███████▏  | 215/300 [01:13<00:30,  2.80it/s]

 72%|███████▏  | 216/300 [01:13<00:27,  3.09it/s]

 72%|███████▏  | 217/300 [01:14<00:31,  2.64it/s]

 73%|███████▎  | 218/300 [01:14<00:26,  3.07it/s]

 73%|███████▎  | 219/300 [01:14<00:28,  2.81it/s]

 73%|███████▎  | 220/300 [01:14<00:25,  3.11it/s]

 74%|███████▎  | 221/300 [01:15<00:28,  2.78it/s]

 74%|███████▍  | 222/300 [01:15<00:25,  3.09it/s]

 74%|███████▍  | 223/300 [01:16<00:27,  2.75it/s]

 75%|███████▍  | 224/300 [01:16<00:25,  2.96it/s]

 75%|███████▌  | 225/300 [01:16<00:26,  2.81it/s]

 75%|███████▌  | 226/300 [01:17<00:24,  3.05it/s]

 76%|███████▌  | 227/300 [01:17<00:26,  2.80it/s]

 76%|███████▌  | 228/300 [01:17<00:23,  3.13it/s]

 76%|███████▋  | 229/300 [01:18<00:25,  2.75it/s]

 77%|███████▋  | 230/300 [01:18<00:24,  2.86it/s]

 77%|███████▋  | 231/300 [01:18<00:25,  2.70it/s]

 77%|███████▋  | 232/300 [01:19<00:24,  2.76it/s]

 78%|███████▊  | 233/300 [01:19<00:27,  2.45it/s]

 78%|███████▊  | 234/300 [01:19<00:23,  2.78it/s]

 78%|███████▊  | 235/300 [01:20<00:24,  2.60it/s]

 79%|███████▊  | 236/300 [01:20<00:23,  2.67it/s]

 79%|███████▉  | 237/300 [01:21<00:24,  2.53it/s]

 79%|███████▉  | 238/300 [01:21<00:21,  2.84it/s]

 80%|███████▉  | 239/300 [01:21<00:21,  2.83it/s]

 80%|████████  | 240/300 [01:22<00:19,  3.15it/s]

 80%|████████  | 241/300 [01:22<00:19,  2.96it/s]

 81%|████████  | 242/300 [01:22<00:19,  3.00it/s]

 81%|████████  | 243/300 [01:23<00:21,  2.68it/s]

 81%|████████▏ | 244/300 [01:23<00:20,  2.79it/s]

 82%|████████▏ | 245/300 [01:24<00:22,  2.48it/s]

 82%|████████▏ | 246/300 [01:24<00:20,  2.63it/s]

 82%|████████▏ | 247/300 [01:24<00:21,  2.50it/s]

 83%|████████▎ | 248/300 [01:25<00:18,  2.85it/s]

 83%|████████▎ | 249/300 [01:25<00:18,  2.73it/s]

 83%|████████▎ | 250/300 [01:25<00:17,  2.85it/s]

 84%|████████▎ | 251/300 [01:26<00:17,  2.76it/s]

 84%|████████▍ | 252/300 [01:26<00:16,  2.96it/s]

 84%|████████▍ | 253/300 [01:26<00:16,  2.81it/s]

 85%|████████▍ | 254/300 [01:27<00:14,  3.18it/s]

 85%|████████▌ | 255/300 [01:27<00:16,  2.75it/s]

 85%|████████▌ | 256/300 [01:27<00:14,  3.09it/s]

 86%|████████▌ | 257/300 [01:28<00:14,  2.89it/s]

 86%|████████▌ | 258/300 [01:28<00:13,  3.16it/s]

 86%|████████▋ | 259/300 [01:28<00:13,  2.94it/s]

 87%|████████▋ | 260/300 [01:29<00:12,  3.14it/s]

 87%|████████▋ | 261/300 [01:29<00:13,  2.97it/s]

 87%|████████▋ | 262/300 [01:29<00:11,  3.26it/s]

 88%|████████▊ | 263/300 [01:30<00:12,  3.00it/s]

 88%|████████▊ | 264/300 [01:30<00:12,  2.99it/s]

 88%|████████▊ | 265/300 [01:30<00:13,  2.59it/s]

 89%|████████▊ | 266/300 [01:31<00:12,  2.68it/s]

 89%|████████▉ | 267/300 [01:31<00:13,  2.42it/s]

 89%|████████▉ | 268/300 [01:32<00:11,  2.78it/s]

 90%|████████▉ | 269/300 [01:32<00:12,  2.46it/s]

 90%|█████████ | 270/300 [01:32<00:11,  2.63it/s]

 90%|█████████ | 271/300 [01:33<00:12,  2.37it/s]

 91%|█████████ | 272/300 [01:33<00:10,  2.62it/s]

 91%|█████████ | 273/300 [01:34<00:11,  2.33it/s]

 91%|█████████▏| 274/300 [01:34<00:09,  2.67it/s]

 92%|█████████▏| 275/300 [01:34<00:10,  2.42it/s]

 92%|█████████▏| 276/300 [01:35<00:09,  2.60it/s]

 92%|█████████▏| 277/300 [01:35<00:09,  2.47it/s]

 93%|█████████▎| 278/300 [01:36<00:07,  2.79it/s]

 93%|█████████▎| 279/300 [01:36<00:07,  2.63it/s]

 93%|█████████▎| 280/300 [01:36<00:07,  2.75it/s]

 94%|█████████▎| 281/300 [01:37<00:07,  2.63it/s]

 94%|█████████▍| 282/300 [01:37<00:06,  2.95it/s]

 94%|█████████▍| 283/300 [01:37<00:06,  2.75it/s]

 95%|█████████▍| 284/300 [01:38<00:05,  3.03it/s]

 95%|█████████▌| 285/300 [01:38<00:05,  2.64it/s]

 95%|█████████▌| 286/300 [01:38<00:05,  2.70it/s]

 96%|█████████▌| 287/300 [01:39<00:04,  2.63it/s]

 96%|█████████▌| 288/300 [01:39<00:03,  3.05it/s]

 96%|█████████▋| 289/300 [01:39<00:03,  2.80it/s]

 97%|█████████▋| 290/300 [01:40<00:03,  2.82it/s]

 97%|█████████▋| 291/300 [01:40<00:03,  2.62it/s]

 97%|█████████▋| 292/300 [01:41<00:02,  2.94it/s]

 98%|█████████▊| 293/300 [01:41<00:02,  2.78it/s]

 98%|█████████▊| 294/300 [01:41<00:01,  3.02it/s]

 98%|█████████▊| 295/300 [01:42<00:01,  2.95it/s]

 99%|█████████▊| 296/300 [01:42<00:01,  3.15it/s]

 99%|█████████▉| 297/300 [01:42<00:01,  2.92it/s]

 99%|█████████▉| 298/300 [01:42<00:00,  3.28it/s]

100%|█████████▉| 299/300 [01:43<00:00,  2.88it/s]

100%|██████████| 300/300 [01:43<00:00,  2.75it/s]
                                                       

                                                 

 73%|███████▎  | 2000/2757 [4:34:32<1:39:59,  7.93s/it]

100%|██████████| 300/300 [01:43<00:00,  2.75it/s]

                                                 Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-2000/tokenizer_config.json.

 73%|███████▎  | 2001/2757 [4:34:46<8:39:03, 41.20s/it]
 73%|███████▎  | 2002/2757 [4:34:54<6:34:09, 31.32s/it]
 73%|███████▎  | 2003/2757 [4:35:02<5:04:47, 24.25s/it]
 73%|███████▎  | 2004/2757 [4:35:10<4:02:00, 19.28s/it]
 73%|███████▎  | 2005/2757 [4:35:17<3:17:12, 15.74s/it]
 73%|███████▎  | 2006/2757 [4:35:25<2:46:16, 13.28s/it]
 73%|███████▎  | 2007/2757 [4:35:33<2:27:25, 11.79s/it]
 73%|███████▎  | 2008/2757 [4:35:41<2:12:05, 10.58s/it]
 73%|███████▎  | 2009/2757 [4:35:48<2:00:45,  9.69s/it]
 73%|███████▎  | 2010/2757 [4:35:57<1:55:55,  9.31s/it]
                                                       

 73%|███████▎  | 2010/2757 [4:35:57<1:55:55,  9.31s/it]
 73%|███████▎  | 2011/2757 [4:36:04<1:50:02,  8.85s/it]
 73%|███████▎  | 2012/2757 [4:36:12<1:45:06,  8.47s/it]
 73%|███████▎  | 2013/2757 [4:36:20<1:42:05,  8.23s/it]
 73%|███████▎  | 2014/2757 [4:36:28<1:40:27,  8.11s/it]
 73%|███████▎  | 2015/2757 [4:36:35<1:38:49,  7.99s/it]
 73%|███████▎  | 2016/2757 [4:36:44<1:40:22,  8.13s/it]
 73%|███████▎  | 2017/2757 [4:36:52<1:39:20,  8.05s/it]
 73%|███████▎  | 2018/2757 [4:37:00<1:39:26,  8.07s/it]
 73%|███████▎  | 2019/2757 [4:37:08<1:39:02,  8.05s/it]
 73%|███████▎  | 2020/2757 [4:37:15<1:37:06,  7.91s/it]
                                                       

 73%|███████▎  | 2020/2757 [4:37:15<1:37:06,  7.91s/it]
 73%|███████▎  | 2021/2757 [4:37:24<1:38:40,  8.04s/it]
 73%|███████▎  | 2022/2757 [4:37:31<1:37:17,  7.94s/it]
 73%|███████▎  | 2023/2757 [4:37:39<1:35:33,  7.81s/it]
 73%|███████▎  | 2024/2757 [4:37:47<1:35:55,  7.85s/it]
 73%|███████▎  | 2025/2757 [4:37:55<1:36:24,  7.90s/it]
 73%|███████▎  | 2026/2757 [4:38:03<1:37:26,  8.00s/it]
 74%|███████▎  | 2027/2757 [4:38:11<1:37:17,  8.00s/it]
 74%|███████▎  | 2028/2757 [4:38:19<1:37:57,  8.06s/it]
 74%|███████▎  | 2029/2757 [4:38:27<1:37:00,  8.00s/it]
 74%|███████▎  | 2030/2757 [4:38:35<1:37:14,  8.03s/it]
                                                       

 74%|███████▎  | 2030/2757 [4:38:35<1:37:14,  8.03s/it]
 74%|███████▎  | 2031/2757 [4:38:43<1:37:23,  8.05s/it]
 74%|███████▎  | 2032/2757 [4:38:51<1:35:32,  7.91s/it]
 74%|███████▎  | 2033/2757 [4:38:59<1:34:31,  7.83s/it]
 74%|███████▍  | 2034/2757 [4:39:06<1:34:28,  7.84s/it]
 74%|███████▍  | 2035/2757 [4:39:15<1:37:06,  8.07s/it]
 74%|███████▍  | 2036/2757 [4:39:23<1:36:27,  8.03s/it]
 74%|███████▍  | 2037/2757 [4:39:31<1:35:18,  7.94s/it]
 74%|███████▍  | 2038/2757 [4:39:39<1:36:29,  8.05s/it]
 74%|███████▍  | 2039/2757 [4:39:47<1:37:51,  8.18s/it]
 74%|███████▍  | 2040/2757 [4:39:56<1:37:33,  8.16s/it]
                                                       

 74%|███████▍  | 2040/2757 [4:39:56<1:37:33,  8.16s/it]
 74%|███████▍  | 2041/2757 [4:40:04<1:36:30,  8.09s/it]
 74%|███████▍  | 2042/2757 [4:40:11<1:35:19,  8.00s/it]
 74%|███████▍  | 2043/2757 [4:40:19<1:34:26,  7.94s/it]
 74%|███████▍  | 2044/2757 [4:40:27<1:33:36,  7.88s/it]
 74%|███████▍  | 2045/2757 [4:40:35<1:34:41,  7.98s/it]
 74%|███████▍  | 2046/2757 [4:40:43<1:33:36,  7.90s/it]
 74%|███████▍  | 2047/2757 [4:40:51<1:35:05,  8.04s/it]
 74%|███████▍  | 2048/2757 [4:40:59<1:34:23,  7.99s/it]
 74%|███████▍  | 2049/2757 [4:41:07<1:33:49,  7.95s/it]
 74%|███████▍  | 2050/2757 [4:41:15<1:33:52,  7.97s/it]
                                                       

 74%|███████▍  | 2050/2757 [4:41:15<1:33:52,  7.97s/it]
 74%|███████▍  | 2051/2757 [4:41:23<1:32:49,  7.89s/it]
 74%|███████▍  | 2052/2757 [4:41:30<1:31:56,  7.82s/it]
 74%|███████▍  | 2053/2757 [4:41:38<1:32:01,  7.84s/it]
 75%|███████▍  | 2054/2757 [4:41:46<1:33:03,  7.94s/it]
 75%|███████▍  | 2055/2757 [4:41:54<1:32:18,  7.89s/it]
 75%|███████▍  | 2056/2757 [4:42:02<1:33:59,  8.04s/it]
 75%|███████▍  | 2057/2757 [4:42:10<1:32:52,  7.96s/it]
 75%|███████▍  | 2058/2757 [4:42:18<1:33:46,  8.05s/it]
 75%|███████▍  | 2059/2757 [4:42:27<1:35:33,  8.21s/it]
 75%|███████▍  | 2060/2757 [4:42:35<1:33:53,  8.08s/it]
                                                       

 75%|███████▍  | 2060/2757 [4:42:35<1:33:53,  8.08s/it]
 75%|███████▍  | 2061/2757 [4:42:43<1:33:35,  8.07s/it]
 75%|███████▍  | 2062/2757 [4:42:51<1:32:25,  7.98s/it]
 75%|███████▍  | 2063/2757 [4:42:59<1:32:39,  8.01s/it]
 75%|███████▍  | 2064/2757 [4:43:07<1:34:47,  8.21s/it]
 75%|███████▍  | 2065/2757 [4:43:15<1:33:17,  8.09s/it]
 75%|███████▍  | 2066/2757 [4:43:23<1:32:11,  8.01s/it]
 75%|███████▍  | 2067/2757 [4:43:31<1:32:51,  8.07s/it]
 75%|███████▌  | 2068/2757 [4:43:39<1:32:08,  8.02s/it]
 75%|███████▌  | 2069/2757 [4:43:47<1:31:40,  8.00s/it]
 75%|███████▌  | 2070/2757 [4:43:55<1:29:27,  7.81s/it]
                                                       

 75%|███████▌  | 2070/2757 [4:43:55<1:29:27,  7.81s/it]
 75%|███████▌  | 2071/2757 [4:44:02<1:28:34,  7.75s/it]
 75%|███████▌  | 2072/2757 [4:44:10<1:28:14,  7.73s/it]
 75%|███████▌  | 2073/2757 [4:44:18<1:29:21,  7.84s/it]
 75%|███████▌  | 2074/2757 [4:44:26<1:29:43,  7.88s/it]
 75%|███████▌  | 2075/2757 [4:44:34<1:28:56,  7.82s/it]
 75%|███████▌  | 2076/2757 [4:44:41<1:28:27,  7.79s/it]
 75%|███████▌  | 2077/2757 [4:44:49<1:29:43,  7.92s/it]
 75%|███████▌  | 2078/2757 [4:44:57<1:29:33,  7.91s/it]
 75%|███████▌  | 2079/2757 [4:45:06<1:31:28,  8.09s/it]
 75%|███████▌  | 2080/2757 [4:45:14<1:32:30,  8.20s/it]
                                                       

 75%|███████▌  | 2080/2757 [4:45:14<1:32:30,  8.20s/it]
 75%|███████▌  | 2081/2757 [4:45:22<1:31:40,  8.14s/it]
 76%|███████▌  | 2082/2757 [4:45:30<1:31:11,  8.11s/it]
 76%|███████▌  | 2083/2757 [4:45:38<1:29:14,  7.94s/it]
 76%|███████▌  | 2084/2757 [4:45:46<1:29:24,  7.97s/it]
 76%|███████▌  | 2085/2757 [4:45:54<1:29:18,  7.97s/it]
 76%|███████▌  | 2086/2757 [4:46:02<1:28:52,  7.95s/it]
 76%|███████▌  | 2087/2757 [4:46:10<1:29:15,  7.99s/it]
 76%|███████▌  | 2088/2757 [4:46:18<1:29:00,  7.98s/it]
 76%|███████▌  | 2089/2757 [4:46:26<1:30:03,  8.09s/it]
 76%|███████▌  | 2090/2757 [4:46:34<1:29:46,  8.08s/it]
                                                       

 76%|███████▌  | 2090/2757 [4:46:34<1:29:46,  8.08s/it]
 76%|███████▌  | 2091/2757 [4:46:43<1:30:13,  8.13s/it]
 76%|███████▌  | 2092/2757 [4:46:51<1:30:40,  8.18s/it]
 76%|███████▌  | 2093/2757 [4:46:59<1:30:30,  8.18s/it]
 76%|███████▌  | 2094/2757 [4:47:07<1:29:10,  8.07s/it]
 76%|███████▌  | 2095/2757 [4:47:15<1:30:09,  8.17s/it]
 76%|███████▌  | 2096/2757 [4:47:23<1:29:20,  8.11s/it]
 76%|███████▌  | 2097/2757 [4:47:31<1:29:03,  8.10s/it]
 76%|███████▌  | 2098/2757 [4:47:40<1:29:45,  8.17s/it]
 76%|███████▌  | 2099/2757 [4:47:48<1:30:00,  8.21s/it]
 76%|███████▌  | 2100/2757 [4:47:56<1:30:44,  8.29s/it]
                                                       

 76%|███████▌  | 2100/2757 [4:47:56<1:30:44,  8.29s/it]
 76%|███████▌  | 2101/2757 [4:48:05<1:31:07,  8.33s/it]
 76%|███████▌  | 2102/2757 [4:48:13<1:31:02,  8.34s/it]
 76%|███████▋  | 2103/2757 [4:48:21<1:29:23,  8.20s/it]
 76%|███████▋  | 2104/2757 [4:48:29<1:27:44,  8.06s/it]
 76%|███████▋  | 2105/2757 [4:48:37<1:28:13,  8.12s/it]
 76%|███████▋  | 2106/2757 [4:48:45<1:27:48,  8.09s/it]
 76%|███████▋  | 2107/2757 [4:48:53<1:26:48,  8.01s/it]
 76%|███████▋  | 2108/2757 [4:49:01<1:26:17,  7.98s/it]
 76%|███████▋  | 2109/2757 [4:49:08<1:24:56,  7.87s/it]
 77%|███████▋  | 2110/2757 [4:49:16<1:24:39,  7.85s/it]
                                                       

 77%|███████▋  | 2110/2757 [4:49:16<1:24:39,  7.85s/it]
 77%|███████▋  | 2111/2757 [4:49:24<1:24:25,  7.84s/it]
 77%|███████▋  | 2112/2757 [4:49:33<1:26:33,  8.05s/it]
 77%|███████▋  | 2113/2757 [4:49:40<1:25:08,  7.93s/it]
 77%|███████▋  | 2114/2757 [4:49:49<1:26:25,  8.06s/it]
 77%|███████▋  | 2115/2757 [4:49:56<1:25:10,  7.96s/it]
 77%|███████▋  | 2116/2757 [4:50:05<1:26:09,  8.06s/it]
 77%|███████▋  | 2117/2757 [4:50:13<1:26:07,  8.07s/it]
 77%|███████▋  | 2118/2757 [4:50:20<1:24:51,  7.97s/it]
 77%|███████▋  | 2119/2757 [4:50:28<1:24:10,  7.92s/it]
 77%|███████▋  | 2120/2757 [4:50:36<1:25:09,  8.02s/it]
                                                       

 77%|███████▋  | 2120/2757 [4:50:36<1:25:09,  8.02s/it]
 77%|███████▋  | 2121/2757 [4:50:45<1:25:16,  8.04s/it]
 77%|███████▋  | 2122/2757 [4:50:53<1:24:42,  8.00s/it]
 77%|███████▋  | 2123/2757 [4:51:00<1:23:08,  7.87s/it]
 77%|███████▋  | 2124/2757 [4:51:08<1:22:55,  7.86s/it]
 77%|███████▋  | 2125/2757 [4:51:15<1:21:56,  7.78s/it]
 77%|███████▋  | 2126/2757 [4:51:23<1:21:47,  7.78s/it]
 77%|███████▋  | 2127/2757 [4:51:31<1:22:25,  7.85s/it]
 77%|███████▋  | 2128/2757 [4:51:39<1:22:42,  7.89s/it]
 77%|███████▋  | 2129/2757 [4:51:47<1:23:13,  7.95s/it]
 77%|███████▋  | 2130/2757 [4:51:55<1:23:08,  7.96s/it]
                                                       

 77%|███████▋  | 2130/2757 [4:51:55<1:23:08,  7.96s/it]
 77%|███████▋  | 2131/2757 [4:52:03<1:21:52,  7.85s/it]
 77%|███████▋  | 2132/2757 [4:52:11<1:22:26,  7.91s/it]
 77%|███████▋  | 2133/2757 [4:52:19<1:22:12,  7.90s/it]
 77%|███████▋  | 2134/2757 [4:52:27<1:22:31,  7.95s/it]
 77%|███████▋  | 2135/2757 [4:52:35<1:21:40,  7.88s/it]
 77%|███████▋  | 2136/2757 [4:52:43<1:21:49,  7.91s/it]
 78%|███████▊  | 2137/2757 [4:52:50<1:21:08,  7.85s/it]
 78%|███████▊  | 2138/2757 [4:52:58<1:21:46,  7.93s/it]
 78%|███████▊  | 2139/2757 [4:53:07<1:23:06,  8.07s/it]
 78%|███████▊  | 2140/2757 [4:53:15<1:22:52,  8.06s/it]
                                                       

 78%|███████▊  | 2140/2757 [4:53:15<1:22:52,  8.06s/it]
 78%|███████▊  | 2141/2757 [4:53:22<1:21:03,  7.90s/it]
 78%|███████▊  | 2142/2757 [4:53:31<1:22:17,  8.03s/it]
 78%|███████▊  | 2143/2757 [4:53:38<1:21:05,  7.92s/it]
 78%|███████▊  | 2144/2757 [4:53:46<1:19:51,  7.82s/it]
 78%|███████▊  | 2145/2757 [4:53:54<1:19:54,  7.83s/it]
 78%|███████▊  | 2146/2757 [4:54:02<1:21:42,  8.02s/it]
 78%|███████▊  | 2147/2757 [4:54:10<1:21:30,  8.02s/it]
 78%|███████▊  | 2148/2757 [4:54:18<1:20:00,  7.88s/it]
 78%|███████▊  | 2149/2757 [4:54:25<1:18:54,  7.79s/it]
 78%|███████▊  | 2150/2757 [4:54:33<1:19:21,  7.84s/it]
                                                       

 78%|███████▊  | 2150/2757 [4:54:33<1:19:21,  7.84s/it]
 78%|███████▊  | 2151/2757 [4:54:42<1:20:12,  7.94s/it]
 78%|███████▊  | 2152/2757 [4:54:49<1:19:31,  7.89s/it]
 78%|███████▊  | 2153/2757 [4:54:58<1:20:33,  8.00s/it]
 78%|███████▊  | 2154/2757 [4:55:05<1:19:42,  7.93s/it]
 78%|███████▊  | 2155/2757 [4:55:14<1:20:50,  8.06s/it]
 78%|███████▊  | 2156/2757 [4:55:21<1:19:16,  7.91s/it]
 78%|███████▊  | 2157/2757 [4:55:30<1:20:44,  8.07s/it]
 78%|███████▊  | 2158/2757 [4:55:38<1:21:00,  8.11s/it]
 78%|███████▊  | 2159/2757 [4:55:46<1:20:15,  8.05s/it]
 78%|███████▊  | 2160/2757 [4:55:54<1:20:42,  8.11s/it]
                                                       

 78%|███████▊  | 2160/2757 [4:55:54<1:20:42,  8.11s/it]
 78%|███████▊  | 2161/2757 [4:56:02<1:19:43,  8.03s/it]
 78%|███████▊  | 2162/2757 [4:56:09<1:17:32,  7.82s/it]
 78%|███████▊  | 2163/2757 [4:56:17<1:18:05,  7.89s/it]
 78%|███████▊  | 2164/2757 [4:56:25<1:18:23,  7.93s/it]
 79%|███████▊  | 2165/2757 [4:56:33<1:18:33,  7.96s/it]
 79%|███████▊  | 2166/2757 [4:56:41<1:18:43,  7.99s/it]
 79%|███████▊  | 2167/2757 [4:56:50<1:19:22,  8.07s/it]
 79%|███████▊  | 2168/2757 [4:56:58<1:18:50,  8.03s/it]
 79%|███████▊  | 2169/2757 [4:57:06<1:19:22,  8.10s/it]
 79%|███████▊  | 2170/2757 [4:57:14<1:19:06,  8.09s/it]
                                                       

 79%|███████▊  | 2170/2757 [4:57:14<1:19:06,  8.09s/it]
 79%|███████▊  | 2171/2757 [4:57:22<1:17:26,  7.93s/it]
 79%|███████▉  | 2172/2757 [4:57:29<1:17:10,  7.92s/it]
 79%|███████▉  | 2173/2757 [4:57:37<1:17:19,  7.94s/it]
 79%|███████▉  | 2174/2757 [4:57:45<1:17:15,  7.95s/it]
 79%|███████▉  | 2175/2757 [4:57:54<1:17:36,  8.00s/it]
 79%|███████▉  | 2176/2757 [4:58:02<1:18:31,  8.11s/it]
 79%|███████▉  | 2177/2757 [4:58:10<1:17:17,  8.00s/it]
 79%|███████▉  | 2178/2757 [4:58:17<1:16:41,  7.95s/it]
 79%|███████▉  | 2179/2757 [4:58:25<1:16:30,  7.94s/it]
 79%|███████▉  | 2180/2757 [4:58:33<1:16:24,  7.95s/it]
                                                       

 79%|███████▉  | 2180/2757 [4:58:33<1:16:24,  7.95s/it]
 79%|███████▉  | 2181/2757 [4:58:41<1:16:24,  7.96s/it]
 79%|███████▉  | 2182/2757 [4:58:49<1:15:31,  7.88s/it]
 79%|███████▉  | 2183/2757 [4:58:57<1:15:36,  7.90s/it]
 79%|███████▉  | 2184/2757 [4:59:05<1:14:41,  7.82s/it]
 79%|███████▉  | 2185/2757 [4:59:12<1:14:44,  7.84s/it]
 79%|███████▉  | 2186/2757 [4:59:21<1:15:06,  7.89s/it]
 79%|███████▉  | 2187/2757 [4:59:28<1:14:44,  7.87s/it]
 79%|███████▉  | 2188/2757 [4:59:36<1:15:22,  7.95s/it]
 79%|███████▉  | 2189/2757 [4:59:45<1:15:42,  8.00s/it]
 79%|███████▉  | 2190/2757 [4:59:53<1:16:37,  8.11s/it]
                                                       

 79%|███████▉  | 2190/2757 [4:59:53<1:16:37,  8.11s/it]
 79%|███████▉  | 2191/2757 [5:00:01<1:17:15,  8.19s/it]
 80%|███████▉  | 2192/2757 [5:00:09<1:16:08,  8.09s/it]
 80%|███████▉  | 2193/2757 [5:00:17<1:15:57,  8.08s/it]
 80%|███████▉  | 2194/2757 [5:00:25<1:14:20,  7.92s/it]
 80%|███████▉  | 2195/2757 [5:00:33<1:14:51,  7.99s/it]
 80%|███████▉  | 2196/2757 [5:00:41<1:13:53,  7.90s/it]
 80%|███████▉  | 2197/2757 [5:00:48<1:12:26,  7.76s/it]
 80%|███████▉  | 2198/2757 [5:00:56<1:11:58,  7.73s/it]
 80%|███████▉  | 2199/2757 [5:01:04<1:13:18,  7.88s/it]
 80%|███████▉  | 2200/2757 [5:01:12<1:14:14,  8.00s/it]
                                                       

 80%|███████▉  | 2200/2757 [5:01:12<1:14:14,  8.00s/it]
 80%|███████▉  | 2201/2757 [5:01:20<1:14:17,  8.02s/it]
 80%|███████▉  | 2202/2757 [5:01:28<1:13:41,  7.97s/it]
 80%|███████▉  | 2203/2757 [5:01:36<1:13:42,  7.98s/it]
 80%|███████▉  | 2204/2757 [5:01:44<1:13:17,  7.95s/it]
 80%|███████▉  | 2205/2757 [5:01:52<1:13:31,  7.99s/it]
 80%|████████  | 2206/2757 [5:02:00<1:13:11,  7.97s/it]
 80%|████████  | 2207/2757 [5:02:07<1:11:31,  7.80s/it]
 80%|████████  | 2208/2757 [5:02:15<1:12:03,  7.88s/it]
 80%|████████  | 2209/2757 [5:02:23<1:12:16,  7.91s/it]
 80%|████████  | 2210/2757 [5:02:31<1:12:14,  7.92s/it]
                                                       

 80%|████████  | 2210/2757 [5:02:31<1:12:14,  7.92s/it]
 80%|████████  | 2211/2757 [5:02:40<1:12:32,  7.97s/it]
 80%|████████  | 2212/2757 [5:02:48<1:13:27,  8.09s/it]
 80%|████████  | 2213/2757 [5:02:56<1:12:39,  8.01s/it]
 80%|████████  | 2214/2757 [5:03:04<1:12:31,  8.01s/it]
 80%|████████  | 2215/2757 [5:03:11<1:11:04,  7.87s/it]
 80%|████████  | 2216/2757 [5:03:19<1:11:23,  7.92s/it]
 80%|████████  | 2217/2757 [5:03:27<1:11:13,  7.91s/it]
 80%|████████  | 2218/2757 [5:03:35<1:10:38,  7.86s/it]
 80%|████████  | 2219/2757 [5:03:43<1:09:58,  7.80s/it]
 81%|████████  | 2220/2757 [5:03:51<1:10:13,  7.85s/it]
                                                       

 81%|████████  | 2220/2757 [5:03:51<1:10:13,  7.85s/it]
 81%|████████  | 2221/2757 [5:03:59<1:11:15,  7.98s/it]
 81%|████████  | 2222/2757 [5:04:07<1:11:11,  7.98s/it]
 81%|████████  | 2223/2757 [5:04:14<1:09:54,  7.85s/it]
 81%|████████  | 2224/2757 [5:04:22<1:10:24,  7.93s/it]
 81%|████████  | 2225/2757 [5:04:30<1:09:48,  7.87s/it]
 81%|████████  | 2226/2757 [5:04:39<1:11:20,  8.06s/it]
 81%|████████  | 2227/2757 [5:04:46<1:09:31,  7.87s/it]
 81%|████████  | 2228/2757 [5:04:54<1:10:10,  7.96s/it]
 81%|████████  | 2229/2757 [5:05:02<1:08:40,  7.80s/it]
 81%|████████  | 2230/2757 [5:05:10<1:09:17,  7.89s/it]
                                                       

 81%|████████  | 2230/2757 [5:05:10<1:09:17,  7.89s/it]
 81%|████████  | 2231/2757 [5:05:18<1:09:15,  7.90s/it]
 81%|████████  | 2232/2757 [5:05:26<1:10:00,  8.00s/it]
 81%|████████  | 2233/2757 [5:05:34<1:09:49,  7.99s/it]
 81%|████████  | 2234/2757 [5:05:42<1:09:22,  7.96s/it]
 81%|████████  | 2235/2757 [5:05:50<1:08:30,  7.87s/it]
 81%|████████  | 2236/2757 [5:05:57<1:08:12,  7.86s/it]
 81%|████████  | 2237/2757 [5:06:06<1:09:01,  7.96s/it]
 81%|████████  | 2238/2757 [5:06:14<1:08:51,  7.96s/it]
 81%|████████  | 2239/2757 [5:06:22<1:09:11,  8.02s/it]
 81%|████████  | 2240/2757 [5:06:30<1:09:41,  8.09s/it]
                                                       

 81%|████████  | 2240/2757 [5:06:30<1:09:41,  8.09s/it]
 81%|████████▏ | 2241/2757 [5:06:38<1:08:56,  8.02s/it]
 81%|████████▏ | 2242/2757 [5:06:46<1:08:38,  8.00s/it]
 81%|████████▏ | 2243/2757 [5:06:53<1:07:24,  7.87s/it]
 81%|████████▏ | 2244/2757 [5:07:01<1:06:20,  7.76s/it]
 81%|████████▏ | 2245/2757 [5:07:09<1:06:17,  7.77s/it]
 81%|████████▏ | 2246/2757 [5:07:16<1:06:23,  7.79s/it]
 82%|████████▏ | 2247/2757 [5:07:25<1:07:44,  7.97s/it]
 82%|████████▏ | 2248/2757 [5:07:33<1:07:41,  7.98s/it]
 82%|████████▏ | 2249/2757 [5:07:41<1:07:59,  8.03s/it]
 82%|████████▏ | 2250/2757 [5:07:49<1:08:09,  8.07s/it]
                                                       
{'eval_loss': '0.4653', 'eval_runtime': '104.8', 'eval_samples_per_second': '22.91', 'eval_steps_per_second': '2.863', 'eval_rewards/chosen': '-3.413', 'eval_rewards/rejected': '-4.97', 'eval_rewards/accuracies': '0.7125', 'eval_rewards/margins': '1.556', 'eval_logps/chosen': '-113.7', 'eval_logps/rejected': '-135.5', 'eval_logits/chosen': '-4.393', 'eval_logits/rejected': '-4.345', 'epoch': '2.176'}
{'loss': '0.2672', 'grad_norm': '0.64', 'learning_rate': '5.595e-05', 'rewards/chosen': '-3.084', 'rewards/rejected': '-5.276', 'rewards/accuracies': '0.8438', 'rewards/margins': '2.193', 'logps/chosen': '-115.2', 'logps/rejected': '-140.9', 'logits/chosen': '-4.258', 'logits/rejected': '-4.195', 'epoch': '2.187'}
{'loss': '0.2488', 'grad_norm': '0.5022', 'learning_rate': '5.52e-05', 'rewards/chosen': '-2.958', 'rewards/rejected': '-5.24', 'rewards/accuracies': '0.8687', 'rewards/margins': '2.282', 'logps/chosen': '-108.5', 'logps/rejected': '-137.5', 'logits/chosen': '-4.378', 'logits/rejected': '-4.255', 'epoch': '2.198'}
{'loss': '0.2526', 'grad_norm': '0.4892', 'learning_rate': '5.445e-05', 'rewards/chosen': '-2.979', 'rewards/rejected': '-5.242', 'rewards/accuracies': '0.8508', 'rewards/margins': '2.262', 'logps/chosen': '-108.2', 'logps/rejected': '-137.7', 'logits/chosen': '-4.429', 'logits/rejected': '-4.317', 'epoch': '2.209'}
{'loss': '0.2479', 'grad_norm': '0.5125', 'learning_rate': '5.37e-05', 'rewards/chosen': '-2.978', 'rewards/rejected': '-5.298', 'rewards/accuracies': '0.8516', 'rewards/margins': '2.321', 'logps/chosen': '-110.8', 'logps/rejected': '-140.5', 'logits/chosen': '-4.358', 'logits/rejected': '-4.336', 'epoch': '2.22'}
{'loss': '0.2748', 'grad_norm': '0.5904', 'learning_rate': '5.295e-05', 'rewards/chosen': '-2.839', 'rewards/rejected': '-4.998', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.159', 'logps/chosen': '-108.7', 'logps/rejected': '-136.3', 'logits/chosen': '-4.396', 'logits/rejected': '-4.374', 'epoch': '2.231'}
{'loss': '0.2631', 'grad_norm': '0.5153', 'learning_rate': '5.221e-05', 'rewards/chosen': '-2.878', 'rewards/rejected': '-5.143', 'rewards/accuracies': '0.8406', 'rewards/margins': '2.264', 'logps/chosen': '-108.4', 'logps/rejected': '-137.3', 'logits/chosen': '-4.383', 'logits/rejected': '-4.335', 'epoch': '2.242'}
{'loss': '0.2632', 'grad_norm': '0.5087', 'learning_rate': '5.146e-05', 'rewards/chosen': '-2.793', 'rewards/rejected': '-4.991', 'rewards/accuracies': '0.8516', 'rewards/margins': '2.198', 'logps/chosen': '-107.8', 'logps/rejected': '-135.7', 'logits/chosen': '-4.283', 'logits/rejected': '-4.199', 'epoch': '2.253'}
{'loss': '0.2556', 'grad_norm': '0.5399', 'learning_rate': '5.071e-05', 'rewards/chosen': '-2.866', 'rewards/rejected': '-5.147', 'rewards/accuracies': '0.857', 'rewards/margins': '2.281', 'logps/chosen': '-106.8', 'logps/rejected': '-136.2', 'logits/chosen': '-4.345', 'logits/rejected': '-4.317', 'epoch': '2.263'}
{'loss': '0.2625', 'grad_norm': '0.4645', 'learning_rate': '4.996e-05', 'rewards/chosen': '-2.911', 'rewards/rejected': '-5.203', 'rewards/accuracies': '0.8477', 'rewards/margins': '2.292', 'logps/chosen': '-106.5', 'logps/rejected': '-136.2', 'logits/chosen': '-4.257', 'logits/rejected': '-4.259', 'epoch': '2.274'}
{'loss': '0.2609', 'grad_norm': '0.3866', 'learning_rate': '4.921e-05', 'rewards/chosen': '-2.991', 'rewards/rejected': '-5.275', 'rewards/accuracies': '0.8461', 'rewards/margins': '2.285', 'logps/chosen': '-111.6', 'logps/rejected': '-139.2', 'logits/chosen': '-4.299', 'logits/rejected': '-4.223', 'epoch': '2.285'}
{'loss': '0.2628', 'grad_norm': '0.5244', 'learning_rate': '4.847e-05', 'rewards/chosen': '-2.894', 'rewards/rejected': '-5.102', 'rewards/accuracies': '0.8578', 'rewards/margins': '2.207', 'logps/chosen': '-105.7', 'logps/rejected': '-132.3', 'logits/chosen': '-4.305', 'logits/rejected': '-4.244', 'epoch': '2.296'}
{'loss': '0.2798', 'grad_norm': '0.5264', 'learning_rate': '4.772e-05', 'rewards/chosen': '-2.842', 'rewards/rejected': '-4.989', 'rewards/accuracies': '0.8211', 'rewards/margins': '2.147', 'logps/chosen': '-107.4', 'logps/rejected': '-135.3', 'logits/chosen': '-4.359', 'logits/rejected': '-4.288', 'epoch': '2.307'}
{'loss': '0.2712', 'grad_norm': '0.619', 'learning_rate': '4.697e-05', 'rewards/chosen': '-2.925', 'rewards/rejected': '-5.061', 'rewards/accuracies': '0.8414', 'rewards/margins': '2.136', 'logps/chosen': '-109.6', 'logps/rejected': '-136.1', 'logits/chosen': '-4.337', 'logits/rejected': '-4.261', 'epoch': '2.318'}
{'loss': '0.2454', 'grad_norm': '0.4767', 'learning_rate': '4.622e-05', 'rewards/chosen': '-2.863', 'rewards/rejected': '-5.176', 'rewards/accuracies': '0.8625', 'rewards/margins': '2.314', 'logps/chosen': '-110.6', 'logps/rejected': '-139.7', 'logits/chosen': '-4.277', 'logits/rejected': '-4.208', 'epoch': '2.329'}
{'loss': '0.2712', 'grad_norm': '0.4791', 'learning_rate': '4.547e-05', 'rewards/chosen': '-2.719', 'rewards/rejected': '-4.824', 'rewards/accuracies': '0.8414', 'rewards/margins': '2.106', 'logps/chosen': '-105.8', 'logps/rejected': '-131.3', 'logits/chosen': '-4.355', 'logits/rejected': '-4.288', 'epoch': '2.34'}
{'loss': '0.2594', 'grad_norm': '0.6803', 'learning_rate': '4.473e-05', 'rewards/chosen': '-2.693', 'rewards/rejected': '-4.999', 'rewards/accuracies': '0.8469', 'rewards/margins': '2.306', 'logps/chosen': '-105.4', 'logps/rejected': '-134.7', 'logits/chosen': '-4.417', 'logits/rejected': '-4.399', 'epoch': '2.35'}
{'loss': '0.2457', 'grad_norm': '0.5753', 'learning_rate': '4.398e-05', 'rewards/chosen': '-2.723', 'rewards/rejected': '-5.072', 'rewards/accuracies': '0.8555', 'rewards/margins': '2.349', 'logps/chosen': '-108.5', 'logps/rejected': '-138', 'logits/chosen': '-4.324', 'logits/rejected': '-4.302', 'epoch': '2.361'}
{'loss': '0.2549', 'grad_norm': '0.5278', 'learning_rate': '4.323e-05', 'rewards/chosen': '-2.749', 'rewards/rejected': '-5.012', 'rewards/accuracies': '0.8547', 'rewards/margins': '2.263', 'logps/chosen': '-109.8', 'logps/rejected': '-138.1', 'logits/chosen': '-4.221', 'logits/rejected': '-4.199', 'epoch': '2.372'}
{'loss': '0.2544', 'grad_norm': '0.6214', 'learning_rate': '4.248e-05', 'rewards/chosen': '-2.788', 'rewards/rejected': '-5.146', 'rewards/accuracies': '0.8523', 'rewards/margins': '2.358', 'logps/chosen': '-109.3', 'logps/rejected': '-137.8', 'logits/chosen': '-4.419', 'logits/rejected': '-4.302', 'epoch': '2.383'}
{'loss': '0.2642', 'grad_norm': '0.6792', 'learning_rate': '4.174e-05', 'rewards/chosen': '-2.842', 'rewards/rejected': '-5.078', 'rewards/accuracies': '0.8438', 'rewards/margins': '2.235', 'logps/chosen': '-107.8', 'logps/rejected': '-135.4', 'logits/chosen': '-4.336', 'logits/rejected': '-4.239', 'epoch': '2.394'}
{'loss': '0.254', 'grad_norm': '0.5071', 'learning_rate': '4.099e-05', 'rewards/chosen': '-2.94', 'rewards/rejected': '-5.256', 'rewards/accuracies': '0.85', 'rewards/margins': '2.316', 'logps/chosen': '-108.2', 'logps/rejected': '-135.4', 'logits/chosen': '-4.275', 'logits/rejected': '-4.161', 'epoch': '2.405'}
{'loss': '0.2531', 'grad_norm': '0.5598', 'learning_rate': '4.024e-05', 'rewards/chosen': '-2.943', 'rewards/rejected': '-5.273', 'rewards/accuracies': '0.8453', 'rewards/margins': '2.33', 'logps/chosen': '-111.8', 'logps/rejected': '-139.2', 'logits/chosen': '-4.343', 'logits/rejected': '-4.282', 'epoch': '2.416'}
{'loss': '0.2632', 'grad_norm': '0.4813', 'learning_rate': '3.949e-05', 'rewards/chosen': '-2.964', 'rewards/rejected': '-5.288', 'rewards/accuracies': '0.8422', 'rewards/margins': '2.324', 'logps/chosen': '-108.8', 'logps/rejected': '-136.8', 'logits/chosen': '-4.238', 'logits/rejected': '-4.215', 'epoch': '2.427'}
{'loss': '0.2669', 'grad_norm': '0.5383', 'learning_rate': '3.874e-05', 'rewards/chosen': '-3.067', 'rewards/rejected': '-5.306', 'rewards/accuracies': '0.8453', 'rewards/margins': '2.239', 'logps/chosen': '-113.3', 'logps/rejected': '-140.9', 'logits/chosen': '-4.169', 'logits/rejected': '-4.063', 'epoch': '2.438'}
{'loss': '0.2641', 'grad_norm': '0.4986', 'learning_rate': '3.8e-05', 'rewards/chosen': '-2.952', 'rewards/rejected': '-5.159', 'rewards/accuracies': '0.8508', 'rewards/margins': '2.207', 'logps/chosen': '-110.6', 'logps/rejected': '-138.2', 'logits/chosen': '-4.118', 'logits/rejected': '-4.037', 'epoch': '2.448'}
 82%|████████▏ | 2250/2757 [5:07:49<1:08:09,  8.07s/it]
 82%|████████▏ | 2251/2757 [5:08:00<1:15:07,  8.91s/it]
 82%|████████▏ | 2252/2757 [5:08:08<1:12:37,  8.63s/it]
 82%|████████▏ | 2253/2757 [5:08:16<1:10:38,  8.41s/it]
 82%|████████▏ | 2254/2757 [5:08:24<1:09:14,  8.26s/it]
 82%|████████▏ | 2255/2757 [5:08:32<1:09:02,  8.25s/it]
 82%|████████▏ | 2256/2757 [5:08:40<1:08:28,  8.20s/it]
 82%|████████▏ | 2257/2757 [5:08:48<1:06:47,  8.02s/it]
 82%|████████▏ | 2258/2757 [5:08:55<1:06:07,  7.95s/it]
 82%|████████▏ | 2259/2757 [5:09:04<1:06:13,  7.98s/it]
 82%|████████▏ | 2260/2757 [5:09:12<1:06:55,  8.08s/it]
                                                       

 82%|████████▏ | 2260/2757 [5:09:12<1:06:55,  8.08s/it]
 82%|████████▏ | 2261/2757 [5:09:20<1:06:34,  8.05s/it]
 82%|████████▏ | 2262/2757 [5:09:27<1:05:07,  7.89s/it]
 82%|████████▏ | 2263/2757 [5:09:36<1:06:13,  8.04s/it]
 82%|████████▏ | 2264/2757 [5:09:44<1:06:15,  8.06s/it]
 82%|████████▏ | 2265/2757 [5:09:52<1:05:48,  8.03s/it]
 82%|████████▏ | 2266/2757 [5:10:00<1:05:27,  8.00s/it]
 82%|████████▏ | 2267/2757 [5:10:08<1:04:50,  7.94s/it]
 82%|████████▏ | 2268/2757 [5:10:15<1:04:26,  7.91s/it]
 82%|████████▏ | 2269/2757 [5:10:23<1:04:13,  7.90s/it]
 82%|████████▏ | 2270/2757 [5:10:31<1:02:34,  7.71s/it]
                                                       

 82%|████████▏ | 2270/2757 [5:10:31<1:02:34,  7.71s/it]
 82%|████████▏ | 2271/2757 [5:10:38<1:01:38,  7.61s/it]
 82%|████████▏ | 2272/2757 [5:10:45<1:01:13,  7.57s/it]
 82%|████████▏ | 2273/2757 [5:10:54<1:02:48,  7.79s/it]
 82%|████████▏ | 2274/2757 [5:11:02<1:04:28,  8.01s/it]
 83%|████████▎ | 2275/2757 [5:11:10<1:04:45,  8.06s/it]
 83%|████████▎ | 2276/2757 [5:11:18<1:04:19,  8.02s/it]
 83%|████████▎ | 2277/2757 [5:11:26<1:04:27,  8.06s/it]
 83%|████████▎ | 2278/2757 [5:11:35<1:04:59,  8.14s/it]
 83%|████████▎ | 2279/2757 [5:11:43<1:04:21,  8.08s/it]
 83%|████████▎ | 2280/2757 [5:11:51<1:04:12,  8.08s/it]
                                                       

 83%|████████▎ | 2280/2757 [5:11:51<1:04:12,  8.08s/it]
 83%|████████▎ | 2281/2757 [5:11:59<1:03:54,  8.05s/it]
 83%|████████▎ | 2282/2757 [5:12:07<1:03:00,  7.96s/it]
 83%|████████▎ | 2283/2757 [5:12:15<1:02:54,  7.96s/it]
 83%|████████▎ | 2284/2757 [5:12:22<1:02:50,  7.97s/it]
 83%|████████▎ | 2285/2757 [5:12:30<1:02:05,  7.89s/it]
 83%|████████▎ | 2286/2757 [5:12:38<1:02:25,  7.95s/it]
 83%|████████▎ | 2287/2757 [5:12:46<1:01:28,  7.85s/it]
 83%|████████▎ | 2288/2757 [5:12:54<1:01:03,  7.81s/it]
 83%|████████▎ | 2289/2757 [5:13:02<1:01:49,  7.93s/it]
 83%|████████▎ | 2290/2757 [5:13:10<1:01:44,  7.93s/it]
                                                       

 83%|████████▎ | 2290/2757 [5:13:10<1:01:44,  7.93s/it]
 83%|████████▎ | 2291/2757 [5:13:18<1:02:49,  8.09s/it]
 83%|████████▎ | 2292/2757 [5:13:26<1:02:01,  8.00s/it]
 83%|████████▎ | 2293/2757 [5:13:34<1:02:34,  8.09s/it]
 83%|████████▎ | 2294/2757 [5:13:42<1:02:07,  8.05s/it]
 83%|████████▎ | 2295/2757 [5:13:50<1:01:44,  8.02s/it]
 83%|████████▎ | 2296/2757 [5:13:58<1:01:28,  8.00s/it]
 83%|████████▎ | 2297/2757 [5:14:06<1:00:39,  7.91s/it]
 83%|████████▎ | 2298/2757 [5:14:14<1:00:32,  7.91s/it]
 83%|████████▎ | 2299/2757 [5:14:22<1:00:18,  7.90s/it]
 83%|████████▎ | 2300/2757 [5:14:30<1:00:44,  7.98s/it]
                                                       

 83%|████████▎ | 2300/2757 [5:14:30<1:00:44,  7.98s/it]
 83%|████████▎ | 2301/2757 [5:14:38<1:00:50,  8.00s/it]
 83%|████████▎ | 2302/2757 [5:14:46<1:00:56,  8.04s/it]
 84%|████████▎ | 2303/2757 [5:14:54<1:00:58,  8.06s/it]
 84%|████████▎ | 2304/2757 [5:15:02<1:00:06,  7.96s/it]
 84%|████████▎ | 2305/2757 [5:15:10<1:00:21,  8.01s/it]
 84%|████████▎ | 2306/2757 [5:15:18<1:00:55,  8.11s/it]
 84%|████████▎ | 2307/2757 [5:15:26<1:00:22,  8.05s/it]
 84%|████████▎ | 2308/2757 [5:15:34<1:00:12,  8.05s/it]
 84%|████████▍ | 2309/2757 [5:15:42<59:19,  7.94s/it]  
 84%|████████▍ | 2310/2757 [5:15:50<59:38,  8.00s/it]
                                                     

 84%|████████▍ | 2310/2757 [5:15:50<59:38,  8.00s/it]
 84%|████████▍ | 2311/2757 [5:15:58<59:18,  7.98s/it]
 84%|████████▍ | 2312/2757 [5:16:06<59:43,  8.05s/it]
 84%|████████▍ | 2313/2757 [5:16:14<58:56,  7.97s/it]
 84%|████████▍ | 2314/2757 [5:16:22<59:10,  8.01s/it]
 84%|████████▍ | 2315/2757 [5:16:30<59:25,  8.07s/it]
 84%|████████▍ | 2316/2757 [5:16:39<1:00:09,  8.18s/it]
 84%|████████▍ | 2317/2757 [5:16:47<59:12,  8.07s/it]  
 84%|████████▍ | 2318/2757 [5:16:55<59:04,  8.07s/it]
 84%|████████▍ | 2319/2757 [5:17:03<58:51,  8.06s/it]
 84%|████████▍ | 2320/2757 [5:17:11<58:18,  8.01s/it]
                                                     

 84%|████████▍ | 2320/2757 [5:17:11<58:18,  8.01s/it]
 84%|████████▍ | 2321/2757 [5:17:18<56:56,  7.84s/it]
 84%|████████▍ | 2322/2757 [5:17:26<57:04,  7.87s/it]
 84%|████████▍ | 2323/2757 [5:17:33<55:55,  7.73s/it]
 84%|████████▍ | 2324/2757 [5:17:42<56:48,  7.87s/it]
 84%|████████▍ | 2325/2757 [5:17:49<56:01,  7.78s/it]
 84%|████████▍ | 2326/2757 [5:17:58<57:42,  8.03s/it]
 84%|████████▍ | 2327/2757 [5:18:06<58:59,  8.23s/it]
 84%|████████▍ | 2328/2757 [5:18:15<58:33,  8.19s/it]
 84%|████████▍ | 2329/2757 [5:18:23<57:53,  8.12s/it]
 85%|████████▍ | 2330/2757 [5:18:31<58:26,  8.21s/it]
                                                     

 85%|████████▍ | 2330/2757 [5:18:31<58:26,  8.21s/it]
 85%|████████▍ | 2331/2757 [5:18:39<58:04,  8.18s/it]
 85%|████████▍ | 2332/2757 [5:18:47<57:11,  8.08s/it]
 85%|████████▍ | 2333/2757 [5:18:55<56:35,  8.01s/it]
 85%|████████▍ | 2334/2757 [5:19:03<56:24,  8.00s/it]
 85%|████████▍ | 2335/2757 [5:19:11<57:07,  8.12s/it]
 85%|████████▍ | 2336/2757 [5:19:19<57:09,  8.15s/it]
 85%|████████▍ | 2337/2757 [5:19:27<56:07,  8.02s/it]
 85%|████████▍ | 2338/2757 [5:19:35<56:16,  8.06s/it]
 85%|████████▍ | 2339/2757 [5:19:43<56:15,  8.08s/it]
 85%|████████▍ | 2340/2757 [5:19:52<57:09,  8.22s/it]
                                                     

 85%|████████▍ | 2340/2757 [5:19:52<57:09,  8.22s/it]
 85%|████████▍ | 2341/2757 [5:20:00<56:27,  8.14s/it]
 85%|████████▍ | 2342/2757 [5:20:08<56:07,  8.11s/it]
 85%|████████▍ | 2343/2757 [5:20:16<57:01,  8.26s/it]
 85%|████████▌ | 2344/2757 [5:20:24<55:44,  8.10s/it]
 85%|████████▌ | 2345/2757 [5:20:32<54:21,  7.92s/it]
 85%|████████▌ | 2346/2757 [5:20:40<54:25,  7.95s/it]
 85%|████████▌ | 2347/2757 [5:20:48<54:26,  7.97s/it]
 85%|████████▌ | 2348/2757 [5:20:56<54:53,  8.05s/it]
 85%|████████▌ | 2349/2757 [5:21:04<55:09,  8.11s/it]
 85%|████████▌ | 2350/2757 [5:21:12<55:10,  8.13s/it]
                                                     

 85%|████████▌ | 2350/2757 [5:21:12<55:10,  8.13s/it]
 85%|████████▌ | 2351/2757 [5:21:20<54:05,  7.99s/it]
 85%|████████▌ | 2352/2757 [5:21:28<54:03,  8.01s/it]
 85%|████████▌ | 2353/2757 [5:21:37<55:39,  8.27s/it]
 85%|████████▌ | 2354/2757 [5:21:45<54:18,  8.09s/it]
 85%|████████▌ | 2355/2757 [5:21:53<53:53,  8.04s/it]
 85%|████████▌ | 2356/2757 [5:22:01<53:50,  8.06s/it]
 85%|████████▌ | 2357/2757 [5:22:09<53:38,  8.05s/it]
 86%|████████▌ | 2358/2757 [5:22:17<53:01,  7.97s/it]
 86%|████████▌ | 2359/2757 [5:22:24<52:45,  7.95s/it]
 86%|████████▌ | 2360/2757 [5:22:32<52:49,  7.98s/it]
                                                     

 86%|████████▌ | 2360/2757 [5:22:32<52:49,  7.98s/it]
 86%|████████▌ | 2361/2757 [5:22:40<52:27,  7.95s/it]
 86%|████████▌ | 2362/2757 [5:22:48<52:05,  7.91s/it]
 86%|████████▌ | 2363/2757 [5:22:56<52:16,  7.96s/it]
 86%|████████▌ | 2364/2757 [5:23:04<52:09,  7.96s/it]
 86%|████████▌ | 2365/2757 [5:23:13<53:17,  8.16s/it]
 86%|████████▌ | 2366/2757 [5:23:21<52:43,  8.09s/it]
 86%|████████▌ | 2367/2757 [5:23:30<53:55,  8.29s/it]
 86%|████████▌ | 2368/2757 [5:23:38<53:08,  8.20s/it]
 86%|████████▌ | 2369/2757 [5:23:45<52:20,  8.09s/it]
 86%|████████▌ | 2370/2757 [5:23:54<52:37,  8.16s/it]
                                                     

 86%|████████▌ | 2370/2757 [5:23:54<52:37,  8.16s/it]
 86%|████████▌ | 2371/2757 [5:24:02<52:00,  8.08s/it]
 86%|████████▌ | 2372/2757 [5:24:10<52:03,  8.11s/it]
 86%|████████▌ | 2373/2757 [5:24:18<51:37,  8.07s/it]
 86%|████████▌ | 2374/2757 [5:24:25<50:48,  7.96s/it]
 86%|████████▌ | 2375/2757 [5:24:34<51:07,  8.03s/it]
 86%|████████▌ | 2376/2757 [5:24:42<51:39,  8.13s/it]
 86%|████████▌ | 2377/2757 [5:24:50<51:37,  8.15s/it]
 86%|████████▋ | 2378/2757 [5:24:58<51:36,  8.17s/it]
 86%|████████▋ | 2379/2757 [5:25:06<51:11,  8.13s/it]
 86%|████████▋ | 2380/2757 [5:25:15<51:23,  8.18s/it]
                                                     

 86%|████████▋ | 2380/2757 [5:25:15<51:23,  8.18s/it]
 86%|████████▋ | 2381/2757 [5:25:23<50:38,  8.08s/it]
 86%|████████▋ | 2382/2757 [5:25:31<50:55,  8.15s/it]
 86%|████████▋ | 2383/2757 [5:25:39<50:47,  8.15s/it]
 86%|████████▋ | 2384/2757 [5:25:48<51:34,  8.30s/it]
 87%|████████▋ | 2385/2757 [5:25:56<51:56,  8.38s/it]
 87%|████████▋ | 2386/2757 [5:26:05<51:45,  8.37s/it]
 87%|████████▋ | 2387/2757 [5:26:12<50:07,  8.13s/it]
 87%|████████▋ | 2388/2757 [5:26:21<50:58,  8.29s/it]
 87%|████████▋ | 2389/2757 [5:26:28<49:12,  8.02s/it]
 87%|████████▋ | 2390/2757 [5:26:36<48:01,  7.85s/it]
                                                     

 87%|████████▋ | 2390/2757 [5:26:36<48:01,  7.85s/it]
 87%|████████▋ | 2391/2757 [5:26:44<48:19,  7.92s/it]
 87%|████████▋ | 2392/2757 [5:26:52<48:42,  8.01s/it]
 87%|████████▋ | 2393/2757 [5:27:00<48:36,  8.01s/it]
 87%|████████▋ | 2394/2757 [5:27:08<48:28,  8.01s/it]
 87%|████████▋ | 2395/2757 [5:27:16<48:15,  8.00s/it]
 87%|████████▋ | 2396/2757 [5:27:24<48:48,  8.11s/it]
 87%|████████▋ | 2397/2757 [5:27:32<48:23,  8.07s/it]
 87%|████████▋ | 2398/2757 [5:27:40<47:19,  7.91s/it]
 87%|████████▋ | 2399/2757 [5:27:48<46:46,  7.84s/it]
 87%|████████▋ | 2400/2757 [5:27:55<46:37,  7.83s/it]
                                                     

 87%|████████▋ | 2400/2757 [5:27:55<46:37,  7.83s/it]
 87%|████████▋ | 2401/2757 [5:28:04<47:15,  7.96s/it]
 87%|████████▋ | 2402/2757 [5:28:11<46:24,  7.84s/it]
 87%|████████▋ | 2403/2757 [5:28:19<46:49,  7.94s/it]
 87%|████████▋ | 2404/2757 [5:28:28<47:08,  8.01s/it]
 87%|████████▋ | 2405/2757 [5:28:35<46:48,  7.98s/it]
 87%|████████▋ | 2406/2757 [5:28:43<46:28,  7.95s/it]
 87%|████████▋ | 2407/2757 [5:28:51<46:40,  8.00s/it]
 87%|████████▋ | 2408/2757 [5:29:00<46:43,  8.03s/it]
 87%|████████▋ | 2409/2757 [5:29:07<45:43,  7.88s/it]
 87%|████████▋ | 2410/2757 [5:29:15<45:31,  7.87s/it]
                                                     

 87%|████████▋ | 2410/2757 [5:29:15<45:31,  7.87s/it]
 87%|████████▋ | 2411/2757 [5:29:23<45:35,  7.91s/it]
 87%|████████▋ | 2412/2757 [5:29:31<45:11,  7.86s/it]
 88%|████████▊ | 2413/2757 [5:29:39<45:41,  7.97s/it]
 88%|████████▊ | 2414/2757 [5:29:47<45:28,  7.95s/it]
 88%|████████▊ | 2415/2757 [5:29:55<45:43,  8.02s/it]
 88%|████████▊ | 2416/2757 [5:30:03<45:53,  8.07s/it]
 88%|████████▊ | 2417/2757 [5:30:11<45:52,  8.10s/it]
 88%|████████▊ | 2418/2757 [5:30:20<46:10,  8.17s/it]
 88%|████████▊ | 2419/2757 [5:30:27<45:15,  8.03s/it]
 88%|████████▊ | 2420/2757 [5:30:35<45:02,  8.02s/it]
                                                     

 88%|████████▊ | 2420/2757 [5:30:35<45:02,  8.02s/it]
 88%|████████▊ | 2421/2757 [5:30:44<45:06,  8.06s/it]
 88%|████████▊ | 2422/2757 [5:30:51<44:36,  7.99s/it]
 88%|████████▊ | 2423/2757 [5:30:59<43:50,  7.88s/it]
 88%|████████▊ | 2424/2757 [5:31:07<43:45,  7.88s/it]
 88%|████████▊ | 2425/2757 [5:31:14<42:57,  7.76s/it]
 88%|████████▊ | 2426/2757 [5:31:22<42:43,  7.75s/it]
 88%|████████▊ | 2427/2757 [5:31:30<43:04,  7.83s/it]
 88%|████████▊ | 2428/2757 [5:31:38<42:34,  7.76s/it]
 88%|████████▊ | 2429/2757 [5:31:46<42:32,  7.78s/it]
 88%|████████▊ | 2430/2757 [5:31:54<42:51,  7.86s/it]
                                                     

 88%|████████▊ | 2430/2757 [5:31:54<42:51,  7.86s/it]
 88%|████████▊ | 2431/2757 [5:32:01<42:06,  7.75s/it]
 88%|████████▊ | 2432/2757 [5:32:09<41:54,  7.74s/it]
 88%|████████▊ | 2433/2757 [5:32:16<41:20,  7.66s/it]
 88%|████████▊ | 2434/2757 [5:32:24<42:01,  7.81s/it]
 88%|████████▊ | 2435/2757 [5:32:32<41:54,  7.81s/it]
 88%|████████▊ | 2436/2757 [5:32:40<42:00,  7.85s/it]
 88%|████████▊ | 2437/2757 [5:32:48<42:08,  7.90s/it]
 88%|████████▊ | 2438/2757 [5:32:56<42:19,  7.96s/it]
 88%|████████▊ | 2439/2757 [5:33:04<42:34,  8.03s/it]
 89%|████████▊ | 2440/2757 [5:33:12<42:19,  8.01s/it]
                                                     

 89%|████████▊ | 2440/2757 [5:33:12<42:19,  8.01s/it]
 89%|████████▊ | 2441/2757 [5:33:21<42:40,  8.10s/it]
 89%|████████▊ | 2442/2757 [5:33:29<42:25,  8.08s/it]
 89%|████████▊ | 2443/2757 [5:33:36<40:59,  7.83s/it]
 89%|████████▊ | 2444/2757 [5:33:44<41:23,  7.94s/it]
 89%|████████▊ | 2445/2757 [5:33:53<41:56,  8.06s/it]
 89%|████████▊ | 2446/2757 [5:34:00<40:56,  7.90s/it]
 89%|████████▉ | 2447/2757 [5:34:08<41:04,  7.95s/it]
 89%|████████▉ | 2448/2757 [5:34:16<40:28,  7.86s/it]
 89%|████████▉ | 2449/2757 [5:34:24<40:50,  7.96s/it]
 89%|████████▉ | 2450/2757 [5:34:31<39:56,  7.81s/it]
                                                     

 89%|████████▉ | 2450/2757 [5:34:31<39:56,  7.81s/it]
 89%|████████▉ | 2451/2757 [5:34:39<39:58,  7.84s/it]
 89%|████████▉ | 2452/2757 [5:34:47<40:07,  7.90s/it]
 89%|████████▉ | 2453/2757 [5:34:55<40:17,  7.95s/it]
 89%|████████▉ | 2454/2757 [5:35:03<39:53,  7.90s/it]
 89%|████████▉ | 2455/2757 [5:35:11<39:43,  7.89s/it]
 89%|████████▉ | 2456/2757 [5:35:19<39:26,  7.86s/it]
 89%|████████▉ | 2457/2757 [5:35:27<39:20,  7.87s/it]
 89%|████████▉ | 2458/2757 [5:35:35<39:57,  8.02s/it]
 89%|████████▉ | 2459/2757 [5:35:43<39:13,  7.90s/it]
 89%|████████▉ | 2460/2757 [5:35:51<39:20,  7.95s/it]
                                                     

 89%|████████▉ | 2460/2757 [5:35:51<39:20,  7.95s/it]
 89%|████████▉ | 2461/2757 [5:35:59<39:32,  8.02s/it]
 89%|████████▉ | 2462/2757 [5:36:07<39:28,  8.03s/it]
 89%|████████▉ | 2463/2757 [5:36:15<39:19,  8.03s/it]
 89%|████████▉ | 2464/2757 [5:36:23<39:03,  8.00s/it]
 89%|████████▉ | 2465/2757 [5:36:31<38:58,  8.01s/it]
 89%|████████▉ | 2466/2757 [5:36:39<38:50,  8.01s/it]
 89%|████████▉ | 2467/2757 [5:36:47<38:49,  8.03s/it]
 90%|████████▉ | 2468/2757 [5:36:56<39:30,  8.20s/it]
 90%|████████▉ | 2469/2757 [5:37:04<38:45,  8.08s/it]
 90%|████████▉ | 2470/2757 [5:37:12<39:26,  8.25s/it]
                                                     

 90%|████████▉ | 2470/2757 [5:37:12<39:26,  8.25s/it]
 90%|████████▉ | 2471/2757 [5:37:20<38:59,  8.18s/it]
 90%|████████▉ | 2472/2757 [5:37:28<38:53,  8.19s/it]
 90%|████████▉ | 2473/2757 [5:37:37<38:46,  8.19s/it]
 90%|████████▉ | 2474/2757 [5:37:44<37:45,  8.01s/it]
 90%|████████▉ | 2475/2757 [5:37:52<37:02,  7.88s/it]
 90%|████████▉ | 2476/2757 [5:38:00<37:32,  8.01s/it]
 90%|████████▉ | 2477/2757 [5:38:08<37:14,  7.98s/it]
 90%|████████▉ | 2478/2757 [5:38:16<37:01,  7.96s/it]
 90%|████████▉ | 2479/2757 [5:38:24<36:23,  7.86s/it]
 90%|████████▉ | 2480/2757 [5:38:32<36:28,  7.90s/it]
                                                     

 90%|████████▉ | 2480/2757 [5:38:32<36:28,  7.90s/it]
 90%|████████▉ | 2481/2757 [5:38:39<36:25,  7.92s/it]
 90%|█████████ | 2482/2757 [5:38:47<36:21,  7.93s/it]
 90%|█████████ | 2483/2757 [5:38:55<35:33,  7.79s/it]
 90%|█████████ | 2484/2757 [5:39:03<35:18,  7.76s/it]
 90%|█████████ | 2485/2757 [5:39:11<35:27,  7.82s/it]
 90%|█████████ | 2486/2757 [5:39:19<36:05,  7.99s/it]
 90%|█████████ | 2487/2757 [5:39:27<36:41,  8.15s/it]
 90%|█████████ | 2488/2757 [5:39:36<36:45,  8.20s/it]
 90%|█████████ | 2489/2757 [5:39:44<36:13,  8.11s/it]
 90%|█████████ | 2490/2757 [5:39:52<35:47,  8.04s/it]
                                                     

 90%|█████████ | 2490/2757 [5:39:52<35:47,  8.04s/it]
 90%|█████████ | 2491/2757 [5:39:59<35:26,  7.99s/it]
 90%|█████████ | 2492/2757 [5:40:07<34:59,  7.92s/it]
 90%|█████████ | 2493/2757 [5:40:15<34:39,  7.88s/it]
 90%|█████████ | 2494/2757 [5:40:23<34:43,  7.92s/it]
 90%|█████████ | 2495/2757 [5:40:32<35:21,  8.10s/it]
 91%|█████████ | 2496/2757 [5:40:40<35:05,  8.07s/it]
 91%|█████████ | 2497/2757 [5:40:47<34:46,  8.02s/it]
 91%|█████████ | 2498/2757 [5:40:55<33:56,  7.86s/it]
 91%|█████████ | 2499/2757 [5:41:03<34:22,  7.99s/it]
 91%|█████████ | 2500/2757 [5:41:11<33:59,  7.94s/it]
                                                     

 91%|█████████ | 2500/2757 [5:41:11<33:59,  7.94s/it]
{'loss': '0.2519', 'grad_norm': '0.5483', 'learning_rate': '3.725e-05', 'rewards/chosen': '-3.012', 'rewards/rejected': '-5.305', 'rewards/accuracies': '0.857', 'rewards/margins': '2.293', 'logps/chosen': '-112.8', 'logps/rejected': '-140.4', 'logits/chosen': '-4.164', 'logits/rejected': '-4.179', 'epoch': '2.459'}
{'loss': '0.2518', 'grad_norm': '0.4968', 'learning_rate': '3.65e-05', 'rewards/chosen': '-3.082', 'rewards/rejected': '-5.411', 'rewards/accuracies': '0.8484', 'rewards/margins': '2.329', 'logps/chosen': '-110.1', 'logps/rejected': '-137.6', 'logits/chosen': '-4.339', 'logits/rejected': '-4.249', 'epoch': '2.47'}
{'loss': '0.2498', 'grad_norm': '0.5726', 'learning_rate': '3.575e-05', 'rewards/chosen': '-3.102', 'rewards/rejected': '-5.512', 'rewards/accuracies': '0.8492', 'rewards/margins': '2.41', 'logps/chosen': '-113.6', 'logps/rejected': '-142.6', 'logits/chosen': '-4.268', 'logits/rejected': '-4.18', 'epoch': '2.481'}
{'loss': '0.26', 'grad_norm': '0.5567', 'learning_rate': '3.5e-05', 'rewards/chosen': '-2.999', 'rewards/rejected': '-5.299', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.3', 'logps/chosen': '-107.5', 'logps/rejected': '-135.8', 'logits/chosen': '-4.303', 'logits/rejected': '-4.221', 'epoch': '2.492'}
{'loss': '0.2627', 'grad_norm': '0.4611', 'learning_rate': '3.426e-05', 'rewards/chosen': '-3.027', 'rewards/rejected': '-5.354', 'rewards/accuracies': '0.8414', 'rewards/margins': '2.327', 'logps/chosen': '-109.5', 'logps/rejected': '-138.5', 'logits/chosen': '-4.257', 'logits/rejected': '-4.285', 'epoch': '2.503'}
{'loss': '0.2664', 'grad_norm': '0.5548', 'learning_rate': '3.351e-05', 'rewards/chosen': '-3.159', 'rewards/rejected': '-5.442', 'rewards/accuracies': '0.8406', 'rewards/margins': '2.282', 'logps/chosen': '-112.9', 'logps/rejected': '-140.6', 'logits/chosen': '-4.131', 'logits/rejected': '-4.09', 'epoch': '2.514'}
{'loss': '0.2697', 'grad_norm': '0.6786', 'learning_rate': '3.276e-05', 'rewards/chosen': '-3.175', 'rewards/rejected': '-5.44', 'rewards/accuracies': '0.8383', 'rewards/margins': '2.265', 'logps/chosen': '-113.3', 'logps/rejected': '-140.3', 'logits/chosen': '-4.26', 'logits/rejected': '-4.25', 'epoch': '2.525'}
{'loss': '0.2606', 'grad_norm': '0.5422', 'learning_rate': '3.201e-05', 'rewards/chosen': '-3.227', 'rewards/rejected': '-5.541', 'rewards/accuracies': '0.843', 'rewards/margins': '2.314', 'logps/chosen': '-112.1', 'logps/rejected': '-142.1', 'logits/chosen': '-4.233', 'logits/rejected': '-4.16', 'epoch': '2.536'}
{'loss': '0.2648', 'grad_norm': '0.6207', 'learning_rate': '3.126e-05', 'rewards/chosen': '-3.262', 'rewards/rejected': '-5.532', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.27', 'logps/chosen': '-110.7', 'logps/rejected': '-138.4', 'logits/chosen': '-4.249', 'logits/rejected': '-4.177', 'epoch': '2.546'}
{'loss': '0.2524', 'grad_norm': '0.6249', 'learning_rate': '3.052e-05', 'rewards/chosen': '-3.352', 'rewards/rejected': '-5.683', 'rewards/accuracies': '0.8562', 'rewards/margins': '2.332', 'logps/chosen': '-115.2', 'logps/rejected': '-144.4', 'logits/chosen': '-4.308', 'logits/rejected': '-4.223', 'epoch': '2.557'}
{'loss': '0.2689', 'grad_norm': '0.6504', 'learning_rate': '2.977e-05', 'rewards/chosen': '-3.239', 'rewards/rejected': '-5.564', 'rewards/accuracies': '0.8398', 'rewards/margins': '2.325', 'logps/chosen': '-111.2', 'logps/rejected': '-141.9', 'logits/chosen': '-4.297', 'logits/rejected': '-4.28', 'epoch': '2.568'}
{'loss': '0.2643', 'grad_norm': '0.6845', 'learning_rate': '2.902e-05', 'rewards/chosen': '-3.097', 'rewards/rejected': '-5.37', 'rewards/accuracies': '0.8336', 'rewards/margins': '2.273', 'logps/chosen': '-111', 'logps/rejected': '-138.2', 'logits/chosen': '-4.262', 'logits/rejected': '-4.167', 'epoch': '2.579'}
{'loss': '0.2551', 'grad_norm': '0.4879', 'learning_rate': '2.827e-05', 'rewards/chosen': '-2.942', 'rewards/rejected': '-5.293', 'rewards/accuracies': '0.8484', 'rewards/margins': '2.351', 'logps/chosen': '-106.5', 'logps/rejected': '-137.2', 'logits/chosen': '-4.409', 'logits/rejected': '-4.352', 'epoch': '2.59'}
{'loss': '0.2424', 'grad_norm': '0.5652', 'learning_rate': '2.752e-05', 'rewards/chosen': '-3.074', 'rewards/rejected': '-5.492', 'rewards/accuracies': '0.8578', 'rewards/margins': '2.419', 'logps/chosen': '-108.9', 'logps/rejected': '-137.6', 'logits/chosen': '-4.418', 'logits/rejected': '-4.38', 'epoch': '2.601'}
{'loss': '0.2535', 'grad_norm': '0.5238', 'learning_rate': '2.678e-05', 'rewards/chosen': '-3.111', 'rewards/rejected': '-5.516', 'rewards/accuracies': '0.8562', 'rewards/margins': '2.405', 'logps/chosen': '-112.7', 'logps/rejected': '-142.6', 'logits/chosen': '-4.35', 'logits/rejected': '-4.247', 'epoch': '2.612'}
{'loss': '0.26', 'grad_norm': '0.519', 'learning_rate': '2.603e-05', 'rewards/chosen': '-3.063', 'rewards/rejected': '-5.384', 'rewards/accuracies': '0.8383', 'rewards/margins': '2.322', 'logps/chosen': '-109.9', 'logps/rejected': '-137.5', 'logits/chosen': '-4.415', 'logits/rejected': '-4.263', 'epoch': '2.623'}
{'loss': '0.2537', 'grad_norm': '0.5785', 'learning_rate': '2.528e-05', 'rewards/chosen': '-3.048', 'rewards/rejected': '-5.36', 'rewards/accuracies': '0.8523', 'rewards/margins': '2.312', 'logps/chosen': '-111.9', 'logps/rejected': '-139.8', 'logits/chosen': '-4.251', 'logits/rejected': '-4.197', 'epoch': '2.633'}
{'loss': '0.2483', 'grad_norm': '0.5177', 'learning_rate': '2.453e-05', 'rewards/chosen': '-3.117', 'rewards/rejected': '-5.505', 'rewards/accuracies': '0.8422', 'rewards/margins': '2.388', 'logps/chosen': '-114.2', 'logps/rejected': '-142.7', 'logits/chosen': '-4.407', 'logits/rejected': '-4.32', 'epoch': '2.644'}
{'loss': '0.2676', 'grad_norm': '0.6798', 'learning_rate': '2.378e-05', 'rewards/chosen': '-2.953', 'rewards/rejected': '-5.227', 'rewards/accuracies': '0.8391', 'rewards/margins': '2.275', 'logps/chosen': '-109.7', 'logps/rejected': '-138.4', 'logits/chosen': '-4.313', 'logits/rejected': '-4.273', 'epoch': '2.655'}
{'loss': '0.2508', 'grad_norm': '0.6013', 'learning_rate': '2.304e-05', 'rewards/chosen': '-2.888', 'rewards/rejected': '-5.234', 'rewards/accuracies': '0.8633', 'rewards/margins': '2.347', 'logps/chosen': '-110.6', 'logps/rejected': '-139.8', 'logits/chosen': '-4.369', 'logits/rejected': '-4.297', 'epoch': '2.666'}
{'loss': '0.2731', 'grad_norm': '0.5708', 'learning_rate': '2.229e-05', 'rewards/chosen': '-2.885', 'rewards/rejected': '-5.138', 'rewards/accuracies': '0.8281', 'rewards/margins': '2.253', 'logps/chosen': '-107.8', 'logps/rejected': '-134.8', 'logits/chosen': '-4.391', 'logits/rejected': '-4.356', 'epoch': '2.677'}
{'loss': '0.2608', 'grad_norm': '0.4749', 'learning_rate': '2.154e-05', 'rewards/chosen': '-2.873', 'rewards/rejected': '-5.198', 'rewards/accuracies': '0.8562', 'rewards/margins': '2.325', 'logps/chosen': '-108.3', 'logps/rejected': '-135.5', 'logits/chosen': '-4.335', 'logits/rejected': '-4.284', 'epoch': '2.688'}
{'loss': '0.2541', 'grad_norm': '0.5339', 'learning_rate': '2.079e-05', 'rewards/chosen': '-2.983', 'rewards/rejected': '-5.276', 'rewards/accuracies': '0.8625', 'rewards/margins': '2.292', 'logps/chosen': '-109.3', 'logps/rejected': '-137.2', 'logits/chosen': '-4.364', 'logits/rejected': '-4.272', 'epoch': '2.699'}
{'loss': '0.258', 'grad_norm': '0.533', 'learning_rate': '2.004e-05', 'rewards/chosen': '-3.023', 'rewards/rejected': '-5.346', 'rewards/accuracies': '0.8586', 'rewards/margins': '2.323', 'logps/chosen': '-109.9', 'logps/rejected': '-138.8', 'logits/chosen': '-4.273', 'logits/rejected': '-4.253', 'epoch': '2.71'}
{'loss': '0.2491', 'grad_norm': '0.4829', 'learning_rate': '1.93e-05', 'rewards/chosen': '-3.075', 'rewards/rejected': '-5.44', 'rewards/accuracies': '0.8445', 'rewards/margins': '2.365', 'logps/chosen': '-114.9', 'logps/rejected': '-143.9', 'logits/chosen': '-4.346', 'logits/rejected': '-4.261', 'epoch': '2.721'}
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)


  0%|          | 0/300 [00:00<?, ?it/s]

  1%|          | 2/300 [00:00<00:52,  5.72it/s]

  1%|          | 3/300 [00:00<01:46,  2.79it/s]

  1%|▏         | 4/300 [00:01<01:34,  3.12it/s]

  2%|▏         | 5/300 [00:01<01:44,  2.83it/s]

  2%|▏         | 6/300 [00:01<01:40,  2.93it/s]

  2%|▏         | 7/300 [00:02<01:55,  2.55it/s]

  3%|▎         | 8/300 [00:02<01:50,  2.64it/s]

  3%|▎         | 9/300 [00:03<01:52,  2.60it/s]

  3%|▎         | 10/300 [00:03<01:48,  2.67it/s]

  4%|▎         | 11/300 [00:03<01:51,  2.59it/s]

  4%|▍         | 12/300 [00:04<01:38,  2.93it/s]

  4%|▍         | 13/300 [00:04<01:43,  2.78it/s]

  5%|▍         | 14/300 [00:04<01:35,  2.98it/s]

  5%|▌         | 15/300 [00:05<01:45,  2.69it/s]

  5%|▌         | 16/300 [00:05<01:34,  3.01it/s]

  6%|▌         | 17/300 [00:05<01:38,  2.87it/s]

  6%|▌         | 18/300 [00:06<01:29,  3.17it/s]

  6%|▋         | 19/300 [00:06<01:39,  2.82it/s]

  7%|▋         | 20/300 [00:06<01:31,  3.05it/s]

  7%|▋         | 21/300 [00:07<01:43,  2.70it/s]

  7%|▋         | 22/300 [00:07<01:31,  3.02it/s]

  8%|▊         | 23/300 [00:07<01:32,  2.99it/s]

  8%|▊         | 24/300 [00:08<01:23,  3.30it/s]

  8%|▊         | 25/300 [00:08<01:35,  2.87it/s]

  9%|▊         | 26/300 [00:08<01:29,  3.06it/s]

  9%|▉         | 27/300 [00:09<01:38,  2.78it/s]

  9%|▉         | 28/300 [00:09<01:27,  3.12it/s]

 10%|▉         | 29/300 [00:09<01:30,  3.00it/s]

 10%|█         | 30/300 [00:10<01:21,  3.30it/s]

 10%|█         | 31/300 [00:10<01:33,  2.86it/s]

 11%|█         | 32/300 [00:10<01:24,  3.17it/s]

 11%|█         | 33/300 [00:11<01:25,  3.12it/s]

 11%|█▏        | 34/300 [00:11<01:19,  3.35it/s]

 12%|█▏        | 35/300 [00:11<01:30,  2.93it/s]

 12%|█▏        | 36/300 [00:12<01:23,  3.14it/s]

 12%|█▏        | 37/300 [00:12<01:32,  2.84it/s]

 13%|█▎        | 38/300 [00:12<01:21,  3.20it/s]

 13%|█▎        | 39/300 [00:13<01:28,  2.95it/s]

 13%|█▎        | 40/300 [00:13<01:20,  3.22it/s]

 14%|█▎        | 41/300 [00:13<01:27,  2.97it/s]

 14%|█▍        | 42/300 [00:14<01:19,  3.26it/s]

 14%|█▍        | 43/300 [00:14<01:23,  3.09it/s]

 15%|█▍        | 44/300 [00:14<01:16,  3.36it/s]

 15%|█▌        | 45/300 [00:15<01:23,  3.07it/s]

 15%|█▌        | 46/300 [00:15<01:24,  3.02it/s]

 16%|█▌        | 47/300 [00:15<01:29,  2.82it/s]

 16%|█▌        | 48/300 [00:16<01:22,  3.05it/s]

 16%|█▋        | 49/300 [00:16<01:23,  3.01it/s]

 17%|█▋        | 50/300 [00:16<01:14,  3.35it/s]

 17%|█▋        | 51/300 [00:17<01:18,  3.19it/s]

 17%|█▋        | 52/300 [00:17<01:15,  3.30it/s]

 18%|█▊        | 53/300 [00:17<01:28,  2.80it/s]

 18%|█▊        | 54/300 [00:18<01:19,  3.10it/s]

 18%|█▊        | 55/300 [00:18<01:26,  2.82it/s]

 19%|█▊        | 56/300 [00:18<01:15,  3.23it/s]

 19%|█▉        | 57/300 [00:19<01:19,  3.04it/s]

 19%|█▉        | 58/300 [00:19<01:21,  2.98it/s]

 20%|█▉        | 59/300 [00:19<01:26,  2.79it/s]

 20%|██        | 60/300 [00:20<01:17,  3.10it/s]

 20%|██        | 61/300 [00:20<01:21,  2.95it/s]

 21%|██        | 62/300 [00:20<01:20,  2.94it/s]

 21%|██        | 63/300 [00:21<01:25,  2.76it/s]

 21%|██▏       | 64/300 [00:21<01:23,  2.82it/s]

 22%|██▏       | 65/300 [00:21<01:27,  2.70it/s]

 22%|██▏       | 66/300 [00:22<01:19,  2.95it/s]

 22%|██▏       | 67/300 [00:22<01:26,  2.69it/s]

 23%|██▎       | 68/300 [00:22<01:18,  2.97it/s]

 23%|██▎       | 69/300 [00:23<01:21,  2.84it/s]

 23%|██▎       | 70/300 [00:23<01:13,  3.15it/s]

 24%|██▎       | 71/300 [00:23<01:15,  3.03it/s]

 24%|██▍       | 72/300 [00:24<01:09,  3.29it/s]

 24%|██▍       | 73/300 [00:24<01:12,  3.13it/s]

 25%|██▍       | 74/300 [00:24<01:08,  3.29it/s]

 25%|██▌       | 75/300 [00:25<01:13,  3.06it/s]

 25%|██▌       | 76/300 [00:25<01:14,  3.00it/s]

 26%|██▌       | 77/300 [00:25<01:23,  2.67it/s]

 26%|██▌       | 78/300 [00:26<01:15,  2.93it/s]

 26%|██▋       | 79/300 [00:26<01:16,  2.87it/s]

 27%|██▋       | 80/300 [00:26<01:12,  3.05it/s]

 27%|██▋       | 81/300 [00:27<01:19,  2.74it/s]

 27%|██▋       | 82/300 [00:27<01:11,  3.05it/s]

 28%|██▊       | 83/300 [00:27<01:14,  2.92it/s]

 28%|██▊       | 84/300 [00:28<01:14,  2.90it/s]

 28%|██▊       | 85/300 [00:28<01:18,  2.72it/s]

 29%|██▊       | 86/300 [00:28<01:10,  3.03it/s]

 29%|██▉       | 87/300 [00:29<01:12,  2.93it/s]

 29%|██▉       | 88/300 [00:29<01:06,  3.18it/s]

 30%|██▉       | 89/300 [00:29<01:10,  2.97it/s]

 30%|███       | 90/300 [00:30<01:05,  3.21it/s]

 30%|███       | 91/300 [00:30<01:09,  3.02it/s]

 31%|███       | 92/300 [00:30<01:03,  3.28it/s]

 31%|███       | 93/300 [00:31<01:09,  3.00it/s]

 31%|███▏      | 94/300 [00:31<01:02,  3.30it/s]

 32%|███▏      | 95/300 [00:31<01:07,  3.02it/s]

 32%|███▏      | 96/300 [00:32<01:03,  3.21it/s]

 32%|███▏      | 97/300 [00:32<01:05,  3.08it/s]

 33%|███▎      | 98/300 [00:32<01:02,  3.25it/s]

 33%|███▎      | 99/300 [00:33<01:07,  2.99it/s]

 33%|███▎      | 100/300 [00:33<01:03,  3.14it/s]

 34%|███▎      | 101/300 [00:33<01:07,  2.95it/s]

 34%|███▍      | 102/300 [00:34<01:05,  3.01it/s]

 34%|███▍      | 103/300 [00:34<01:09,  2.82it/s]

 35%|███▍      | 104/300 [00:34<01:02,  3.14it/s]

 35%|███▌      | 105/300 [00:35<01:05,  2.98it/s]

 35%|███▌      | 106/300 [00:35<01:00,  3.18it/s]

 36%|███▌      | 107/300 [00:35<01:05,  2.96it/s]

 36%|███▌      | 108/300 [00:36<01:02,  3.06it/s]

 36%|███▋      | 109/300 [00:36<01:06,  2.88it/s]

 37%|███▋      | 110/300 [00:36<01:02,  3.03it/s]

 37%|███▋      | 111/300 [00:37<01:11,  2.65it/s]

 37%|███▋      | 112/300 [00:37<01:05,  2.85it/s]

 38%|███▊      | 113/300 [00:37<01:07,  2.77it/s]

 38%|███▊      | 114/300 [00:38<01:06,  2.79it/s]

 38%|███▊      | 115/300 [00:38<01:09,  2.65it/s]

 39%|███▊      | 116/300 [00:38<01:01,  2.98it/s]

 39%|███▉      | 117/300 [00:39<01:03,  2.90it/s]

 39%|███▉      | 118/300 [00:39<00:58,  3.13it/s]

 40%|███▉      | 119/300 [00:39<01:00,  3.00it/s]

 40%|████      | 120/300 [00:40<00:59,  3.03it/s]

 40%|████      | 121/300 [00:40<01:01,  2.89it/s]

 41%|████      | 122/300 [00:40<00:57,  3.08it/s]

 41%|████      | 123/300 [00:41<01:01,  2.87it/s]

 41%|████▏     | 124/300 [00:41<00:57,  3.04it/s]

 42%|████▏     | 125/300 [00:41<00:59,  2.94it/s]

 42%|████▏     | 126/300 [00:42<00:59,  2.91it/s]

 42%|████▏     | 127/300 [00:42<01:01,  2.79it/s]

 43%|████▎     | 128/300 [00:42<00:57,  2.99it/s]

 43%|████▎     | 129/300 [00:43<00:59,  2.87it/s]

 43%|████▎     | 130/300 [00:43<00:53,  3.15it/s]

 44%|████▎     | 131/300 [00:44<01:00,  2.78it/s]

 44%|████▍     | 132/300 [00:44<00:55,  3.03it/s]

 44%|████▍     | 133/300 [00:44<01:05,  2.54it/s]

 45%|████▍     | 134/300 [00:45<00:58,  2.81it/s]

 45%|████▌     | 135/300 [00:45<01:00,  2.74it/s]

 45%|████▌     | 136/300 [00:45<00:53,  3.04it/s]

 46%|████▌     | 137/300 [00:46<00:54,  3.01it/s]

 46%|████▌     | 138/300 [00:46<00:50,  3.23it/s]

 46%|████▋     | 139/300 [00:46<00:50,  3.17it/s]

 47%|████▋     | 140/300 [00:46<00:46,  3.43it/s]

 47%|████▋     | 141/300 [00:47<00:48,  3.26it/s]

 47%|████▋     | 142/300 [00:47<00:50,  3.12it/s]

 48%|████▊     | 143/300 [00:48<00:53,  2.93it/s]

 48%|████▊     | 144/300 [00:48<00:48,  3.23it/s]

 48%|████▊     | 145/300 [00:48<00:48,  3.17it/s]

 49%|████▊     | 146/300 [00:48<00:46,  3.32it/s]

 49%|████▉     | 147/300 [00:49<00:53,  2.87it/s]

 49%|████▉     | 148/300 [00:49<00:50,  3.04it/s]

 50%|████▉     | 149/300 [00:50<00:53,  2.81it/s]

 50%|█████     | 150/300 [00:50<00:49,  3.01it/s]

 50%|█████     | 151/300 [00:50<00:51,  2.89it/s]

 51%|█████     | 152/300 [00:50<00:46,  3.20it/s]

 51%|█████     | 153/300 [00:51<00:47,  3.08it/s]

 51%|█████▏    | 154/300 [00:51<00:43,  3.35it/s]

 52%|█████▏    | 155/300 [00:51<00:46,  3.14it/s]

 52%|█████▏    | 156/300 [00:52<00:41,  3.47it/s]

 52%|█████▏    | 157/300 [00:52<00:46,  3.09it/s]

 53%|█████▎    | 158/300 [00:52<00:43,  3.24it/s]

 53%|█████▎    | 159/300 [00:53<00:50,  2.78it/s]

 53%|█████▎    | 160/300 [00:53<00:46,  3.01it/s]

 54%|█████▎    | 161/300 [00:54<00:54,  2.54it/s]

 54%|█████▍    | 162/300 [00:54<00:52,  2.63it/s]

 54%|█████▍    | 163/300 [00:54<00:57,  2.38it/s]

 55%|█████▍    | 164/300 [00:55<00:54,  2.50it/s]

 55%|█████▌    | 165/300 [00:55<00:53,  2.51it/s]

 55%|█████▌    | 166/300 [00:55<00:51,  2.63it/s]

 56%|█████▌    | 167/300 [00:56<00:54,  2.42it/s]

 56%|█████▌    | 168/300 [00:56<00:48,  2.74it/s]

 56%|█████▋    | 169/300 [00:57<00:53,  2.47it/s]

 57%|█████▋    | 170/300 [00:57<00:50,  2.57it/s]

 57%|█████▋    | 171/300 [00:57<00:50,  2.53it/s]

 57%|█████▋    | 172/300 [00:58<00:44,  2.87it/s]

 58%|█████▊    | 173/300 [00:58<00:46,  2.71it/s]

 58%|█████▊    | 174/300 [00:58<00:41,  3.04it/s]

 58%|█████▊    | 175/300 [00:59<00:44,  2.78it/s]

 59%|█████▊    | 176/300 [00:59<00:39,  3.11it/s]

 59%|█████▉    | 177/300 [00:59<00:44,  2.77it/s]

 59%|█████▉    | 178/300 [01:00<00:38,  3.20it/s]

 60%|█████▉    | 179/300 [01:00<00:41,  2.94it/s]

 60%|██████    | 180/300 [01:00<00:37,  3.21it/s]

 60%|██████    | 181/300 [01:01<00:41,  2.88it/s]

 61%|██████    | 182/300 [01:01<00:37,  3.19it/s]

 61%|██████    | 183/300 [01:01<00:40,  2.86it/s]

 61%|██████▏   | 184/300 [01:02<00:40,  2.86it/s]

 62%|██████▏   | 185/300 [01:02<00:42,  2.68it/s]

 62%|██████▏   | 186/300 [01:03<00:39,  2.88it/s]

 62%|██████▏   | 187/300 [01:03<00:39,  2.86it/s]

 63%|██████▎   | 188/300 [01:03<00:35,  3.14it/s]

 63%|██████▎   | 189/300 [01:04<00:38,  2.85it/s]

 63%|██████▎   | 190/300 [01:04<00:35,  3.09it/s]

 64%|██████▎   | 191/300 [01:04<00:37,  2.94it/s]

 64%|██████▍   | 192/300 [01:05<00:36,  2.97it/s]

 64%|██████▍   | 193/300 [01:05<00:41,  2.57it/s]

 65%|██████▍   | 194/300 [01:05<00:40,  2.65it/s]

 65%|██████▌   | 195/300 [01:06<00:41,  2.55it/s]

 65%|██████▌   | 196/300 [01:06<00:37,  2.77it/s]

 66%|██████▌   | 197/300 [01:07<00:39,  2.62it/s]

 66%|██████▌   | 198/300 [01:07<00:35,  2.87it/s]

 66%|██████▋   | 199/300 [01:07<00:36,  2.78it/s]

 67%|██████▋   | 200/300 [01:07<00:33,  2.98it/s]

 67%|██████▋   | 201/300 [01:08<00:35,  2.78it/s]

 67%|██████▋   | 202/300 [01:08<00:31,  3.07it/s]

 68%|██████▊   | 203/300 [01:08<00:33,  2.93it/s]

 68%|██████▊   | 204/300 [01:09<00:29,  3.22it/s]

 68%|██████▊   | 205/300 [01:09<00:31,  2.99it/s]

 69%|██████▊   | 206/300 [01:09<00:27,  3.39it/s]

 69%|██████▉   | 207/300 [01:10<00:32,  2.88it/s]

 69%|██████▉   | 208/300 [01:10<00:31,  2.91it/s]

 70%|██████▉   | 209/300 [01:11<00:32,  2.77it/s]

 70%|███████   | 210/300 [01:11<00:30,  2.94it/s]

 70%|███████   | 211/300 [01:11<00:33,  2.62it/s]

 71%|███████   | 212/300 [01:12<00:29,  2.94it/s]

 71%|███████   | 213/300 [01:12<00:34,  2.53it/s]

 71%|███████▏  | 214/300 [01:12<00:30,  2.82it/s]

 72%|███████▏  | 215/300 [01:13<00:31,  2.68it/s]

 72%|███████▏  | 216/300 [01:13<00:28,  2.99it/s]

 72%|███████▏  | 217/300 [01:13<00:32,  2.58it/s]

 73%|███████▎  | 218/300 [01:14<00:27,  3.02it/s]

 73%|███████▎  | 219/300 [01:14<00:30,  2.63it/s]

 73%|███████▎  | 220/300 [01:14<00:27,  2.95it/s]

 74%|███████▎  | 221/300 [01:15<00:29,  2.69it/s]

 74%|███████▍  | 222/300 [01:15<00:25,  3.00it/s]

 74%|███████▍  | 223/300 [01:16<00:28,  2.70it/s]

 75%|███████▍  | 224/300 [01:16<00:26,  2.92it/s]

 75%|███████▌  | 225/300 [01:16<00:26,  2.79it/s]

 75%|███████▌  | 226/300 [01:17<00:24,  3.03it/s]

 76%|███████▌  | 227/300 [01:17<00:26,  2.78it/s]

 76%|███████▌  | 228/300 [01:17<00:23,  3.11it/s]

 76%|███████▋  | 229/300 [01:18<00:25,  2.75it/s]

 77%|███████▋  | 230/300 [01:18<00:24,  2.85it/s]

 77%|███████▋  | 231/300 [01:18<00:25,  2.70it/s]

 77%|███████▋  | 232/300 [01:19<00:24,  2.77it/s]

 78%|███████▊  | 233/300 [01:19<00:27,  2.46it/s]

 78%|███████▊  | 234/300 [01:19<00:23,  2.82it/s]

 78%|███████▊  | 235/300 [01:20<00:24,  2.64it/s]

 79%|███████▊  | 236/300 [01:20<00:23,  2.70it/s]

 79%|███████▉  | 237/300 [01:21<00:24,  2.60it/s]

 79%|███████▉  | 238/300 [01:21<00:21,  2.91it/s]

 80%|███████▉  | 239/300 [01:21<00:21,  2.88it/s]

 80%|████████  | 240/300 [01:22<00:18,  3.20it/s]

 80%|████████  | 241/300 [01:22<00:19,  3.02it/s]

 81%|████████  | 242/300 [01:22<00:18,  3.06it/s]

 81%|████████  | 243/300 [01:23<00:20,  2.72it/s]

 81%|████████▏ | 244/300 [01:23<00:19,  2.82it/s]

 82%|████████▏ | 245/300 [01:23<00:20,  2.67it/s]

 82%|████████▏ | 246/300 [01:24<00:19,  2.76it/s]

 82%|████████▏ | 247/300 [01:24<00:20,  2.59it/s]

 83%|████████▎ | 248/300 [01:24<00:17,  2.93it/s]

 83%|████████▎ | 249/300 [01:25<00:18,  2.80it/s]

 83%|████████▎ | 250/300 [01:25<00:17,  2.92it/s]

 84%|████████▎ | 251/300 [01:26<00:18,  2.59it/s]

 84%|████████▍ | 252/300 [01:26<00:16,  2.83it/s]

 84%|████████▍ | 253/300 [01:26<00:17,  2.74it/s]

 85%|████████▍ | 254/300 [01:26<00:14,  3.12it/s]

 85%|████████▌ | 255/300 [01:27<00:16,  2.75it/s]

 85%|████████▌ | 256/300 [01:27<00:14,  3.10it/s]

 86%|████████▌ | 257/300 [01:28<00:14,  2.93it/s]

 86%|████████▌ | 258/300 [01:28<00:13,  3.20it/s]

 86%|████████▋ | 259/300 [01:28<00:13,  2.97it/s]

 87%|████████▋ | 260/300 [01:28<00:12,  3.16it/s]

 87%|████████▋ | 261/300 [01:29<00:13,  2.99it/s]

 87%|████████▋ | 262/300 [01:29<00:11,  3.26it/s]

 88%|████████▊ | 263/300 [01:29<00:12,  3.00it/s]

 88%|████████▊ | 264/300 [01:30<00:11,  3.00it/s]

 88%|████████▊ | 265/300 [01:30<00:13,  2.59it/s]

 89%|████████▊ | 266/300 [01:31<00:12,  2.69it/s]

 89%|████████▉ | 267/300 [01:31<00:13,  2.43it/s]

 89%|████████▉ | 268/300 [01:31<00:11,  2.79it/s]

 90%|████████▉ | 269/300 [01:32<00:12,  2.47it/s]

 90%|█████████ | 270/300 [01:32<00:11,  2.65it/s]

 90%|█████████ | 271/300 [01:33<00:12,  2.39it/s]

 91%|█████████ | 272/300 [01:33<00:10,  2.65it/s]

 91%|█████████ | 273/300 [01:34<00:11,  2.39it/s]

 91%|█████████▏| 274/300 [01:34<00:09,  2.73it/s]

 92%|█████████▏| 275/300 [01:34<00:10,  2.45it/s]

 92%|█████████▏| 276/300 [01:35<00:09,  2.63it/s]

 92%|█████████▏| 277/300 [01:35<00:09,  2.53it/s]

 93%|█████████▎| 278/300 [01:35<00:07,  2.85it/s]

 93%|█████████▎| 279/300 [01:36<00:07,  2.81it/s]

 93%|█████████▎| 280/300 [01:36<00:06,  2.90it/s]

 94%|█████████▎| 281/300 [01:36<00:06,  2.72it/s]

 94%|█████████▍| 282/300 [01:37<00:05,  3.03it/s]

 94%|█████████▍| 283/300 [01:37<00:06,  2.80it/s]

 95%|█████████▍| 284/300 [01:37<00:05,  3.06it/s]

 95%|█████████▌| 285/300 [01:38<00:05,  2.65it/s]

 95%|█████████▌| 286/300 [01:38<00:05,  2.71it/s]

 96%|█████████▌| 287/300 [01:39<00:04,  2.62it/s]

 96%|█████████▌| 288/300 [01:39<00:03,  3.04it/s]

 96%|█████████▋| 289/300 [01:39<00:03,  2.79it/s]

 97%|█████████▋| 290/300 [01:40<00:03,  2.81it/s]

 97%|█████████▋| 291/300 [01:40<00:03,  2.61it/s]

 97%|█████████▋| 292/300 [01:40<00:02,  2.93it/s]

 98%|█████████▊| 293/300 [01:41<00:02,  2.67it/s]

 98%|█████████▊| 294/300 [01:41<00:02,  2.91it/s]

 98%|█████████▊| 295/300 [01:41<00:01,  2.56it/s]

 99%|█████████▊| 296/300 [01:42<00:01,  2.83it/s]

 99%|█████████▉| 297/300 [01:42<00:01,  2.72it/s]

 99%|█████████▉| 298/300 [01:42<00:00,  3.11it/s]

100%|█████████▉| 299/300 [01:43<00:00,  2.77it/s]

100%|██████████| 300/300 [01:43<00:00,  2.83it/s]
                                                     

                                                 

 91%|█████████ | 2500/2757 [5:42:56<33:59,  7.94s/it]

100%|██████████| 300/300 [01:43<00:00,  2.83it/s]

                                                 Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-2500/tokenizer_config.json.

 91%|█████████ | 2501/2757 [5:43:08<2:53:55, 40.76s/it]
 91%|█████████ | 2502/2757 [5:43:17<2:12:09, 31.09s/it]
 91%|█████████ | 2503/2757 [5:43:25<1:41:53, 24.07s/it]
 91%|█████████ | 2504/2757 [5:43:32<1:20:55, 19.19s/it]
 91%|█████████ | 2505/2757 [5:43:41<1:06:48, 15.91s/it]
 91%|█████████ | 2506/2757 [5:43:48<56:19, 13.46s/it]  
 91%|█████████ | 2507/2757 [5:43:56<49:16, 11.82s/it]
 91%|█████████ | 2508/2757 [5:44:05<44:48, 10.80s/it]
 91%|█████████ | 2509/2757 [5:44:13<41:18,  9.99s/it]
 91%|█████████ | 2510/2757 [5:44:21<39:05,  9.50s/it]
                                                     

 91%|█████████ | 2510/2757 [5:44:21<39:05,  9.50s/it]
 91%|█████████ | 2511/2757 [5:44:29<36:46,  8.97s/it]
 91%|█████████ | 2512/2757 [5:44:37<35:59,  8.81s/it]
 91%|█████████ | 2513/2757 [5:44:46<35:07,  8.64s/it]
 91%|█████████ | 2514/2757 [5:44:54<34:23,  8.49s/it]
 91%|█████████ | 2515/2757 [5:45:02<34:02,  8.44s/it]
 91%|█████████▏| 2516/2757 [5:45:10<33:27,  8.33s/it]
 91%|█████████▏| 2517/2757 [5:45:18<32:51,  8.21s/it]
 91%|█████████▏| 2518/2757 [5:45:26<32:04,  8.05s/it]
 91%|█████████▏| 2519/2757 [5:45:34<31:53,  8.04s/it]
 91%|█████████▏| 2520/2757 [5:45:42<31:43,  8.03s/it]
                                                     

 91%|█████████▏| 2520/2757 [5:45:42<31:43,  8.03s/it]
 91%|█████████▏| 2521/2757 [5:45:50<32:01,  8.14s/it]
 91%|█████████▏| 2522/2757 [5:45:59<31:59,  8.17s/it]
 92%|█████████▏| 2523/2757 [5:46:06<31:19,  8.03s/it]
 92%|█████████▏| 2524/2757 [5:46:14<31:05,  8.01s/it]
 92%|█████████▏| 2525/2757 [5:46:22<30:43,  7.94s/it]
 92%|█████████▏| 2526/2757 [5:46:30<31:00,  8.06s/it]
 92%|█████████▏| 2527/2757 [5:46:38<30:57,  8.08s/it]
 92%|█████████▏| 2528/2757 [5:46:46<30:30,  7.99s/it]
 92%|█████████▏| 2529/2757 [5:46:54<30:18,  7.98s/it]
 92%|█████████▏| 2530/2757 [5:47:01<29:27,  7.79s/it]
                                                     

 92%|█████████▏| 2530/2757 [5:47:01<29:27,  7.79s/it]
 92%|█████████▏| 2531/2757 [5:47:09<29:30,  7.83s/it]
 92%|█████████▏| 2532/2757 [5:47:17<29:37,  7.90s/it]
 92%|█████████▏| 2533/2757 [5:47:26<29:47,  7.98s/it]
 92%|█████████▏| 2534/2757 [5:47:33<29:15,  7.87s/it]
 92%|█████████▏| 2535/2757 [5:47:41<29:29,  7.97s/it]
 92%|█████████▏| 2536/2757 [5:47:49<29:14,  7.94s/it]
 92%|█████████▏| 2537/2757 [5:47:57<29:10,  7.96s/it]
 92%|█████████▏| 2538/2757 [5:48:05<28:47,  7.89s/it]
 92%|█████████▏| 2539/2757 [5:48:13<29:01,  7.99s/it]
 92%|█████████▏| 2540/2757 [5:48:21<29:03,  8.03s/it]
                                                     

 92%|█████████▏| 2540/2757 [5:48:21<29:03,  8.03s/it]
 92%|█████████▏| 2541/2757 [5:48:30<29:35,  8.22s/it]
 92%|█████████▏| 2542/2757 [5:48:37<28:30,  7.96s/it]
 92%|█████████▏| 2543/2757 [5:48:45<27:56,  7.83s/it]
 92%|█████████▏| 2544/2757 [5:48:53<28:07,  7.92s/it]
 92%|█████████▏| 2545/2757 [5:49:01<28:05,  7.95s/it]
 92%|█████████▏| 2546/2757 [5:49:09<28:16,  8.04s/it]
 92%|█████████▏| 2547/2757 [5:49:17<28:01,  8.01s/it]
 92%|█████████▏| 2548/2757 [5:49:26<28:16,  8.12s/it]
 92%|█████████▏| 2549/2757 [5:49:34<27:58,  8.07s/it]
 92%|█████████▏| 2550/2757 [5:49:42<27:50,  8.07s/it]
                                                     

 92%|█████████▏| 2550/2757 [5:49:42<27:50,  8.07s/it]
 93%|█████████▎| 2551/2757 [5:49:50<27:24,  7.99s/it]
 93%|█████████▎| 2552/2757 [5:49:58<27:29,  8.05s/it]
 93%|█████████▎| 2553/2757 [5:50:06<27:34,  8.11s/it]
 93%|█████████▎| 2554/2757 [5:50:14<27:34,  8.15s/it]
 93%|█████████▎| 2555/2757 [5:50:22<27:03,  8.04s/it]
 93%|█████████▎| 2556/2757 [5:50:30<26:53,  8.03s/it]
 93%|█████████▎| 2557/2757 [5:50:38<26:56,  8.08s/it]
 93%|█████████▎| 2558/2757 [5:50:46<26:27,  7.98s/it]
 93%|█████████▎| 2559/2757 [5:50:54<26:04,  7.90s/it]
 93%|█████████▎| 2560/2757 [5:51:02<26:17,  8.01s/it]
                                                     

 93%|█████████▎| 2560/2757 [5:51:02<26:17,  8.01s/it]
 93%|█████████▎| 2561/2757 [5:51:10<25:50,  7.91s/it]
 93%|█████████▎| 2562/2757 [5:51:18<25:53,  7.96s/it]
 93%|█████████▎| 2563/2757 [5:51:26<26:22,  8.16s/it]
 93%|█████████▎| 2564/2757 [5:51:35<26:19,  8.18s/it]
 93%|█████████▎| 2565/2757 [5:51:42<25:46,  8.05s/it]
 93%|█████████▎| 2566/2757 [5:51:50<25:14,  7.93s/it]
 93%|█████████▎| 2567/2757 [5:51:58<25:04,  7.92s/it]
 93%|█████████▎| 2568/2757 [5:52:06<25:15,  8.02s/it]
 93%|█████████▎| 2569/2757 [5:52:14<24:47,  7.91s/it]
 93%|█████████▎| 2570/2757 [5:52:22<24:32,  7.88s/it]
                                                     

 93%|█████████▎| 2570/2757 [5:52:22<24:32,  7.88s/it]
 93%|█████████▎| 2571/2757 [5:52:29<24:28,  7.90s/it]
 93%|█████████▎| 2572/2757 [5:52:37<24:13,  7.86s/it]
 93%|█████████▎| 2573/2757 [5:52:45<23:53,  7.79s/it]
 93%|█████████▎| 2574/2757 [5:52:53<24:09,  7.92s/it]
 93%|█████████▎| 2575/2757 [5:53:01<24:12,  7.98s/it]
 93%|█████████▎| 2576/2757 [5:53:09<24:21,  8.08s/it]
 93%|█████████▎| 2577/2757 [5:53:18<24:28,  8.16s/it]
 94%|█████████▎| 2578/2757 [5:53:26<24:02,  8.06s/it]
 94%|█████████▎| 2579/2757 [5:53:34<24:04,  8.12s/it]
 94%|█████████▎| 2580/2757 [5:53:42<23:47,  8.07s/it]
                                                     

 94%|█████████▎| 2580/2757 [5:53:42<23:47,  8.07s/it]
 94%|█████████▎| 2581/2757 [5:53:50<23:29,  8.01s/it]
 94%|█████████▎| 2582/2757 [5:53:58<23:12,  7.96s/it]
 94%|█████████▎| 2583/2757 [5:54:06<23:19,  8.05s/it]
 94%|█████████▎| 2584/2757 [5:54:14<22:59,  7.97s/it]
 94%|█████████▍| 2585/2757 [5:54:21<22:44,  7.93s/it]
 94%|█████████▍| 2586/2757 [5:54:29<22:38,  7.94s/it]
 94%|█████████▍| 2587/2757 [5:54:38<23:24,  8.26s/it]
 94%|█████████▍| 2588/2757 [5:54:47<23:15,  8.25s/it]
 94%|█████████▍| 2589/2757 [5:54:55<23:04,  8.24s/it]
 94%|█████████▍| 2590/2757 [5:55:03<22:55,  8.23s/it]
                                                     

 94%|█████████▍| 2590/2757 [5:55:03<22:55,  8.23s/it]
 94%|█████████▍| 2591/2757 [5:55:11<22:31,  8.14s/it]
 94%|█████████▍| 2592/2757 [5:55:19<22:29,  8.18s/it]
 94%|█████████▍| 2593/2757 [5:55:27<22:04,  8.08s/it]
 94%|█████████▍| 2594/2757 [5:55:35<21:27,  7.90s/it]
 94%|█████████▍| 2595/2757 [5:55:42<21:16,  7.88s/it]
 94%|█████████▍| 2596/2757 [5:55:50<21:02,  7.84s/it]
 94%|█████████▍| 2597/2757 [5:55:58<21:04,  7.90s/it]
 94%|█████████▍| 2598/2757 [5:56:07<21:16,  8.03s/it]
 94%|█████████▍| 2599/2757 [5:56:14<20:53,  7.93s/it]
 94%|█████████▍| 2600/2757 [5:56:23<21:03,  8.05s/it]
                                                     

 94%|█████████▍| 2600/2757 [5:56:23<21:03,  8.05s/it]
 94%|█████████▍| 2601/2757 [5:56:31<20:54,  8.04s/it]
 94%|█████████▍| 2602/2757 [5:56:39<20:46,  8.04s/it]
 94%|█████████▍| 2603/2757 [5:56:47<20:48,  8.11s/it]
 94%|█████████▍| 2604/2757 [5:56:55<20:24,  8.00s/it]
 94%|█████████▍| 2605/2757 [5:57:03<20:12,  7.98s/it]
 95%|█████████▍| 2606/2757 [5:57:11<20:10,  8.02s/it]
 95%|█████████▍| 2607/2757 [5:57:18<19:50,  7.94s/it]
 95%|█████████▍| 2608/2757 [5:57:26<19:17,  7.77s/it]
 95%|█████████▍| 2609/2757 [5:57:34<19:26,  7.88s/it]
 95%|█████████▍| 2610/2757 [5:57:42<19:34,  7.99s/it]
                                                     

 95%|█████████▍| 2610/2757 [5:57:42<19:34,  7.99s/it]
 95%|█████████▍| 2611/2757 [5:57:51<19:39,  8.08s/it]
 95%|█████████▍| 2612/2757 [5:57:59<19:37,  8.12s/it]
 95%|█████████▍| 2613/2757 [5:58:07<19:26,  8.10s/it]
 95%|█████████▍| 2614/2757 [5:58:15<19:10,  8.04s/it]
 95%|█████████▍| 2615/2757 [5:58:22<18:49,  7.96s/it]
 95%|█████████▍| 2616/2757 [5:58:30<18:38,  7.93s/it]
 95%|█████████▍| 2617/2757 [5:58:38<18:20,  7.86s/it]
 95%|█████████▍| 2618/2757 [5:58:46<18:32,  8.00s/it]
 95%|█████████▍| 2619/2757 [5:58:55<18:39,  8.12s/it]
 95%|█████████▌| 2620/2757 [5:59:02<18:14,  7.99s/it]
                                                     

 95%|█████████▌| 2620/2757 [5:59:02<18:14,  7.99s/it]
 95%|█████████▌| 2621/2757 [5:59:10<18:03,  7.96s/it]
 95%|█████████▌| 2622/2757 [5:59:19<18:15,  8.11s/it]
 95%|█████████▌| 2623/2757 [5:59:27<18:22,  8.23s/it]
 95%|█████████▌| 2624/2757 [5:59:36<18:22,  8.29s/it]
 95%|█████████▌| 2625/2757 [5:59:44<18:06,  8.23s/it]
 95%|█████████▌| 2626/2757 [5:59:51<17:33,  8.04s/it]
 95%|█████████▌| 2627/2757 [6:00:00<17:31,  8.09s/it]
 95%|█████████▌| 2628/2757 [6:00:07<17:12,  8.01s/it]
 95%|█████████▌| 2629/2757 [6:00:16<17:16,  8.10s/it]
 95%|█████████▌| 2630/2757 [6:00:24<16:59,  8.03s/it]
                                                     

 95%|█████████▌| 2630/2757 [6:00:24<16:59,  8.03s/it]
 95%|█████████▌| 2631/2757 [6:00:32<16:51,  8.03s/it]
 95%|█████████▌| 2632/2757 [6:00:40<16:50,  8.08s/it]
 96%|█████████▌| 2633/2757 [6:00:48<16:35,  8.03s/it]
 96%|█████████▌| 2634/2757 [6:00:56<16:31,  8.06s/it]
 96%|█████████▌| 2635/2757 [6:01:03<16:06,  7.92s/it]
 96%|█████████▌| 2636/2757 [6:01:11<16:00,  7.94s/it]
 96%|█████████▌| 2637/2757 [6:01:20<15:59,  8.00s/it]
 96%|█████████▌| 2638/2757 [6:01:28<15:49,  7.98s/it]
 96%|█████████▌| 2639/2757 [6:01:35<15:32,  7.90s/it]
 96%|█████████▌| 2640/2757 [6:01:43<15:18,  7.85s/it]
                                                     

 96%|█████████▌| 2640/2757 [6:01:43<15:18,  7.85s/it]
 96%|█████████▌| 2641/2757 [6:01:51<15:17,  7.91s/it]
 96%|█████████▌| 2642/2757 [6:01:59<15:09,  7.91s/it]
 96%|█████████▌| 2643/2757 [6:02:07<15:07,  7.96s/it]
 96%|█████████▌| 2644/2757 [6:02:15<15:12,  8.07s/it]
 96%|█████████▌| 2645/2757 [6:02:23<14:44,  7.90s/it]
 96%|█████████▌| 2646/2757 [6:02:31<14:44,  7.97s/it]
 96%|█████████▌| 2647/2757 [6:02:39<14:31,  7.92s/it]
 96%|█████████▌| 2648/2757 [6:02:47<14:35,  8.03s/it]
 96%|█████████▌| 2649/2757 [6:02:55<14:27,  8.04s/it]
 96%|█████████▌| 2650/2757 [6:03:03<14:16,  8.00s/it]
                                                     

 96%|█████████▌| 2650/2757 [6:03:03<14:16,  8.00s/it]
 96%|█████████▌| 2651/2757 [6:03:11<14:00,  7.93s/it]
 96%|█████████▌| 2652/2757 [6:03:19<14:04,  8.04s/it]
 96%|█████████▌| 2653/2757 [6:03:27<13:56,  8.04s/it]
 96%|█████████▋| 2654/2757 [6:03:35<13:42,  7.99s/it]
 96%|█████████▋| 2655/2757 [6:03:44<13:50,  8.14s/it]
 96%|█████████▋| 2656/2757 [6:03:52<13:40,  8.12s/it]
 96%|█████████▋| 2657/2757 [6:03:59<13:22,  8.03s/it]
 96%|█████████▋| 2658/2757 [6:04:08<13:25,  8.14s/it]
 96%|█████████▋| 2659/2757 [6:04:16<13:13,  8.09s/it]
 96%|█████████▋| 2660/2757 [6:04:23<12:52,  7.96s/it]
                                                     

 96%|█████████▋| 2660/2757 [6:04:23<12:52,  7.96s/it]
 97%|█████████▋| 2661/2757 [6:04:31<12:38,  7.90s/it]
 97%|█████████▋| 2662/2757 [6:04:39<12:38,  7.99s/it]
 97%|█████████▋| 2663/2757 [6:04:47<12:25,  7.93s/it]
 97%|█████████▋| 2664/2757 [6:04:55<12:06,  7.81s/it]
 97%|█████████▋| 2665/2757 [6:05:03<12:01,  7.84s/it]
 97%|█████████▋| 2666/2757 [6:05:11<12:11,  8.04s/it]
 97%|█████████▋| 2667/2757 [6:05:20<12:13,  8.15s/it]
 97%|█████████▋| 2668/2757 [6:05:27<11:48,  7.96s/it]
 97%|█████████▋| 2669/2757 [6:05:35<11:35,  7.90s/it]
 97%|█████████▋| 2670/2757 [6:05:43<11:22,  7.84s/it]
                                                     

 97%|█████████▋| 2670/2757 [6:05:43<11:22,  7.84s/it]
 97%|█████████▋| 2671/2757 [6:05:50<11:12,  7.82s/it]
 97%|█████████▋| 2672/2757 [6:05:58<11:03,  7.80s/it]
 97%|█████████▋| 2673/2757 [6:06:06<11:03,  7.89s/it]
 97%|█████████▋| 2674/2757 [6:06:14<10:58,  7.93s/it]
 97%|█████████▋| 2675/2757 [6:06:22<10:48,  7.91s/it]
 97%|█████████▋| 2676/2757 [6:06:30<10:37,  7.87s/it]
 97%|█████████▋| 2677/2757 [6:06:38<10:32,  7.90s/it]
 97%|█████████▋| 2678/2757 [6:06:46<10:22,  7.88s/it]
 97%|█████████▋| 2679/2757 [6:06:54<10:18,  7.93s/it]
 97%|█████████▋| 2680/2757 [6:07:02<10:17,  8.02s/it]
                                                     

 97%|█████████▋| 2680/2757 [6:07:02<10:17,  8.02s/it]
 97%|█████████▋| 2681/2757 [6:07:10<10:14,  8.09s/it]
 97%|█████████▋| 2682/2757 [6:07:18<10:08,  8.12s/it]
 97%|█████████▋| 2683/2757 [6:07:26<09:50,  7.97s/it]
 97%|█████████▋| 2684/2757 [6:07:34<09:52,  8.11s/it]
 97%|█████████▋| 2685/2757 [6:07:43<09:43,  8.11s/it]
 97%|█████████▋| 2686/2757 [6:07:51<09:41,  8.19s/it]
 97%|█████████▋| 2687/2757 [6:07:59<09:21,  8.02s/it]
 97%|█████████▋| 2688/2757 [6:08:06<09:06,  7.92s/it]
 98%|█████████▊| 2689/2757 [6:08:14<08:55,  7.88s/it]
 98%|█████████▊| 2690/2757 [6:08:22<08:49,  7.91s/it]
                                                     

 98%|█████████▊| 2690/2757 [6:08:22<08:49,  7.91s/it]
 98%|█████████▊| 2691/2757 [6:08:30<08:48,  8.01s/it]
 98%|█████████▊| 2692/2757 [6:08:38<08:38,  7.98s/it]
 98%|█████████▊| 2693/2757 [6:08:46<08:32,  8.01s/it]
 98%|█████████▊| 2694/2757 [6:08:54<08:19,  7.93s/it]
 98%|█████████▊| 2695/2757 [6:09:02<08:16,  8.02s/it]
 98%|█████████▊| 2696/2757 [6:09:10<08:06,  7.98s/it]
 98%|█████████▊| 2697/2757 [6:09:18<07:57,  7.95s/it]
 98%|█████████▊| 2698/2757 [6:09:26<07:47,  7.92s/it]
 98%|█████████▊| 2699/2757 [6:09:34<07:42,  7.97s/it]
 98%|█████████▊| 2700/2757 [6:09:42<07:32,  7.95s/it]
                                                     

 98%|█████████▊| 2700/2757 [6:09:42<07:32,  7.95s/it]
 98%|█████████▊| 2701/2757 [6:09:50<07:31,  8.07s/it]
 98%|█████████▊| 2702/2757 [6:09:58<07:20,  8.01s/it]
 98%|█████████▊| 2703/2757 [6:10:06<07:13,  8.03s/it]
 98%|█████████▊| 2704/2757 [6:10:14<06:59,  7.91s/it]
 98%|█████████▊| 2705/2757 [6:10:22<06:57,  8.04s/it]
 98%|█████████▊| 2706/2757 [6:10:30<06:43,  7.91s/it]
 98%|█████████▊| 2707/2757 [6:10:38<06:41,  8.02s/it]
 98%|█████████▊| 2708/2757 [6:10:46<06:33,  8.03s/it]
 98%|█████████▊| 2709/2757 [6:10:54<06:23,  7.99s/it]
 98%|█████████▊| 2710/2757 [6:11:02<06:13,  7.94s/it]
                                                     

 98%|█████████▊| 2710/2757 [6:11:02<06:13,  7.94s/it]
 98%|█████████▊| 2711/2757 [6:11:10<06:03,  7.90s/it]
 98%|█████████▊| 2712/2757 [6:11:17<05:54,  7.87s/it]
 98%|█████████▊| 2713/2757 [6:11:25<05:48,  7.91s/it]
 98%|█████████▊| 2714/2757 [6:11:34<05:45,  8.04s/it]
 98%|█████████▊| 2715/2757 [6:11:41<05:31,  7.90s/it]
 99%|█████████▊| 2716/2757 [6:11:49<05:22,  7.88s/it]
 99%|█████████▊| 2717/2757 [6:11:57<05:17,  7.94s/it]
 99%|█████████▊| 2718/2757 [6:12:05<05:12,  8.00s/it]
 99%|█████████▊| 2719/2757 [6:12:13<05:04,  8.02s/it]
 99%|█████████▊| 2720/2757 [6:12:21<04:55,  7.99s/it]
                                                     

 99%|█████████▊| 2720/2757 [6:12:21<04:55,  7.99s/it]
 99%|█████████▊| 2721/2757 [6:12:29<04:42,  7.84s/it]
 99%|█████████▊| 2722/2757 [6:12:36<04:30,  7.73s/it]
 99%|█████████▉| 2723/2757 [6:12:44<04:25,  7.81s/it]
 99%|█████████▉| 2724/2757 [6:12:52<04:18,  7.84s/it]
 99%|█████████▉| 2725/2757 [6:13:00<04:11,  7.86s/it]
 99%|█████████▉| 2726/2757 [6:13:08<04:04,  7.89s/it]
 99%|█████████▉| 2727/2757 [6:13:16<03:57,  7.92s/it]
 99%|█████████▉| 2728/2757 [6:13:24<03:50,  7.95s/it]
 99%|█████████▉| 2729/2757 [6:13:32<03:45,  8.04s/it]
 99%|█████████▉| 2730/2757 [6:13:40<03:34,  7.95s/it]
                                                     

 99%|█████████▉| 2730/2757 [6:13:40<03:34,  7.95s/it]
 99%|█████████▉| 2731/2757 [6:13:48<03:27,  7.99s/it]
 99%|█████████▉| 2732/2757 [6:13:56<03:21,  8.07s/it]
 99%|█████████▉| 2733/2757 [6:14:04<03:12,  8.00s/it]
 99%|█████████▉| 2734/2757 [6:14:12<03:02,  7.93s/it]
 99%|█████████▉| 2735/2757 [6:14:20<02:52,  7.86s/it]
 99%|█████████▉| 2736/2757 [6:14:27<02:43,  7.77s/it]
 99%|█████████▉| 2737/2757 [6:14:36<02:41,  8.09s/it]
 99%|█████████▉| 2738/2757 [6:14:44<02:33,  8.07s/it]
 99%|█████████▉| 2739/2757 [6:14:52<02:21,  7.88s/it]
 99%|█████████▉| 2740/2757 [6:14:59<02:13,  7.84s/it]
                                                     

 99%|█████████▉| 2740/2757 [6:14:59<02:13,  7.84s/it]
 99%|█████████▉| 2741/2757 [6:15:07<02:05,  7.86s/it]
 99%|█████████▉| 2742/2757 [6:15:15<01:58,  7.87s/it]
 99%|█████████▉| 2743/2757 [6:15:23<01:50,  7.92s/it]
100%|█████████▉| 2744/2757 [6:15:31<01:44,  8.03s/it]
100%|█████████▉| 2745/2757 [6:15:39<01:36,  8.01s/it]
100%|█████████▉| 2746/2757 [6:15:48<01:29,  8.15s/it]
100%|█████████▉| 2747/2757 [6:15:56<01:21,  8.18s/it]
100%|█████████▉| 2748/2757 [6:16:03<01:11,  7.94s/it]
100%|█████████▉| 2749/2757 [6:16:12<01:04,  8.10s/it]
100%|█████████▉| 2750/2757 [6:16:20<00:56,  8.05s/it]
                                                     
{'eval_loss': '0.4573', 'eval_runtime': '104.7', 'eval_samples_per_second': '22.93', 'eval_steps_per_second': '2.867', 'eval_rewards/chosen': '-3.349', 'eval_rewards/rejected': '-4.964', 'eval_rewards/accuracies': '0.7217', 'eval_rewards/margins': '1.615', 'eval_logps/chosen': '-113.1', 'eval_logps/rejected': '-135.5', 'eval_logits/chosen': '-4.366', 'eval_logits/rejected': '-4.316', 'epoch': '2.721'}
{'loss': '0.2396', 'grad_norm': '0.5165', 'learning_rate': '1.855e-05', 'rewards/chosen': '-3.146', 'rewards/rejected': '-5.601', 'rewards/accuracies': '0.8609', 'rewards/margins': '2.456', 'logps/chosen': '-113.9', 'logps/rejected': '-145.7', 'logits/chosen': '-4.321', 'logits/rejected': '-4.34', 'epoch': '2.731'}
{'loss': '0.251', 'grad_norm': '0.5313', 'learning_rate': '1.78e-05', 'rewards/chosen': '-3.095', 'rewards/rejected': '-5.495', 'rewards/accuracies': '0.8547', 'rewards/margins': '2.4', 'logps/chosen': '-112', 'logps/rejected': '-144.2', 'logits/chosen': '-4.337', 'logits/rejected': '-4.252', 'epoch': '2.742'}
{'loss': '0.2669', 'grad_norm': '0.6009', 'learning_rate': '1.705e-05', 'rewards/chosen': '-3.144', 'rewards/rejected': '-5.451', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.306', 'logps/chosen': '-112.2', 'logps/rejected': '-141.5', 'logits/chosen': '-4.345', 'logits/rejected': '-4.265', 'epoch': '2.753'}
{'loss': '0.2554', 'grad_norm': '0.5517', 'learning_rate': '1.631e-05', 'rewards/chosen': '-3.108', 'rewards/rejected': '-5.491', 'rewards/accuracies': '0.8422', 'rewards/margins': '2.383', 'logps/chosen': '-113.3', 'logps/rejected': '-140.6', 'logits/chosen': '-4.365', 'logits/rejected': '-4.302', 'epoch': '2.764'}
{'loss': '0.2582', 'grad_norm': '0.6179', 'learning_rate': '1.556e-05', 'rewards/chosen': '-3.127', 'rewards/rejected': '-5.444', 'rewards/accuracies': '0.8375', 'rewards/margins': '2.317', 'logps/chosen': '-113.7', 'logps/rejected': '-143.8', 'logits/chosen': '-4.218', 'logits/rejected': '-4.297', 'epoch': '2.775'}
{'loss': '0.2668', 'grad_norm': '0.5837', 'learning_rate': '1.481e-05', 'rewards/chosen': '-3.097', 'rewards/rejected': '-5.393', 'rewards/accuracies': '0.8422', 'rewards/margins': '2.297', 'logps/chosen': '-107.7', 'logps/rejected': '-136.1', 'logits/chosen': '-4.289', 'logits/rejected': '-4.224', 'epoch': '2.786'}
{'loss': '0.2648', 'grad_norm': '0.6681', 'learning_rate': '1.406e-05', 'rewards/chosen': '-3.148', 'rewards/rejected': '-5.445', 'rewards/accuracies': '0.8406', 'rewards/margins': '2.297', 'logps/chosen': '-112.1', 'logps/rejected': '-140.3', 'logits/chosen': '-4.223', 'logits/rejected': '-4.183', 'epoch': '2.797'}
{'loss': '0.2467', 'grad_norm': '0.5711', 'learning_rate': '1.331e-05', 'rewards/chosen': '-3.118', 'rewards/rejected': '-5.44', 'rewards/accuracies': '0.8547', 'rewards/margins': '2.322', 'logps/chosen': '-113.7', 'logps/rejected': '-142.5', 'logits/chosen': '-4.21', 'logits/rejected': '-4.185', 'epoch': '2.808'}
{'loss': '0.262', 'grad_norm': '0.5574', 'learning_rate': '1.257e-05', 'rewards/chosen': '-3.077', 'rewards/rejected': '-5.374', 'rewards/accuracies': '0.8438', 'rewards/margins': '2.297', 'logps/chosen': '-109', 'logps/rejected': '-136.3', 'logits/chosen': '-4.284', 'logits/rejected': '-4.183', 'epoch': '2.819'}
{'loss': '0.2469', 'grad_norm': '0.549', 'learning_rate': '1.182e-05', 'rewards/chosen': '-3.052', 'rewards/rejected': '-5.457', 'rewards/accuracies': '0.8555', 'rewards/margins': '2.405', 'logps/chosen': '-111.3', 'logps/rejected': '-140.8', 'logits/chosen': '-4.32', 'logits/rejected': '-4.266', 'epoch': '2.829'}
{'loss': '0.2493', 'grad_norm': '0.4994', 'learning_rate': '1.107e-05', 'rewards/chosen': '-3.06', 'rewards/rejected': '-5.434', 'rewards/accuracies': '0.8508', 'rewards/margins': '2.374', 'logps/chosen': '-108.5', 'logps/rejected': '-137.1', 'logits/chosen': '-4.284', 'logits/rejected': '-4.174', 'epoch': '2.84'}
{'loss': '0.2597', 'grad_norm': '0.596', 'learning_rate': '1.032e-05', 'rewards/chosen': '-3.036', 'rewards/rejected': '-5.337', 'rewards/accuracies': '0.8578', 'rewards/margins': '2.3', 'logps/chosen': '-107.8', 'logps/rejected': '-135.4', 'logits/chosen': '-4.267', 'logits/rejected': '-4.113', 'epoch': '2.851'}
{'loss': '0.2591', 'grad_norm': '0.5377', 'learning_rate': '9.574e-06', 'rewards/chosen': '-3.168', 'rewards/rejected': '-5.573', 'rewards/accuracies': '0.8469', 'rewards/margins': '2.405', 'logps/chosen': '-111.8', 'logps/rejected': '-141.1', 'logits/chosen': '-4.322', 'logits/rejected': '-4.283', 'epoch': '2.862'}
{'loss': '0.2559', 'grad_norm': '0.6059', 'learning_rate': '8.826e-06', 'rewards/chosen': '-3.091', 'rewards/rejected': '-5.442', 'rewards/accuracies': '0.8352', 'rewards/margins': '2.351', 'logps/chosen': '-111.7', 'logps/rejected': '-141', 'logits/chosen': '-4.247', 'logits/rejected': '-4.217', 'epoch': '2.873'}
{'loss': '0.2493', 'grad_norm': '0.6502', 'learning_rate': '8.078e-06', 'rewards/chosen': '-3.102', 'rewards/rejected': '-5.487', 'rewards/accuracies': '0.8523', 'rewards/margins': '2.385', 'logps/chosen': '-112.2', 'logps/rejected': '-142.6', 'logits/chosen': '-4.256', 'logits/rejected': '-4.207', 'epoch': '2.884'}
{'loss': '0.2628', 'grad_norm': '0.524', 'learning_rate': '7.33e-06', 'rewards/chosen': '-3.17', 'rewards/rejected': '-5.479', 'rewards/accuracies': '0.8336', 'rewards/margins': '2.309', 'logps/chosen': '-113', 'logps/rejected': '-140.3', 'logits/chosen': '-4.29', 'logits/rejected': '-4.22', 'epoch': '2.895'}
{'loss': '0.2616', 'grad_norm': '0.6188', 'learning_rate': '6.582e-06', 'rewards/chosen': '-3.144', 'rewards/rejected': '-5.507', 'rewards/accuracies': '0.8484', 'rewards/margins': '2.363', 'logps/chosen': '-111.8', 'logps/rejected': '-140.9', 'logits/chosen': '-4.309', 'logits/rejected': '-4.274', 'epoch': '2.906'}
{'loss': '0.2506', 'grad_norm': '0.66', 'learning_rate': '5.834e-06', 'rewards/chosen': '-3.092', 'rewards/rejected': '-5.493', 'rewards/accuracies': '0.8562', 'rewards/margins': '2.402', 'logps/chosen': '-112.2', 'logps/rejected': '-143.6', 'logits/chosen': '-4.265', 'logits/rejected': '-4.203', 'epoch': '2.916'}
{'loss': '0.2568', 'grad_norm': '0.4364', 'learning_rate': '5.086e-06', 'rewards/chosen': '-3.115', 'rewards/rejected': '-5.444', 'rewards/accuracies': '0.8406', 'rewards/margins': '2.328', 'logps/chosen': '-110', 'logps/rejected': '-139.2', 'logits/chosen': '-4.176', 'logits/rejected': '-4.107', 'epoch': '2.927'}
{'loss': '0.2593', 'grad_norm': '0.5918', 'learning_rate': '4.338e-06', 'rewards/chosen': '-3.071', 'rewards/rejected': '-5.375', 'rewards/accuracies': '0.8438', 'rewards/margins': '2.304', 'logps/chosen': '-112.2', 'logps/rejected': '-139.6', 'logits/chosen': '-4.195', 'logits/rejected': '-4.221', 'epoch': '2.938'}
{'loss': '0.2563', 'grad_norm': '0.4884', 'learning_rate': '3.59e-06', 'rewards/chosen': '-3.157', 'rewards/rejected': '-5.468', 'rewards/accuracies': '0.8375', 'rewards/margins': '2.311', 'logps/chosen': '-113.7', 'logps/rejected': '-140.7', 'logits/chosen': '-4.237', 'logits/rejected': '-4.152', 'epoch': '2.949'}
{'loss': '0.2622', 'grad_norm': '0.6017', 'learning_rate': '2.842e-06', 'rewards/chosen': '-3.124', 'rewards/rejected': '-5.479', 'rewards/accuracies': '0.8359', 'rewards/margins': '2.355', 'logps/chosen': '-112.5', 'logps/rejected': '-141.4', 'logits/chosen': '-4.274', 'logits/rejected': '-4.203', 'epoch': '2.96'}
{'loss': '0.2499', 'grad_norm': '0.6066', 'learning_rate': '2.094e-06', 'rewards/chosen': '-3.07', 'rewards/rejected': '-5.482', 'rewards/accuracies': '0.8547', 'rewards/margins': '2.412', 'logps/chosen': '-112.7', 'logps/rejected': '-142.3', 'logits/chosen': '-4.194', 'logits/rejected': '-4.145', 'epoch': '2.971'}
{'loss': '0.2538', 'grad_norm': '0.6238', 'learning_rate': '1.346e-06', 'rewards/chosen': '-3.098', 'rewards/rejected': '-5.456', 'rewards/accuracies': '0.8492', 'rewards/margins': '2.358', 'logps/chosen': '-111.6', 'logps/rejected': '-141.5', 'logits/chosen': '-4.227', 'logits/rejected': '-4.174', 'epoch': '2.982'}
{'loss': '0.2607', 'grad_norm': '0.6461', 'learning_rate': '5.984e-07', 'rewards/chosen': '-3.128', 'rewards/rejected': '-5.369', 'rewards/accuracies': '0.8492', 'rewards/margins': '2.241', 'logps/chosen': '-109.2', 'logps/rejected': '-137.4', 'logits/chosen': '-4.289', 'logits/rejected': '-4.216', 'epoch': '2.993'}
100%|█████████▉| 2750/2757 [6:16:20<00:56,  8.05s/it]
100%|█████████▉| 2751/2757 [6:16:31<00:54,  9.02s/it]
100%|█████████▉| 2752/2757 [6:16:39<00:43,  8.63s/it]
100%|█████████▉| 2753/2757 [6:16:47<00:33,  8.38s/it]
100%|█████████▉| 2754/2757 [6:16:54<00:24,  8.20s/it]
100%|█████████▉| 2755/2757 [6:17:02<00:16,  8.16s/it]
100%|█████████▉| 2756/2757 [6:17:10<00:08,  8.06s/it]
100%|██████████| 2757/2757 [6:17:16<00:00,  7.44s/it]
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:172: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:202: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/workspace/post-training-experiments/dpo/DPO_SmolLM135M/.venv/lib/python3.12/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)


  0%|          | 0/300 [00:00<?, ?it/s]

  1%|          | 2/300 [00:00<00:52,  5.70it/s]

  1%|          | 3/300 [00:00<01:46,  2.80it/s]

  1%|▏         | 4/300 [00:01<01:34,  3.12it/s]

  2%|▏         | 5/300 [00:01<01:44,  2.81it/s]

  2%|▏         | 6/300 [00:01<01:40,  2.92it/s]

  2%|▏         | 7/300 [00:02<01:55,  2.54it/s]

  3%|▎         | 8/300 [00:02<01:50,  2.64it/s]

  3%|▎         | 9/300 [00:03<01:52,  2.59it/s]

  3%|▎         | 10/300 [00:03<01:49,  2.66it/s]

  4%|▎         | 11/300 [00:03<01:51,  2.59it/s]

  4%|▍         | 12/300 [00:04<01:39,  2.91it/s]

  4%|▍         | 13/300 [00:04<01:49,  2.63it/s]

  5%|▍         | 14/300 [00:04<01:40,  2.86it/s]

  5%|▌         | 15/300 [00:05<01:49,  2.61it/s]

  5%|▌         | 16/300 [00:05<01:36,  2.94it/s]

  6%|▌         | 17/300 [00:06<01:39,  2.83it/s]

  6%|▌         | 18/300 [00:06<01:30,  3.13it/s]

  6%|▋         | 19/300 [00:06<01:40,  2.80it/s]

  7%|▋         | 20/300 [00:07<01:32,  3.03it/s]

  7%|▋         | 21/300 [00:07<01:50,  2.53it/s]

  7%|▋         | 22/300 [00:07<01:36,  2.87it/s]

  8%|▊         | 23/300 [00:08<01:35,  2.89it/s]

  8%|▊         | 24/300 [00:08<01:26,  3.21it/s]

  8%|▊         | 25/300 [00:08<01:37,  2.81it/s]

  9%|▊         | 26/300 [00:09<01:30,  3.01it/s]

  9%|▉         | 27/300 [00:09<01:41,  2.70it/s]

  9%|▉         | 28/300 [00:09<01:29,  3.05it/s]

 10%|▉         | 29/300 [00:10<01:31,  2.95it/s]

 10%|█         | 30/300 [00:10<01:22,  3.26it/s]

 10%|█         | 31/300 [00:10<01:25,  3.14it/s]

 11%|█         | 32/300 [00:10<01:18,  3.41it/s]

 11%|█         | 33/300 [00:11<01:21,  3.27it/s]

 11%|█▏        | 34/300 [00:11<01:17,  3.45it/s]

 12%|█▏        | 35/300 [00:11<01:23,  3.19it/s]

 12%|█▏        | 36/300 [00:12<01:19,  3.34it/s]

 12%|█▏        | 37/300 [00:12<01:27,  3.01it/s]

 13%|█▎        | 38/300 [00:12<01:18,  3.35it/s]

 13%|█▎        | 39/300 [00:13<01:27,  2.97it/s]

 13%|█▎        | 40/300 [00:13<01:19,  3.27it/s]

 14%|█▎        | 41/300 [00:13<01:26,  3.01it/s]

 14%|█▍        | 42/300 [00:14<01:18,  3.30it/s]

 14%|█▍        | 43/300 [00:14<01:21,  3.14it/s]

 15%|█▍        | 44/300 [00:14<01:15,  3.41it/s]

 15%|█▌        | 45/300 [00:15<01:22,  3.10it/s]

 15%|█▌        | 46/300 [00:15<01:23,  3.04it/s]

 16%|█▌        | 47/300 [00:15<01:29,  2.83it/s]

 16%|█▌        | 48/300 [00:16<01:22,  3.06it/s]

 16%|█▋        | 49/300 [00:16<01:23,  3.02it/s]

 17%|█▋        | 50/300 [00:16<01:14,  3.35it/s]

 17%|█▋        | 51/300 [00:17<01:17,  3.21it/s]

 17%|█▋        | 52/300 [00:17<01:14,  3.31it/s]

 18%|█▊        | 53/300 [00:17<01:27,  2.81it/s]

 18%|█▊        | 54/300 [00:18<01:18,  3.11it/s]

 18%|█▊        | 55/300 [00:18<01:30,  2.70it/s]

 19%|█▊        | 56/300 [00:18<01:18,  3.12it/s]

 19%|█▉        | 57/300 [00:19<01:21,  3.00it/s]

 19%|█▉        | 58/300 [00:19<01:22,  2.95it/s]

 20%|█▉        | 59/300 [00:19<01:27,  2.75it/s]

 20%|██        | 60/300 [00:20<01:18,  3.07it/s]

 20%|██        | 61/300 [00:20<01:21,  2.93it/s]

 21%|██        | 62/300 [00:20<01:26,  2.74it/s]

 21%|██        | 63/300 [00:21<01:29,  2.64it/s]

 21%|██▏       | 64/300 [00:21<01:26,  2.72it/s]

 22%|██▏       | 65/300 [00:22<01:29,  2.62it/s]

 22%|██▏       | 66/300 [00:22<01:20,  2.89it/s]

 22%|██▏       | 67/300 [00:22<01:27,  2.65it/s]

 23%|██▎       | 68/300 [00:23<01:19,  2.94it/s]

 23%|██▎       | 69/300 [00:23<01:21,  2.82it/s]

 23%|██▎       | 70/300 [00:23<01:13,  3.12it/s]

 24%|██▎       | 71/300 [00:23<01:15,  3.02it/s]

 24%|██▍       | 72/300 [00:24<01:09,  3.28it/s]

 24%|██▍       | 73/300 [00:24<01:12,  3.13it/s]

 25%|██▍       | 74/300 [00:24<01:08,  3.31it/s]

 25%|██▌       | 75/300 [00:25<01:16,  2.94it/s]

 25%|██▌       | 76/300 [00:25<01:16,  2.92it/s]

 26%|██▌       | 77/300 [00:26<01:21,  2.74it/s]

 26%|██▌       | 78/300 [00:26<01:14,  2.99it/s]

 26%|██▋       | 79/300 [00:26<01:16,  2.90it/s]

 27%|██▋       | 80/300 [00:26<01:11,  3.06it/s]

 27%|██▋       | 81/300 [00:27<01:19,  2.75it/s]

 27%|██▋       | 82/300 [00:27<01:11,  3.05it/s]

 28%|██▊       | 83/300 [00:28<01:13,  2.94it/s]

 28%|██▊       | 84/300 [00:28<01:14,  2.90it/s]

 28%|██▊       | 85/300 [00:28<01:19,  2.72it/s]

 29%|██▊       | 86/300 [00:29<01:10,  3.01it/s]

 29%|██▉       | 87/300 [00:29<01:13,  2.90it/s]

 29%|██▉       | 88/300 [00:29<01:07,  3.14it/s]

 30%|██▉       | 89/300 [00:30<01:11,  2.95it/s]

 30%|███       | 90/300 [00:30<01:06,  3.17it/s]

 30%|███       | 91/300 [00:30<01:09,  2.99it/s]

 31%|███       | 92/300 [00:30<01:03,  3.26it/s]

 31%|███       | 93/300 [00:31<01:09,  2.98it/s]

 31%|███▏      | 94/300 [00:31<01:02,  3.27it/s]

 32%|███▏      | 95/300 [00:32<01:10,  2.91it/s]

 32%|███▏      | 96/300 [00:32<01:05,  3.13it/s]

 32%|███▏      | 97/300 [00:32<01:06,  3.04it/s]

 33%|███▎      | 98/300 [00:32<01:02,  3.22it/s]

 33%|███▎      | 99/300 [00:33<01:07,  2.98it/s]

 33%|███▎      | 100/300 [00:33<01:03,  3.15it/s]

 34%|███▎      | 101/300 [00:33<01:07,  2.95it/s]

 34%|███▍      | 102/300 [00:34<01:05,  3.01it/s]

 34%|███▍      | 103/300 [00:34<01:08,  2.88it/s]

 35%|███▍      | 104/300 [00:34<01:01,  3.20it/s]

 35%|███▌      | 105/300 [00:35<01:04,  3.02it/s]

 35%|███▌      | 106/300 [00:35<01:00,  3.22it/s]

 36%|███▌      | 107/300 [00:35<01:04,  3.00it/s]

 36%|███▌      | 108/300 [00:36<01:02,  3.08it/s]

 36%|███▋      | 109/300 [00:36<01:06,  2.89it/s]

 37%|███▋      | 110/300 [00:36<01:02,  3.05it/s]

 37%|███▋      | 111/300 [00:37<01:11,  2.66it/s]

 37%|███▋      | 112/300 [00:37<01:05,  2.86it/s]

 38%|███▊      | 113/300 [00:38<01:17,  2.42it/s]

 38%|███▊      | 114/300 [00:38<01:13,  2.53it/s]

 38%|███▊      | 115/300 [00:39<01:16,  2.43it/s]

 39%|███▊      | 116/300 [00:39<01:06,  2.77it/s]

 39%|███▉      | 117/300 [00:39<01:10,  2.59it/s]

 39%|███▉      | 118/300 [00:39<01:03,  2.87it/s]

 40%|███▉      | 119/300 [00:40<01:04,  2.82it/s]

 40%|████      | 120/300 [00:40<01:02,  2.89it/s]

 40%|████      | 121/300 [00:41<01:03,  2.82it/s]

 41%|████      | 122/300 [00:41<00:58,  3.05it/s]

 41%|████      | 123/300 [00:41<01:02,  2.85it/s]

 41%|████▏     | 124/300 [00:42<00:58,  3.03it/s]

 42%|████▏     | 125/300 [00:42<00:59,  2.93it/s]

 42%|████▏     | 126/300 [00:42<00:59,  2.91it/s]

 42%|████▏     | 127/300 [00:43<01:02,  2.77it/s]

 43%|████▎     | 128/300 [00:43<00:57,  2.98it/s]

 43%|████▎     | 129/300 [00:43<00:59,  2.86it/s]

 43%|████▎     | 130/300 [00:44<00:54,  3.14it/s]

 44%|████▎     | 131/300 [00:44<01:00,  2.78it/s]

 44%|████▍     | 132/300 [00:44<00:55,  3.03it/s]

 44%|████▍     | 133/300 [00:45<01:02,  2.68it/s]

 45%|████▍     | 134/300 [00:45<00:56,  2.93it/s]

 45%|████▌     | 135/300 [00:45<00:58,  2.80it/s]

 45%|████▌     | 136/300 [00:46<00:52,  3.09it/s]

 46%|████▌     | 137/300 [00:46<00:53,  3.03it/s]

 46%|████▌     | 138/300 [00:46<00:49,  3.25it/s]

 46%|████▋     | 139/300 [00:47<00:50,  3.19it/s]

 47%|████▋     | 140/300 [00:47<00:46,  3.44it/s]

 47%|████▋     | 141/300 [00:47<00:48,  3.27it/s]

 47%|████▋     | 142/300 [00:47<00:50,  3.13it/s]

 48%|████▊     | 143/300 [00:48<00:53,  2.94it/s]

 48%|████▊     | 144/300 [00:48<00:48,  3.24it/s]

 48%|████▊     | 145/300 [00:48<00:48,  3.18it/s]

 49%|████▊     | 146/300 [00:49<00:46,  3.33it/s]

 49%|████▉     | 147/300 [00:49<00:53,  2.88it/s]

 49%|████▉     | 148/300 [00:49<00:49,  3.04it/s]

 50%|████▉     | 149/300 [00:50<00:53,  2.80it/s]

 50%|█████     | 150/300 [00:50<00:50,  3.00it/s]

 50%|█████     | 151/300 [00:51<00:51,  2.88it/s]

 51%|█████     | 152/300 [00:51<00:46,  3.19it/s]

 51%|█████     | 153/300 [00:51<00:47,  3.09it/s]

 51%|█████▏    | 154/300 [00:51<00:43,  3.36it/s]

 52%|█████▏    | 155/300 [00:52<00:45,  3.17it/s]

 52%|█████▏    | 156/300 [00:52<00:41,  3.49it/s]

 52%|█████▏    | 157/300 [00:52<00:45,  3.12it/s]

 53%|█████▎    | 158/300 [00:53<00:43,  3.26it/s]

 53%|█████▎    | 159/300 [00:53<00:50,  2.79it/s]

 53%|█████▎    | 160/300 [00:53<00:46,  3.02it/s]

 54%|█████▎    | 161/300 [00:54<00:50,  2.76it/s]

 54%|█████▍    | 162/300 [00:54<00:49,  2.79it/s]

 54%|█████▍    | 163/300 [00:55<00:55,  2.47it/s]

 55%|█████▍    | 164/300 [00:55<00:52,  2.58it/s]

 55%|█████▌    | 165/300 [00:55<00:52,  2.56it/s]

 55%|█████▌    | 166/300 [00:56<00:50,  2.67it/s]

 56%|█████▌    | 167/300 [00:56<00:54,  2.44it/s]

 56%|█████▌    | 168/300 [00:56<00:48,  2.75it/s]

 56%|█████▋    | 169/300 [00:57<00:52,  2.48it/s]

 57%|█████▋    | 170/300 [00:57<00:50,  2.58it/s]

 57%|█████▋    | 171/300 [00:58<00:50,  2.54it/s]

 57%|█████▋    | 172/300 [00:58<00:44,  2.87it/s]

 58%|█████▊    | 173/300 [00:58<00:46,  2.71it/s]

 58%|█████▊    | 174/300 [00:59<00:41,  3.05it/s]

 58%|█████▊    | 175/300 [00:59<00:46,  2.69it/s]

 59%|█████▊    | 176/300 [00:59<00:40,  3.03it/s]

 59%|█████▉    | 177/300 [01:00<00:44,  2.73it/s]

 59%|█████▉    | 178/300 [01:00<00:38,  3.16it/s]

 60%|█████▉    | 179/300 [01:00<00:41,  2.92it/s]

 60%|██████    | 180/300 [01:01<00:37,  3.19it/s]

 60%|██████    | 181/300 [01:01<00:41,  2.88it/s]

 61%|██████    | 182/300 [01:01<00:37,  3.18it/s]

 61%|██████    | 183/300 [01:02<00:40,  2.91it/s]

 61%|██████▏   | 184/300 [01:02<00:40,  2.90it/s]

 62%|██████▏   | 185/300 [01:02<00:42,  2.71it/s]

 62%|██████▏   | 186/300 [01:03<00:39,  2.90it/s]

 62%|██████▏   | 187/300 [01:03<00:39,  2.87it/s]

 63%|██████▎   | 188/300 [01:03<00:35,  3.15it/s]

 63%|██████▎   | 189/300 [01:04<00:38,  2.86it/s]

 63%|██████▎   | 190/300 [01:04<00:35,  3.10it/s]

 64%|██████▎   | 191/300 [01:04<00:37,  2.89it/s]

 64%|██████▍   | 192/300 [01:05<00:36,  2.94it/s]

 64%|██████▍   | 193/300 [01:05<00:41,  2.56it/s]

 65%|██████▍   | 194/300 [01:06<00:40,  2.64it/s]

 65%|██████▌   | 195/300 [01:06<00:43,  2.42it/s]

 65%|██████▌   | 196/300 [01:06<00:39,  2.66it/s]

 66%|██████▌   | 197/300 [01:07<00:39,  2.64it/s]

 66%|██████▌   | 198/300 [01:07<00:35,  2.87it/s]

 66%|██████▋   | 199/300 [01:07<00:37,  2.73it/s]

 67%|██████▋   | 200/300 [01:08<00:34,  2.93it/s]

 67%|██████▋   | 201/300 [01:08<00:37,  2.63it/s]

 67%|██████▋   | 202/300 [01:08<00:33,  2.93it/s]

 68%|██████▊   | 203/300 [01:09<00:34,  2.85it/s]

 68%|██████▊   | 204/300 [01:09<00:30,  3.15it/s]

 68%|██████▊   | 205/300 [01:09<00:32,  2.97it/s]

 69%|██████▊   | 206/300 [01:10<00:28,  3.35it/s]

 69%|██████▉   | 207/300 [01:10<00:31,  2.91it/s]

 69%|██████▉   | 208/300 [01:10<00:31,  2.93it/s]

 70%|██████▉   | 209/300 [01:11<00:32,  2.78it/s]

 70%|███████   | 210/300 [01:11<00:29,  3.01it/s]

 70%|███████   | 211/300 [01:12<00:33,  2.66it/s]

 71%|███████   | 212/300 [01:12<00:29,  2.97it/s]

 71%|███████   | 213/300 [01:12<00:31,  2.72it/s]

 71%|███████▏  | 214/300 [01:13<00:28,  2.97it/s]

 72%|███████▏  | 215/300 [01:13<00:30,  2.77it/s]

 72%|███████▏  | 216/300 [01:13<00:27,  3.07it/s]

 72%|███████▏  | 217/300 [01:14<00:30,  2.75it/s]

 73%|███████▎  | 218/300 [01:14<00:26,  3.15it/s]

 73%|███████▎  | 219/300 [01:14<00:28,  2.86it/s]

 73%|███████▎  | 220/300 [01:15<00:25,  3.14it/s]

 74%|███████▎  | 221/300 [01:15<00:28,  2.80it/s]

 74%|███████▍  | 222/300 [01:15<00:25,  3.10it/s]

 74%|███████▍  | 223/300 [01:16<00:27,  2.75it/s]

 75%|███████▍  | 224/300 [01:16<00:25,  2.97it/s]

 75%|███████▌  | 225/300 [01:16<00:26,  2.82it/s]

 75%|███████▌  | 226/300 [01:17<00:24,  3.05it/s]

 76%|███████▌  | 227/300 [01:17<00:26,  2.80it/s]

 76%|███████▌  | 228/300 [01:17<00:23,  3.13it/s]

 76%|███████▋  | 229/300 [01:18<00:25,  2.76it/s]

 77%|███████▋  | 230/300 [01:18<00:24,  2.86it/s]

 77%|███████▋  | 231/300 [01:19<00:25,  2.71it/s]

 77%|███████▋  | 232/300 [01:19<00:24,  2.77it/s]

 78%|███████▊  | 233/300 [01:19<00:27,  2.47it/s]

 78%|███████▊  | 234/300 [01:20<00:23,  2.82it/s]

 78%|███████▊  | 235/300 [01:20<00:24,  2.63it/s]

 79%|███████▊  | 236/300 [01:20<00:23,  2.69it/s]

 79%|███████▉  | 237/300 [01:21<00:24,  2.58it/s]

 79%|███████▉  | 238/300 [01:21<00:21,  2.91it/s]

 80%|███████▉  | 239/300 [01:21<00:21,  2.83it/s]

 80%|████████  | 240/300 [01:22<00:18,  3.16it/s]

 80%|████████  | 241/300 [01:22<00:19,  3.05it/s]

 81%|████████  | 242/300 [01:22<00:18,  3.09it/s]

 81%|████████  | 243/300 [01:23<00:20,  2.73it/s]

 81%|████████▏ | 244/300 [01:23<00:19,  2.82it/s]

 82%|████████▏ | 245/300 [01:24<00:20,  2.67it/s]

 82%|████████▏ | 246/300 [01:24<00:19,  2.77it/s]

 82%|████████▏ | 247/300 [01:24<00:20,  2.59it/s]

 83%|████████▎ | 248/300 [01:25<00:17,  2.92it/s]

 83%|████████▎ | 249/300 [01:25<00:18,  2.78it/s]

 83%|████████▎ | 250/300 [01:25<00:17,  2.89it/s]

 84%|████████▎ | 251/300 [01:26<00:17,  2.78it/s]

 84%|████████▍ | 252/300 [01:26<00:16,  2.99it/s]

 84%|████████▍ | 253/300 [01:26<00:17,  2.73it/s]

 85%|████████▍ | 254/300 [01:27<00:14,  3.11it/s]

 85%|████████▌ | 255/300 [01:27<00:16,  2.80it/s]

 85%|████████▌ | 256/300 [01:27<00:14,  3.14it/s]

 86%|████████▌ | 257/300 [01:28<00:14,  2.95it/s]

 86%|████████▌ | 258/300 [01:28<00:13,  3.21it/s]

 86%|████████▋ | 259/300 [01:28<00:14,  2.93it/s]

 87%|████████▋ | 260/300 [01:29<00:12,  3.13it/s]

 87%|████████▋ | 261/300 [01:29<00:13,  2.96it/s]

 87%|████████▋ | 262/300 [01:29<00:11,  3.26it/s]

 88%|████████▊ | 263/300 [01:30<00:12,  3.00it/s]

 88%|████████▊ | 264/300 [01:30<00:12,  2.99it/s]

 88%|████████▊ | 265/300 [01:30<00:13,  2.59it/s]

 89%|████████▊ | 266/300 [01:31<00:12,  2.68it/s]

 89%|████████▉ | 267/300 [01:31<00:13,  2.42it/s]

 89%|████████▉ | 268/300 [01:32<00:11,  2.78it/s]

 90%|████████▉ | 269/300 [01:32<00:12,  2.46it/s]

 90%|█████████ | 270/300 [01:32<00:11,  2.62it/s]

 90%|█████████ | 271/300 [01:33<00:12,  2.37it/s]

 91%|█████████ | 272/300 [01:33<00:10,  2.63it/s]

 91%|█████████ | 273/300 [01:34<00:11,  2.38it/s]

 91%|█████████▏| 274/300 [01:34<00:09,  2.72it/s]

 92%|█████████▏| 275/300 [01:34<00:10,  2.44it/s]

 92%|█████████▏| 276/300 [01:35<00:09,  2.62it/s]

 92%|█████████▏| 277/300 [01:35<00:09,  2.53it/s]

 93%|█████████▎| 278/300 [01:35<00:07,  2.85it/s]

 93%|█████████▎| 279/300 [01:36<00:07,  2.80it/s]

 93%|█████████▎| 280/300 [01:36<00:06,  2.89it/s]

 94%|█████████▎| 281/300 [01:37<00:06,  2.72it/s]

 94%|█████████▍| 282/300 [01:37<00:05,  3.02it/s]

 94%|█████████▍| 283/300 [01:37<00:06,  2.79it/s]

 95%|█████████▍| 284/300 [01:37<00:05,  3.06it/s]

 95%|█████████▌| 285/300 [01:38<00:05,  2.66it/s]

 95%|█████████▌| 286/300 [01:38<00:05,  2.71it/s]

 96%|█████████▌| 287/300 [01:39<00:05,  2.59it/s]

 96%|█████████▌| 288/300 [01:39<00:03,  3.01it/s]

 96%|█████████▋| 289/300 [01:39<00:04,  2.68it/s]

 97%|█████████▋| 290/300 [01:40<00:03,  2.73it/s]

 97%|█████████▋| 291/300 [01:40<00:03,  2.47it/s]

 97%|█████████▋| 292/300 [01:40<00:02,  2.80it/s]

 98%|█████████▊| 293/300 [01:41<00:02,  2.70it/s]

 98%|█████████▊| 294/300 [01:41<00:02,  2.95it/s]

 98%|█████████▊| 295/300 [01:42<00:01,  2.89it/s]

 99%|█████████▊| 296/300 [01:42<00:01,  3.11it/s]

 99%|█████████▉| 297/300 [01:42<00:01,  2.88it/s]

 99%|█████████▉| 298/300 [01:42<00:00,  3.25it/s]

100%|█████████▉| 299/300 [01:43<00:00,  2.53it/s]

100%|██████████| 300/300 [01:43<00:00,  2.57it/s]
                                                     

                                                 

100%|██████████| 2757/2757 [6:19:01<00:00,  7.44s/it]

100%|██████████| 300/300 [01:44<00:00,  2.57it/s]

                                                 Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/checkpoint-2757/tokenizer_config.json.

                                                     

100%|██████████| 2757/2757 [6:19:31<00:00,  7.44s/it]
100%|██████████| 2757/2757 [6:19:32<00:00,  8.26s/it]
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/final/tokenizer_config.json.
Unsloth: Restored added_tokens_decoder metadata in models/dpo_default/merged/tokenizer_config.json.
{'eval_loss': '0.4576', 'eval_runtime': '104.9', 'eval_samples_per_second': '22.88', 'eval_steps_per_second': '2.86', 'eval_rewards/chosen': '-3.332', 'eval_rewards/rejected': '-4.978', 'eval_rewards/accuracies': '0.7233', 'eval_rewards/margins': '1.646', 'eval_logps/chosen': '-112.9', 'eval_logps/rejected': '-135.6', 'eval_logits/chosen': '-4.33', 'eval_logits/rejected': '-4.281', 'epoch': '3'}
{'train_runtime': '2.277e+04', 'train_samples_per_second': '15.49', 'train_steps_per_second': '0.121', 'train_loss': '0.4124', 'epoch': '3'}
Saved adapter to models/dpo_default/final
Found HuggingFace hub cache directory: /workspace/.cache/huggingface/hub
Checking cache directory for required files...

Unsloth: Copying 1 files from cache to `models/dpo_default/merged`:   0%|          | 0/1 [00:00<?, ?it/s]
Unsloth: Copying 1 files from cache to `models/dpo_default/merged`: 100%|██████████| 1/1 [00:02<00:00,  2.20s/it]
Unsloth: Copying 1 files from cache to `models/dpo_default/merged`: 100%|██████████| 1/1 [00:02<00:00,  2.20s/it]
Successfully copied all 1 files from cache to `models/dpo_default/merged`
Checking cache directory for required files...
Cache check failed: tokenizer.model not found in local cache.
Not all required files found in cache. Will proceed with downloading.

Unsloth: Preparing safetensor model files:   0%|          | 0/1 [00:00<?, ?it/s]
Unsloth: Preparing safetensor model files: 100%|██████████| 1/1 [00:00<00:00, 37117.73it/s]

Unsloth: Merging weights into 16bit:   0%|          | 0/1 [00:00<?, ?it/s]
Unsloth: Merging weights into 16bit: 100%|██████████| 1/1 [00:02<00:00,  2.43s/it]
Unsloth: Merging weights into 16bit: 100%|██████████| 1/1 [00:02<00:00,  2.43s/it]
Unsloth: Merge process complete. Saved to `/workspace/post-training-experiments/dpo/DPO_SmolLM135M/models/dpo_default/merged`
Saved merged model to models/dpo_default/merged
== Step 2: post-dpo diversity (compare to baseline above) ==
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
`torch_dtype` is deprecated! Use `dtype` instead!
Skipping import of cpp extensions due to incompatible torch version. Please upgrade to torch >= 2.11.0 (found 2.10.0+cu128).

Loading weights:   0%|          | 0/272 [00:00<?, ?it/s]
Loading weights: 100%|██████████| 272/272 [00:00<00:00, 3064.64it/s]
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
Both `max_new_tokens` (=512) and `max_length`(=2048) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
[1/100] 16 responses  avg_len=815 chars
    t=0.3  ### Q1 **Question:** What metrics are used to control page layout in the dataset...
    t=0.3  ### Q1 **Question:** What metrics are used to control page layout in the dataset...
    t=0.3  ### Q1 **Question:** What metrics are used to control page layout in the dataset...
    t=0.3  ### Q1 **Question:** What does Figure 4 and 5 illustrate?  **Answer:** Figure 4 ...
    t=0.5  ### Q1 **Question:** What metrics are used to control page layout in the dataset...
    t=0.5  ### Q1 **Question:** What does Figure 4 and Table 4 illustrate?  **Answer:** Fig...
    t=0.5  ### Q1 **Question:** What does Figure 4 and 5 illustrate?  **Answer:** Figure 4 ...
    t=0.5  ### Q1 **Question:** What does Figure 4 and Figure 5 illustrate?  **Answer:** Fi...
    t=0.7  **Question:** What metrics are used to control page layout in the dataset?  **An...
    t=0.7  *'default' is a value that is empty if none. *Figure 4 and Figure 5 provide an o...
    t=0.7  ### Q1 **Question:** What metrics are used to control the page layout of the dat...
    t=0.7  *['How is the 'default' value defined in the context of the API format?'] *What ...
    t=1.0  ### Q1 **Question:** What metrics are used to control page layout in the dataset...
    t=1.0  ### Q1 **Question:** What metrics are used to control page layout in the dataset...
    t=1.0  *'default' represents the default value, empty if none *Figure 5 illustrates the...
    t=1.0  ### Q1 **Question:** What does Figure 4 and Table 4 illustrate?  **Answer:** Fig...
[2/100] 16 responses  avg_len=240 chars
    t=0.3  [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
    t=0.3  [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {" timedelt...
    t=0.3  [{"subject": "AoA", "relation": "is a", "object": "second entry of path estimate...
    t=0.3  [{"subject": "AoA", "relation": "is a", "object": "second entry of path estimate...
    t=0.5  [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
    t=0.5  [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {" timedelt...
    t=0.5  [{"subject": "AoA", "relation": "is a", "object": "second entry of path estimate...
    t=0.5  [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {" timedelt...
    t=0.7  [{"subject": "Option", "relation": "is a declaration in RLang", "object": "presy...
    t=0.7  [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
    t=0.7  [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"subject":...
    t=0.7  [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
    t=1.0  [{"subject": "Option", "relation": "describes", "object": "sequence of actions s...
    t=1.0  [{"subject": "AoA", "relation": "is a specification of", " object": "sequences o...
    t=1.0  [{"subject": "AoA", "relation": "is a entry of", "object": "RLang"}, {"day": "Op...
    t=1.0  [{"subject": "Option", "relation": "is a declaration in RLang", "object": "pecul...
[3/100] 16 responses  avg_len=218 chars
[4/100] 16 responses  avg_len=823 chars
[5/100] 16 responses  avg_len=880 chars
[6/100] 16 responses  avg_len=276 chars
[7/100] 16 responses  avg_len=152 chars
[8/100] 16 responses  avg_len=295 chars
[9/100] 16 responses  avg_len=230 chars
[10/100] 16 responses  avg_len=311 chars
[11/100] 16 responses  avg_len=741 chars
[12/100] 16 responses  avg_len=865 chars
[13/100] 16 responses  avg_len=133 chars
[14/100] 16 responses  avg_len=736 chars
[15/100] 16 responses  avg_len=327 chars
[16/100] 16 responses  avg_len=1030 chars
[17/100] 16 responses  avg_len=702 chars
[18/100] 16 responses  avg_len=1030 chars
[19/100] 16 responses  avg_len=297 chars
[20/100] 16 responses  avg_len=319 chars
[21/100] 16 responses  avg_len=420 chars
[22/100] 16 responses  avg_len=70 chars
[23/100] 16 responses  avg_len=320 chars
[24/100] 16 responses  avg_len=25 chars
[25/100] 16 responses  avg_len=68 chars
[26/100] 16 responses  avg_len=299 chars
[27/100] 16 responses  avg_len=720 chars
[28/100] 16 responses  avg_len=1254 chars
[29/100] 16 responses  avg_len=137 chars
[30/100] 16 responses  avg_len=353 chars
[31/100] 16 responses  avg_len=1093 chars
[32/100] 16 responses  avg_len=603 chars
[33/100] 16 responses  avg_len=75 chars
[34/100] 16 responses  avg_len=109 chars
[35/100] 16 responses  avg_len=1307 chars
[36/100] 16 responses  avg_len=750 chars
[37/100] 16 responses  avg_len=714 chars
[38/100] 16 responses  avg_len=179 chars
[39/100] 16 responses  avg_len=284 chars
[40/100] 16 responses  avg_len=120 chars
[41/100] 16 responses  avg_len=765 chars
[42/100] 16 responses  avg_len=259 chars
[43/100] 16 responses  avg_len=481 chars
[44/100] 16 responses  avg_len=241 chars
[45/100] 16 responses  avg_len=95 chars
[46/100] 16 responses  avg_len=320 chars
[47/100] 16 responses  avg_len=391 chars
[48/100] 16 responses  avg_len=1030 chars
[49/100] 16 responses  avg_len=283 chars
[50/100] 16 responses  avg_len=822 chars
[51/100] 16 responses  avg_len=788 chars
[52/100] 16 responses  avg_len=741 chars
[53/100] 16 responses  avg_len=294 chars
[54/100] 16 responses  avg_len=120 chars
[55/100] 16 responses  avg_len=550 chars
[56/100] 16 responses  avg_len=1174 chars
[57/100] 16 responses  avg_len=205 chars
[58/100] 16 responses  avg_len=190 chars
[59/100] 16 responses  avg_len=373 chars
[60/100] 16 responses  avg_len=674 chars
[61/100] 16 responses  avg_len=1170 chars
[62/100] 16 responses  avg_len=693 chars
[63/100] 16 responses  avg_len=245 chars
[64/100] 16 responses  avg_len=705 chars
[65/100] 16 responses  avg_len=161 chars
[66/100] 16 responses  avg_len=247 chars
[67/100] 16 responses  avg_len=846 chars
[68/100] 16 responses  avg_len=816 chars
[69/100] 16 responses  avg_len=276 chars
[70/100] 16 responses  avg_len=527 chars
[71/100] 16 responses  avg_len=253 chars
[72/100] 16 responses  avg_len=1074 chars
[73/100] 16 responses  avg_len=324 chars
[74/100] 16 responses  avg_len=110 chars
[75/100] 16 responses  avg_len=337 chars
[76/100] 16 responses  avg_len=703 chars
[77/100] 16 responses  avg_len=1396 chars
[78/100] 16 responses  avg_len=449 chars
[79/100] 16 responses  avg_len=592 chars
[80/100] 16 responses  avg_len=156 chars
[81/100] 16 responses  avg_len=243 chars
[82/100] 16 responses  avg_len=581 chars
[83/100] 16 responses  avg_len=272 chars
[84/100] 16 responses  avg_len=152 chars
[85/100] 16 responses  avg_len=75 chars
[86/100] 16 responses  avg_len=199 chars
[87/100] 16 responses  avg_len=306 chars
[88/100] 16 responses  avg_len=306 chars
[89/100] 16 responses  avg_len=1197 chars
[90/100] 16 responses  avg_len=689 chars
[91/100] 16 responses  avg_len=597 chars
[92/100] 16 responses  avg_len=677 chars
[93/100] 16 responses  avg_len=165 chars
[94/100] 16 responses  avg_len=274 chars
[95/100] 16 responses  avg_len=416 chars
[96/100] 16 responses  avg_len=287 chars
[97/100] 16 responses  avg_len=308 chars
[98/100] 16 responses  avg_len=62 chars
[99/100] 16 responses  avg_len=69 chars
[100/100] 16 responses  avg_len=129 chars

Saved 100 records to evals/dpo_default_n100_r4.jsonl
Skipping import of cpp extensions due to incompatible torch version. Please upgrade to torch >= 2.11.0 (found 2.10.0+cu128).
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Loading SBERT model: sentence-transformers/all-MiniLM-L6-v2 ...

Loading weights:   0%|          | 0/103 [00:00<?, ?it/s]
Loading weights:  46%|████▌     | 47/103 [00:00<00:00, 409.92it/s]
Loading weights:  91%|█████████▏| 94/103 [00:00<00:00, 414.36it/s]
Loading weights: 100%|██████████| 103/103 [00:00<00:00, 450.62it/s]
BertModel LOAD REPORT from: sentence-transformers/all-MiniLM-L6-v2
Key                     | Status     |  | 
------------------------+------------+--+-
embeddings.position_ids | UNEXPECTED |  | 

Notes:
- UNEXPECTED:	can be ignored when loading from different task/architecture; not ok if you expect identical arch.
Scoring 100 records ...
[1/100]  ead=0.2215  sbert=0.4202  vendi=4.7004
[2/100]  ead=0.1539  sbert=0.3031  vendi=3.0089
[3/100]  ead=0.1791  sbert=0.3479  vendi=3.7807
[4/100]  ead=0.2852  sbert=0.5217  vendi=6.8363
[5/100]  ead=0.2914  sbert=0.5311  vendi=6.9277
[6/100]  ead=0.0645  sbert=0.1331  vendi=1.4444
[7/100]  ead=0.1304  sbert=0.2600  vendi=2.8699
[8/100]  ead=0.0776  sbert=0.1590  vendi=1.9956
[9/100]  ead=0.0983  sbert=0.1994  vendi=2.3667
[10/100]  ead=0.0731  sbert=0.1503  vendi=1.9737
[11/100]  ead=0.0827  sbert=0.1691  vendi=2.0416
[12/100]  ead=0.0821  sbert=0.1679  vendi=2.0866
[13/100]  ead=0.0655  sbert=0.1352  vendi=1.7483
[14/100]  ead=0.0807  sbert=0.1652  vendi=2.1535
[15/100]  ead=0.1073  sbert=0.2166  vendi=2.4434
[16/100]  ead=0.3385  sbert=0.5999  vendi=8.1126
[17/100]  ead=0.0539  sbert=0.1119  vendi=1.7672
[18/100]  ead=0.3339  sbert=0.5934  vendi=8.0090
[19/100]  ead=0.2089  sbert=0.3991  vendi=3.5623
[20/100]  ead=0.1009  sbert=0.2043  vendi=2.3543
[21/100]  ead=0.0806  sbert=0.1650  vendi=1.9491
[22/100]  ead=0.1957  sbert=0.3766  vendi=2.7339
[23/100]  ead=0.3114  sbert=0.5609  vendi=5.9426
[24/100]  ead=0.1103  sbert=0.2223  vendi=1.7700
[25/100]  ead=0.0882  sbert=0.1798  vendi=1.6982
[26/100]  ead=0.1494  sbert=0.2950  vendi=3.0703
[27/100]  ead=0.1351  sbert=0.2688  vendi=3.2119
[28/100]  ead=0.2144  sbert=0.4084  vendi=5.0096
[29/100]  ead=0.0538  sbert=0.1117  vendi=1.5520
[30/100]  ead=0.0033  sbert=0.0070  vendi=1.0495
[31/100]  ead=0.2240  sbert=0.4243  vendi=5.3786
[32/100]  ead=0.2644  sbert=0.4895  vendi=5.5140
[33/100]  ead=0.1268  sbert=0.2533  vendi=2.8508
[34/100]  ead=0.0572  sbert=0.1185  vendi=1.5461
[35/100]  ead=0.2483  sbert=0.4640  vendi=4.7925
[36/100]  ead=0.0421  sbert=0.0880  vendi=1.5689
[37/100]  ead=0.0832  sbert=0.1701  vendi=2.1962
[38/100]  ead=0.0302  sbert=0.0634  vendi=1.3600
[39/100]  ead=0.0455  sbert=0.0948  vendi=1.6311
[40/100]  ead=0.0178  sbert=0.0376  vendi=1.1938
[41/100]  ead=0.0478  sbert=0.0996  vendi=1.6916
[42/100]  ead=0.0623  sbert=0.1289  vendi=1.8174
[43/100]  ead=0.0325  sbert=0.0681  vendi=1.4654
[44/100]  ead=0.1723  sbert=0.3359  vendi=3.1290
[45/100]  ead=0.1127  sbert=0.2269  vendi=2.2373
[46/100]  ead=0.0968  sbert=0.1966  vendi=2.1951
[47/100]  ead=0.0870  sbert=0.1775  vendi=2.2215
[48/100]  ead=0.1293  sbert=0.2581  vendi=3.1035
[49/100]  ead=0.1105  sbert=0.2227  vendi=2.5606
[50/100]  ead=0.0722  sbert=0.1485  vendi=2.0503
[51/100]  ead=0.0434  sbert=0.0905  vendi=1.6095
[52/100]  ead=0.1231  sbert=0.2464  vendi=2.9946
[53/100]  ead=0.0189  sbert=0.0399  vendi=1.2115
[54/100]  ead=0.2173  sbert=0.4132  vendi=3.7740
[55/100]  ead=0.0714  sbert=0.1468  vendi=1.9421
[56/100]  ead=0.0795  sbert=0.1629  vendi=2.1548
[57/100]  ead=0.0646  sbert=0.1334  vendi=1.8689
[58/100]  ead=0.2422  sbert=0.4542  vendi=2.8661
[59/100]  ead=0.1103  sbert=0.2224  vendi=2.3809
[60/100]  ead=0.0845  sbert=0.1727  vendi=2.2505
[61/100]  ead=0.2839  sbert=0.5197  vendi=6.7955
[62/100]  ead=0.0455  sbert=0.0948  vendi=1.6370
[63/100]  ead=0.0812  sbert=0.1662  vendi=1.9496
[64/100]  ead=0.0175  sbert=0.0369  vendi=1.2523
[65/100]  ead=0.2150  sbert=0.4094  vendi=2.5259
[66/100]  ead=0.1168  sbert=0.2345  vendi=2.2798
[67/100]  ead=0.1980  sbert=0.3807  vendi=4.4107
[68/100]  ead=0.0632  sbert=0.1306  vendi=1.9128
[69/100]  ead=0.0313  sbert=0.0657  vendi=1.3866
[70/100]  ead=0.0799  sbert=0.1637  vendi=2.2017
[71/100]  ead=0.0780  sbert=0.1600  vendi=2.0474
[72/100]  ead=0.2311  sbert=0.4360  vendi=5.4815
[73/100]  ead=0.1055  sbert=0.2132  vendi=2.1208
[74/100]  ead=0.0918  sbert=0.1869  vendi=1.9718
[75/100]  ead=0.0958  sbert=0.1945  vendi=2.1223
[76/100]  ead=0.0597  sbert=0.1235  vendi=1.8748
[77/100]  ead=0.0671  sbert=0.1384  vendi=1.9527
[78/100]  ead=0.1057  sbert=0.2136  vendi=2.4193
[79/100]  ead=0.0185  sbert=0.0391  vendi=1.2546
[80/100]  ead=0.0971  sbert=0.1972  vendi=2.0882
[81/100]  ead=0.1035  sbert=0.2094  vendi=2.3673
[82/100]  ead=0.1347  sbert=0.2680  vendi=3.1583
[83/100]  ead=0.0241  sbert=0.0508  vendi=1.3213
[84/100]  ead=0.0043  sbert=0.0092  vendi=1.0509
[85/100]  ead=0.2164  sbert=0.4118  vendi=2.9182
[86/100]  ead=0.1350  sbert=0.2685  vendi=2.6217
[87/100]  ead=0.0750  sbert=0.1541  vendi=2.0077
[88/100]  ead=0.0538  sbert=0.1116  vendi=1.5855
[89/100]  ead=0.2717  sbert=0.5008  vendi=6.5359
[90/100]  ead=0.0692  sbert=0.1426  vendi=2.0181
[91/100]  ead=0.2908  sbert=0.5302  vendi=6.5217
[92/100]  ead=0.1777  sbert=0.3454  vendi=3.8662
[93/100]  ead=0.2116  sbert=0.4037  vendi=3.1923
[94/100]  ead=0.0679  sbert=0.1400  vendi=1.8048
[95/100]  ead=0.0586  sbert=0.1213  vendi=1.6027
[96/100]  ead=0.0668  sbert=0.1378  vendi=1.7351
[97/100]  ead=0.1519  sbert=0.2994  vendi=3.0876
[98/100]  ead=0.2075  sbert=0.3967  vendi=2.8306
[99/100]  ead=0.0000  sbert=0.0000  vendi=1.0000
[100/100]  ead=0.0407  sbert=0.0850  vendi=1.4648

--- summary ---
  ead         mean=0.1193  min=0.0000  max=0.3385
  sbert       mean=0.2322  min=0.0000  max=0.5999
  vendi       mean=2.7410  min=1.0000  max=8.1126

Saved to evals/dpo_default_n100_r4_diversity.jsonl
==========================================
 Done. Compare the two diversity summaries:
   baseline: evals/baseline_smollm_135M_neuraltxt_v1_n100_r4.jsonl
   dpo_default: evals/dpo_default_n100_r4.jsonl
 A large drop in SBERT/Vendi => mode collapse.
 Merged model: models/dpo_default/merged
==========================================