LMCocktail-Mistral-7B-v1/Alpaca_eval/chatgpt_fn_--LMCocktail-Mistral-7B-v1/alpaca_eval_log.txt


Chunking for generation:   0%|          | 0/13 [00:00<?, ?it/s]INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.43s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.01s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.22s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/64 [00:00<?, ?it/s][A

  2%|▏         | 1/64 [00:26<27:51, 26.54s/it][A

  3%|▎         | 2/64 [00:52<27:18, 26.43s/it][A

  5%|▍         | 3/64 [01:18<26:38, 26.21s/it][A

  6%|▋         | 4/64 [01:51<28:42, 28.71s/it][A

  8%|▊         | 5/64 [02:08<23:59, 24.39s/it][A

  9%|▉         | 6/64 [02:23<20:42, 21.42s/it][A

 11%|█         | 7/64 [02:38<18:08, 19.09s/it][A

 12%|█▎        | 8/64 [02:48<15:14, 16.33s/it][A

 14%|█▍        | 9/64 [03:10<16:38, 18.15s/it][A

 16%|█▌        | 10/64 [03:38<19:01, 21.14s/it][A

 17%|█▋        | 11/64 [03:42<14:07, 15.99s/it][A

 19%|█▉        | 12/64 [03:59<13:58, 16.12s/it][A

 20%|██        | 13/64 [04:14<13:33, 15.96s/it][A

 22%|██▏       | 14/64 [04:33<13:57, 16.76s/it][A

 23%|██▎       | 15/64 [04:48<13:23, 16.40s/it][A

 25%|██▌       | 16/64 [05:24<17:38, 22.06s/it][A

 27%|██▋       | 17/64 [05:47<17:30, 22.35s/it][A

 28%|██▊       | 18/64 [06:11<17:35, 22.94s/it][A

 30%|██▉       | 19/64 [06:15<13:02, 17.39s/it][A

 31%|███▏      | 20/64 [06:40<14:13, 19.39s/it][A

 33%|███▎      | 21/64 [06:59<13:55, 19.43s/it][A

 34%|███▍      | 22/64 [07:21<14:05, 20.13s/it][A

 36%|███▌      | 23/64 [07:27<10:52, 15.91s/it][A

 38%|███▊      | 24/64 [07:43<10:34, 15.87s/it][A

 39%|███▉      | 25/64 [07:44<07:30, 11.55s/it][A

 41%|████      | 26/64 [08:08<09:35, 15.15s/it][A

 42%|████▏     | 27/64 [08:15<07:51, 12.75s/it][A

 44%|████▍     | 28/64 [08:37<09:23, 15.66s/it][A

 45%|████▌     | 29/64 [08:57<09:48, 16.81s/it][A

 47%|████▋     | 30/64 [09:12<09:18, 16.41s/it][A

 48%|████▊     | 31/64 [09:41<11:02, 20.08s/it][A

 50%|█████     | 32/64 [09:57<10:08, 19.03s/it][A

 52%|█████▏    | 33/64 [10:23<10:51, 21.01s/it][A

 53%|█████▎    | 34/64 [10:41<09:58, 19.96s/it][A

 55%|█████▍    | 35/64 [10:45<07:22, 15.26s/it][A

 56%|█████▋    | 36/64 [10:53<06:07, 13.11s/it][A

 58%|█████▊    | 37/64 [11:14<07:00, 15.57s/it][A

 59%|█████▉    | 38/64 [11:31<06:55, 15.96s/it][A

 61%|██████    | 39/64 [11:43<06:08, 14.74s/it][A

 62%|██████▎   | 40/64 [11:57<05:50, 14.62s/it][A

 64%|██████▍   | 41/64 [12:07<05:04, 13.24s/it][A

 66%|██████▌   | 42/64 [12:23<05:08, 14.04s/it][A

 67%|██████▋   | 43/64 [12:45<05:44, 16.40s/it][A

 69%|██████▉   | 44/64 [12:53<04:36, 13.85s/it][A

 70%|███████   | 45/64 [13:15<05:05, 16.10s/it][A

 72%|███████▏  | 46/64 [13:33<05:02, 16.79s/it][A

 73%|███████▎  | 47/64 [13:52<04:54, 17.33s/it][A

 75%|███████▌  | 48/64 [14:14<05:00, 18.81s/it][A

 77%|███████▋  | 49/64 [14:21<03:50, 15.35s/it][A

 78%|███████▊  | 50/64 [14:40<03:50, 16.49s/it][A

 80%|███████▉  | 51/64 [14:46<02:54, 13.42s/it][A

 81%|████████▏ | 52/64 [14:51<02:09, 10.80s/it][A

 83%|████████▎ | 53/64 [15:17<02:47, 15.23s/it][A

 84%|████████▍ | 54/64 [15:36<02:44, 16.48s/it][A

 86%|████████▌ | 55/64 [15:48<02:16, 15.20s/it][A

 88%|████████▊ | 56/64 [16:06<02:07, 15.99s/it][A

 89%|████████▉ | 57/64 [16:26<02:00, 17.17s/it][A

 91%|█████████ | 58/64 [16:50<01:55, 19.31s/it][A

 92%|█████████▏| 59/64 [17:03<01:27, 17.43s/it][A

 94%|█████████▍| 60/64 [17:10<00:57, 14.26s/it][A

 95%|█████████▌| 61/64 [17:30<00:47, 15.81s/it][A

 97%|█████████▋| 62/64 [17:48<00:32, 16.45s/it][A

 98%|█████████▊| 63/64 [18:11<00:18, 18.37s/it][A

100%|██████████| 64/64 [18:31<00:00, 18.90s/it][A
100%|██████████| 64/64 [18:31<00:00, 17.36s/it]
INFO:root:Time for 64 completions: 1111.1 seconds

Chunking for generation:   8%|▊         | 1/13 [18:45<3:45:08, 1125.70s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
INFO:root:Found 64. Only generating 64 .
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.39s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.12s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.31s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/64 [00:00<?, ?it/s][A

  2%|▏         | 1/64 [00:18<19:19, 18.40s/it][A

  3%|▎         | 2/64 [00:28<13:49, 13.37s/it][A

  5%|▍         | 3/64 [00:44<14:55, 14.68s/it][A

  6%|▋         | 4/64 [01:06<17:24, 17.41s/it][A

  8%|▊         | 5/64 [01:30<19:34, 19.90s/it][A

  9%|▉         | 6/64 [01:40<16:06, 16.66s/it][A

 11%|█         | 7/64 [01:52<14:10, 14.92s/it][A

 12%|█▎        | 8/64 [01:57<10:58, 11.76s/it][A

 14%|█▍        | 9/64 [02:15<12:37, 13.77s/it][A

 16%|█▌        | 10/64 [02:38<14:53, 16.54s/it][A

 17%|█▋        | 11/64 [03:00<16:16, 18.43s/it][A

 19%|█▉        | 12/64 [03:11<13:58, 16.12s/it][A

 20%|██        | 13/64 [03:33<15:13, 17.91s/it][A

 22%|██▏       | 14/64 [04:03<17:52, 21.45s/it][A

 23%|██▎       | 15/64 [04:16<15:22, 18.83s/it][A

 25%|██▌       | 16/64 [04:32<14:31, 18.15s/it][A

 27%|██▋       | 17/64 [04:48<13:38, 17.42s/it][A

 28%|██▊       | 18/64 [05:13<15:07, 19.72s/it][A

 30%|██▉       | 19/64 [05:36<15:30, 20.69s/it][A

 31%|███▏      | 20/64 [06:02<16:22, 22.33s/it][A

 33%|███▎      | 21/64 [06:22<15:31, 21.67s/it][A

 34%|███▍      | 22/64 [06:46<15:33, 22.23s/it][A

 36%|███▌      | 23/64 [07:06<14:46, 21.62s/it][A

 38%|███▊      | 24/64 [07:36<16:03, 24.09s/it][A

 39%|███▉      | 25/64 [07:41<12:02, 18.52s/it][A

 41%|████      | 26/64 [08:06<12:53, 20.37s/it][A

 42%|████▏     | 27/64 [08:31<13:21, 21.67s/it][A

 44%|████▍     | 28/64 [08:35<09:57, 16.59s/it][A

 45%|████▌     | 29/64 [08:54<10:06, 17.32s/it][A

 47%|████▋     | 30/64 [08:59<07:35, 13.39s/it][A

 48%|████▊     | 31/64 [09:17<08:13, 14.96s/it][A

 50%|█████     | 32/64 [09:33<08:09, 15.29s/it][A

 52%|█████▏    | 33/64 [09:57<09:11, 17.78s/it][A

 53%|█████▎    | 34/64 [10:04<07:16, 14.56s/it][A

 55%|█████▍    | 35/64 [10:17<06:48, 14.08s/it][A

 56%|█████▋    | 36/64 [10:28<06:07, 13.12s/it][A

 58%|█████▊    | 37/64 [10:53<07:32, 16.75s/it][A

 59%|█████▉    | 38/64 [11:01<06:08, 14.18s/it][A

 61%|██████    | 39/64 [11:06<04:41, 11.24s/it][A

 62%|██████▎   | 40/64 [11:28<05:50, 14.61s/it][A

 64%|██████▍   | 41/64 [11:41<05:23, 14.08s/it][A

 66%|██████▌   | 42/64 [11:45<04:01, 10.98s/it][A

 67%|██████▋   | 43/64 [12:00<04:21, 12.45s/it][A

 69%|██████▉   | 44/64 [12:18<04:40, 14.02s/it][A

 70%|███████   | 45/64 [12:32<04:27, 14.08s/it][A

 72%|███████▏  | 46/64 [12:42<03:50, 12.82s/it][A

 73%|███████▎  | 47/64 [13:33<06:49, 24.11s/it][A

 75%|███████▌  | 48/64 [13:52<06:04, 22.81s/it][A

 77%|███████▋  | 49/64 [14:02<04:43, 18.91s/it][A

 78%|███████▊  | 50/64 [14:10<03:37, 15.52s/it][A

 80%|███████▉  | 51/64 [14:19<02:55, 13.52s/it][A

 81%|████████▏ | 52/64 [14:42<03:15, 16.31s/it][A

 83%|████████▎ | 53/64 [14:59<03:02, 16.61s/it][A

 84%|████████▍ | 54/64 [15:17<02:50, 17.04s/it][A

 86%|████████▌ | 55/64 [15:26<02:12, 14.72s/it][A

 88%|████████▊ | 56/64 [15:45<02:07, 15.89s/it][A

 89%|████████▉ | 57/64 [15:47<01:21, 11.69s/it][A

 91%|█████████ | 58/64 [15:51<00:55,  9.31s/it][A

 92%|█████████▏| 59/64 [15:55<00:39,  7.85s/it][A

 94%|█████████▍| 60/64 [16:08<00:37,  9.34s/it][A

 95%|█████████▌| 61/64 [16:29<00:38, 12.92s/it][A

 97%|█████████▋| 62/64 [16:40<00:24, 12.21s/it][A

 98%|█████████▊| 63/64 [16:53<00:12, 12.43s/it][A

100%|██████████| 64/64 [17:08<00:00, 13.28s/it][A
100%|██████████| 64/64 [17:08<00:00, 16.07s/it]
INFO:root:Time for 64 completions: 1028.3 seconds

Chunking for generation:  15%|█▌        | 2/13 [36:01<3:16:38, 1072.61s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
INFO:root:Found 128. Only generating 64 .
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.26s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.00s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.19s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/64 [00:00<?, ?it/s][A

  2%|▏         | 1/64 [00:03<03:47,  3.61s/it][A

  3%|▎         | 2/64 [00:31<18:25, 17.83s/it][A

  5%|▍         | 3/64 [00:55<21:10, 20.83s/it][A

  6%|▋         | 4/64 [01:11<18:54, 18.90s/it][A

  8%|▊         | 5/64 [01:25<16:37, 16.90s/it][A

  9%|▉         | 6/64 [01:27<11:38, 12.04s/it][A

 11%|█         | 7/64 [01:45<13:20, 14.04s/it][A

 12%|█▎        | 8/64 [01:55<11:47, 12.64s/it][A

 14%|█▍        | 9/64 [02:31<18:09, 19.81s/it][A

 16%|█▌        | 10/64 [02:52<18:14, 20.27s/it][A

 17%|█▋        | 11/64 [03:33<23:28, 26.58s/it][A

 19%|█▉        | 12/64 [03:52<20:59, 24.21s/it][A

 20%|██        | 13/64 [04:09<18:57, 22.30s/it][A

 22%|██▏       | 14/64 [04:15<14:22, 17.25s/it][A

 23%|██▎       | 15/64 [04:26<12:39, 15.50s/it][A

 25%|██▌       | 16/64 [04:43<12:44, 15.92s/it][A

 27%|██▋       | 17/64 [04:49<10:04, 12.87s/it][A

 28%|██▊       | 18/64 [05:08<11:17, 14.73s/it][A

 30%|██▉       | 19/64 [05:20<10:18, 13.74s/it][A

 31%|███▏      | 20/64 [05:38<11:04, 15.10s/it][A

 33%|███▎      | 21/64 [06:56<24:24, 34.05s/it][A

 34%|███▍      | 22/64 [07:15<20:41, 29.55s/it][A

 36%|███▌      | 23/64 [07:22<15:31, 22.73s/it][A

 38%|███▊      | 24/64 [07:49<15:58, 23.97s/it][A

 39%|███▉      | 25/64 [08:13<15:39, 24.09s/it][A

 41%|████      | 26/64 [09:02<19:55, 31.45s/it][A

 42%|████▏     | 27/64 [09:45<21:29, 34.86s/it][A

 44%|████▍     | 28/64 [09:50<15:32, 25.89s/it][A

 45%|████▌     | 29/64 [10:31<17:49, 30.56s/it][A

 47%|████▋     | 30/64 [10:47<14:48, 26.14s/it][A

 48%|████▊     | 31/64 [10:52<10:53, 19.80s/it][A

 50%|█████     | 32/64 [10:54<07:40, 14.39s/it][A

 52%|█████▏    | 33/64 [11:18<08:56, 17.30s/it][A

 53%|█████▎    | 34/64 [11:25<07:10, 14.34s/it][A

 55%|█████▍    | 35/64 [11:59<09:47, 20.25s/it][A

 56%|█████▋    | 36/64 [12:06<07:35, 16.26s/it][A

 58%|█████▊    | 37/64 [12:16<06:25, 14.29s/it][A

 59%|█████▉    | 38/64 [12:37<07:06, 16.39s/it][A

 61%|██████    | 39/64 [12:50<06:23, 15.32s/it][A

 62%|██████▎   | 40/64 [13:04<05:59, 14.97s/it][A

 64%|██████▍   | 41/64 [13:05<04:04, 10.65s/it][A

 66%|██████▌   | 42/64 [13:17<04:06, 11.22s/it][A

 67%|██████▋   | 43/64 [13:32<04:19, 12.35s/it][A

 69%|██████▉   | 44/64 [14:18<07:24, 22.21s/it][A

 70%|███████   | 45/64 [14:25<05:40, 17.90s/it][A

 72%|███████▏  | 46/64 [14:35<04:36, 15.35s/it][A

 73%|███████▎  | 47/64 [14:40<03:28, 12.29s/it][A

 75%|███████▌  | 48/64 [14:56<03:35, 13.49s/it][A

 77%|███████▋  | 49/64 [15:38<05:28, 21.88s/it][A

 78%|███████▊  | 50/64 [16:04<05:23, 23.09s/it][A

 80%|███████▉  | 51/64 [16:13<04:07, 19.03s/it][A

 81%|████████▏ | 52/64 [16:54<05:07, 25.63s/it][A

 83%|████████▎ | 53/64 [17:15<04:26, 24.20s/it][A

 84%|████████▍ | 54/64 [17:31<03:36, 21.64s/it][A

 86%|████████▌ | 55/64 [18:18<04:23, 29.27s/it][A

 88%|████████▊ | 56/64 [18:42<03:42, 27.80s/it][A

 89%|████████▉ | 57/64 [18:58<02:48, 24.10s/it][A

 91%|█████████ | 58/64 [19:17<02:16, 22.73s/it][A

 92%|█████████▏| 59/64 [19:37<01:49, 21.91s/it][A

 94%|█████████▍| 60/64 [20:05<01:34, 23.61s/it][A

 95%|█████████▌| 61/64 [20:38<01:19, 26.58s/it][A

 97%|█████████▋| 62/64 [20:47<00:42, 21.35s/it][A

 98%|█████████▊| 63/64 [20:49<00:15, 15.36s/it][A

100%|██████████| 64/64 [21:00<00:00, 14.24s/it][A
100%|██████████| 64/64 [21:00<00:00, 19.70s/it]
INFO:root:Time for 64 completions: 1260.9 seconds

Chunking for generation:  23%|██▎       | 3/13 [57:08<3:13:37, 1161.71s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
INFO:root:Found 192. Only generating 64 .
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.43s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.12s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.32s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/64 [00:00<?, ?it/s][A

  2%|▏         | 1/64 [00:11<12:31, 11.93s/it][A

  3%|▎         | 2/64 [00:47<26:45, 25.89s/it][A

  5%|▍         | 3/64 [01:06<23:06, 22.73s/it][A

  6%|▋         | 4/64 [01:07<14:06, 14.11s/it][A

  8%|▊         | 5/64 [01:24<14:56, 15.20s/it][A

  9%|▉         | 6/64 [01:38<14:21, 14.86s/it][A

 11%|█         | 7/64 [01:45<11:32, 12.15s/it][A

 14%|█▍        | 9/64 [02:04<10:03, 10.97s/it][A

 16%|█▌        | 10/64 [02:25<12:04, 13.43s/it][A

 17%|█▋        | 11/64 [02:30<09:54, 11.21s/it][A

 19%|█▉        | 12/64 [03:43<24:36, 28.40s/it][A

 20%|██        | 13/64 [04:11<24:05, 28.34s/it][A

 22%|██▏       | 14/64 [04:32<21:43, 26.07s/it][A

 23%|██▎       | 15/64 [05:08<23:43, 29.05s/it][A

 25%|██▌       | 16/64 [05:12<17:19, 21.65s/it][A

 27%|██▋       | 17/64 [05:54<21:36, 27.58s/it][A

 28%|██▊       | 18/64 [06:12<18:59, 24.76s/it][A

 30%|██▉       | 19/64 [06:16<13:51, 18.49s/it][A

 31%|███▏      | 20/64 [06:21<10:40, 14.56s/it][A

 33%|███▎      | 21/64 [06:40<11:19, 15.79s/it][A

 34%|███▍      | 22/64 [07:09<13:52, 19.82s/it][A

 36%|███▌      | 23/64 [07:33<14:28, 21.17s/it][A

 38%|███▊      | 24/64 [07:56<14:26, 21.65s/it][A

 39%|███▉      | 25/64 [08:20<14:36, 22.48s/it][A

 41%|████      | 26/64 [09:17<20:42, 32.69s/it][A

 42%|████▏     | 27/64 [09:43<18:58, 30.78s/it][A

 44%|████▍     | 28/64 [10:03<16:24, 27.35s/it][A

 45%|████▌     | 29/64 [10:32<16:21, 28.05s/it][A

 47%|████▋     | 30/64 [11:02<16:13, 28.64s/it][A

 48%|████▊     | 31/64 [11:10<12:15, 22.28s/it][A

 50%|█████     | 32/64 [11:35<12:21, 23.16s/it][A

 52%|█████▏    | 33/64 [11:48<10:24, 20.15s/it][A

 53%|█████▎    | 34/64 [11:54<07:54, 15.82s/it][A

 55%|█████▍    | 35/64 [12:21<09:18, 19.26s/it][A

 56%|█████▋    | 36/64 [12:36<08:27, 18.11s/it][A

 58%|█████▊    | 37/64 [13:16<10:59, 24.41s/it][A

 59%|█████▉    | 38/64 [13:39<10:30, 24.25s/it][A

 61%|██████    | 39/64 [13:44<07:41, 18.45s/it][A

 62%|██████▎   | 40/64 [14:44<12:20, 30.84s/it][A

 64%|██████▍   | 41/64 [14:52<09:07, 23.81s/it][A

 66%|██████▌   | 42/64 [14:58<06:50, 18.67s/it][A

 67%|██████▋   | 43/64 [15:01<04:52, 13.91s/it][A

 69%|██████▉   | 44/64 [15:33<06:25, 19.29s/it][A

 70%|███████   | 45/64 [16:00<06:53, 21.75s/it][A

 72%|███████▏  | 46/64 [16:36<07:44, 25.78s/it][A

 73%|███████▎  | 47/64 [16:57<06:56, 24.53s/it][A

 75%|███████▌  | 48/64 [17:19<06:17, 23.62s/it][A

 77%|███████▋  | 49/64 [17:42<05:53, 23.56s/it][A

 78%|███████▊  | 50/64 [18:09<05:45, 24.70s/it][A

 80%|███████▉  | 51/64 [18:16<04:09, 19.18s/it][A

 81%|████████▏ | 52/64 [18:50<04:43, 23.65s/it][A

 83%|████████▎ | 53/64 [19:07<03:57, 21.63s/it][A

 84%|████████▍ | 54/64 [19:23<03:20, 20.07s/it][A

 86%|████████▌ | 55/64 [19:53<03:28, 23.12s/it][A

 88%|████████▊ | 56/64 [20:18<03:09, 23.63s/it][A

 89%|████████▉ | 57/64 [20:54<03:10, 27.25s/it][A

 91%|█████████ | 58/64 [20:58<02:01, 20.32s/it][A

 92%|█████████▏| 59/64 [21:16<01:38, 19.63s/it][A

 94%|█████████▍| 60/64 [21:24<01:04, 16.11s/it][A

 95%|█████████▌| 61/64 [21:50<00:56, 18.96s/it][A

 97%|█████████▋| 62/64 [22:18<00:43, 21.84s/it][A

 98%|█████████▊| 63/64 [22:42<00:22, 22.50s/it][A

100%|██████████| 64/64 [23:02<00:00, 21.78s/it][A
100%|██████████| 64/64 [23:02<00:00, 21.61s/it]
INFO:root:Time for 64 completions: 1382.8 seconds

Chunking for generation:  31%|███       | 4/13 [1:20:18<3:07:46, 1251.80s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
INFO:root:Found 256. Only generating 64 .
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.39s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.09s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.29s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/64 [00:00<?, ?it/s][A

  2%|▏         | 1/64 [00:26<27:32, 26.22s/it][A

  3%|▎         | 2/64 [00:46<23:30, 22.74s/it][A

  5%|▍         | 3/64 [00:58<18:03, 17.77s/it][A

  6%|▋         | 4/64 [01:15<17:25, 17.43s/it][A

  8%|▊         | 5/64 [01:50<23:28, 23.87s/it][A

  9%|▉         | 6/64 [01:57<17:27, 18.06s/it][A

 11%|█         | 7/64 [01:58<11:59, 12.62s/it][A

 12%|█▎        | 8/64 [02:27<16:42, 17.89s/it][A

 14%|█▍        | 9/64 [02:41<15:01, 16.38s/it][A

 16%|█▌        | 10/64 [03:14<19:21, 21.52s/it][A

 17%|█▋        | 11/64 [03:29<17:14, 19.52s/it][A

 19%|█▉        | 12/64 [03:48<16:47, 19.37s/it][A

 20%|██        | 13/64 [03:51<12:25, 14.62s/it][A

 22%|██▏       | 14/64 [04:26<17:14, 20.69s/it][A

 23%|██▎       | 15/64 [05:28<27:07, 33.22s/it][A

 25%|██▌       | 16/64 [06:41<36:01, 45.03s/it][A

 27%|██▋       | 17/64 [06:53<27:37, 35.28s/it][A

 28%|██▊       | 18/64 [07:15<23:58, 31.28s/it][A

 30%|██▉       | 19/64 [07:40<21:53, 29.18s/it][A

 31%|███▏      | 20/64 [07:54<18:11, 24.81s/it][A

 33%|███▎      | 21/64 [08:28<19:41, 27.47s/it][A

 34%|███▍      | 22/64 [08:44<16:54, 24.14s/it][A

 36%|███▌      | 23/64 [09:19<18:36, 27.22s/it][A

 38%|███▊      | 24/64 [09:51<19:06, 28.67s/it][A

 39%|███▉      | 25/64 [10:13<17:27, 26.85s/it][A

 41%|████      | 26/64 [10:43<17:33, 27.72s/it][A

 42%|████▏     | 27/64 [11:09<16:42, 27.09s/it][A

 44%|████▍     | 28/64 [11:43<17:29, 29.15s/it][A

 45%|████▌     | 29/64 [12:57<24:57, 42.79s/it][A

 47%|████▋     | 30/64 [13:02<17:44, 31.31s/it][A

 48%|████▊     | 31/64 [13:08<13:00, 23.65s/it][A

 50%|█████     | 32/64 [13:34<13:00, 24.38s/it][A

 52%|█████▏    | 33/64 [13:56<12:17, 23.78s/it][A

 53%|█████▎    | 34/64 [14:28<13:10, 26.36s/it][A

 55%|█████▍    | 35/64 [14:46<11:31, 23.86s/it][A

 56%|█████▋    | 36/64 [14:57<09:13, 19.79s/it][A

 58%|█████▊    | 37/64 [15:03<07:03, 15.70s/it][A

 59%|█████▉    | 38/64 [15:10<05:40, 13.10s/it][A

 61%|██████    | 39/64 [15:32<06:37, 15.90s/it][A

 62%|██████▎   | 40/64 [15:59<07:37, 19.04s/it][A

 64%|██████▍   | 41/64 [16:03<05:38, 14.70s/it][A

 66%|██████▌   | 42/64 [16:23<05:58, 16.31s/it][A

 67%|██████▋   | 43/64 [17:03<08:07, 23.23s/it][A

 69%|██████▉   | 44/64 [17:23<07:28, 22.44s/it][A

 70%|███████   | 45/64 [17:43<06:52, 21.69s/it][A

 72%|███████▏  | 46/64 [18:07<06:41, 22.31s/it][A

 73%|███████▎  | 47/64 [18:21<05:37, 19.84s/it][A

 75%|███████▌  | 48/64 [18:53<06:16, 23.53s/it][A

 77%|███████▋  | 49/64 [19:14<05:41, 22.74s/it][A

 78%|███████▊  | 50/64 [19:36<05:15, 22.54s/it][A

 80%|███████▉  | 51/64 [19:55<04:40, 21.57s/it][A

 81%|████████▏ | 52/64 [20:17<04:19, 21.61s/it][A

 83%|████████▎ | 53/64 [20:37<03:51, 21.06s/it][A

 84%|████████▍ | 54/64 [20:51<03:10, 19.06s/it][A

 86%|████████▌ | 55/64 [21:30<03:44, 24.99s/it][A

 88%|████████▊ | 56/64 [21:32<02:24, 18.08s/it][A

 89%|████████▉ | 57/64 [21:58<02:23, 20.54s/it][A

 91%|█████████ | 58/64 [22:16<01:58, 19.73s/it][A

 92%|█████████▏| 59/64 [22:23<01:19, 15.90s/it][A

 94%|█████████▍| 60/64 [22:44<01:08, 17.23s/it][A

 95%|█████████▌| 61/64 [23:09<00:59, 19.84s/it][A

 97%|█████████▋| 62/64 [23:15<00:30, 15.45s/it][A

 98%|█████████▊| 63/64 [23:21<00:12, 12.72s/it][A

100%|██████████| 64/64 [24:03<00:00, 21.53s/it][A
100%|██████████| 64/64 [24:03<00:00, 22.56s/it]
INFO:root:Time for 64 completions: 1443.6 seconds

Chunking for generation:  38%|███▊      | 5/13 [1:44:29<2:56:28, 1323.51s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
INFO:root:Found 320. Only generating 64 .
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.46s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.15s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.35s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/64 [00:00<?, ?it/s][A

  2%|▏         | 1/64 [00:37<39:01, 37.16s/it][A

  3%|▎         | 2/64 [00:42<19:04, 18.46s/it][A

  5%|▍         | 3/64 [01:03<19:53, 19.57s/it][A

  6%|▋         | 4/64 [01:13<15:48, 15.80s/it][A

  8%|▊         | 5/64 [02:08<29:36, 30.12s/it][A

  9%|▉         | 6/64 [02:22<23:49, 24.64s/it][A

 11%|█         | 7/64 [02:37<20:16, 21.34s/it][A

 12%|█▎        | 8/64 [02:58<19:49, 21.24s/it][A

 14%|█▍        | 9/64 [03:26<21:18, 23.24s/it][A

 16%|█▌        | 10/64 [03:44<19:39, 21.84s/it][A

 17%|█▋        | 11/64 [04:08<19:54, 22.53s/it][A

 19%|█▉        | 12/64 [04:26<18:15, 21.06s/it][A

 20%|██        | 13/64 [04:29<13:12, 15.54s/it][A

 22%|██▏       | 14/64 [04:30<09:24, 11.28s/it][A

 23%|██▎       | 15/64 [05:10<16:07, 19.75s/it][A

 25%|██▌       | 16/64 [05:20<13:27, 16.82s/it][A

 27%|██▋       | 17/64 [05:39<13:47, 17.60s/it][A

 28%|██▊       | 18/64 [05:50<11:59, 15.64s/it][A

 30%|██▉       | 19/64 [06:12<13:06, 17.48s/it][A

 31%|███▏      | 20/64 [06:23<11:27, 15.62s/it][A

 33%|███▎      | 21/64 [06:26<08:19, 11.61s/it][A

 34%|███▍      | 22/64 [06:42<09:06, 13.01s/it][A

 36%|███▌      | 23/64 [07:20<14:01, 20.53s/it][A

 38%|███▊      | 24/64 [07:39<13:24, 20.11s/it][A

 39%|███▉      | 25/64 [07:57<12:36, 19.39s/it][A

 41%|████      | 26/64 [08:05<10:04, 15.92s/it][A

 42%|████▏     | 27/64 [08:21<09:48, 15.91s/it][A

 44%|████▍     | 28/64 [08:41<10:16, 17.13s/it][A

 45%|████▌     | 29/64 [08:52<08:57, 15.36s/it][A

 47%|████▋     | 30/64 [08:57<07:03, 12.44s/it][A

 48%|████▊     | 31/64 [09:02<05:28,  9.97s/it][A

 50%|█████     | 32/64 [09:23<07:04, 13.26s/it][A

 52%|█████▏    | 33/64 [09:31<06:02, 11.70s/it][A

 53%|█████▎    | 34/64 [10:01<08:39, 17.30s/it][A

 55%|█████▍    | 35/64 [10:48<12:42, 26.29s/it][A

 56%|█████▋    | 36/64 [11:05<10:56, 23.44s/it][A

 58%|█████▊    | 37/64 [11:14<08:39, 19.22s/it][A

 59%|█████▉    | 38/64 [11:39<09:02, 20.86s/it][A

 61%|██████    | 39/64 [12:01<08:51, 21.27s/it][A

 62%|██████▎   | 40/64 [12:11<07:04, 17.70s/it][A

 64%|██████▍   | 41/64 [12:27<06:35, 17.19s/it][A

 66%|██████▌   | 42/64 [13:45<13:03, 35.62s/it][A

 67%|██████▋   | 43/64 [13:50<09:13, 26.37s/it][A

 69%|██████▉   | 44/64 [14:13<08:28, 25.42s/it][A

 70%|███████   | 45/64 [14:29<07:09, 22.58s/it][A

 72%|███████▏  | 46/64 [14:46<06:16, 20.92s/it][A

 73%|███████▎  | 47/64 [14:57<05:03, 17.83s/it][A

 75%|███████▌  | 48/64 [15:12<04:33, 17.11s/it][A

 77%|███████▋  | 49/64 [15:21<03:40, 14.70s/it][A

 78%|███████▊  | 50/64 [16:15<06:08, 26.29s/it][A

 80%|███████▉  | 51/64 [16:20<04:18, 19.86s/it][A

 81%|████████▏ | 52/64 [16:21<02:53, 14.44s/it][A

 83%|████████▎ | 53/64 [16:35<02:37, 14.32s/it][A

 84%|████████▍ | 54/64 [16:48<02:17, 13.77s/it][A

 86%|████████▌ | 55/64 [17:10<02:26, 16.26s/it][A

 88%|████████▊ | 56/64 [17:14<01:40, 12.51s/it][A

 89%|████████▉ | 57/64 [17:37<01:50, 15.84s/it][A

 91%|█████████ | 58/64 [17:48<01:25, 14.24s/it][A

 92%|█████████▏| 59/64 [17:52<00:56, 11.33s/it][A

 94%|█████████▍| 60/64 [18:13<00:56, 14.11s/it][A

 95%|█████████▌| 61/64 [18:27<00:42, 14.03s/it][A

 97%|█████████▋| 62/64 [18:59<00:39, 19.51s/it][A

 98%|█████████▊| 63/64 [19:12<00:17, 17.63s/it][A

100%|██████████| 64/64 [19:22<00:00, 15.17s/it][A
100%|██████████| 64/64 [19:22<00:00, 18.16s/it]
INFO:root:Time for 64 completions: 1162.3 seconds

Chunking for generation:  46%|████▌     | 6/13 [2:03:58<2:28:17, 1271.14s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
INFO:root:Found 384. Only generating 64 .
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.36s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.09s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.29s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/64 [00:00<?, ?it/s][A

  2%|▏         | 1/64 [00:04<04:38,  4.43s/it][A

  3%|▎         | 2/64 [00:24<13:50, 13.40s/it][A

  5%|▍         | 3/64 [00:47<18:03, 17.77s/it][A

  6%|▋         | 4/64 [00:52<12:45, 12.76s/it][A

  8%|▊         | 5/64 [01:08<13:38, 13.88s/it][A

  9%|▉         | 6/64 [01:09<09:14,  9.57s/it][A

 11%|█         | 7/64 [01:36<14:26, 15.20s/it][A

 12%|█▎        | 8/64 [01:45<12:29, 13.38s/it][A

 14%|█▍        | 9/64 [02:03<13:33, 14.78s/it][A

 16%|█▌        | 10/64 [02:16<12:57, 14.39s/it][A

 17%|█▋        | 11/64 [02:53<18:36, 21.06s/it][A

 19%|█▉        | 12/64 [03:14<18:21, 21.19s/it][A

 20%|██        | 13/64 [03:39<18:54, 22.25s/it][A

 22%|██▏       | 14/64 [04:00<18:21, 22.02s/it][A

 23%|██▎       | 15/64 [04:15<16:04, 19.69s/it][A

 25%|██▌       | 16/64 [05:00<21:51, 27.32s/it][A

 27%|██▋       | 17/64 [05:14<18:25, 23.52s/it][A

 28%|██▊       | 18/64 [05:27<15:37, 20.39s/it][A

 30%|██▉       | 19/64 [05:35<12:21, 16.48s/it][A

 31%|███▏      | 20/64 [05:47<11:04, 15.09s/it][A

 33%|███▎      | 21/64 [05:50<08:21, 11.67s/it][A

 34%|███▍      | 22/64 [06:04<08:39, 12.38s/it][A

 36%|███▌      | 23/64 [06:15<08:12, 12.01s/it][A

 38%|███▊      | 24/64 [06:29<08:24, 12.61s/it][A

 39%|███▉      | 25/64 [06:31<06:07,  9.42s/it][A

 41%|████      | 26/64 [07:09<11:15, 17.77s/it][A

 42%|████▏     | 27/64 [08:08<18:35, 30.15s/it][A

 44%|████▍     | 28/64 [08:25<15:50, 26.40s/it][A

 45%|████▌     | 29/64 [08:43<13:50, 23.74s/it][A

 47%|████▋     | 30/64 [09:16<15:04, 26.59s/it][A

 48%|████▊     | 31/64 [09:52<16:13, 29.49s/it][A

 50%|█████     | 32/64 [10:14<14:25, 27.05s/it][A

 52%|█████▏    | 33/64 [10:31<12:29, 24.18s/it][A

 53%|█████▎    | 34/64 [10:44<10:26, 20.87s/it][A

 55%|█████▍    | 35/64 [10:55<08:36, 17.82s/it][A

 56%|█████▋    | 36/64 [10:58<06:17, 13.48s/it][A

 58%|█████▊    | 37/64 [11:02<04:43, 10.51s/it][A

 59%|█████▉    | 38/64 [11:07<03:53,  8.99s/it][A

 61%|██████    | 39/64 [11:33<05:51, 14.06s/it][A

 62%|██████▎   | 40/64 [11:37<04:25, 11.05s/it][A

 64%|██████▍   | 41/64 [11:40<03:13,  8.39s/it][A

 66%|██████▌   | 42/64 [12:05<04:59, 13.61s/it][A

 67%|██████▋   | 43/64 [12:47<07:40, 21.91s/it][A

 69%|██████▉   | 44/64 [13:02<06:37, 19.89s/it][A

 70%|███████   | 45/64 [13:22<06:17, 19.88s/it][A

 72%|███████▏  | 46/64 [13:32<05:06, 17.04s/it][A

 73%|███████▎  | 47/64 [13:37<03:47, 13.36s/it][A

 75%|███████▌  | 48/64 [14:05<04:45, 17.87s/it][A

 77%|███████▋  | 49/64 [14:09<03:23, 13.59s/it][A

 78%|███████▊  | 50/64 [14:44<04:40, 20.07s/it][A

 80%|███████▉  | 51/64 [15:16<05:07, 23.63s/it][A

 81%|████████▏ | 52/64 [15:38<04:37, 23.09s/it][A

 83%|████████▎ | 53/64 [15:40<03:04, 16.78s/it][A

 84%|████████▍ | 54/64 [15:48<02:22, 14.22s/it][A

 86%|████████▌ | 55/64 [15:52<01:39, 11.07s/it][A

 88%|████████▊ | 56/64 [16:10<01:44, 13.09s/it][A

 89%|████████▉ | 57/64 [16:17<01:19, 11.37s/it][A

 91%|█████████ | 58/64 [16:30<01:11, 11.87s/it][A

 92%|█████████▏| 59/64 [16:51<01:12, 14.60s/it][A

 94%|█████████▍| 60/64 [16:59<00:50, 12.61s/it][A

 95%|█████████▌| 61/64 [17:06<00:33, 11.06s/it][A

 97%|█████████▋| 62/64 [17:34<00:31, 15.92s/it][A

 98%|█████████▊| 63/64 [17:40<00:13, 13.15s/it][A

100%|██████████| 64/64 [17:44<00:00, 10.37s/it][A
100%|██████████| 64/64 [17:44<00:00, 16.64s/it]
INFO:root:Time for 64 completions: 1064.8 seconds

Chunking for generation:  54%|█████▍    | 7/13 [2:21:50<2:00:35, 1205.98s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
INFO:root:Found 448. Only generating 64 .
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.44s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.14s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.34s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/64 [00:00<?, ?it/s][A

  2%|▏         | 1/64 [00:02<02:38,  2.51s/it][A

  3%|▎         | 2/64 [00:35<20:59, 20.31s/it][A

  5%|▍         | 3/64 [01:00<22:55, 22.55s/it][A

  6%|▋         | 4/64 [01:54<34:48, 34.82s/it][A

  8%|▊         | 5/64 [02:19<30:50, 31.37s/it][A

  9%|▉         | 6/64 [02:36<25:42, 26.59s/it][A

 11%|█         | 7/64 [02:52<21:47, 22.93s/it][A

 12%|█▎        | 8/64 [02:56<15:45, 16.88s/it][A

 14%|█▍        | 9/64 [03:16<16:37, 18.13s/it][A

 16%|█▌        | 10/64 [03:18<11:35, 12.88s/it][A

 17%|█▋        | 11/64 [03:23<09:18, 10.54s/it][A

 19%|█▉        | 12/64 [03:37<10:12, 11.77s/it][A

 20%|██        | 13/64 [03:48<09:46, 11.49s/it][A

 22%|██▏       | 14/64 [03:49<06:59,  8.38s/it][A

 23%|██▎       | 15/64 [03:50<04:55,  6.02s/it][A

 25%|██▌       | 16/64 [04:04<06:38,  8.30s/it][A

 27%|██▋       | 17/64 [04:18<07:59, 10.21s/it][A

 28%|██▊       | 18/64 [04:26<07:19,  9.55s/it][A

 30%|██▉       | 19/64 [04:36<07:08,  9.53s/it][A

 31%|███▏      | 20/64 [04:54<08:53, 12.13s/it][A

 33%|███▎      | 21/64 [05:10<09:35, 13.39s/it][A

 34%|███▍      | 22/64 [05:19<08:18, 11.87s/it][A

 36%|███▌      | 23/64 [06:35<21:19, 31.20s/it][A

 38%|███▊      | 24/64 [06:39<15:25, 23.15s/it][A

 39%|███▉      | 25/64 [06:51<12:48, 19.70s/it][A

 41%|████      | 26/64 [06:52<08:55, 14.09s/it][A

 42%|████▏     | 27/64 [07:04<08:22, 13.57s/it][A

 44%|████▍     | 28/64 [07:05<05:54,  9.86s/it][A

 45%|████▌     | 29/64 [07:21<06:47, 11.65s/it][A

 47%|████▋     | 30/64 [07:28<05:51, 10.34s/it][A

 48%|████▊     | 31/64 [08:03<09:44, 17.71s/it][A

 50%|█████     | 32/64 [08:07<07:15, 13.60s/it][A

 52%|█████▏    | 33/64 [08:17<06:29, 12.55s/it][A

 53%|█████▎    | 34/64 [08:23<05:13, 10.44s/it][A

 55%|█████▍    | 35/64 [09:04<09:31, 19.71s/it][A

 56%|█████▋    | 36/64 [09:32<10:14, 21.96s/it][A

 58%|█████▊    | 37/64 [09:41<08:10, 18.15s/it][A

 59%|█████▉    | 38/64 [09:49<06:36, 15.24s/it][A

 61%|██████    | 39/64 [10:05<06:23, 15.36s/it][A

 62%|██████▎   | 40/64 [10:43<08:49, 22.06s/it][A

 64%|██████▍   | 41/64 [10:48<06:35, 17.21s/it][A

 66%|██████▌   | 42/64 [10:55<05:11, 14.14s/it][A

 67%|██████▋   | 43/64 [11:18<05:52, 16.79s/it][A

 69%|██████▉   | 44/64 [11:27<04:45, 14.27s/it][A

 70%|███████   | 45/64 [11:30<03:25, 10.79s/it][A

 72%|███████▏  | 46/64 [11:42<03:22, 11.27s/it][A

 73%|███████▎  | 47/64 [11:46<02:32,  9.00s/it][A

 75%|███████▌  | 48/64 [12:04<03:10, 11.90s/it][A

 77%|███████▋  | 49/64 [12:32<04:10, 16.69s/it][A

 78%|███████▊  | 50/64 [12:55<04:19, 18.57s/it][A

 80%|███████▉  | 51/64 [13:08<03:37, 16.73s/it][A

 81%|████████▏ | 52/64 [13:26<03:26, 17.23s/it][A

 83%|████████▎ | 53/64 [13:38<02:52, 15.64s/it][A

 84%|████████▍ | 54/64 [13:42<02:00, 12.09s/it][A

 86%|████████▌ | 55/64 [14:00<02:05, 13.90s/it][A

 88%|████████▊ | 56/64 [14:18<02:01, 15.25s/it][A

 89%|████████▉ | 57/64 [14:42<02:05, 17.89s/it][A

 91%|█████████ | 58/64 [14:51<01:30, 15.09s/it][A

 92%|█████████▏| 59/64 [15:00<01:06, 13.32s/it][A

 94%|█████████▍| 60/64 [15:24<01:06, 16.67s/it][A

 95%|█████████▌| 61/64 [15:41<00:49, 16.57s/it][A

 97%|█████████▋| 62/64 [15:43<00:24, 12.41s/it][A

 98%|█████████▊| 63/64 [15:52<00:11, 11.22s/it][A

100%|██████████| 64/64 [16:16<00:00, 15.05s/it][A
100%|██████████| 64/64 [16:16<00:00, 15.26s/it]
INFO:root:Time for 64 completions: 976.4 seconds

Chunking for generation:  62%|██████▏   | 8/13 [2:38:14<1:34:35, 1135.18s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
INFO:root:Found 512. Only generating 64 .
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.40s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.10s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.29s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/64 [00:00<?, ?it/s][A

  2%|▏         | 1/64 [00:40<42:08, 40.14s/it][A

  3%|▎         | 2/64 [00:43<19:12, 18.59s/it][A

  5%|▍         | 3/64 [00:57<16:40, 16.40s/it][A

  6%|▋         | 4/64 [01:26<21:21, 21.36s/it][A

  8%|▊         | 5/64 [01:41<18:41, 19.01s/it][A

  9%|▉         | 6/64 [02:03<19:17, 19.95s/it][A

 11%|█         | 7/64 [02:14<16:21, 17.23s/it][A

 12%|█▎        | 8/64 [02:36<17:24, 18.65s/it][A

 14%|█▍        | 9/64 [02:59<18:20, 20.01s/it][A

 16%|█▌        | 10/64 [03:20<18:24, 20.46s/it][A

 17%|█▋        | 11/64 [03:56<22:11, 25.13s/it][A

 19%|█▉        | 12/64 [04:13<19:41, 22.72s/it][A

 20%|██        | 13/64 [04:45<21:43, 25.56s/it][A

 22%|██▏       | 14/64 [04:57<17:43, 21.27s/it][A

 23%|██▎       | 15/64 [06:16<31:38, 38.75s/it][A

 25%|██▌       | 16/64 [06:20<22:32, 28.17s/it][A

 27%|██▋       | 17/64 [06:25<16:42, 21.34s/it][A

 28%|██▊       | 18/64 [06:47<16:30, 21.52s/it][A

 30%|██▉       | 19/64 [06:58<13:45, 18.35s/it][A

 31%|███▏      | 20/64 [07:07<11:21, 15.50s/it][A

 33%|███▎      | 21/64 [07:29<12:36, 17.60s/it][A

 34%|███▍      | 22/64 [07:59<14:49, 21.19s/it][A

 36%|███▌      | 23/64 [08:09<12:14, 17.91s/it][A

 38%|███▊      | 24/64 [08:42<14:52, 22.32s/it][A

 39%|███▉      | 25/64 [09:30<19:37, 30.20s/it][A

 41%|████      | 26/64 [09:33<13:49, 21.83s/it][A

 42%|████▏     | 27/64 [10:26<19:17, 31.28s/it][A

 44%|████▍     | 28/64 [11:07<20:37, 34.37s/it][A

 45%|████▌     | 29/64 [11:13<14:58, 25.66s/it][A

 47%|████▋     | 30/64 [11:27<12:32, 22.12s/it][A

 48%|████▊     | 31/64 [11:29<08:56, 16.26s/it][A

 50%|█████     | 32/64 [12:11<12:41, 23.81s/it][A

 52%|█████▏    | 33/64 [12:29<11:30, 22.27s/it][A

 53%|█████▎    | 34/64 [12:33<08:17, 16.57s/it][A

 55%|█████▍    | 35/64 [12:43<07:06, 14.70s/it][A

 56%|█████▋    | 36/64 [12:57<06:41, 14.35s/it][A

 58%|█████▊    | 37/64 [13:17<07:17, 16.19s/it][A

 59%|█████▉    | 38/64 [13:20<05:21, 12.35s/it][A

 61%|██████    | 39/64 [13:57<08:09, 19.59s/it][A

 62%|██████▎   | 40/64 [14:25<08:49, 22.05s/it][A

 64%|██████▍   | 41/64 [14:30<06:34, 17.14s/it][A

 66%|██████▌   | 42/64 [14:40<05:30, 15.02s/it][A

 67%|██████▋   | 43/64 [14:49<04:32, 12.96s/it][A

 69%|██████▉   | 44/64 [14:52<03:23, 10.18s/it][A

 70%|███████   | 45/64 [15:15<04:26, 14.02s/it][A

 72%|███████▏  | 46/64 [15:43<05:26, 18.14s/it][A

 73%|███████▎  | 47/64 [15:57<04:48, 16.96s/it][A

 75%|███████▌  | 48/64 [16:12<04:20, 16.25s/it][A

 77%|███████▋  | 49/64 [16:33<04:26, 17.80s/it][A

 78%|███████▊  | 50/64 [17:16<05:54, 25.32s/it][A

 80%|███████▉  | 51/64 [17:44<05:40, 26.18s/it][A

 81%|████████▏ | 52/64 [18:00<04:36, 23.06s/it][A

 83%|████████▎ | 53/64 [18:17<03:54, 21.29s/it][A

 84%|████████▍ | 54/64 [18:32<03:14, 19.42s/it][A

 86%|████████▌ | 55/64 [18:52<02:54, 19.41s/it][A

 88%|████████▊ | 56/64 [19:14<02:43, 20.40s/it][A

 89%|████████▉ | 57/64 [19:29<02:10, 18.57s/it][A

 91%|█████████ | 58/64 [19:41<01:40, 16.73s/it][A

 92%|█████████▏| 59/64 [20:14<01:48, 21.71s/it][A

 94%|█████████▍| 60/64 [20:20<01:07, 16.96s/it][A

 95%|█████████▌| 61/64 [20:45<00:58, 19.41s/it][A

 97%|█████████▋| 62/64 [21:03<00:37, 18.92s/it][A

 98%|█████████▊| 63/64 [21:20<00:18, 18.29s/it][A

100%|██████████| 64/64 [21:26<00:00, 14.57s/it][A
100%|██████████| 64/64 [21:26<00:00, 20.10s/it]
INFO:root:Time for 64 completions: 1286.4 seconds

Chunking for generation:  69%|██████▉   | 9/13 [2:59:47<1:18:58, 1184.67s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
INFO:root:Found 576. Only generating 64 .
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.49s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.18s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.38s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/64 [00:00<?, ?it/s][A

  2%|▏         | 1/64 [00:27<28:40, 27.32s/it][A

  3%|▎         | 2/64 [01:44<58:28, 56.59s/it][A

  5%|▍         | 3/64 [02:10<43:31, 42.82s/it][A

  6%|▋         | 4/64 [02:33<34:43, 34.72s/it][A

  8%|▊         | 5/64 [02:56<30:02, 30.55s/it][A

  9%|▉         | 6/64 [03:24<28:39, 29.64s/it][A

 11%|█         | 7/64 [03:44<25:18, 26.63s/it][A

 12%|█▎        | 8/64 [04:13<25:38, 27.47s/it][A

 14%|█▍        | 9/64 [04:48<27:11, 29.66s/it][A

 16%|█▌        | 10/64 [05:06<23:32, 26.16s/it][A

 17%|█▋        | 11/64 [05:29<22:19, 25.27s/it][A

 19%|█▉        | 12/64 [05:37<17:20, 20.00s/it][A

 20%|██        | 13/64 [05:44<13:34, 15.97s/it][A

 22%|██▏       | 14/64 [06:03<14:02, 16.85s/it][A

 23%|██▎       | 15/64 [06:22<14:22, 17.60s/it][A

 25%|██▌       | 16/64 [06:47<15:40, 19.59s/it][A

 27%|██▋       | 17/64 [07:05<15:07, 19.31s/it][A

 28%|██▊       | 18/64 [07:23<14:31, 18.94s/it][A

 30%|██▉       | 19/64 [07:34<12:22, 16.51s/it][A

 31%|███▏      | 20/64 [08:09<16:06, 21.96s/it][A

 33%|███▎      | 21/64 [08:20<13:31, 18.88s/it][A

 34%|███▍      | 22/64 [08:23<09:51, 14.08s/it][A

 36%|███▌      | 23/64 [08:25<07:06, 10.41s/it][A

 38%|███▊      | 24/64 [08:37<07:08, 10.71s/it][A

 39%|███▉      | 25/64 [08:39<05:16,  8.12s/it][A

 41%|████      | 26/64 [09:02<08:02, 12.69s/it][A

 42%|████▏     | 27/64 [09:09<06:40, 10.83s/it][A

 44%|████▍     | 28/64 [09:21<06:44, 11.23s/it][A

 45%|████▌     | 29/64 [09:34<06:55, 11.88s/it][A

 47%|████▋     | 30/64 [09:48<07:06, 12.56s/it][A

 48%|████▊     | 31/64 [10:06<07:48, 14.20s/it][A

 50%|█████     | 32/64 [10:49<12:07, 22.73s/it][A

 52%|█████▏    | 33/64 [10:52<08:40, 16.81s/it][A

 53%|█████▎    | 34/64 [11:19<09:57, 19.93s/it][A

 55%|█████▍    | 35/64 [11:26<07:41, 15.92s/it][A

 56%|█████▋    | 36/64 [11:50<08:36, 18.43s/it][A

 58%|█████▊    | 37/64 [11:55<06:32, 14.54s/it][A

 59%|█████▉    | 38/64 [12:09<06:09, 14.21s/it][A

 61%|██████    | 39/64 [12:34<07:17, 17.49s/it][A

 62%|██████▎   | 40/64 [12:47<06:30, 16.26s/it][A

 64%|██████▍   | 41/64 [12:52<04:56, 12.91s/it][A

 66%|██████▌   | 42/64 [13:00<04:05, 11.18s/it][A

 67%|██████▋   | 43/64 [13:23<05:11, 14.83s/it][A

 69%|██████▉   | 44/64 [13:40<05:08, 15.41s/it][A

 70%|███████   | 45/64 [13:43<03:42, 11.70s/it][A

 72%|███████▏  | 46/64 [13:56<03:38, 12.15s/it][A

 73%|███████▎  | 47/64 [14:01<02:48,  9.90s/it][A

 75%|███████▌  | 48/64 [14:13<02:51, 10.73s/it][A

 77%|███████▋  | 49/64 [15:30<07:37, 30.52s/it][A

 78%|███████▊  | 50/64 [15:44<05:56, 25.48s/it][A

 80%|███████▉  | 51/64 [15:45<03:56, 18.19s/it][A

 81%|████████▏ | 52/64 [15:47<02:41, 13.47s/it][A

 83%|████████▎ | 53/64 [15:52<01:59, 10.83s/it][A

 84%|████████▍ | 54/64 [15:54<01:21,  8.10s/it][A

 86%|████████▌ | 55/64 [15:55<00:55,  6.12s/it][A

 88%|████████▊ | 56/64 [15:59<00:44,  5.53s/it][A

 89%|████████▉ | 57/64 [16:01<00:29,  4.25s/it][A

 91%|█████████ | 58/64 [16:05<00:25,  4.23s/it][A

 92%|█████████▏| 59/64 [16:06<00:16,  3.39s/it][A

 94%|█████████▍| 60/64 [16:08<00:11,  2.85s/it][A

 95%|█████████▌| 61/64 [16:13<00:10,  3.42s/it][A

 97%|█████████▋| 62/64 [16:15<00:05,  2.98s/it][A

 98%|█████████▊| 63/64 [16:15<00:02,  2.13s/it][A

100%|██████████| 64/64 [16:22<00:00,  3.58s/it][A
100%|██████████| 64/64 [16:22<00:00, 15.35s/it]
INFO:root:Time for 64 completions: 982.2 seconds

Chunking for generation:  77%|███████▋  | 10/13 [3:16:17<56:13, 1124.40s/it] INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
INFO:root:Found 640. Only generating 64 .
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.44s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.13s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.33s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/64 [00:00<?, ?it/s][A

  2%|▏         | 1/64 [00:01<02:00,  1.91s/it][A

  3%|▎         | 2/64 [00:13<07:54,  7.66s/it][A

  5%|▍         | 3/64 [00:31<12:38, 12.43s/it][A

  6%|▋         | 4/64 [00:38<10:02, 10.04s/it][A

  8%|▊         | 5/64 [00:56<12:54, 13.13s/it][A

  9%|▉         | 6/64 [02:14<33:51, 35.03s/it][A

 11%|█         | 7/64 [02:29<27:00, 28.43s/it][A

 12%|█▎        | 8/64 [02:39<21:19, 22.85s/it][A

 14%|█▍        | 9/64 [02:50<17:21, 18.93s/it][A

 16%|█▌        | 10/64 [02:54<13:02, 14.48s/it][A

 17%|█▋        | 11/64 [03:22<16:21, 18.51s/it][A

 19%|█▉        | 12/64 [03:28<12:48, 14.78s/it][A

 20%|██        | 13/64 [03:42<12:22, 14.57s/it][A

 22%|██▏       | 14/64 [04:07<14:42, 17.66s/it][A

 23%|██▎       | 15/64 [04:13<11:29, 14.06s/it][A

 25%|██▌       | 16/64 [04:36<13:25, 16.78s/it][A

 27%|██▋       | 17/64 [04:38<09:37, 12.29s/it][A

 28%|██▊       | 18/64 [04:39<06:46,  8.84s/it][A

 30%|██▉       | 19/64 [04:42<05:22,  7.17s/it][A

 31%|███▏      | 20/64 [04:58<07:11,  9.81s/it][A

 33%|███▎      | 21/64 [04:59<05:08,  7.17s/it][A

 34%|███▍      | 22/64 [04:59<03:37,  5.17s/it][A

 36%|███▌      | 23/64 [05:02<02:57,  4.33s/it][A

 38%|███▊      | 24/64 [05:07<03:06,  4.67s/it][A

 39%|███▉      | 25/64 [05:19<04:31,  6.95s/it][A

 41%|████      | 26/64 [05:46<08:06, 12.80s/it][A

 42%|████▏     | 27/64 [05:46<05:35,  9.07s/it][A

 44%|████▍     | 28/64 [05:48<04:04,  6.78s/it][A

 45%|████▌     | 29/64 [05:49<02:56,  5.04s/it][A

 47%|████▋     | 30/64 [05:50<02:13,  3.92s/it][A

 48%|████▊     | 31/64 [05:51<01:40,  3.04s/it][A

 50%|█████     | 32/64 [06:12<04:31,  8.48s/it][A

 52%|█████▏    | 33/64 [06:14<03:24,  6.61s/it][A

 53%|█████▎    | 34/64 [06:15<02:26,  4.90s/it][A

 55%|█████▍    | 35/64 [06:20<02:18,  4.77s/it][A

 56%|█████▋    | 36/64 [06:24<02:09,  4.62s/it][A

 58%|█████▊    | 37/64 [06:28<02:00,  4.47s/it][A

 59%|█████▉    | 38/64 [06:29<01:30,  3.50s/it][A

 61%|██████    | 39/64 [06:34<01:32,  3.69s/it][A

 62%|██████▎   | 40/64 [06:36<01:22,  3.42s/it][A

 64%|██████▍   | 41/64 [06:39<01:13,  3.20s/it][A

 66%|██████▌   | 42/64 [06:52<02:12,  6.04s/it][A

 67%|██████▋   | 43/64 [07:20<04:26, 12.68s/it][A

 69%|██████▉   | 44/64 [07:41<05:03, 15.20s/it][A

 70%|███████   | 45/64 [08:06<05:44, 18.15s/it][A

 72%|███████▏  | 46/64 [08:15<04:35, 15.32s/it][A

 73%|███████▎  | 47/64 [08:30<04:20, 15.34s/it][A

 75%|███████▌  | 48/64 [08:56<04:56, 18.54s/it][A

 77%|███████▋  | 49/64 [09:12<04:28, 17.90s/it][A

 78%|███████▊  | 50/64 [09:30<04:10, 17.87s/it][A

 80%|███████▉  | 51/64 [09:46<03:45, 17.33s/it][A

 81%|████████▏ | 52/64 [10:03<03:26, 17.24s/it][A

 83%|████████▎ | 53/64 [10:19<03:05, 16.85s/it][A

 84%|████████▍ | 54/64 [10:20<02:01, 12.12s/it][A

 86%|████████▌ | 55/64 [10:46<02:26, 16.24s/it][A

 88%|████████▊ | 56/64 [11:18<02:47, 20.98s/it][A

 89%|████████▉ | 57/64 [11:32<02:10, 18.66s/it][A

 91%|█████████ | 58/64 [11:34<01:22, 13.81s/it][A

 92%|█████████▏| 59/64 [11:37<00:52, 10.46s/it][A

 94%|█████████▍| 60/64 [11:42<00:35,  8.92s/it][A

 95%|█████████▌| 61/64 [11:48<00:24,  8.01s/it][A

 97%|█████████▋| 62/64 [12:05<00:21, 10.69s/it][A

 98%|█████████▊| 63/64 [12:10<00:09,  9.11s/it][A

100%|██████████| 64/64 [12:28<00:00, 11.57s/it][A
100%|██████████| 64/64 [12:28<00:00, 11.69s/it]
INFO:root:Time for 64 completions: 748.1 seconds

Chunking for generation:  85%|████████▍ | 11/13 [3:28:52<33:42, 1011.41s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
INFO:root:Found 704. Only generating 64 .
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.40s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.11s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.30s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/64 [00:00<?, ?it/s][A

  2%|▏         | 1/64 [00:15<16:16, 15.50s/it][A

  3%|▎         | 2/64 [00:16<07:24,  7.18s/it][A

  5%|▍         | 3/64 [00:39<14:26, 14.21s/it][A

  6%|▋         | 4/64 [00:42<09:41,  9.69s/it][A

  8%|▊         | 5/64 [00:43<06:28,  6.58s/it][A

  9%|▉         | 6/64 [00:45<04:55,  5.09s/it][A

 11%|█         | 7/64 [01:07<10:02, 10.57s/it][A

 12%|█▎        | 8/64 [01:08<06:59,  7.49s/it][A

 14%|█▍        | 9/64 [01:16<07:12,  7.87s/it][A

 16%|█▌        | 10/64 [01:17<05:01,  5.59s/it][A

 17%|█▋        | 11/64 [01:20<04:12,  4.77s/it][A

 19%|█▉        | 12/64 [01:28<04:56,  5.70s/it][A

 20%|██        | 13/64 [01:29<03:39,  4.30s/it][A

 22%|██▏       | 14/64 [01:32<03:18,  3.98s/it][A

 23%|██▎       | 15/64 [01:35<03:00,  3.69s/it][A

 25%|██▌       | 16/64 [01:43<03:59,  4.99s/it][A

 27%|██▋       | 17/64 [01:45<03:06,  3.96s/it][A

 28%|██▊       | 18/64 [02:07<07:11,  9.38s/it][A

 30%|██▉       | 19/64 [02:16<06:59,  9.32s/it][A

 31%|███▏      | 20/64 [02:16<04:54,  6.70s/it][A

 33%|███▎      | 21/64 [02:25<05:12,  7.27s/it][A

 34%|███▍      | 22/64 [02:49<08:38, 12.35s/it][A

 36%|███▌      | 23/64 [03:04<09:02, 13.23s/it][A

 38%|███▊      | 24/64 [03:32<11:39, 17.48s/it][A

 39%|███▉      | 25/64 [03:47<10:59, 16.91s/it][A

 41%|████      | 26/64 [04:07<11:08, 17.58s/it][A

 42%|████▏     | 27/64 [04:30<12:01, 19.50s/it][A

 44%|████▍     | 28/64 [04:51<11:50, 19.74s/it][A

 45%|████▌     | 29/64 [05:18<12:49, 21.99s/it][A

 47%|████▋     | 30/64 [05:42<12:51, 22.69s/it][A

 48%|████▊     | 31/64 [06:00<11:40, 21.22s/it][A

 50%|█████     | 32/64 [06:19<10:55, 20.47s/it][A

 52%|█████▏    | 33/64 [06:50<12:17, 23.78s/it][A

 53%|█████▎    | 34/64 [07:17<12:18, 24.61s/it][A

 55%|█████▍    | 35/64 [07:41<11:52, 24.56s/it][A

 56%|█████▋    | 36/64 [08:03<10:59, 23.56s/it][A

 58%|█████▊    | 37/64 [08:33<11:32, 25.65s/it][A

 59%|█████▉    | 38/64 [08:55<10:40, 24.62s/it][A

 61%|██████    | 39/64 [09:16<09:48, 23.55s/it][A

 62%|██████▎   | 40/64 [09:40<09:27, 23.66s/it][A

 64%|██████▍   | 41/64 [10:02<08:47, 22.95s/it][A

 66%|██████▌   | 42/64 [10:08<06:35, 17.98s/it][A

 67%|██████▋   | 43/64 [10:19<05:34, 15.92s/it][A

 69%|██████▉   | 44/64 [10:28<04:38, 13.92s/it][A

 70%|███████   | 45/64 [10:38<04:02, 12.78s/it][A

 72%|███████▏  | 46/64 [11:00<04:36, 15.35s/it][A

 73%|███████▎  | 47/64 [11:15<04:20, 15.32s/it][A

 75%|███████▌  | 48/64 [11:38<04:43, 17.70s/it][A

 77%|███████▋  | 49/64 [11:53<04:10, 16.67s/it][A

 78%|███████▊  | 50/64 [12:15<04:18, 18.44s/it][A

 80%|███████▉  | 51/64 [12:38<04:16, 19.69s/it][A

 81%|████████▏ | 52/64 [12:56<03:49, 19.12s/it][A

 83%|████████▎ | 53/64 [13:16<03:34, 19.54s/it][A

 84%|████████▍ | 54/64 [13:48<03:52, 23.21s/it][A

 86%|████████▌ | 55/64 [14:06<03:15, 21.74s/it][A

 88%|████████▊ | 56/64 [14:28<02:53, 21.72s/it][A

 89%|████████▉ | 57/64 [14:49<02:30, 21.45s/it][A

 91%|█████████ | 58/64 [15:03<01:55, 19.21s/it][A

 92%|█████████▏| 59/64 [15:28<01:45, 21.11s/it][A

 94%|█████████▍| 60/64 [15:34<01:06, 16.55s/it][A

 95%|█████████▌| 61/64 [15:41<00:40, 13.65s/it][A

 97%|█████████▋| 62/64 [15:57<00:28, 14.27s/it][A

 98%|█████████▊| 63/64 [16:19<00:16, 16.62s/it][A

100%|██████████| 64/64 [16:40<00:00, 17.93s/it][A
100%|██████████| 64/64 [16:40<00:00, 15.63s/it]
INFO:root:Time for 64 completions: 1000.2 seconds

Chunking for generation:  92%|█████████▏| 12/13 [3:45:39<16:50, 1010.18s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
INFO:root:Found 768. Only generating 37 .
INFO:root:Using `huggingface_local_completions` on 37 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.


Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s][A

Loading checkpoint shards:  50%|█████     | 1/2 [00:04<00:04,  4.34s/it][A

Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.08s/it][A
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00,  3.27s/it]
INFO:root:Model memory: 15.020343296 GB
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}


  0%|          | 0/37 [00:00<?, ?it/s][A

  3%|▎         | 1/37 [00:18<10:55, 18.21s/it][A

  5%|▌         | 2/37 [00:34<09:57, 17.08s/it][A

  8%|▊         | 3/37 [00:51<09:33, 16.88s/it][A

 11%|█         | 4/37 [01:24<12:53, 23.45s/it][A

 14%|█▎        | 5/37 [01:47<12:26, 23.32s/it][A

 16%|█▌        | 6/37 [02:06<11:11, 21.65s/it][A

 19%|█▉        | 7/37 [02:30<11:20, 22.69s/it][A

 22%|██▏       | 8/37 [02:54<11:02, 22.85s/it][A

 24%|██▍       | 9/37 [03:22<11:27, 24.56s/it][A

 27%|██▋       | 10/37 [03:50<11:31, 25.59s/it][A

 30%|██▉       | 11/37 [04:10<10:22, 23.93s/it][A

 32%|███▏      | 12/37 [04:37<10:21, 24.85s/it][A

 35%|███▌      | 13/37 [04:52<08:43, 21.80s/it][A

 38%|███▊      | 14/37 [05:09<07:49, 20.41s/it][A

 41%|████      | 15/37 [05:24<06:55, 18.91s/it][A

 43%|████▎     | 16/37 [05:46<06:56, 19.82s/it][A

 46%|████▌     | 17/37 [06:17<07:40, 23.01s/it][A

 49%|████▊     | 18/37 [06:37<06:58, 22.03s/it][A

 51%|█████▏    | 19/37 [06:56<06:24, 21.35s/it][A

 54%|█████▍    | 20/37 [07:09<05:20, 18.87s/it][A

 57%|█████▋    | 21/37 [07:20<04:23, 16.44s/it][A

 59%|█████▉    | 22/37 [07:36<04:04, 16.32s/it][A

 62%|██████▏   | 23/37 [08:00<04:18, 18.49s/it][A

 65%|██████▍   | 24/37 [08:32<04:55, 22.73s/it][A

 68%|██████▊   | 25/37 [08:37<03:26, 17.22s/it][A

 70%|███████   | 26/37 [08:42<02:29, 13.58s/it][A

 73%|███████▎  | 27/37 [08:49<01:56, 11.66s/it][A

 76%|███████▌  | 28/37 [09:08<02:05, 13.93s/it][A

 78%|███████▊  | 29/37 [09:22<01:51, 13.95s/it][A

 81%|████████  | 30/37 [09:50<02:06, 18.01s/it][A

 84%|████████▍ | 31/37 [10:09<01:50, 18.48s/it][A

 86%|████████▋ | 32/37 [10:24<01:26, 17.31s/it][A

 89%|████████▉ | 33/37 [11:11<01:45, 26.38s/it][A

 92%|█████████▏| 34/37 [11:54<01:33, 31.26s/it][A

 95%|█████████▍| 35/37 [12:28<01:04, 32.20s/it][A

 97%|█████████▋| 36/37 [13:04<00:33, 33.08s/it][A

100%|██████████| 37/37 [13:38<00:00, 33.61s/it][A
100%|██████████| 37/37 [13:38<00:00, 22.13s/it]
INFO:root:Time for 37 completions: 819.0 seconds

Chunking for generation: 100%|██████████| 13/13 [3:59:25<00:00, 954.40s/it] 
Chunking for generation: 100%|██████████| 13/13 [3:59:25<00:00, 1105.07s/it]
INFO:root:Evaluating the LMCocktail-7B-v1 outputs.
INFO:root:Creating the annotator from `chatgpt_fn`.
INFO:root:Saving annotations to `/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json`.
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.

Annotation chunk:   0%|          | 0/7 [00:00<?, ?it/s]INFO:root:Annotating 128 examples with chatgpt_fn
INFO:root:Using `openai_completions` on 128 prompts using gpt-3.5-turbo-16k-0613.
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5


prompt_batches:   0%|          | 0/128 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   1%|          | 1/128 [00:01<02:19,  1.10s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   3%|▎         | 4/128 [00:01<00:32,  3.79it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   5%|▍         | 6/128 [00:02<00:44,  2.72it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   5%|▌         | 7/128 [00:02<00:38,  3.12it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   8%|▊         | 10/128 [00:02<00:21,  5.40it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   9%|▉         | 12/128 [00:05<00:59,  1.96it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  16%|█▌        | 20/128 [00:07<00:42,  2.53it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  20%|█▉        | 25/128 [00:09<00:36,  2.80it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  23%|██▎       | 30/128 [00:09<00:26,  3.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  24%|██▍       | 31/128 [00:09<00:26,  3.62it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  26%|██▌       | 33/128 [00:10<00:26,  3.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  28%|██▊       | 36/128 [00:11<00:23,  3.87it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  30%|██▉       | 38/128 [00:11<00:19,  4.60it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  30%|███       | 39/128 [00:11<00:21,  4.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  31%|███▏      | 40/128 [00:11<00:19,  4.54it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  32%|███▏      | 41/128 [00:12<00:24,  3.52it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  33%|███▎      | 42/128 [00:12<00:22,  3.91it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  34%|███▍      | 44/128 [00:12<00:16,  5.03it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  35%|███▌      | 45/128 [00:12<00:16,  4.95it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  36%|███▌      | 46/128 [00:13<00:22,  3.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  38%|███▊      | 48/128 [00:13<00:17,  4.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  38%|███▊      | 49/128 [00:13<00:15,  4.94it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  39%|███▉      | 50/128 [00:14<00:15,  4.96it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  40%|███▉      | 51/128 [00:14<00:22,  3.47it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  41%|████▏     | 53/128 [00:14<00:16,  4.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  42%|████▏     | 54/128 [00:15<00:14,  5.05it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  43%|████▎     | 55/128 [00:15<00:14,  5.14it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  44%|████▍     | 56/128 [00:15<00:19,  3.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  45%|████▌     | 58/128 [00:15<00:15,  4.51it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  46%|████▌     | 59/128 [00:16<00:18,  3.82it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  48%|████▊     | 61/128 [00:16<00:17,  3.87it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  49%|████▉     | 63/128 [00:17<00:12,  5.22it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  50%|█████     | 64/128 [00:18<00:29,  2.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  54%|█████▍    | 69/128 [00:18<00:12,  4.70it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  55%|█████▍    | 70/128 [00:19<00:13,  4.27it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  56%|█████▋    | 72/128 [00:19<00:13,  4.14it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  58%|█████▊    | 74/128 [00:19<00:10,  5.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  59%|█████▊    | 75/128 [00:20<00:12,  4.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  60%|██████    | 77/128 [00:20<00:13,  3.78it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  62%|██████▎   | 80/128 [00:21<00:09,  5.30it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  63%|██████▎   | 81/128 [00:21<00:08,  5.33it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  64%|██████▍   | 82/128 [00:21<00:12,  3.67it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  65%|██████▍   | 83/128 [00:22<00:11,  4.02it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  66%|██████▋   | 85/128 [00:22<00:12,  3.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  68%|██████▊   | 87/128 [00:23<00:09,  4.16it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  70%|██████▉   | 89/128 [00:23<00:08,  4.76it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  71%|███████   | 91/128 [00:23<00:07,  4.87it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  72%|███████▏  | 92/128 [00:24<00:09,  3.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  73%|███████▎  | 93/128 [00:26<00:21,  1.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  79%|███████▉  | 101/128 [00:26<00:05,  4.76it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  80%|████████  | 103/128 [00:27<00:05,  4.30it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  83%|████████▎ | 106/128 [00:27<00:04,  4.89it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  84%|████████▎ | 107/128 [00:28<00:04,  4.26it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  84%|████████▍ | 108/128 [00:28<00:04,  4.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  87%|████████▋ | 111/128 [00:28<00:03,  5.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  88%|████████▊ | 112/128 [00:29<00:03,  4.05it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  88%|████████▊ | 113/128 [00:29<00:03,  3.95it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  91%|█████████ | 116/128 [00:29<00:02,  5.38it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  91%|█████████▏| 117/128 [00:30<00:02,  3.99it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:openai._base_client:Retrying request to /chat/completions in 0.750557 seconds
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  92%|█████████▏| 118/128 [00:36<00:13,  1.36s/it][A
prompt_batches: 100%|██████████| 128/128 [00:36<00:00,  3.53it/s]
INFO:root:Completed 128 examples in 36.5 seconds.
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.

Annotation chunk:  14%|█▍        | 1/7 [00:36<03:41, 36.98s/it]INFO:root:Annotating 127 examples with chatgpt_fn
INFO:root:Using `openai_completions` on 127 prompts using gpt-3.5-turbo-16k-0613.
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5


prompt_batches:   0%|          | 0/127 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   1%|          | 1/127 [00:01<03:41,  1.76s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   5%|▍         | 6/127 [00:02<00:37,  3.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   6%|▌         | 7/127 [00:02<00:32,  3.64it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   7%|▋         | 9/127 [00:02<00:25,  4.60it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   9%|▊         | 11/127 [00:03<00:31,  3.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  11%|█         | 14/127 [00:03<00:24,  4.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  12%|█▏        | 15/127 [00:03<00:22,  5.01it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  13%|█▎        | 16/127 [00:04<00:29,  3.72it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  14%|█▍        | 18/127 [00:04<00:21,  4.97it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  15%|█▍        | 19/127 [00:05<00:30,  3.52it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  17%|█▋        | 21/127 [00:05<00:21,  4.85it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  17%|█▋        | 22/127 [00:05<00:19,  5.34it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  18%|█▊        | 23/127 [00:06<00:33,  3.12it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  20%|█▉        | 25/127 [00:06<00:24,  4.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  21%|██▏       | 27/127 [00:06<00:17,  5.64it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  22%|██▏       | 28/127 [00:07<00:35,  2.79it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  26%|██▌       | 33/127 [00:08<00:23,  4.08it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  27%|██▋       | 34/127 [00:08<00:21,  4.38it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  28%|██▊       | 36/127 [00:09<00:18,  4.80it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  30%|██▉       | 38/127 [00:10<00:28,  3.07it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  34%|███▍      | 43/127 [00:11<00:22,  3.69it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  38%|███▊      | 48/127 [00:12<00:18,  4.26it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  39%|███▉      | 50/127 [00:12<00:16,  4.56it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  42%|████▏     | 53/127 [00:13<00:17,  4.30it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  43%|████▎     | 55/127 [00:15<00:29,  2.47it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  50%|████▉     | 63/127 [00:15<00:12,  5.03it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  51%|█████     | 65/127 [00:16<00:12,  4.90it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  52%|█████▏    | 66/127 [00:16<00:14,  4.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  54%|█████▎    | 68/127 [00:16<00:11,  5.03it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  55%|█████▌    | 70/127 [00:17<00:11,  4.84it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  56%|█████▌    | 71/127 [00:17<00:12,  4.52it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  57%|█████▋    | 72/127 [00:17<00:12,  4.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  57%|█████▋    | 73/127 [00:18<00:14,  3.79it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  59%|█████▉    | 75/127 [00:18<00:09,  5.36it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  60%|█████▉    | 76/127 [00:18<00:10,  4.74it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  61%|██████    | 77/127 [00:19<00:13,  3.76it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  61%|██████▏   | 78/127 [00:19<00:14,  3.27it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  64%|██████▍   | 81/127 [00:19<00:08,  5.21it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  65%|██████▍   | 82/127 [00:20<00:10,  4.38it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  65%|██████▌   | 83/127 [00:20<00:12,  3.67it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  68%|██████▊   | 86/127 [00:20<00:08,  4.93it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  69%|██████▊   | 87/127 [00:21<00:10,  3.86it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  69%|██████▉   | 88/127 [00:21<00:09,  4.18it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  70%|███████   | 89/127 [00:21<00:09,  4.18it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  72%|███████▏  | 92/127 [00:22<00:07,  4.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  73%|███████▎  | 93/127 [00:22<00:07,  4.51it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  75%|███████▍  | 95/127 [00:23<00:06,  4.81it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  76%|███████▋  | 97/127 [00:23<00:07,  3.94it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  77%|███████▋  | 98/127 [00:24<00:08,  3.37it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  79%|███████▊  | 100/127 [00:24<00:05,  4.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  80%|████████  | 102/127 [00:24<00:05,  4.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  81%|████████  | 103/127 [00:25<00:05,  4.04it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  82%|████████▏ | 104/127 [00:25<00:05,  4.01it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  83%|████████▎ | 106/127 [00:25<00:03,  5.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  84%|████████▍ | 107/127 [00:25<00:04,  4.25it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  85%|████████▌ | 108/127 [00:26<00:05,  3.37it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  86%|████████▌ | 109/127 [00:26<00:05,  3.44it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  87%|████████▋ | 110/127 [00:27<00:06,  2.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  89%|████████▉ | 113/127 [00:27<00:03,  4.33it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  91%|█████████ | 115/127 [00:28<00:02,  4.42it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  92%|█████████▏| 117/127 [00:28<00:01,  5.12it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  93%|█████████▎| 118/127 [00:28<00:02,  4.10it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  94%|█████████▍| 120/127 [00:29<00:01,  4.44it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  95%|█████████▌| 121/127 [00:29<00:01,  4.88it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  96%|█████████▌| 122/127 [00:29<00:01,  4.81it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  97%|█████████▋| 123/127 [00:30<00:01,  3.42it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  98%|█████████▊| 125/127 [00:30<00:00,  5.04it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  99%|█████████▉| 126/127 [00:30<00:00,  5.49it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches: 100%|██████████| 127/127 [00:30<00:00,  3.95it/s][A
prompt_batches: 100%|██████████| 127/127 [00:30<00:00,  4.12it/s]
INFO:root:Completed 127 examples in 31.2 seconds.
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.

Annotation chunk:  29%|██▊       | 2/7 [01:08<02:49, 33.83s/it]INFO:root:Annotating 128 examples with chatgpt_fn
INFO:root:Using `openai_completions` on 128 prompts using gpt-3.5-turbo-16k-0613.
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5


prompt_batches:   0%|          | 0/128 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   1%|          | 1/128 [00:01<02:38,  1.25s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   5%|▍         | 6/128 [00:02<00:39,  3.06it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   5%|▌         | 7/128 [00:02<00:34,  3.50it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   6%|▋         | 8/128 [00:02<00:33,  3.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   7%|▋         | 9/128 [00:02<00:28,  4.22it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   9%|▊         | 11/128 [00:03<00:32,  3.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   9%|▉         | 12/128 [00:03<00:27,  4.17it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  10%|█         | 13/128 [00:03<00:33,  3.43it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  12%|█▎        | 16/128 [00:04<00:26,  4.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  14%|█▍        | 18/128 [00:04<00:23,  4.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  15%|█▍        | 19/128 [00:05<00:28,  3.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  16%|█▋        | 21/128 [00:05<00:22,  4.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  18%|█▊        | 23/128 [00:05<00:22,  4.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  19%|█▉        | 24/128 [00:06<00:19,  5.27it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  20%|█▉        | 25/128 [00:06<00:27,  3.70it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  22%|██▏       | 28/128 [00:07<00:22,  4.39it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  23%|██▎       | 30/128 [00:07<00:20,  4.69it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  24%|██▍       | 31/128 [00:08<00:24,  3.89it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  26%|██▌       | 33/128 [00:08<00:21,  4.40it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  27%|██▋       | 34/128 [00:08<00:22,  4.10it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  27%|██▋       | 35/128 [00:09<00:26,  3.53it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  30%|██▉       | 38/128 [00:09<00:17,  5.10it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  30%|███       | 39/128 [00:09<00:23,  3.82it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  31%|███▏      | 40/128 [00:10<00:20,  4.26it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  32%|███▏      | 41/128 [00:10<00:22,  3.89it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  34%|███▍      | 44/128 [00:12<00:45,  1.84it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  41%|████▏     | 53/128 [00:13<00:16,  4.46it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  42%|████▏     | 54/128 [00:13<00:15,  4.65it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  44%|████▍     | 56/128 [00:13<00:13,  5.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  45%|████▍     | 57/128 [00:14<00:14,  4.81it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  45%|████▌     | 58/128 [00:14<00:19,  3.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  47%|████▋     | 60/128 [00:14<00:14,  4.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  48%|████▊     | 62/128 [00:15<00:13,  5.01it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  49%|████▉     | 63/128 [00:16<00:20,  3.14it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  50%|█████     | 64/128 [00:16<00:23,  2.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  53%|█████▎    | 68/128 [00:17<00:14,  4.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  55%|█████▍    | 70/128 [00:17<00:15,  3.85it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  57%|█████▋    | 73/128 [00:18<00:11,  4.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  58%|█████▊    | 74/128 [00:18<00:12,  4.44it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  59%|█████▊    | 75/128 [00:18<00:11,  4.55it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  59%|█████▉    | 76/128 [00:19<00:11,  4.38it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  61%|██████    | 78/128 [00:19<00:10,  4.57it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  62%|██████▏   | 79/128 [00:20<00:14,  3.49it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  63%|██████▎   | 81/128 [00:20<00:10,  4.62it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  64%|██████▍   | 82/128 [00:20<00:09,  4.98it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  65%|██████▍   | 83/128 [00:20<00:09,  4.82it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  66%|██████▌   | 84/128 [00:21<00:12,  3.61it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  67%|██████▋   | 86/128 [00:21<00:08,  4.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  69%|██████▉   | 88/128 [00:21<00:08,  4.70it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  70%|██████▉   | 89/128 [00:22<00:10,  3.66it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  71%|███████   | 91/128 [00:22<00:09,  4.01it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  72%|███████▏  | 92/128 [00:22<00:08,  4.06it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  73%|███████▎  | 94/128 [00:23<00:08,  3.85it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  75%|███████▌  | 96/128 [00:23<00:07,  4.51it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  76%|███████▌  | 97/128 [00:24<00:07,  4.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  77%|███████▋  | 99/128 [00:24<00:06,  4.60it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  78%|███████▊  | 100/128 [00:24<00:05,  5.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  79%|███████▉  | 101/128 [00:25<00:09,  2.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  83%|████████▎ | 106/128 [00:26<00:05,  4.33it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  84%|████████▍ | 108/128 [00:26<00:04,  4.43it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  85%|████████▌ | 109/128 [00:27<00:04,  3.84it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  87%|████████▋ | 111/128 [00:27<00:03,  4.33it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  88%|████████▊ | 112/128 [00:27<00:03,  4.49it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  88%|████████▊ | 113/128 [00:28<00:04,  3.61it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  90%|████████▉ | 115/128 [00:28<00:02,  5.14it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  91%|█████████ | 116/128 [00:28<00:02,  4.61it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  91%|█████████▏| 117/128 [00:28<00:02,  4.85it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  92%|█████████▏| 118/128 [00:29<00:02,  3.64it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  93%|█████████▎| 119/128 [00:29<00:02,  4.18it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  95%|█████████▍| 121/128 [00:29<00:01,  3.80it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  96%|█████████▌| 123/128 [00:30<00:01,  4.32it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  97%|█████████▋| 124/128 [00:30<00:00,  4.51it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  98%|█████████▊| 126/128 [00:30<00:00,  4.54it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  99%|█████████▉| 127/128 [00:31<00:00,  3.39it/s][A
prompt_batches: 100%|██████████| 128/128 [00:31<00:00,  4.07it/s]
INFO:root:Completed 128 examples in 31.8 seconds.
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.

Annotation chunk:  43%|████▎     | 3/7 [01:40<02:12, 33.09s/it]INFO:root:Annotating 128 examples with chatgpt_fn
INFO:root:Using `openai_completions` on 128 prompts using gpt-3.5-turbo-16k-0613.
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5


prompt_batches:   0%|          | 0/128 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   1%|          | 1/128 [00:01<02:23,  1.13s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   2%|▏         | 2/128 [00:01<01:14,  1.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   5%|▍         | 6/128 [00:02<00:33,  3.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   5%|▌         | 7/128 [00:02<00:34,  3.52it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   8%|▊         | 10/128 [00:02<00:19,  6.05it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   9%|▉         | 12/128 [00:03<00:30,  3.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  12%|█▏        | 15/128 [00:03<00:19,  5.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  13%|█▎        | 17/128 [00:04<00:24,  4.45it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  15%|█▍        | 19/128 [00:04<00:24,  4.44it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  16%|█▌        | 20/128 [00:04<00:22,  4.86it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  16%|█▋        | 21/128 [00:05<00:29,  3.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  18%|█▊        | 23/128 [00:05<00:26,  3.94it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  19%|█▉        | 24/128 [00:06<00:24,  4.21it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  20%|██        | 26/128 [00:06<00:21,  4.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  21%|██        | 27/128 [00:06<00:22,  4.54it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  22%|██▏       | 28/128 [00:06<00:23,  4.24it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  23%|██▎       | 30/128 [00:07<00:25,  3.91it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  26%|██▌       | 33/128 [00:08<00:21,  4.49it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  27%|██▋       | 34/128 [00:08<00:20,  4.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  27%|██▋       | 35/128 [00:08<00:25,  3.60it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  30%|██▉       | 38/128 [00:09<00:21,  4.10it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  31%|███▏      | 40/128 [00:09<00:19,  4.60it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  32%|███▏      | 41/128 [00:30<05:38,  3.89s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  86%|████████▌ | 110/128 [00:30<00:03,  4.74it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  93%|█████████▎| 119/128 [00:32<00:01,  4.76it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  98%|█████████▊| 126/128 [00:34<00:00,  4.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"

prompt_batches: 100%|██████████| 128/128 [00:34<00:00,  3.70it/s]
INFO:root:Completed 128 examples in 34.9 seconds.
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.

Annotation chunk:  57%|█████▋    | 4/7 [02:16<01:41, 34.00s/it]INFO:root:Annotating 127 examples with chatgpt_fn
INFO:root:Using `openai_completions` on 127 prompts using gpt-3.5-turbo-16k-0613.
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5


prompt_batches:   0%|          | 0/127 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   1%|          | 1/127 [00:01<02:42,  1.29s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   5%|▍         | 6/127 [00:02<00:37,  3.21it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   6%|▌         | 7/127 [00:02<00:36,  3.33it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   9%|▊         | 11/127 [00:03<00:30,  3.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  10%|█         | 13/127 [00:03<00:24,  4.74it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  12%|█▏        | 15/127 [00:03<00:18,  5.92it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  13%|█▎        | 17/127 [00:04<00:29,  3.72it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  15%|█▍        | 19/127 [00:04<00:23,  4.65it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  17%|█▋        | 21/127 [00:05<00:30,  3.42it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  18%|█▊        | 23/127 [00:05<00:23,  4.51it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  20%|█▉        | 25/127 [00:06<00:24,  4.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  20%|██        | 26/127 [00:08<00:56,  1.79it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  28%|██▊       | 35/127 [00:09<00:23,  3.87it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  30%|██▉       | 38/127 [00:09<00:18,  4.92it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  31%|███▏      | 40/127 [00:10<00:21,  4.12it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  32%|███▏      | 41/127 [00:10<00:21,  3.96it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  34%|███▍      | 43/127 [00:10<00:16,  4.97it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  35%|███▌      | 45/127 [00:11<00:19,  4.25it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  36%|███▌      | 46/127 [00:12<00:21,  3.73it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  39%|███▉      | 50/127 [00:12<00:16,  4.64it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  40%|████      | 51/127 [00:13<00:17,  4.32it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  41%|████      | 52/127 [00:13<00:16,  4.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  43%|████▎     | 55/127 [00:13<00:14,  4.90it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  44%|████▍     | 56/127 [00:14<00:16,  4.44it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  45%|████▍     | 57/127 [00:14<00:16,  4.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  46%|████▋     | 59/127 [00:14<00:14,  4.76it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  47%|████▋     | 60/127 [00:14<00:14,  4.64it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  48%|████▊     | 61/127 [00:15<00:14,  4.64it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  49%|████▉     | 62/127 [00:15<00:17,  3.78it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  50%|█████     | 64/127 [00:15<00:14,  4.40it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  51%|█████     | 65/127 [00:16<00:15,  3.90it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  53%|█████▎    | 67/127 [00:16<00:14,  4.11it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  54%|█████▍    | 69/127 [00:16<00:11,  4.95it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  55%|█████▌    | 70/127 [00:17<00:13,  4.37it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  56%|█████▌    | 71/127 [00:17<00:11,  4.93it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  57%|█████▋    | 72/127 [00:17<00:12,  4.42it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  57%|█████▋    | 73/127 [00:17<00:11,  4.91it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  58%|█████▊    | 74/127 [00:17<00:09,  5.66it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  59%|█████▉    | 75/127 [00:18<00:15,  3.29it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  60%|█████▉    | 76/127 [00:18<00:14,  3.42it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  61%|██████    | 77/127 [00:19<00:23,  2.16it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  64%|██████▍   | 81/127 [00:20<00:10,  4.22it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  65%|██████▌   | 83/127 [00:22<00:23,  1.87it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  72%|███████▏  | 92/127 [00:23<00:08,  4.28it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  73%|███████▎  | 93/127 [00:23<00:07,  4.26it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  75%|███████▍  | 95/127 [00:23<00:07,  4.51it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  76%|███████▋  | 97/127 [00:24<00:06,  4.46it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  77%|███████▋  | 98/127 [00:24<00:06,  4.36it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  79%|███████▊  | 100/127 [00:25<00:06,  4.06it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  80%|████████  | 102/127 [00:25<00:05,  4.82it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  81%|████████  | 103/127 [00:25<00:04,  4.80it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  82%|████████▏ | 104/127 [00:25<00:05,  4.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  83%|████████▎ | 105/127 [00:26<00:06,  3.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  85%|████████▌ | 108/127 [00:26<00:03,  5.14it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  86%|████████▌ | 109/127 [00:27<00:04,  4.09it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  87%|████████▋ | 110/127 [00:27<00:04,  3.92it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  87%|████████▋ | 111/127 [00:27<00:04,  3.70it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  90%|████████▉ | 114/127 [00:28<00:02,  5.00it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  91%|█████████ | 115/127 [00:28<00:03,  3.90it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  91%|█████████▏| 116/127 [00:28<00:02,  4.43it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  92%|█████████▏| 117/127 [00:29<00:02,  3.55it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  93%|█████████▎| 118/127 [00:29<00:02,  4.06it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  94%|█████████▎| 119/127 [00:29<00:01,  4.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  94%|█████████▍| 120/127 [00:30<00:02,  3.49it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  96%|█████████▌| 122/127 [00:30<00:01,  4.76it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  97%|█████████▋| 123/127 [00:30<00:00,  4.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  98%|█████████▊| 125/127 [00:31<00:00,  3.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  99%|█████████▉| 126/127 [00:31<00:00,  3.90it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches: 100%|██████████| 127/127 [00:32<00:00,  2.98it/s][A
prompt_batches: 100%|██████████| 127/127 [00:32<00:00,  3.97it/s]
INFO:root:Completed 127 examples in 32.4 seconds.
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.

Annotation chunk:  71%|███████▏  | 5/7 [02:49<01:07, 33.58s/it]INFO:root:Annotating 128 examples with chatgpt_fn
INFO:root:Using `openai_completions` on 128 prompts using gpt-3.5-turbo-16k-0613.
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5


prompt_batches:   0%|          | 0/128 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   1%|          | 1/128 [00:01<02:51,  1.35s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   2%|▏         | 2/128 [00:01<01:21,  1.55it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   5%|▍         | 6/128 [00:02<00:36,  3.31it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   6%|▋         | 8/128 [00:02<00:25,  4.66it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   8%|▊         | 10/128 [00:02<00:20,  5.82it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   9%|▉         | 12/128 [00:03<00:30,  3.80it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  11%|█         | 14/128 [00:03<00:25,  4.47it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  12%|█▎        | 16/128 [00:04<00:29,  3.80it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  14%|█▍        | 18/128 [00:04<00:22,  4.99it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  15%|█▍        | 19/128 [00:04<00:21,  5.09it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  16%|█▌        | 20/128 [00:04<00:19,  5.61it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  16%|█▋        | 21/128 [00:05<00:32,  3.28it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  18%|█▊        | 23/128 [00:05<00:24,  4.23it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  20%|█▉        | 25/128 [00:06<00:21,  4.82it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  20%|██        | 26/128 [00:06<00:23,  4.27it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  21%|██        | 27/128 [00:06<00:21,  4.66it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  22%|██▏       | 28/128 [00:06<00:22,  4.37it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  23%|██▎       | 29/128 [00:07<00:21,  4.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  23%|██▎       | 30/128 [00:07<00:21,  4.54it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  24%|██▍       | 31/128 [00:07<00:21,  4.46it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  25%|██▌       | 32/128 [00:07<00:21,  4.37it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  26%|██▌       | 33/128 [00:07<00:20,  4.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  27%|██▋       | 34/128 [00:08<00:27,  3.44it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  28%|██▊       | 36/128 [00:08<00:18,  5.10it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  29%|██▉       | 37/128 [00:08<00:17,  5.25it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  30%|██▉       | 38/128 [00:09<00:22,  4.02it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  30%|███       | 39/128 [00:09<00:26,  3.41it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  32%|███▏      | 41/128 [00:10<00:21,  4.09it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  34%|███▎      | 43/128 [00:10<00:20,  4.06it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  34%|███▍      | 44/128 [00:10<00:18,  4.61it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  36%|███▌      | 46/128 [00:10<00:15,  5.22it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  37%|███▋      | 47/128 [00:11<00:14,  5.65it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  38%|███▊      | 48/128 [00:11<00:19,  4.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  39%|███▉      | 50/128 [00:11<00:14,  5.45it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  40%|███▉      | 51/128 [00:12<00:17,  4.34it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  41%|████▏     | 53/128 [00:12<00:20,  3.62it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  43%|████▎     | 55/128 [00:12<00:15,  4.70it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  44%|████▍     | 56/128 [00:13<00:15,  4.57it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  45%|████▌     | 58/128 [00:15<00:37,  1.85it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  52%|█████▏    | 67/128 [00:16<00:14,  4.12it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  53%|█████▎    | 68/128 [00:16<00:14,  4.01it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  56%|█████▋    | 72/128 [00:17<00:12,  4.34it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  57%|█████▋    | 73/128 [00:17<00:12,  4.24it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  59%|█████▊    | 75/128 [00:18<00:11,  4.48it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  60%|██████    | 77/128 [00:18<00:12,  4.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  62%|██████▎   | 80/128 [00:19<00:09,  5.16it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  63%|██████▎   | 81/128 [00:19<00:08,  5.27it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  64%|██████▍   | 82/128 [00:19<00:09,  4.71it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  65%|██████▍   | 83/128 [00:19<00:10,  4.38it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  66%|██████▌   | 84/128 [00:20<00:14,  3.05it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  68%|██████▊   | 87/128 [00:21<00:10,  3.94it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  70%|██████▉   | 89/128 [00:21<00:08,  4.83it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  71%|███████   | 91/128 [00:21<00:08,  4.37it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  72%|███████▏  | 92/128 [00:22<00:09,  4.00it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  73%|███████▎  | 94/128 [00:22<00:08,  3.92it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  75%|███████▌  | 96/128 [00:23<00:07,  4.47it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  76%|███████▌  | 97/128 [00:23<00:08,  3.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  78%|███████▊  | 100/128 [00:24<00:06,  4.49it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  79%|███████▉  | 101/128 [00:24<00:05,  4.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  80%|███████▉  | 102/128 [00:24<00:07,  3.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  82%|████████▏ | 105/128 [00:25<00:04,  4.90it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  83%|████████▎ | 106/128 [00:25<00:04,  4.93it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  84%|████████▎ | 107/128 [00:25<00:04,  4.35it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  85%|████████▌ | 109/128 [00:26<00:04,  4.35it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  86%|████████▌ | 110/128 [00:26<00:03,  4.57it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  87%|████████▋ | 111/128 [00:26<00:03,  4.38it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  88%|████████▊ | 112/128 [00:26<00:03,  4.87it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  88%|████████▊ | 113/128 [00:27<00:04,  3.55it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  90%|████████▉ | 115/128 [00:27<00:02,  5.09it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  91%|█████████ | 116/128 [00:27<00:02,  4.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  92%|█████████▏| 118/128 [00:28<00:02,  3.55it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  95%|█████████▍| 121/128 [00:28<00:01,  4.88it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  95%|█████████▌| 122/128 [00:28<00:01,  5.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  96%|█████████▌| 123/128 [00:29<00:01,  4.05it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  97%|█████████▋| 124/128 [00:29<00:00,  4.57it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  98%|█████████▊| 125/128 [00:29<00:00,  4.98it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  98%|█████████▊| 126/128 [00:29<00:00,  5.47it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  99%|█████████▉| 127/128 [00:29<00:00,  4.81it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches: 100%|██████████| 128/128 [00:30<00:00,  3.33it/s][A
prompt_batches: 100%|██████████| 128/128 [00:30<00:00,  4.19it/s]
INFO:root:Completed 128 examples in 30.9 seconds.
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.

Annotation chunk:  86%|████████▌ | 6/7 [03:20<00:32, 32.81s/it]INFO:root:Annotating 37 examples with chatgpt_fn
INFO:root:Using `openai_completions` on 37 prompts using gpt-3.5-turbo-16k-0613.
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5


prompt_batches:   0%|          | 0/37 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:   3%|▎         | 1/37 [00:01<00:50,  1.39s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  11%|█         | 4/37 [00:01<00:10,  3.19it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  16%|█▌        | 6/37 [00:02<00:10,  2.86it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  19%|█▉        | 7/37 [00:02<00:10,  2.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  30%|██▉       | 11/37 [00:03<00:06,  4.02it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  32%|███▏      | 12/37 [00:03<00:06,  3.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  43%|████▎     | 16/37 [00:04<00:04,  4.23it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  46%|████▌     | 17/37 [00:04<00:05,  3.96it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  54%|█████▍    | 20/37 [00:05<00:02,  5.93it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  59%|█████▉    | 22/37 [00:05<00:03,  4.35it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  62%|██████▏   | 23/37 [00:06<00:03,  4.62it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  70%|███████   | 26/37 [00:06<00:02,  4.35it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  73%|███████▎  | 27/37 [00:06<00:02,  4.54it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  78%|███████▊  | 29/37 [00:07<00:01,  4.74it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  84%|████████▍ | 31/37 [00:08<00:01,  3.79it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  92%|█████████▏| 34/37 [00:08<00:00,  5.16it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches:  97%|█████████▋| 36/37 [00:09<00:00,  3.73it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"


prompt_batches: 100%|██████████| 37/37 [00:09<00:00,  3.85it/s][A
prompt_batches: 100%|██████████| 37/37 [00:09<00:00,  3.90it/s]
INFO:root:Completed 37 examples in 9.8 seconds.
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.

Annotation chunk: 100%|██████████| 7/7 [03:30<00:00, 25.42s/it]
Annotation chunk: 100%|██████████| 7/7 [03:30<00:00, 30.08s/it]
/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/metrics.py:26: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '1.5' has dtype incompatible with int64, please explicitly cast to a compatible dtype first.
  series_preferences[series_preferences == 0] = 1.5
INFO:root:Saving all results to output/chatgpt_fn_--LMCocktail-7B-v1
INFO:root:Saving result to the precomputed leaderboard at /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/leaderboards/data_AlpacaEval/chatgpt_fn_leaderboard.csv
                       win_rate  standard_error  n_total  avg_length
gpt4                      73.79            1.54      805        1365
LMCocktail-7B-v1          73.54            1.55      805        1870
claude                    70.37            1.60      805        1082
chatgpt                   66.09            1.66      805         811
wizardlm-13b              65.16            1.67      805         985
vicuna-13b                64.10            1.69      805        1037
guanaco-65b               62.36            1.71      805        1249
oasst-rlhf-llama-33b      62.05            1.71      805        1079
alpaca-farm-ppo-human     60.25            1.72      805         803
falcon-40b-instruct       56.52            1.74      805         662
text_davinci_003          50.00            0.00      805         307
alpaca-7b                 45.22            1.74      805         396
text_davinci_001          28.07            1.56      805         296