2238 lines
182 KiB
Plaintext
2238 lines
182 KiB
Plaintext
|
||
Chunking for generation: 0%| | 0/13 [00:00<?, ?it/s]INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.43s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.01s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.22s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/64 [00:00<?, ?it/s][A
|
||
|
||
2%|▏ | 1/64 [00:26<27:51, 26.54s/it][A
|
||
|
||
3%|▎ | 2/64 [00:52<27:18, 26.43s/it][A
|
||
|
||
5%|▍ | 3/64 [01:18<26:38, 26.21s/it][A
|
||
|
||
6%|▋ | 4/64 [01:51<28:42, 28.71s/it][A
|
||
|
||
8%|▊ | 5/64 [02:08<23:59, 24.39s/it][A
|
||
|
||
9%|▉ | 6/64 [02:23<20:42, 21.42s/it][A
|
||
|
||
11%|█ | 7/64 [02:38<18:08, 19.09s/it][A
|
||
|
||
12%|█▎ | 8/64 [02:48<15:14, 16.33s/it][A
|
||
|
||
14%|█▍ | 9/64 [03:10<16:38, 18.15s/it][A
|
||
|
||
16%|█▌ | 10/64 [03:38<19:01, 21.14s/it][A
|
||
|
||
17%|█▋ | 11/64 [03:42<14:07, 15.99s/it][A
|
||
|
||
19%|█▉ | 12/64 [03:59<13:58, 16.12s/it][A
|
||
|
||
20%|██ | 13/64 [04:14<13:33, 15.96s/it][A
|
||
|
||
22%|██▏ | 14/64 [04:33<13:57, 16.76s/it][A
|
||
|
||
23%|██▎ | 15/64 [04:48<13:23, 16.40s/it][A
|
||
|
||
25%|██▌ | 16/64 [05:24<17:38, 22.06s/it][A
|
||
|
||
27%|██▋ | 17/64 [05:47<17:30, 22.35s/it][A
|
||
|
||
28%|██▊ | 18/64 [06:11<17:35, 22.94s/it][A
|
||
|
||
30%|██▉ | 19/64 [06:15<13:02, 17.39s/it][A
|
||
|
||
31%|███▏ | 20/64 [06:40<14:13, 19.39s/it][A
|
||
|
||
33%|███▎ | 21/64 [06:59<13:55, 19.43s/it][A
|
||
|
||
34%|███▍ | 22/64 [07:21<14:05, 20.13s/it][A
|
||
|
||
36%|███▌ | 23/64 [07:27<10:52, 15.91s/it][A
|
||
|
||
38%|███▊ | 24/64 [07:43<10:34, 15.87s/it][A
|
||
|
||
39%|███▉ | 25/64 [07:44<07:30, 11.55s/it][A
|
||
|
||
41%|████ | 26/64 [08:08<09:35, 15.15s/it][A
|
||
|
||
42%|████▏ | 27/64 [08:15<07:51, 12.75s/it][A
|
||
|
||
44%|████▍ | 28/64 [08:37<09:23, 15.66s/it][A
|
||
|
||
45%|████▌ | 29/64 [08:57<09:48, 16.81s/it][A
|
||
|
||
47%|████▋ | 30/64 [09:12<09:18, 16.41s/it][A
|
||
|
||
48%|████▊ | 31/64 [09:41<11:02, 20.08s/it][A
|
||
|
||
50%|█████ | 32/64 [09:57<10:08, 19.03s/it][A
|
||
|
||
52%|█████▏ | 33/64 [10:23<10:51, 21.01s/it][A
|
||
|
||
53%|█████▎ | 34/64 [10:41<09:58, 19.96s/it][A
|
||
|
||
55%|█████▍ | 35/64 [10:45<07:22, 15.26s/it][A
|
||
|
||
56%|█████▋ | 36/64 [10:53<06:07, 13.11s/it][A
|
||
|
||
58%|█████▊ | 37/64 [11:14<07:00, 15.57s/it][A
|
||
|
||
59%|█████▉ | 38/64 [11:31<06:55, 15.96s/it][A
|
||
|
||
61%|██████ | 39/64 [11:43<06:08, 14.74s/it][A
|
||
|
||
62%|██████▎ | 40/64 [11:57<05:50, 14.62s/it][A
|
||
|
||
64%|██████▍ | 41/64 [12:07<05:04, 13.24s/it][A
|
||
|
||
66%|██████▌ | 42/64 [12:23<05:08, 14.04s/it][A
|
||
|
||
67%|██████▋ | 43/64 [12:45<05:44, 16.40s/it][A
|
||
|
||
69%|██████▉ | 44/64 [12:53<04:36, 13.85s/it][A
|
||
|
||
70%|███████ | 45/64 [13:15<05:05, 16.10s/it][A
|
||
|
||
72%|███████▏ | 46/64 [13:33<05:02, 16.79s/it][A
|
||
|
||
73%|███████▎ | 47/64 [13:52<04:54, 17.33s/it][A
|
||
|
||
75%|███████▌ | 48/64 [14:14<05:00, 18.81s/it][A
|
||
|
||
77%|███████▋ | 49/64 [14:21<03:50, 15.35s/it][A
|
||
|
||
78%|███████▊ | 50/64 [14:40<03:50, 16.49s/it][A
|
||
|
||
80%|███████▉ | 51/64 [14:46<02:54, 13.42s/it][A
|
||
|
||
81%|████████▏ | 52/64 [14:51<02:09, 10.80s/it][A
|
||
|
||
83%|████████▎ | 53/64 [15:17<02:47, 15.23s/it][A
|
||
|
||
84%|████████▍ | 54/64 [15:36<02:44, 16.48s/it][A
|
||
|
||
86%|████████▌ | 55/64 [15:48<02:16, 15.20s/it][A
|
||
|
||
88%|████████▊ | 56/64 [16:06<02:07, 15.99s/it][A
|
||
|
||
89%|████████▉ | 57/64 [16:26<02:00, 17.17s/it][A
|
||
|
||
91%|█████████ | 58/64 [16:50<01:55, 19.31s/it][A
|
||
|
||
92%|█████████▏| 59/64 [17:03<01:27, 17.43s/it][A
|
||
|
||
94%|█████████▍| 60/64 [17:10<00:57, 14.26s/it][A
|
||
|
||
95%|█████████▌| 61/64 [17:30<00:47, 15.81s/it][A
|
||
|
||
97%|█████████▋| 62/64 [17:48<00:32, 16.45s/it][A
|
||
|
||
98%|█████████▊| 63/64 [18:11<00:18, 18.37s/it][A
|
||
|
||
100%|██████████| 64/64 [18:31<00:00, 18.90s/it][A
|
||
100%|██████████| 64/64 [18:31<00:00, 17.36s/it]
|
||
INFO:root:Time for 64 completions: 1111.1 seconds
|
||
|
||
Chunking for generation: 8%|▊ | 1/13 [18:45<3:45:08, 1125.70s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
|
||
INFO:root:Found 64. Only generating 64 .
|
||
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.39s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.12s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.31s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/64 [00:00<?, ?it/s][A
|
||
|
||
2%|▏ | 1/64 [00:18<19:19, 18.40s/it][A
|
||
|
||
3%|▎ | 2/64 [00:28<13:49, 13.37s/it][A
|
||
|
||
5%|▍ | 3/64 [00:44<14:55, 14.68s/it][A
|
||
|
||
6%|▋ | 4/64 [01:06<17:24, 17.41s/it][A
|
||
|
||
8%|▊ | 5/64 [01:30<19:34, 19.90s/it][A
|
||
|
||
9%|▉ | 6/64 [01:40<16:06, 16.66s/it][A
|
||
|
||
11%|█ | 7/64 [01:52<14:10, 14.92s/it][A
|
||
|
||
12%|█▎ | 8/64 [01:57<10:58, 11.76s/it][A
|
||
|
||
14%|█▍ | 9/64 [02:15<12:37, 13.77s/it][A
|
||
|
||
16%|█▌ | 10/64 [02:38<14:53, 16.54s/it][A
|
||
|
||
17%|█▋ | 11/64 [03:00<16:16, 18.43s/it][A
|
||
|
||
19%|█▉ | 12/64 [03:11<13:58, 16.12s/it][A
|
||
|
||
20%|██ | 13/64 [03:33<15:13, 17.91s/it][A
|
||
|
||
22%|██▏ | 14/64 [04:03<17:52, 21.45s/it][A
|
||
|
||
23%|██▎ | 15/64 [04:16<15:22, 18.83s/it][A
|
||
|
||
25%|██▌ | 16/64 [04:32<14:31, 18.15s/it][A
|
||
|
||
27%|██▋ | 17/64 [04:48<13:38, 17.42s/it][A
|
||
|
||
28%|██▊ | 18/64 [05:13<15:07, 19.72s/it][A
|
||
|
||
30%|██▉ | 19/64 [05:36<15:30, 20.69s/it][A
|
||
|
||
31%|███▏ | 20/64 [06:02<16:22, 22.33s/it][A
|
||
|
||
33%|███▎ | 21/64 [06:22<15:31, 21.67s/it][A
|
||
|
||
34%|███▍ | 22/64 [06:46<15:33, 22.23s/it][A
|
||
|
||
36%|███▌ | 23/64 [07:06<14:46, 21.62s/it][A
|
||
|
||
38%|███▊ | 24/64 [07:36<16:03, 24.09s/it][A
|
||
|
||
39%|███▉ | 25/64 [07:41<12:02, 18.52s/it][A
|
||
|
||
41%|████ | 26/64 [08:06<12:53, 20.37s/it][A
|
||
|
||
42%|████▏ | 27/64 [08:31<13:21, 21.67s/it][A
|
||
|
||
44%|████▍ | 28/64 [08:35<09:57, 16.59s/it][A
|
||
|
||
45%|████▌ | 29/64 [08:54<10:06, 17.32s/it][A
|
||
|
||
47%|████▋ | 30/64 [08:59<07:35, 13.39s/it][A
|
||
|
||
48%|████▊ | 31/64 [09:17<08:13, 14.96s/it][A
|
||
|
||
50%|█████ | 32/64 [09:33<08:09, 15.29s/it][A
|
||
|
||
52%|█████▏ | 33/64 [09:57<09:11, 17.78s/it][A
|
||
|
||
53%|█████▎ | 34/64 [10:04<07:16, 14.56s/it][A
|
||
|
||
55%|█████▍ | 35/64 [10:17<06:48, 14.08s/it][A
|
||
|
||
56%|█████▋ | 36/64 [10:28<06:07, 13.12s/it][A
|
||
|
||
58%|█████▊ | 37/64 [10:53<07:32, 16.75s/it][A
|
||
|
||
59%|█████▉ | 38/64 [11:01<06:08, 14.18s/it][A
|
||
|
||
61%|██████ | 39/64 [11:06<04:41, 11.24s/it][A
|
||
|
||
62%|██████▎ | 40/64 [11:28<05:50, 14.61s/it][A
|
||
|
||
64%|██████▍ | 41/64 [11:41<05:23, 14.08s/it][A
|
||
|
||
66%|██████▌ | 42/64 [11:45<04:01, 10.98s/it][A
|
||
|
||
67%|██████▋ | 43/64 [12:00<04:21, 12.45s/it][A
|
||
|
||
69%|██████▉ | 44/64 [12:18<04:40, 14.02s/it][A
|
||
|
||
70%|███████ | 45/64 [12:32<04:27, 14.08s/it][A
|
||
|
||
72%|███████▏ | 46/64 [12:42<03:50, 12.82s/it][A
|
||
|
||
73%|███████▎ | 47/64 [13:33<06:49, 24.11s/it][A
|
||
|
||
75%|███████▌ | 48/64 [13:52<06:04, 22.81s/it][A
|
||
|
||
77%|███████▋ | 49/64 [14:02<04:43, 18.91s/it][A
|
||
|
||
78%|███████▊ | 50/64 [14:10<03:37, 15.52s/it][A
|
||
|
||
80%|███████▉ | 51/64 [14:19<02:55, 13.52s/it][A
|
||
|
||
81%|████████▏ | 52/64 [14:42<03:15, 16.31s/it][A
|
||
|
||
83%|████████▎ | 53/64 [14:59<03:02, 16.61s/it][A
|
||
|
||
84%|████████▍ | 54/64 [15:17<02:50, 17.04s/it][A
|
||
|
||
86%|████████▌ | 55/64 [15:26<02:12, 14.72s/it][A
|
||
|
||
88%|████████▊ | 56/64 [15:45<02:07, 15.89s/it][A
|
||
|
||
89%|████████▉ | 57/64 [15:47<01:21, 11.69s/it][A
|
||
|
||
91%|█████████ | 58/64 [15:51<00:55, 9.31s/it][A
|
||
|
||
92%|█████████▏| 59/64 [15:55<00:39, 7.85s/it][A
|
||
|
||
94%|█████████▍| 60/64 [16:08<00:37, 9.34s/it][A
|
||
|
||
95%|█████████▌| 61/64 [16:29<00:38, 12.92s/it][A
|
||
|
||
97%|█████████▋| 62/64 [16:40<00:24, 12.21s/it][A
|
||
|
||
98%|█████████▊| 63/64 [16:53<00:12, 12.43s/it][A
|
||
|
||
100%|██████████| 64/64 [17:08<00:00, 13.28s/it][A
|
||
100%|██████████| 64/64 [17:08<00:00, 16.07s/it]
|
||
INFO:root:Time for 64 completions: 1028.3 seconds
|
||
|
||
Chunking for generation: 15%|█▌ | 2/13 [36:01<3:16:38, 1072.61s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
|
||
INFO:root:Found 128. Only generating 64 .
|
||
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.26s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.00s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.19s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/64 [00:00<?, ?it/s][A
|
||
|
||
2%|▏ | 1/64 [00:03<03:47, 3.61s/it][A
|
||
|
||
3%|▎ | 2/64 [00:31<18:25, 17.83s/it][A
|
||
|
||
5%|▍ | 3/64 [00:55<21:10, 20.83s/it][A
|
||
|
||
6%|▋ | 4/64 [01:11<18:54, 18.90s/it][A
|
||
|
||
8%|▊ | 5/64 [01:25<16:37, 16.90s/it][A
|
||
|
||
9%|▉ | 6/64 [01:27<11:38, 12.04s/it][A
|
||
|
||
11%|█ | 7/64 [01:45<13:20, 14.04s/it][A
|
||
|
||
12%|█▎ | 8/64 [01:55<11:47, 12.64s/it][A
|
||
|
||
14%|█▍ | 9/64 [02:31<18:09, 19.81s/it][A
|
||
|
||
16%|█▌ | 10/64 [02:52<18:14, 20.27s/it][A
|
||
|
||
17%|█▋ | 11/64 [03:33<23:28, 26.58s/it][A
|
||
|
||
19%|█▉ | 12/64 [03:52<20:59, 24.21s/it][A
|
||
|
||
20%|██ | 13/64 [04:09<18:57, 22.30s/it][A
|
||
|
||
22%|██▏ | 14/64 [04:15<14:22, 17.25s/it][A
|
||
|
||
23%|██▎ | 15/64 [04:26<12:39, 15.50s/it][A
|
||
|
||
25%|██▌ | 16/64 [04:43<12:44, 15.92s/it][A
|
||
|
||
27%|██▋ | 17/64 [04:49<10:04, 12.87s/it][A
|
||
|
||
28%|██▊ | 18/64 [05:08<11:17, 14.73s/it][A
|
||
|
||
30%|██▉ | 19/64 [05:20<10:18, 13.74s/it][A
|
||
|
||
31%|███▏ | 20/64 [05:38<11:04, 15.10s/it][A
|
||
|
||
33%|███▎ | 21/64 [06:56<24:24, 34.05s/it][A
|
||
|
||
34%|███▍ | 22/64 [07:15<20:41, 29.55s/it][A
|
||
|
||
36%|███▌ | 23/64 [07:22<15:31, 22.73s/it][A
|
||
|
||
38%|███▊ | 24/64 [07:49<15:58, 23.97s/it][A
|
||
|
||
39%|███▉ | 25/64 [08:13<15:39, 24.09s/it][A
|
||
|
||
41%|████ | 26/64 [09:02<19:55, 31.45s/it][A
|
||
|
||
42%|████▏ | 27/64 [09:45<21:29, 34.86s/it][A
|
||
|
||
44%|████▍ | 28/64 [09:50<15:32, 25.89s/it][A
|
||
|
||
45%|████▌ | 29/64 [10:31<17:49, 30.56s/it][A
|
||
|
||
47%|████▋ | 30/64 [10:47<14:48, 26.14s/it][A
|
||
|
||
48%|████▊ | 31/64 [10:52<10:53, 19.80s/it][A
|
||
|
||
50%|█████ | 32/64 [10:54<07:40, 14.39s/it][A
|
||
|
||
52%|█████▏ | 33/64 [11:18<08:56, 17.30s/it][A
|
||
|
||
53%|█████▎ | 34/64 [11:25<07:10, 14.34s/it][A
|
||
|
||
55%|█████▍ | 35/64 [11:59<09:47, 20.25s/it][A
|
||
|
||
56%|█████▋ | 36/64 [12:06<07:35, 16.26s/it][A
|
||
|
||
58%|█████▊ | 37/64 [12:16<06:25, 14.29s/it][A
|
||
|
||
59%|█████▉ | 38/64 [12:37<07:06, 16.39s/it][A
|
||
|
||
61%|██████ | 39/64 [12:50<06:23, 15.32s/it][A
|
||
|
||
62%|██████▎ | 40/64 [13:04<05:59, 14.97s/it][A
|
||
|
||
64%|██████▍ | 41/64 [13:05<04:04, 10.65s/it][A
|
||
|
||
66%|██████▌ | 42/64 [13:17<04:06, 11.22s/it][A
|
||
|
||
67%|██████▋ | 43/64 [13:32<04:19, 12.35s/it][A
|
||
|
||
69%|██████▉ | 44/64 [14:18<07:24, 22.21s/it][A
|
||
|
||
70%|███████ | 45/64 [14:25<05:40, 17.90s/it][A
|
||
|
||
72%|███████▏ | 46/64 [14:35<04:36, 15.35s/it][A
|
||
|
||
73%|███████▎ | 47/64 [14:40<03:28, 12.29s/it][A
|
||
|
||
75%|███████▌ | 48/64 [14:56<03:35, 13.49s/it][A
|
||
|
||
77%|███████▋ | 49/64 [15:38<05:28, 21.88s/it][A
|
||
|
||
78%|███████▊ | 50/64 [16:04<05:23, 23.09s/it][A
|
||
|
||
80%|███████▉ | 51/64 [16:13<04:07, 19.03s/it][A
|
||
|
||
81%|████████▏ | 52/64 [16:54<05:07, 25.63s/it][A
|
||
|
||
83%|████████▎ | 53/64 [17:15<04:26, 24.20s/it][A
|
||
|
||
84%|████████▍ | 54/64 [17:31<03:36, 21.64s/it][A
|
||
|
||
86%|████████▌ | 55/64 [18:18<04:23, 29.27s/it][A
|
||
|
||
88%|████████▊ | 56/64 [18:42<03:42, 27.80s/it][A
|
||
|
||
89%|████████▉ | 57/64 [18:58<02:48, 24.10s/it][A
|
||
|
||
91%|█████████ | 58/64 [19:17<02:16, 22.73s/it][A
|
||
|
||
92%|█████████▏| 59/64 [19:37<01:49, 21.91s/it][A
|
||
|
||
94%|█████████▍| 60/64 [20:05<01:34, 23.61s/it][A
|
||
|
||
95%|█████████▌| 61/64 [20:38<01:19, 26.58s/it][A
|
||
|
||
97%|█████████▋| 62/64 [20:47<00:42, 21.35s/it][A
|
||
|
||
98%|█████████▊| 63/64 [20:49<00:15, 15.36s/it][A
|
||
|
||
100%|██████████| 64/64 [21:00<00:00, 14.24s/it][A
|
||
100%|██████████| 64/64 [21:00<00:00, 19.70s/it]
|
||
INFO:root:Time for 64 completions: 1260.9 seconds
|
||
|
||
Chunking for generation: 23%|██▎ | 3/13 [57:08<3:13:37, 1161.71s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
|
||
INFO:root:Found 192. Only generating 64 .
|
||
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.43s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.12s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.32s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/64 [00:00<?, ?it/s][A
|
||
|
||
2%|▏ | 1/64 [00:11<12:31, 11.93s/it][A
|
||
|
||
3%|▎ | 2/64 [00:47<26:45, 25.89s/it][A
|
||
|
||
5%|▍ | 3/64 [01:06<23:06, 22.73s/it][A
|
||
|
||
6%|▋ | 4/64 [01:07<14:06, 14.11s/it][A
|
||
|
||
8%|▊ | 5/64 [01:24<14:56, 15.20s/it][A
|
||
|
||
9%|▉ | 6/64 [01:38<14:21, 14.86s/it][A
|
||
|
||
11%|█ | 7/64 [01:45<11:32, 12.15s/it][A
|
||
|
||
14%|█▍ | 9/64 [02:04<10:03, 10.97s/it][A
|
||
|
||
16%|█▌ | 10/64 [02:25<12:04, 13.43s/it][A
|
||
|
||
17%|█▋ | 11/64 [02:30<09:54, 11.21s/it][A
|
||
|
||
19%|█▉ | 12/64 [03:43<24:36, 28.40s/it][A
|
||
|
||
20%|██ | 13/64 [04:11<24:05, 28.34s/it][A
|
||
|
||
22%|██▏ | 14/64 [04:32<21:43, 26.07s/it][A
|
||
|
||
23%|██▎ | 15/64 [05:08<23:43, 29.05s/it][A
|
||
|
||
25%|██▌ | 16/64 [05:12<17:19, 21.65s/it][A
|
||
|
||
27%|██▋ | 17/64 [05:54<21:36, 27.58s/it][A
|
||
|
||
28%|██▊ | 18/64 [06:12<18:59, 24.76s/it][A
|
||
|
||
30%|██▉ | 19/64 [06:16<13:51, 18.49s/it][A
|
||
|
||
31%|███▏ | 20/64 [06:21<10:40, 14.56s/it][A
|
||
|
||
33%|███▎ | 21/64 [06:40<11:19, 15.79s/it][A
|
||
|
||
34%|███▍ | 22/64 [07:09<13:52, 19.82s/it][A
|
||
|
||
36%|███▌ | 23/64 [07:33<14:28, 21.17s/it][A
|
||
|
||
38%|███▊ | 24/64 [07:56<14:26, 21.65s/it][A
|
||
|
||
39%|███▉ | 25/64 [08:20<14:36, 22.48s/it][A
|
||
|
||
41%|████ | 26/64 [09:17<20:42, 32.69s/it][A
|
||
|
||
42%|████▏ | 27/64 [09:43<18:58, 30.78s/it][A
|
||
|
||
44%|████▍ | 28/64 [10:03<16:24, 27.35s/it][A
|
||
|
||
45%|████▌ | 29/64 [10:32<16:21, 28.05s/it][A
|
||
|
||
47%|████▋ | 30/64 [11:02<16:13, 28.64s/it][A
|
||
|
||
48%|████▊ | 31/64 [11:10<12:15, 22.28s/it][A
|
||
|
||
50%|█████ | 32/64 [11:35<12:21, 23.16s/it][A
|
||
|
||
52%|█████▏ | 33/64 [11:48<10:24, 20.15s/it][A
|
||
|
||
53%|█████▎ | 34/64 [11:54<07:54, 15.82s/it][A
|
||
|
||
55%|█████▍ | 35/64 [12:21<09:18, 19.26s/it][A
|
||
|
||
56%|█████▋ | 36/64 [12:36<08:27, 18.11s/it][A
|
||
|
||
58%|█████▊ | 37/64 [13:16<10:59, 24.41s/it][A
|
||
|
||
59%|█████▉ | 38/64 [13:39<10:30, 24.25s/it][A
|
||
|
||
61%|██████ | 39/64 [13:44<07:41, 18.45s/it][A
|
||
|
||
62%|██████▎ | 40/64 [14:44<12:20, 30.84s/it][A
|
||
|
||
64%|██████▍ | 41/64 [14:52<09:07, 23.81s/it][A
|
||
|
||
66%|██████▌ | 42/64 [14:58<06:50, 18.67s/it][A
|
||
|
||
67%|██████▋ | 43/64 [15:01<04:52, 13.91s/it][A
|
||
|
||
69%|██████▉ | 44/64 [15:33<06:25, 19.29s/it][A
|
||
|
||
70%|███████ | 45/64 [16:00<06:53, 21.75s/it][A
|
||
|
||
72%|███████▏ | 46/64 [16:36<07:44, 25.78s/it][A
|
||
|
||
73%|███████▎ | 47/64 [16:57<06:56, 24.53s/it][A
|
||
|
||
75%|███████▌ | 48/64 [17:19<06:17, 23.62s/it][A
|
||
|
||
77%|███████▋ | 49/64 [17:42<05:53, 23.56s/it][A
|
||
|
||
78%|███████▊ | 50/64 [18:09<05:45, 24.70s/it][A
|
||
|
||
80%|███████▉ | 51/64 [18:16<04:09, 19.18s/it][A
|
||
|
||
81%|████████▏ | 52/64 [18:50<04:43, 23.65s/it][A
|
||
|
||
83%|████████▎ | 53/64 [19:07<03:57, 21.63s/it][A
|
||
|
||
84%|████████▍ | 54/64 [19:23<03:20, 20.07s/it][A
|
||
|
||
86%|████████▌ | 55/64 [19:53<03:28, 23.12s/it][A
|
||
|
||
88%|████████▊ | 56/64 [20:18<03:09, 23.63s/it][A
|
||
|
||
89%|████████▉ | 57/64 [20:54<03:10, 27.25s/it][A
|
||
|
||
91%|█████████ | 58/64 [20:58<02:01, 20.32s/it][A
|
||
|
||
92%|█████████▏| 59/64 [21:16<01:38, 19.63s/it][A
|
||
|
||
94%|█████████▍| 60/64 [21:24<01:04, 16.11s/it][A
|
||
|
||
95%|█████████▌| 61/64 [21:50<00:56, 18.96s/it][A
|
||
|
||
97%|█████████▋| 62/64 [22:18<00:43, 21.84s/it][A
|
||
|
||
98%|█████████▊| 63/64 [22:42<00:22, 22.50s/it][A
|
||
|
||
100%|██████████| 64/64 [23:02<00:00, 21.78s/it][A
|
||
100%|██████████| 64/64 [23:02<00:00, 21.61s/it]
|
||
INFO:root:Time for 64 completions: 1382.8 seconds
|
||
|
||
Chunking for generation: 31%|███ | 4/13 [1:20:18<3:07:46, 1251.80s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
|
||
INFO:root:Found 256. Only generating 64 .
|
||
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.39s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.09s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.29s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/64 [00:00<?, ?it/s][A
|
||
|
||
2%|▏ | 1/64 [00:26<27:32, 26.22s/it][A
|
||
|
||
3%|▎ | 2/64 [00:46<23:30, 22.74s/it][A
|
||
|
||
5%|▍ | 3/64 [00:58<18:03, 17.77s/it][A
|
||
|
||
6%|▋ | 4/64 [01:15<17:25, 17.43s/it][A
|
||
|
||
8%|▊ | 5/64 [01:50<23:28, 23.87s/it][A
|
||
|
||
9%|▉ | 6/64 [01:57<17:27, 18.06s/it][A
|
||
|
||
11%|█ | 7/64 [01:58<11:59, 12.62s/it][A
|
||
|
||
12%|█▎ | 8/64 [02:27<16:42, 17.89s/it][A
|
||
|
||
14%|█▍ | 9/64 [02:41<15:01, 16.38s/it][A
|
||
|
||
16%|█▌ | 10/64 [03:14<19:21, 21.52s/it][A
|
||
|
||
17%|█▋ | 11/64 [03:29<17:14, 19.52s/it][A
|
||
|
||
19%|█▉ | 12/64 [03:48<16:47, 19.37s/it][A
|
||
|
||
20%|██ | 13/64 [03:51<12:25, 14.62s/it][A
|
||
|
||
22%|██▏ | 14/64 [04:26<17:14, 20.69s/it][A
|
||
|
||
23%|██▎ | 15/64 [05:28<27:07, 33.22s/it][A
|
||
|
||
25%|██▌ | 16/64 [06:41<36:01, 45.03s/it][A
|
||
|
||
27%|██▋ | 17/64 [06:53<27:37, 35.28s/it][A
|
||
|
||
28%|██▊ | 18/64 [07:15<23:58, 31.28s/it][A
|
||
|
||
30%|██▉ | 19/64 [07:40<21:53, 29.18s/it][A
|
||
|
||
31%|███▏ | 20/64 [07:54<18:11, 24.81s/it][A
|
||
|
||
33%|███▎ | 21/64 [08:28<19:41, 27.47s/it][A
|
||
|
||
34%|███▍ | 22/64 [08:44<16:54, 24.14s/it][A
|
||
|
||
36%|███▌ | 23/64 [09:19<18:36, 27.22s/it][A
|
||
|
||
38%|███▊ | 24/64 [09:51<19:06, 28.67s/it][A
|
||
|
||
39%|███▉ | 25/64 [10:13<17:27, 26.85s/it][A
|
||
|
||
41%|████ | 26/64 [10:43<17:33, 27.72s/it][A
|
||
|
||
42%|████▏ | 27/64 [11:09<16:42, 27.09s/it][A
|
||
|
||
44%|████▍ | 28/64 [11:43<17:29, 29.15s/it][A
|
||
|
||
45%|████▌ | 29/64 [12:57<24:57, 42.79s/it][A
|
||
|
||
47%|████▋ | 30/64 [13:02<17:44, 31.31s/it][A
|
||
|
||
48%|████▊ | 31/64 [13:08<13:00, 23.65s/it][A
|
||
|
||
50%|█████ | 32/64 [13:34<13:00, 24.38s/it][A
|
||
|
||
52%|█████▏ | 33/64 [13:56<12:17, 23.78s/it][A
|
||
|
||
53%|█████▎ | 34/64 [14:28<13:10, 26.36s/it][A
|
||
|
||
55%|█████▍ | 35/64 [14:46<11:31, 23.86s/it][A
|
||
|
||
56%|█████▋ | 36/64 [14:57<09:13, 19.79s/it][A
|
||
|
||
58%|█████▊ | 37/64 [15:03<07:03, 15.70s/it][A
|
||
|
||
59%|█████▉ | 38/64 [15:10<05:40, 13.10s/it][A
|
||
|
||
61%|██████ | 39/64 [15:32<06:37, 15.90s/it][A
|
||
|
||
62%|██████▎ | 40/64 [15:59<07:37, 19.04s/it][A
|
||
|
||
64%|██████▍ | 41/64 [16:03<05:38, 14.70s/it][A
|
||
|
||
66%|██████▌ | 42/64 [16:23<05:58, 16.31s/it][A
|
||
|
||
67%|██████▋ | 43/64 [17:03<08:07, 23.23s/it][A
|
||
|
||
69%|██████▉ | 44/64 [17:23<07:28, 22.44s/it][A
|
||
|
||
70%|███████ | 45/64 [17:43<06:52, 21.69s/it][A
|
||
|
||
72%|███████▏ | 46/64 [18:07<06:41, 22.31s/it][A
|
||
|
||
73%|███████▎ | 47/64 [18:21<05:37, 19.84s/it][A
|
||
|
||
75%|███████▌ | 48/64 [18:53<06:16, 23.53s/it][A
|
||
|
||
77%|███████▋ | 49/64 [19:14<05:41, 22.74s/it][A
|
||
|
||
78%|███████▊ | 50/64 [19:36<05:15, 22.54s/it][A
|
||
|
||
80%|███████▉ | 51/64 [19:55<04:40, 21.57s/it][A
|
||
|
||
81%|████████▏ | 52/64 [20:17<04:19, 21.61s/it][A
|
||
|
||
83%|████████▎ | 53/64 [20:37<03:51, 21.06s/it][A
|
||
|
||
84%|████████▍ | 54/64 [20:51<03:10, 19.06s/it][A
|
||
|
||
86%|████████▌ | 55/64 [21:30<03:44, 24.99s/it][A
|
||
|
||
88%|████████▊ | 56/64 [21:32<02:24, 18.08s/it][A
|
||
|
||
89%|████████▉ | 57/64 [21:58<02:23, 20.54s/it][A
|
||
|
||
91%|█████████ | 58/64 [22:16<01:58, 19.73s/it][A
|
||
|
||
92%|█████████▏| 59/64 [22:23<01:19, 15.90s/it][A
|
||
|
||
94%|█████████▍| 60/64 [22:44<01:08, 17.23s/it][A
|
||
|
||
95%|█████████▌| 61/64 [23:09<00:59, 19.84s/it][A
|
||
|
||
97%|█████████▋| 62/64 [23:15<00:30, 15.45s/it][A
|
||
|
||
98%|█████████▊| 63/64 [23:21<00:12, 12.72s/it][A
|
||
|
||
100%|██████████| 64/64 [24:03<00:00, 21.53s/it][A
|
||
100%|██████████| 64/64 [24:03<00:00, 22.56s/it]
|
||
INFO:root:Time for 64 completions: 1443.6 seconds
|
||
|
||
Chunking for generation: 38%|███▊ | 5/13 [1:44:29<2:56:28, 1323.51s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
|
||
INFO:root:Found 320. Only generating 64 .
|
||
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.46s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.15s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.35s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/64 [00:00<?, ?it/s][A
|
||
|
||
2%|▏ | 1/64 [00:37<39:01, 37.16s/it][A
|
||
|
||
3%|▎ | 2/64 [00:42<19:04, 18.46s/it][A
|
||
|
||
5%|▍ | 3/64 [01:03<19:53, 19.57s/it][A
|
||
|
||
6%|▋ | 4/64 [01:13<15:48, 15.80s/it][A
|
||
|
||
8%|▊ | 5/64 [02:08<29:36, 30.12s/it][A
|
||
|
||
9%|▉ | 6/64 [02:22<23:49, 24.64s/it][A
|
||
|
||
11%|█ | 7/64 [02:37<20:16, 21.34s/it][A
|
||
|
||
12%|█▎ | 8/64 [02:58<19:49, 21.24s/it][A
|
||
|
||
14%|█▍ | 9/64 [03:26<21:18, 23.24s/it][A
|
||
|
||
16%|█▌ | 10/64 [03:44<19:39, 21.84s/it][A
|
||
|
||
17%|█▋ | 11/64 [04:08<19:54, 22.53s/it][A
|
||
|
||
19%|█▉ | 12/64 [04:26<18:15, 21.06s/it][A
|
||
|
||
20%|██ | 13/64 [04:29<13:12, 15.54s/it][A
|
||
|
||
22%|██▏ | 14/64 [04:30<09:24, 11.28s/it][A
|
||
|
||
23%|██▎ | 15/64 [05:10<16:07, 19.75s/it][A
|
||
|
||
25%|██▌ | 16/64 [05:20<13:27, 16.82s/it][A
|
||
|
||
27%|██▋ | 17/64 [05:39<13:47, 17.60s/it][A
|
||
|
||
28%|██▊ | 18/64 [05:50<11:59, 15.64s/it][A
|
||
|
||
30%|██▉ | 19/64 [06:12<13:06, 17.48s/it][A
|
||
|
||
31%|███▏ | 20/64 [06:23<11:27, 15.62s/it][A
|
||
|
||
33%|███▎ | 21/64 [06:26<08:19, 11.61s/it][A
|
||
|
||
34%|███▍ | 22/64 [06:42<09:06, 13.01s/it][A
|
||
|
||
36%|███▌ | 23/64 [07:20<14:01, 20.53s/it][A
|
||
|
||
38%|███▊ | 24/64 [07:39<13:24, 20.11s/it][A
|
||
|
||
39%|███▉ | 25/64 [07:57<12:36, 19.39s/it][A
|
||
|
||
41%|████ | 26/64 [08:05<10:04, 15.92s/it][A
|
||
|
||
42%|████▏ | 27/64 [08:21<09:48, 15.91s/it][A
|
||
|
||
44%|████▍ | 28/64 [08:41<10:16, 17.13s/it][A
|
||
|
||
45%|████▌ | 29/64 [08:52<08:57, 15.36s/it][A
|
||
|
||
47%|████▋ | 30/64 [08:57<07:03, 12.44s/it][A
|
||
|
||
48%|████▊ | 31/64 [09:02<05:28, 9.97s/it][A
|
||
|
||
50%|█████ | 32/64 [09:23<07:04, 13.26s/it][A
|
||
|
||
52%|█████▏ | 33/64 [09:31<06:02, 11.70s/it][A
|
||
|
||
53%|█████▎ | 34/64 [10:01<08:39, 17.30s/it][A
|
||
|
||
55%|█████▍ | 35/64 [10:48<12:42, 26.29s/it][A
|
||
|
||
56%|█████▋ | 36/64 [11:05<10:56, 23.44s/it][A
|
||
|
||
58%|█████▊ | 37/64 [11:14<08:39, 19.22s/it][A
|
||
|
||
59%|█████▉ | 38/64 [11:39<09:02, 20.86s/it][A
|
||
|
||
61%|██████ | 39/64 [12:01<08:51, 21.27s/it][A
|
||
|
||
62%|██████▎ | 40/64 [12:11<07:04, 17.70s/it][A
|
||
|
||
64%|██████▍ | 41/64 [12:27<06:35, 17.19s/it][A
|
||
|
||
66%|██████▌ | 42/64 [13:45<13:03, 35.62s/it][A
|
||
|
||
67%|██████▋ | 43/64 [13:50<09:13, 26.37s/it][A
|
||
|
||
69%|██████▉ | 44/64 [14:13<08:28, 25.42s/it][A
|
||
|
||
70%|███████ | 45/64 [14:29<07:09, 22.58s/it][A
|
||
|
||
72%|███████▏ | 46/64 [14:46<06:16, 20.92s/it][A
|
||
|
||
73%|███████▎ | 47/64 [14:57<05:03, 17.83s/it][A
|
||
|
||
75%|███████▌ | 48/64 [15:12<04:33, 17.11s/it][A
|
||
|
||
77%|███████▋ | 49/64 [15:21<03:40, 14.70s/it][A
|
||
|
||
78%|███████▊ | 50/64 [16:15<06:08, 26.29s/it][A
|
||
|
||
80%|███████▉ | 51/64 [16:20<04:18, 19.86s/it][A
|
||
|
||
81%|████████▏ | 52/64 [16:21<02:53, 14.44s/it][A
|
||
|
||
83%|████████▎ | 53/64 [16:35<02:37, 14.32s/it][A
|
||
|
||
84%|████████▍ | 54/64 [16:48<02:17, 13.77s/it][A
|
||
|
||
86%|████████▌ | 55/64 [17:10<02:26, 16.26s/it][A
|
||
|
||
88%|████████▊ | 56/64 [17:14<01:40, 12.51s/it][A
|
||
|
||
89%|████████▉ | 57/64 [17:37<01:50, 15.84s/it][A
|
||
|
||
91%|█████████ | 58/64 [17:48<01:25, 14.24s/it][A
|
||
|
||
92%|█████████▏| 59/64 [17:52<00:56, 11.33s/it][A
|
||
|
||
94%|█████████▍| 60/64 [18:13<00:56, 14.11s/it][A
|
||
|
||
95%|█████████▌| 61/64 [18:27<00:42, 14.03s/it][A
|
||
|
||
97%|█████████▋| 62/64 [18:59<00:39, 19.51s/it][A
|
||
|
||
98%|█████████▊| 63/64 [19:12<00:17, 17.63s/it][A
|
||
|
||
100%|██████████| 64/64 [19:22<00:00, 15.17s/it][A
|
||
100%|██████████| 64/64 [19:22<00:00, 18.16s/it]
|
||
INFO:root:Time for 64 completions: 1162.3 seconds
|
||
|
||
Chunking for generation: 46%|████▌ | 6/13 [2:03:58<2:28:17, 1271.14s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
|
||
INFO:root:Found 384. Only generating 64 .
|
||
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.36s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.09s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.29s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/64 [00:00<?, ?it/s][A
|
||
|
||
2%|▏ | 1/64 [00:04<04:38, 4.43s/it][A
|
||
|
||
3%|▎ | 2/64 [00:24<13:50, 13.40s/it][A
|
||
|
||
5%|▍ | 3/64 [00:47<18:03, 17.77s/it][A
|
||
|
||
6%|▋ | 4/64 [00:52<12:45, 12.76s/it][A
|
||
|
||
8%|▊ | 5/64 [01:08<13:38, 13.88s/it][A
|
||
|
||
9%|▉ | 6/64 [01:09<09:14, 9.57s/it][A
|
||
|
||
11%|█ | 7/64 [01:36<14:26, 15.20s/it][A
|
||
|
||
12%|█▎ | 8/64 [01:45<12:29, 13.38s/it][A
|
||
|
||
14%|█▍ | 9/64 [02:03<13:33, 14.78s/it][A
|
||
|
||
16%|█▌ | 10/64 [02:16<12:57, 14.39s/it][A
|
||
|
||
17%|█▋ | 11/64 [02:53<18:36, 21.06s/it][A
|
||
|
||
19%|█▉ | 12/64 [03:14<18:21, 21.19s/it][A
|
||
|
||
20%|██ | 13/64 [03:39<18:54, 22.25s/it][A
|
||
|
||
22%|██▏ | 14/64 [04:00<18:21, 22.02s/it][A
|
||
|
||
23%|██▎ | 15/64 [04:15<16:04, 19.69s/it][A
|
||
|
||
25%|██▌ | 16/64 [05:00<21:51, 27.32s/it][A
|
||
|
||
27%|██▋ | 17/64 [05:14<18:25, 23.52s/it][A
|
||
|
||
28%|██▊ | 18/64 [05:27<15:37, 20.39s/it][A
|
||
|
||
30%|██▉ | 19/64 [05:35<12:21, 16.48s/it][A
|
||
|
||
31%|███▏ | 20/64 [05:47<11:04, 15.09s/it][A
|
||
|
||
33%|███▎ | 21/64 [05:50<08:21, 11.67s/it][A
|
||
|
||
34%|███▍ | 22/64 [06:04<08:39, 12.38s/it][A
|
||
|
||
36%|███▌ | 23/64 [06:15<08:12, 12.01s/it][A
|
||
|
||
38%|███▊ | 24/64 [06:29<08:24, 12.61s/it][A
|
||
|
||
39%|███▉ | 25/64 [06:31<06:07, 9.42s/it][A
|
||
|
||
41%|████ | 26/64 [07:09<11:15, 17.77s/it][A
|
||
|
||
42%|████▏ | 27/64 [08:08<18:35, 30.15s/it][A
|
||
|
||
44%|████▍ | 28/64 [08:25<15:50, 26.40s/it][A
|
||
|
||
45%|████▌ | 29/64 [08:43<13:50, 23.74s/it][A
|
||
|
||
47%|████▋ | 30/64 [09:16<15:04, 26.59s/it][A
|
||
|
||
48%|████▊ | 31/64 [09:52<16:13, 29.49s/it][A
|
||
|
||
50%|█████ | 32/64 [10:14<14:25, 27.05s/it][A
|
||
|
||
52%|█████▏ | 33/64 [10:31<12:29, 24.18s/it][A
|
||
|
||
53%|█████▎ | 34/64 [10:44<10:26, 20.87s/it][A
|
||
|
||
55%|█████▍ | 35/64 [10:55<08:36, 17.82s/it][A
|
||
|
||
56%|█████▋ | 36/64 [10:58<06:17, 13.48s/it][A
|
||
|
||
58%|█████▊ | 37/64 [11:02<04:43, 10.51s/it][A
|
||
|
||
59%|█████▉ | 38/64 [11:07<03:53, 8.99s/it][A
|
||
|
||
61%|██████ | 39/64 [11:33<05:51, 14.06s/it][A
|
||
|
||
62%|██████▎ | 40/64 [11:37<04:25, 11.05s/it][A
|
||
|
||
64%|██████▍ | 41/64 [11:40<03:13, 8.39s/it][A
|
||
|
||
66%|██████▌ | 42/64 [12:05<04:59, 13.61s/it][A
|
||
|
||
67%|██████▋ | 43/64 [12:47<07:40, 21.91s/it][A
|
||
|
||
69%|██████▉ | 44/64 [13:02<06:37, 19.89s/it][A
|
||
|
||
70%|███████ | 45/64 [13:22<06:17, 19.88s/it][A
|
||
|
||
72%|███████▏ | 46/64 [13:32<05:06, 17.04s/it][A
|
||
|
||
73%|███████▎ | 47/64 [13:37<03:47, 13.36s/it][A
|
||
|
||
75%|███████▌ | 48/64 [14:05<04:45, 17.87s/it][A
|
||
|
||
77%|███████▋ | 49/64 [14:09<03:23, 13.59s/it][A
|
||
|
||
78%|███████▊ | 50/64 [14:44<04:40, 20.07s/it][A
|
||
|
||
80%|███████▉ | 51/64 [15:16<05:07, 23.63s/it][A
|
||
|
||
81%|████████▏ | 52/64 [15:38<04:37, 23.09s/it][A
|
||
|
||
83%|████████▎ | 53/64 [15:40<03:04, 16.78s/it][A
|
||
|
||
84%|████████▍ | 54/64 [15:48<02:22, 14.22s/it][A
|
||
|
||
86%|████████▌ | 55/64 [15:52<01:39, 11.07s/it][A
|
||
|
||
88%|████████▊ | 56/64 [16:10<01:44, 13.09s/it][A
|
||
|
||
89%|████████▉ | 57/64 [16:17<01:19, 11.37s/it][A
|
||
|
||
91%|█████████ | 58/64 [16:30<01:11, 11.87s/it][A
|
||
|
||
92%|█████████▏| 59/64 [16:51<01:12, 14.60s/it][A
|
||
|
||
94%|█████████▍| 60/64 [16:59<00:50, 12.61s/it][A
|
||
|
||
95%|█████████▌| 61/64 [17:06<00:33, 11.06s/it][A
|
||
|
||
97%|█████████▋| 62/64 [17:34<00:31, 15.92s/it][A
|
||
|
||
98%|█████████▊| 63/64 [17:40<00:13, 13.15s/it][A
|
||
|
||
100%|██████████| 64/64 [17:44<00:00, 10.37s/it][A
|
||
100%|██████████| 64/64 [17:44<00:00, 16.64s/it]
|
||
INFO:root:Time for 64 completions: 1064.8 seconds
|
||
|
||
Chunking for generation: 54%|█████▍ | 7/13 [2:21:50<2:00:35, 1205.98s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
|
||
INFO:root:Found 448. Only generating 64 .
|
||
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.44s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.14s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.34s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/64 [00:00<?, ?it/s][A
|
||
|
||
2%|▏ | 1/64 [00:02<02:38, 2.51s/it][A
|
||
|
||
3%|▎ | 2/64 [00:35<20:59, 20.31s/it][A
|
||
|
||
5%|▍ | 3/64 [01:00<22:55, 22.55s/it][A
|
||
|
||
6%|▋ | 4/64 [01:54<34:48, 34.82s/it][A
|
||
|
||
8%|▊ | 5/64 [02:19<30:50, 31.37s/it][A
|
||
|
||
9%|▉ | 6/64 [02:36<25:42, 26.59s/it][A
|
||
|
||
11%|█ | 7/64 [02:52<21:47, 22.93s/it][A
|
||
|
||
12%|█▎ | 8/64 [02:56<15:45, 16.88s/it][A
|
||
|
||
14%|█▍ | 9/64 [03:16<16:37, 18.13s/it][A
|
||
|
||
16%|█▌ | 10/64 [03:18<11:35, 12.88s/it][A
|
||
|
||
17%|█▋ | 11/64 [03:23<09:18, 10.54s/it][A
|
||
|
||
19%|█▉ | 12/64 [03:37<10:12, 11.77s/it][A
|
||
|
||
20%|██ | 13/64 [03:48<09:46, 11.49s/it][A
|
||
|
||
22%|██▏ | 14/64 [03:49<06:59, 8.38s/it][A
|
||
|
||
23%|██▎ | 15/64 [03:50<04:55, 6.02s/it][A
|
||
|
||
25%|██▌ | 16/64 [04:04<06:38, 8.30s/it][A
|
||
|
||
27%|██▋ | 17/64 [04:18<07:59, 10.21s/it][A
|
||
|
||
28%|██▊ | 18/64 [04:26<07:19, 9.55s/it][A
|
||
|
||
30%|██▉ | 19/64 [04:36<07:08, 9.53s/it][A
|
||
|
||
31%|███▏ | 20/64 [04:54<08:53, 12.13s/it][A
|
||
|
||
33%|███▎ | 21/64 [05:10<09:35, 13.39s/it][A
|
||
|
||
34%|███▍ | 22/64 [05:19<08:18, 11.87s/it][A
|
||
|
||
36%|███▌ | 23/64 [06:35<21:19, 31.20s/it][A
|
||
|
||
38%|███▊ | 24/64 [06:39<15:25, 23.15s/it][A
|
||
|
||
39%|███▉ | 25/64 [06:51<12:48, 19.70s/it][A
|
||
|
||
41%|████ | 26/64 [06:52<08:55, 14.09s/it][A
|
||
|
||
42%|████▏ | 27/64 [07:04<08:22, 13.57s/it][A
|
||
|
||
44%|████▍ | 28/64 [07:05<05:54, 9.86s/it][A
|
||
|
||
45%|████▌ | 29/64 [07:21<06:47, 11.65s/it][A
|
||
|
||
47%|████▋ | 30/64 [07:28<05:51, 10.34s/it][A
|
||
|
||
48%|████▊ | 31/64 [08:03<09:44, 17.71s/it][A
|
||
|
||
50%|█████ | 32/64 [08:07<07:15, 13.60s/it][A
|
||
|
||
52%|█████▏ | 33/64 [08:17<06:29, 12.55s/it][A
|
||
|
||
53%|█████▎ | 34/64 [08:23<05:13, 10.44s/it][A
|
||
|
||
55%|█████▍ | 35/64 [09:04<09:31, 19.71s/it][A
|
||
|
||
56%|█████▋ | 36/64 [09:32<10:14, 21.96s/it][A
|
||
|
||
58%|█████▊ | 37/64 [09:41<08:10, 18.15s/it][A
|
||
|
||
59%|█████▉ | 38/64 [09:49<06:36, 15.24s/it][A
|
||
|
||
61%|██████ | 39/64 [10:05<06:23, 15.36s/it][A
|
||
|
||
62%|██████▎ | 40/64 [10:43<08:49, 22.06s/it][A
|
||
|
||
64%|██████▍ | 41/64 [10:48<06:35, 17.21s/it][A
|
||
|
||
66%|██████▌ | 42/64 [10:55<05:11, 14.14s/it][A
|
||
|
||
67%|██████▋ | 43/64 [11:18<05:52, 16.79s/it][A
|
||
|
||
69%|██████▉ | 44/64 [11:27<04:45, 14.27s/it][A
|
||
|
||
70%|███████ | 45/64 [11:30<03:25, 10.79s/it][A
|
||
|
||
72%|███████▏ | 46/64 [11:42<03:22, 11.27s/it][A
|
||
|
||
73%|███████▎ | 47/64 [11:46<02:32, 9.00s/it][A
|
||
|
||
75%|███████▌ | 48/64 [12:04<03:10, 11.90s/it][A
|
||
|
||
77%|███████▋ | 49/64 [12:32<04:10, 16.69s/it][A
|
||
|
||
78%|███████▊ | 50/64 [12:55<04:19, 18.57s/it][A
|
||
|
||
80%|███████▉ | 51/64 [13:08<03:37, 16.73s/it][A
|
||
|
||
81%|████████▏ | 52/64 [13:26<03:26, 17.23s/it][A
|
||
|
||
83%|████████▎ | 53/64 [13:38<02:52, 15.64s/it][A
|
||
|
||
84%|████████▍ | 54/64 [13:42<02:00, 12.09s/it][A
|
||
|
||
86%|████████▌ | 55/64 [14:00<02:05, 13.90s/it][A
|
||
|
||
88%|████████▊ | 56/64 [14:18<02:01, 15.25s/it][A
|
||
|
||
89%|████████▉ | 57/64 [14:42<02:05, 17.89s/it][A
|
||
|
||
91%|█████████ | 58/64 [14:51<01:30, 15.09s/it][A
|
||
|
||
92%|█████████▏| 59/64 [15:00<01:06, 13.32s/it][A
|
||
|
||
94%|█████████▍| 60/64 [15:24<01:06, 16.67s/it][A
|
||
|
||
95%|█████████▌| 61/64 [15:41<00:49, 16.57s/it][A
|
||
|
||
97%|█████████▋| 62/64 [15:43<00:24, 12.41s/it][A
|
||
|
||
98%|█████████▊| 63/64 [15:52<00:11, 11.22s/it][A
|
||
|
||
100%|██████████| 64/64 [16:16<00:00, 15.05s/it][A
|
||
100%|██████████| 64/64 [16:16<00:00, 15.26s/it]
|
||
INFO:root:Time for 64 completions: 976.4 seconds
|
||
|
||
Chunking for generation: 62%|██████▏ | 8/13 [2:38:14<1:34:35, 1135.18s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
|
||
INFO:root:Found 512. Only generating 64 .
|
||
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.40s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.10s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.29s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/64 [00:00<?, ?it/s][A
|
||
|
||
2%|▏ | 1/64 [00:40<42:08, 40.14s/it][A
|
||
|
||
3%|▎ | 2/64 [00:43<19:12, 18.59s/it][A
|
||
|
||
5%|▍ | 3/64 [00:57<16:40, 16.40s/it][A
|
||
|
||
6%|▋ | 4/64 [01:26<21:21, 21.36s/it][A
|
||
|
||
8%|▊ | 5/64 [01:41<18:41, 19.01s/it][A
|
||
|
||
9%|▉ | 6/64 [02:03<19:17, 19.95s/it][A
|
||
|
||
11%|█ | 7/64 [02:14<16:21, 17.23s/it][A
|
||
|
||
12%|█▎ | 8/64 [02:36<17:24, 18.65s/it][A
|
||
|
||
14%|█▍ | 9/64 [02:59<18:20, 20.01s/it][A
|
||
|
||
16%|█▌ | 10/64 [03:20<18:24, 20.46s/it][A
|
||
|
||
17%|█▋ | 11/64 [03:56<22:11, 25.13s/it][A
|
||
|
||
19%|█▉ | 12/64 [04:13<19:41, 22.72s/it][A
|
||
|
||
20%|██ | 13/64 [04:45<21:43, 25.56s/it][A
|
||
|
||
22%|██▏ | 14/64 [04:57<17:43, 21.27s/it][A
|
||
|
||
23%|██▎ | 15/64 [06:16<31:38, 38.75s/it][A
|
||
|
||
25%|██▌ | 16/64 [06:20<22:32, 28.17s/it][A
|
||
|
||
27%|██▋ | 17/64 [06:25<16:42, 21.34s/it][A
|
||
|
||
28%|██▊ | 18/64 [06:47<16:30, 21.52s/it][A
|
||
|
||
30%|██▉ | 19/64 [06:58<13:45, 18.35s/it][A
|
||
|
||
31%|███▏ | 20/64 [07:07<11:21, 15.50s/it][A
|
||
|
||
33%|███▎ | 21/64 [07:29<12:36, 17.60s/it][A
|
||
|
||
34%|███▍ | 22/64 [07:59<14:49, 21.19s/it][A
|
||
|
||
36%|███▌ | 23/64 [08:09<12:14, 17.91s/it][A
|
||
|
||
38%|███▊ | 24/64 [08:42<14:52, 22.32s/it][A
|
||
|
||
39%|███▉ | 25/64 [09:30<19:37, 30.20s/it][A
|
||
|
||
41%|████ | 26/64 [09:33<13:49, 21.83s/it][A
|
||
|
||
42%|████▏ | 27/64 [10:26<19:17, 31.28s/it][A
|
||
|
||
44%|████▍ | 28/64 [11:07<20:37, 34.37s/it][A
|
||
|
||
45%|████▌ | 29/64 [11:13<14:58, 25.66s/it][A
|
||
|
||
47%|████▋ | 30/64 [11:27<12:32, 22.12s/it][A
|
||
|
||
48%|████▊ | 31/64 [11:29<08:56, 16.26s/it][A
|
||
|
||
50%|█████ | 32/64 [12:11<12:41, 23.81s/it][A
|
||
|
||
52%|█████▏ | 33/64 [12:29<11:30, 22.27s/it][A
|
||
|
||
53%|█████▎ | 34/64 [12:33<08:17, 16.57s/it][A
|
||
|
||
55%|█████▍ | 35/64 [12:43<07:06, 14.70s/it][A
|
||
|
||
56%|█████▋ | 36/64 [12:57<06:41, 14.35s/it][A
|
||
|
||
58%|█████▊ | 37/64 [13:17<07:17, 16.19s/it][A
|
||
|
||
59%|█████▉ | 38/64 [13:20<05:21, 12.35s/it][A
|
||
|
||
61%|██████ | 39/64 [13:57<08:09, 19.59s/it][A
|
||
|
||
62%|██████▎ | 40/64 [14:25<08:49, 22.05s/it][A
|
||
|
||
64%|██████▍ | 41/64 [14:30<06:34, 17.14s/it][A
|
||
|
||
66%|██████▌ | 42/64 [14:40<05:30, 15.02s/it][A
|
||
|
||
67%|██████▋ | 43/64 [14:49<04:32, 12.96s/it][A
|
||
|
||
69%|██████▉ | 44/64 [14:52<03:23, 10.18s/it][A
|
||
|
||
70%|███████ | 45/64 [15:15<04:26, 14.02s/it][A
|
||
|
||
72%|███████▏ | 46/64 [15:43<05:26, 18.14s/it][A
|
||
|
||
73%|███████▎ | 47/64 [15:57<04:48, 16.96s/it][A
|
||
|
||
75%|███████▌ | 48/64 [16:12<04:20, 16.25s/it][A
|
||
|
||
77%|███████▋ | 49/64 [16:33<04:26, 17.80s/it][A
|
||
|
||
78%|███████▊ | 50/64 [17:16<05:54, 25.32s/it][A
|
||
|
||
80%|███████▉ | 51/64 [17:44<05:40, 26.18s/it][A
|
||
|
||
81%|████████▏ | 52/64 [18:00<04:36, 23.06s/it][A
|
||
|
||
83%|████████▎ | 53/64 [18:17<03:54, 21.29s/it][A
|
||
|
||
84%|████████▍ | 54/64 [18:32<03:14, 19.42s/it][A
|
||
|
||
86%|████████▌ | 55/64 [18:52<02:54, 19.41s/it][A
|
||
|
||
88%|████████▊ | 56/64 [19:14<02:43, 20.40s/it][A
|
||
|
||
89%|████████▉ | 57/64 [19:29<02:10, 18.57s/it][A
|
||
|
||
91%|█████████ | 58/64 [19:41<01:40, 16.73s/it][A
|
||
|
||
92%|█████████▏| 59/64 [20:14<01:48, 21.71s/it][A
|
||
|
||
94%|█████████▍| 60/64 [20:20<01:07, 16.96s/it][A
|
||
|
||
95%|█████████▌| 61/64 [20:45<00:58, 19.41s/it][A
|
||
|
||
97%|█████████▋| 62/64 [21:03<00:37, 18.92s/it][A
|
||
|
||
98%|█████████▊| 63/64 [21:20<00:18, 18.29s/it][A
|
||
|
||
100%|██████████| 64/64 [21:26<00:00, 14.57s/it][A
|
||
100%|██████████| 64/64 [21:26<00:00, 20.10s/it]
|
||
INFO:root:Time for 64 completions: 1286.4 seconds
|
||
|
||
Chunking for generation: 69%|██████▉ | 9/13 [2:59:47<1:18:58, 1184.67s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
|
||
INFO:root:Found 576. Only generating 64 .
|
||
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.49s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.18s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.38s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/64 [00:00<?, ?it/s][A
|
||
|
||
2%|▏ | 1/64 [00:27<28:40, 27.32s/it][A
|
||
|
||
3%|▎ | 2/64 [01:44<58:28, 56.59s/it][A
|
||
|
||
5%|▍ | 3/64 [02:10<43:31, 42.82s/it][A
|
||
|
||
6%|▋ | 4/64 [02:33<34:43, 34.72s/it][A
|
||
|
||
8%|▊ | 5/64 [02:56<30:02, 30.55s/it][A
|
||
|
||
9%|▉ | 6/64 [03:24<28:39, 29.64s/it][A
|
||
|
||
11%|█ | 7/64 [03:44<25:18, 26.63s/it][A
|
||
|
||
12%|█▎ | 8/64 [04:13<25:38, 27.47s/it][A
|
||
|
||
14%|█▍ | 9/64 [04:48<27:11, 29.66s/it][A
|
||
|
||
16%|█▌ | 10/64 [05:06<23:32, 26.16s/it][A
|
||
|
||
17%|█▋ | 11/64 [05:29<22:19, 25.27s/it][A
|
||
|
||
19%|█▉ | 12/64 [05:37<17:20, 20.00s/it][A
|
||
|
||
20%|██ | 13/64 [05:44<13:34, 15.97s/it][A
|
||
|
||
22%|██▏ | 14/64 [06:03<14:02, 16.85s/it][A
|
||
|
||
23%|██▎ | 15/64 [06:22<14:22, 17.60s/it][A
|
||
|
||
25%|██▌ | 16/64 [06:47<15:40, 19.59s/it][A
|
||
|
||
27%|██▋ | 17/64 [07:05<15:07, 19.31s/it][A
|
||
|
||
28%|██▊ | 18/64 [07:23<14:31, 18.94s/it][A
|
||
|
||
30%|██▉ | 19/64 [07:34<12:22, 16.51s/it][A
|
||
|
||
31%|███▏ | 20/64 [08:09<16:06, 21.96s/it][A
|
||
|
||
33%|███▎ | 21/64 [08:20<13:31, 18.88s/it][A
|
||
|
||
34%|███▍ | 22/64 [08:23<09:51, 14.08s/it][A
|
||
|
||
36%|███▌ | 23/64 [08:25<07:06, 10.41s/it][A
|
||
|
||
38%|███▊ | 24/64 [08:37<07:08, 10.71s/it][A
|
||
|
||
39%|███▉ | 25/64 [08:39<05:16, 8.12s/it][A
|
||
|
||
41%|████ | 26/64 [09:02<08:02, 12.69s/it][A
|
||
|
||
42%|████▏ | 27/64 [09:09<06:40, 10.83s/it][A
|
||
|
||
44%|████▍ | 28/64 [09:21<06:44, 11.23s/it][A
|
||
|
||
45%|████▌ | 29/64 [09:34<06:55, 11.88s/it][A
|
||
|
||
47%|████▋ | 30/64 [09:48<07:06, 12.56s/it][A
|
||
|
||
48%|████▊ | 31/64 [10:06<07:48, 14.20s/it][A
|
||
|
||
50%|█████ | 32/64 [10:49<12:07, 22.73s/it][A
|
||
|
||
52%|█████▏ | 33/64 [10:52<08:40, 16.81s/it][A
|
||
|
||
53%|█████▎ | 34/64 [11:19<09:57, 19.93s/it][A
|
||
|
||
55%|█████▍ | 35/64 [11:26<07:41, 15.92s/it][A
|
||
|
||
56%|█████▋ | 36/64 [11:50<08:36, 18.43s/it][A
|
||
|
||
58%|█████▊ | 37/64 [11:55<06:32, 14.54s/it][A
|
||
|
||
59%|█████▉ | 38/64 [12:09<06:09, 14.21s/it][A
|
||
|
||
61%|██████ | 39/64 [12:34<07:17, 17.49s/it][A
|
||
|
||
62%|██████▎ | 40/64 [12:47<06:30, 16.26s/it][A
|
||
|
||
64%|██████▍ | 41/64 [12:52<04:56, 12.91s/it][A
|
||
|
||
66%|██████▌ | 42/64 [13:00<04:05, 11.18s/it][A
|
||
|
||
67%|██████▋ | 43/64 [13:23<05:11, 14.83s/it][A
|
||
|
||
69%|██████▉ | 44/64 [13:40<05:08, 15.41s/it][A
|
||
|
||
70%|███████ | 45/64 [13:43<03:42, 11.70s/it][A
|
||
|
||
72%|███████▏ | 46/64 [13:56<03:38, 12.15s/it][A
|
||
|
||
73%|███████▎ | 47/64 [14:01<02:48, 9.90s/it][A
|
||
|
||
75%|███████▌ | 48/64 [14:13<02:51, 10.73s/it][A
|
||
|
||
77%|███████▋ | 49/64 [15:30<07:37, 30.52s/it][A
|
||
|
||
78%|███████▊ | 50/64 [15:44<05:56, 25.48s/it][A
|
||
|
||
80%|███████▉ | 51/64 [15:45<03:56, 18.19s/it][A
|
||
|
||
81%|████████▏ | 52/64 [15:47<02:41, 13.47s/it][A
|
||
|
||
83%|████████▎ | 53/64 [15:52<01:59, 10.83s/it][A
|
||
|
||
84%|████████▍ | 54/64 [15:54<01:21, 8.10s/it][A
|
||
|
||
86%|████████▌ | 55/64 [15:55<00:55, 6.12s/it][A
|
||
|
||
88%|████████▊ | 56/64 [15:59<00:44, 5.53s/it][A
|
||
|
||
89%|████████▉ | 57/64 [16:01<00:29, 4.25s/it][A
|
||
|
||
91%|█████████ | 58/64 [16:05<00:25, 4.23s/it][A
|
||
|
||
92%|█████████▏| 59/64 [16:06<00:16, 3.39s/it][A
|
||
|
||
94%|█████████▍| 60/64 [16:08<00:11, 2.85s/it][A
|
||
|
||
95%|█████████▌| 61/64 [16:13<00:10, 3.42s/it][A
|
||
|
||
97%|█████████▋| 62/64 [16:15<00:05, 2.98s/it][A
|
||
|
||
98%|█████████▊| 63/64 [16:15<00:02, 2.13s/it][A
|
||
|
||
100%|██████████| 64/64 [16:22<00:00, 3.58s/it][A
|
||
100%|██████████| 64/64 [16:22<00:00, 15.35s/it]
|
||
INFO:root:Time for 64 completions: 982.2 seconds
|
||
|
||
Chunking for generation: 77%|███████▋ | 10/13 [3:16:17<56:13, 1124.40s/it] INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
|
||
INFO:root:Found 640. Only generating 64 .
|
||
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.44s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.13s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.33s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/64 [00:00<?, ?it/s][A
|
||
|
||
2%|▏ | 1/64 [00:01<02:00, 1.91s/it][A
|
||
|
||
3%|▎ | 2/64 [00:13<07:54, 7.66s/it][A
|
||
|
||
5%|▍ | 3/64 [00:31<12:38, 12.43s/it][A
|
||
|
||
6%|▋ | 4/64 [00:38<10:02, 10.04s/it][A
|
||
|
||
8%|▊ | 5/64 [00:56<12:54, 13.13s/it][A
|
||
|
||
9%|▉ | 6/64 [02:14<33:51, 35.03s/it][A
|
||
|
||
11%|█ | 7/64 [02:29<27:00, 28.43s/it][A
|
||
|
||
12%|█▎ | 8/64 [02:39<21:19, 22.85s/it][A
|
||
|
||
14%|█▍ | 9/64 [02:50<17:21, 18.93s/it][A
|
||
|
||
16%|█▌ | 10/64 [02:54<13:02, 14.48s/it][A
|
||
|
||
17%|█▋ | 11/64 [03:22<16:21, 18.51s/it][A
|
||
|
||
19%|█▉ | 12/64 [03:28<12:48, 14.78s/it][A
|
||
|
||
20%|██ | 13/64 [03:42<12:22, 14.57s/it][A
|
||
|
||
22%|██▏ | 14/64 [04:07<14:42, 17.66s/it][A
|
||
|
||
23%|██▎ | 15/64 [04:13<11:29, 14.06s/it][A
|
||
|
||
25%|██▌ | 16/64 [04:36<13:25, 16.78s/it][A
|
||
|
||
27%|██▋ | 17/64 [04:38<09:37, 12.29s/it][A
|
||
|
||
28%|██▊ | 18/64 [04:39<06:46, 8.84s/it][A
|
||
|
||
30%|██▉ | 19/64 [04:42<05:22, 7.17s/it][A
|
||
|
||
31%|███▏ | 20/64 [04:58<07:11, 9.81s/it][A
|
||
|
||
33%|███▎ | 21/64 [04:59<05:08, 7.17s/it][A
|
||
|
||
34%|███▍ | 22/64 [04:59<03:37, 5.17s/it][A
|
||
|
||
36%|███▌ | 23/64 [05:02<02:57, 4.33s/it][A
|
||
|
||
38%|███▊ | 24/64 [05:07<03:06, 4.67s/it][A
|
||
|
||
39%|███▉ | 25/64 [05:19<04:31, 6.95s/it][A
|
||
|
||
41%|████ | 26/64 [05:46<08:06, 12.80s/it][A
|
||
|
||
42%|████▏ | 27/64 [05:46<05:35, 9.07s/it][A
|
||
|
||
44%|████▍ | 28/64 [05:48<04:04, 6.78s/it][A
|
||
|
||
45%|████▌ | 29/64 [05:49<02:56, 5.04s/it][A
|
||
|
||
47%|████▋ | 30/64 [05:50<02:13, 3.92s/it][A
|
||
|
||
48%|████▊ | 31/64 [05:51<01:40, 3.04s/it][A
|
||
|
||
50%|█████ | 32/64 [06:12<04:31, 8.48s/it][A
|
||
|
||
52%|█████▏ | 33/64 [06:14<03:24, 6.61s/it][A
|
||
|
||
53%|█████▎ | 34/64 [06:15<02:26, 4.90s/it][A
|
||
|
||
55%|█████▍ | 35/64 [06:20<02:18, 4.77s/it][A
|
||
|
||
56%|█████▋ | 36/64 [06:24<02:09, 4.62s/it][A
|
||
|
||
58%|█████▊ | 37/64 [06:28<02:00, 4.47s/it][A
|
||
|
||
59%|█████▉ | 38/64 [06:29<01:30, 3.50s/it][A
|
||
|
||
61%|██████ | 39/64 [06:34<01:32, 3.69s/it][A
|
||
|
||
62%|██████▎ | 40/64 [06:36<01:22, 3.42s/it][A
|
||
|
||
64%|██████▍ | 41/64 [06:39<01:13, 3.20s/it][A
|
||
|
||
66%|██████▌ | 42/64 [06:52<02:12, 6.04s/it][A
|
||
|
||
67%|██████▋ | 43/64 [07:20<04:26, 12.68s/it][A
|
||
|
||
69%|██████▉ | 44/64 [07:41<05:03, 15.20s/it][A
|
||
|
||
70%|███████ | 45/64 [08:06<05:44, 18.15s/it][A
|
||
|
||
72%|███████▏ | 46/64 [08:15<04:35, 15.32s/it][A
|
||
|
||
73%|███████▎ | 47/64 [08:30<04:20, 15.34s/it][A
|
||
|
||
75%|███████▌ | 48/64 [08:56<04:56, 18.54s/it][A
|
||
|
||
77%|███████▋ | 49/64 [09:12<04:28, 17.90s/it][A
|
||
|
||
78%|███████▊ | 50/64 [09:30<04:10, 17.87s/it][A
|
||
|
||
80%|███████▉ | 51/64 [09:46<03:45, 17.33s/it][A
|
||
|
||
81%|████████▏ | 52/64 [10:03<03:26, 17.24s/it][A
|
||
|
||
83%|████████▎ | 53/64 [10:19<03:05, 16.85s/it][A
|
||
|
||
84%|████████▍ | 54/64 [10:20<02:01, 12.12s/it][A
|
||
|
||
86%|████████▌ | 55/64 [10:46<02:26, 16.24s/it][A
|
||
|
||
88%|████████▊ | 56/64 [11:18<02:47, 20.98s/it][A
|
||
|
||
89%|████████▉ | 57/64 [11:32<02:10, 18.66s/it][A
|
||
|
||
91%|█████████ | 58/64 [11:34<01:22, 13.81s/it][A
|
||
|
||
92%|█████████▏| 59/64 [11:37<00:52, 10.46s/it][A
|
||
|
||
94%|█████████▍| 60/64 [11:42<00:35, 8.92s/it][A
|
||
|
||
95%|█████████▌| 61/64 [11:48<00:24, 8.01s/it][A
|
||
|
||
97%|█████████▋| 62/64 [12:05<00:21, 10.69s/it][A
|
||
|
||
98%|█████████▊| 63/64 [12:10<00:09, 9.11s/it][A
|
||
|
||
100%|██████████| 64/64 [12:28<00:00, 11.57s/it][A
|
||
100%|██████████| 64/64 [12:28<00:00, 11.69s/it]
|
||
INFO:root:Time for 64 completions: 748.1 seconds
|
||
|
||
Chunking for generation: 85%|████████▍ | 11/13 [3:28:52<33:42, 1011.41s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
|
||
INFO:root:Found 704. Only generating 64 .
|
||
INFO:root:Using `huggingface_local_completions` on 64 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.40s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.11s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.30s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/64 [00:00<?, ?it/s][A
|
||
|
||
2%|▏ | 1/64 [00:15<16:16, 15.50s/it][A
|
||
|
||
3%|▎ | 2/64 [00:16<07:24, 7.18s/it][A
|
||
|
||
5%|▍ | 3/64 [00:39<14:26, 14.21s/it][A
|
||
|
||
6%|▋ | 4/64 [00:42<09:41, 9.69s/it][A
|
||
|
||
8%|▊ | 5/64 [00:43<06:28, 6.58s/it][A
|
||
|
||
9%|▉ | 6/64 [00:45<04:55, 5.09s/it][A
|
||
|
||
11%|█ | 7/64 [01:07<10:02, 10.57s/it][A
|
||
|
||
12%|█▎ | 8/64 [01:08<06:59, 7.49s/it][A
|
||
|
||
14%|█▍ | 9/64 [01:16<07:12, 7.87s/it][A
|
||
|
||
16%|█▌ | 10/64 [01:17<05:01, 5.59s/it][A
|
||
|
||
17%|█▋ | 11/64 [01:20<04:12, 4.77s/it][A
|
||
|
||
19%|█▉ | 12/64 [01:28<04:56, 5.70s/it][A
|
||
|
||
20%|██ | 13/64 [01:29<03:39, 4.30s/it][A
|
||
|
||
22%|██▏ | 14/64 [01:32<03:18, 3.98s/it][A
|
||
|
||
23%|██▎ | 15/64 [01:35<03:00, 3.69s/it][A
|
||
|
||
25%|██▌ | 16/64 [01:43<03:59, 4.99s/it][A
|
||
|
||
27%|██▋ | 17/64 [01:45<03:06, 3.96s/it][A
|
||
|
||
28%|██▊ | 18/64 [02:07<07:11, 9.38s/it][A
|
||
|
||
30%|██▉ | 19/64 [02:16<06:59, 9.32s/it][A
|
||
|
||
31%|███▏ | 20/64 [02:16<04:54, 6.70s/it][A
|
||
|
||
33%|███▎ | 21/64 [02:25<05:12, 7.27s/it][A
|
||
|
||
34%|███▍ | 22/64 [02:49<08:38, 12.35s/it][A
|
||
|
||
36%|███▌ | 23/64 [03:04<09:02, 13.23s/it][A
|
||
|
||
38%|███▊ | 24/64 [03:32<11:39, 17.48s/it][A
|
||
|
||
39%|███▉ | 25/64 [03:47<10:59, 16.91s/it][A
|
||
|
||
41%|████ | 26/64 [04:07<11:08, 17.58s/it][A
|
||
|
||
42%|████▏ | 27/64 [04:30<12:01, 19.50s/it][A
|
||
|
||
44%|████▍ | 28/64 [04:51<11:50, 19.74s/it][A
|
||
|
||
45%|████▌ | 29/64 [05:18<12:49, 21.99s/it][A
|
||
|
||
47%|████▋ | 30/64 [05:42<12:51, 22.69s/it][A
|
||
|
||
48%|████▊ | 31/64 [06:00<11:40, 21.22s/it][A
|
||
|
||
50%|█████ | 32/64 [06:19<10:55, 20.47s/it][A
|
||
|
||
52%|█████▏ | 33/64 [06:50<12:17, 23.78s/it][A
|
||
|
||
53%|█████▎ | 34/64 [07:17<12:18, 24.61s/it][A
|
||
|
||
55%|█████▍ | 35/64 [07:41<11:52, 24.56s/it][A
|
||
|
||
56%|█████▋ | 36/64 [08:03<10:59, 23.56s/it][A
|
||
|
||
58%|█████▊ | 37/64 [08:33<11:32, 25.65s/it][A
|
||
|
||
59%|█████▉ | 38/64 [08:55<10:40, 24.62s/it][A
|
||
|
||
61%|██████ | 39/64 [09:16<09:48, 23.55s/it][A
|
||
|
||
62%|██████▎ | 40/64 [09:40<09:27, 23.66s/it][A
|
||
|
||
64%|██████▍ | 41/64 [10:02<08:47, 22.95s/it][A
|
||
|
||
66%|██████▌ | 42/64 [10:08<06:35, 17.98s/it][A
|
||
|
||
67%|██████▋ | 43/64 [10:19<05:34, 15.92s/it][A
|
||
|
||
69%|██████▉ | 44/64 [10:28<04:38, 13.92s/it][A
|
||
|
||
70%|███████ | 45/64 [10:38<04:02, 12.78s/it][A
|
||
|
||
72%|███████▏ | 46/64 [11:00<04:36, 15.35s/it][A
|
||
|
||
73%|███████▎ | 47/64 [11:15<04:20, 15.32s/it][A
|
||
|
||
75%|███████▌ | 48/64 [11:38<04:43, 17.70s/it][A
|
||
|
||
77%|███████▋ | 49/64 [11:53<04:10, 16.67s/it][A
|
||
|
||
78%|███████▊ | 50/64 [12:15<04:18, 18.44s/it][A
|
||
|
||
80%|███████▉ | 51/64 [12:38<04:16, 19.69s/it][A
|
||
|
||
81%|████████▏ | 52/64 [12:56<03:49, 19.12s/it][A
|
||
|
||
83%|████████▎ | 53/64 [13:16<03:34, 19.54s/it][A
|
||
|
||
84%|████████▍ | 54/64 [13:48<03:52, 23.21s/it][A
|
||
|
||
86%|████████▌ | 55/64 [14:06<03:15, 21.74s/it][A
|
||
|
||
88%|████████▊ | 56/64 [14:28<02:53, 21.72s/it][A
|
||
|
||
89%|████████▉ | 57/64 [14:49<02:30, 21.45s/it][A
|
||
|
||
91%|█████████ | 58/64 [15:03<01:55, 19.21s/it][A
|
||
|
||
92%|█████████▏| 59/64 [15:28<01:45, 21.11s/it][A
|
||
|
||
94%|█████████▍| 60/64 [15:34<01:06, 16.55s/it][A
|
||
|
||
95%|█████████▌| 61/64 [15:41<00:40, 13.65s/it][A
|
||
|
||
97%|█████████▋| 62/64 [15:57<00:28, 14.27s/it][A
|
||
|
||
98%|█████████▊| 63/64 [16:19<00:16, 16.62s/it][A
|
||
|
||
100%|██████████| 64/64 [16:40<00:00, 17.93s/it][A
|
||
100%|██████████| 64/64 [16:40<00:00, 15.63s/it]
|
||
INFO:root:Time for 64 completions: 1000.2 seconds
|
||
|
||
Chunking for generation: 92%|█████████▏| 12/13 [3:45:39<16:50, 1010.18s/it]INFO:root:Loading outputs from output/chatgpt_fn_--LMCocktail-7B-v1/model_outputs.json
|
||
INFO:root:Found 768. Only generating 37 .
|
||
INFO:root:Using `huggingface_local_completions` on 37 prompts using /media/hangyu5/Home/Documents/Hugging-Face/LM_cocktail/LMCocktail-7B-v1.
|
||
|
||
|
||
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s][A
|
||
|
||
Loading checkpoint shards: 50%|█████ | 1/2 [00:04<00:04, 4.34s/it][A
|
||
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.08s/it][A
|
||
Loading checkpoint shards: 100%|██████████| 2/2 [00:06<00:00, 3.27s/it]
|
||
INFO:root:Model memory: 15.020343296 GB
|
||
INFO:root:Kwargs to completion: {'do_sample': True, 'model_kwargs': {'torch_dtype': torch.bfloat16, 'device_map': 'auto'}, 'batch_size': 1, 'max_new_tokens': 2048, 'temperature': 0.7, 'top_p': 1.0}
|
||
|
||
|
||
0%| | 0/37 [00:00<?, ?it/s][A
|
||
|
||
3%|▎ | 1/37 [00:18<10:55, 18.21s/it][A
|
||
|
||
5%|▌ | 2/37 [00:34<09:57, 17.08s/it][A
|
||
|
||
8%|▊ | 3/37 [00:51<09:33, 16.88s/it][A
|
||
|
||
11%|█ | 4/37 [01:24<12:53, 23.45s/it][A
|
||
|
||
14%|█▎ | 5/37 [01:47<12:26, 23.32s/it][A
|
||
|
||
16%|█▌ | 6/37 [02:06<11:11, 21.65s/it][A
|
||
|
||
19%|█▉ | 7/37 [02:30<11:20, 22.69s/it][A
|
||
|
||
22%|██▏ | 8/37 [02:54<11:02, 22.85s/it][A
|
||
|
||
24%|██▍ | 9/37 [03:22<11:27, 24.56s/it][A
|
||
|
||
27%|██▋ | 10/37 [03:50<11:31, 25.59s/it][A
|
||
|
||
30%|██▉ | 11/37 [04:10<10:22, 23.93s/it][A
|
||
|
||
32%|███▏ | 12/37 [04:37<10:21, 24.85s/it][A
|
||
|
||
35%|███▌ | 13/37 [04:52<08:43, 21.80s/it][A
|
||
|
||
38%|███▊ | 14/37 [05:09<07:49, 20.41s/it][A
|
||
|
||
41%|████ | 15/37 [05:24<06:55, 18.91s/it][A
|
||
|
||
43%|████▎ | 16/37 [05:46<06:56, 19.82s/it][A
|
||
|
||
46%|████▌ | 17/37 [06:17<07:40, 23.01s/it][A
|
||
|
||
49%|████▊ | 18/37 [06:37<06:58, 22.03s/it][A
|
||
|
||
51%|█████▏ | 19/37 [06:56<06:24, 21.35s/it][A
|
||
|
||
54%|█████▍ | 20/37 [07:09<05:20, 18.87s/it][A
|
||
|
||
57%|█████▋ | 21/37 [07:20<04:23, 16.44s/it][A
|
||
|
||
59%|█████▉ | 22/37 [07:36<04:04, 16.32s/it][A
|
||
|
||
62%|██████▏ | 23/37 [08:00<04:18, 18.49s/it][A
|
||
|
||
65%|██████▍ | 24/37 [08:32<04:55, 22.73s/it][A
|
||
|
||
68%|██████▊ | 25/37 [08:37<03:26, 17.22s/it][A
|
||
|
||
70%|███████ | 26/37 [08:42<02:29, 13.58s/it][A
|
||
|
||
73%|███████▎ | 27/37 [08:49<01:56, 11.66s/it][A
|
||
|
||
76%|███████▌ | 28/37 [09:08<02:05, 13.93s/it][A
|
||
|
||
78%|███████▊ | 29/37 [09:22<01:51, 13.95s/it][A
|
||
|
||
81%|████████ | 30/37 [09:50<02:06, 18.01s/it][A
|
||
|
||
84%|████████▍ | 31/37 [10:09<01:50, 18.48s/it][A
|
||
|
||
86%|████████▋ | 32/37 [10:24<01:26, 17.31s/it][A
|
||
|
||
89%|████████▉ | 33/37 [11:11<01:45, 26.38s/it][A
|
||
|
||
92%|█████████▏| 34/37 [11:54<01:33, 31.26s/it][A
|
||
|
||
95%|█████████▍| 35/37 [12:28<01:04, 32.20s/it][A
|
||
|
||
97%|█████████▋| 36/37 [13:04<00:33, 33.08s/it][A
|
||
|
||
100%|██████████| 37/37 [13:38<00:00, 33.61s/it][A
|
||
100%|██████████| 37/37 [13:38<00:00, 22.13s/it]
|
||
INFO:root:Time for 37 completions: 819.0 seconds
|
||
|
||
Chunking for generation: 100%|██████████| 13/13 [3:59:25<00:00, 954.40s/it]
|
||
Chunking for generation: 100%|██████████| 13/13 [3:59:25<00:00, 1105.07s/it]
|
||
INFO:root:Evaluating the LMCocktail-7B-v1 outputs.
|
||
INFO:root:Creating the annotator from `chatgpt_fn`.
|
||
INFO:root:Saving annotations to `/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json`.
|
||
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
|
||
Annotation chunk: 0%| | 0/7 [00:00<?, ?it/s]INFO:root:Annotating 128 examples with chatgpt_fn
|
||
INFO:root:Using `openai_completions` on 128 prompts using gpt-3.5-turbo-16k-0613.
|
||
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5
|
||
|
||
|
||
prompt_batches: 0%| | 0/128 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 1%| | 1/128 [00:01<02:19, 1.10s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 3%|▎ | 4/128 [00:01<00:32, 3.79it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 5%|▍ | 6/128 [00:02<00:44, 2.72it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 5%|▌ | 7/128 [00:02<00:38, 3.12it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 8%|▊ | 10/128 [00:02<00:21, 5.40it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 9%|▉ | 12/128 [00:05<00:59, 1.96it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 16%|█▌ | 20/128 [00:07<00:42, 2.53it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 20%|█▉ | 25/128 [00:09<00:36, 2.80it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 23%|██▎ | 30/128 [00:09<00:26, 3.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 24%|██▍ | 31/128 [00:09<00:26, 3.62it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 26%|██▌ | 33/128 [00:10<00:26, 3.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 28%|██▊ | 36/128 [00:11<00:23, 3.87it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 30%|██▉ | 38/128 [00:11<00:19, 4.60it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 30%|███ | 39/128 [00:11<00:21, 4.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 31%|███▏ | 40/128 [00:11<00:19, 4.54it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 32%|███▏ | 41/128 [00:12<00:24, 3.52it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 33%|███▎ | 42/128 [00:12<00:22, 3.91it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 34%|███▍ | 44/128 [00:12<00:16, 5.03it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 35%|███▌ | 45/128 [00:12<00:16, 4.95it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 36%|███▌ | 46/128 [00:13<00:22, 3.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 38%|███▊ | 48/128 [00:13<00:17, 4.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 38%|███▊ | 49/128 [00:13<00:15, 4.94it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 39%|███▉ | 50/128 [00:14<00:15, 4.96it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 40%|███▉ | 51/128 [00:14<00:22, 3.47it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 41%|████▏ | 53/128 [00:14<00:16, 4.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 42%|████▏ | 54/128 [00:15<00:14, 5.05it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 43%|████▎ | 55/128 [00:15<00:14, 5.14it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 44%|████▍ | 56/128 [00:15<00:19, 3.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 45%|████▌ | 58/128 [00:15<00:15, 4.51it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 46%|████▌ | 59/128 [00:16<00:18, 3.82it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 48%|████▊ | 61/128 [00:16<00:17, 3.87it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 49%|████▉ | 63/128 [00:17<00:12, 5.22it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 50%|█████ | 64/128 [00:18<00:29, 2.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 54%|█████▍ | 69/128 [00:18<00:12, 4.70it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 55%|█████▍ | 70/128 [00:19<00:13, 4.27it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 56%|█████▋ | 72/128 [00:19<00:13, 4.14it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 58%|█████▊ | 74/128 [00:19<00:10, 5.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 59%|█████▊ | 75/128 [00:20<00:12, 4.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 60%|██████ | 77/128 [00:20<00:13, 3.78it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 62%|██████▎ | 80/128 [00:21<00:09, 5.30it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 63%|██████▎ | 81/128 [00:21<00:08, 5.33it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 64%|██████▍ | 82/128 [00:21<00:12, 3.67it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 65%|██████▍ | 83/128 [00:22<00:11, 4.02it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 66%|██████▋ | 85/128 [00:22<00:12, 3.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 68%|██████▊ | 87/128 [00:23<00:09, 4.16it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 70%|██████▉ | 89/128 [00:23<00:08, 4.76it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 71%|███████ | 91/128 [00:23<00:07, 4.87it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 72%|███████▏ | 92/128 [00:24<00:09, 3.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 73%|███████▎ | 93/128 [00:26<00:21, 1.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 79%|███████▉ | 101/128 [00:26<00:05, 4.76it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 80%|████████ | 103/128 [00:27<00:05, 4.30it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 83%|████████▎ | 106/128 [00:27<00:04, 4.89it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 84%|████████▎ | 107/128 [00:28<00:04, 4.26it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 84%|████████▍ | 108/128 [00:28<00:04, 4.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 87%|████████▋ | 111/128 [00:28<00:03, 5.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 88%|████████▊ | 112/128 [00:29<00:03, 4.05it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 88%|████████▊ | 113/128 [00:29<00:03, 3.95it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 91%|█████████ | 116/128 [00:29<00:02, 5.38it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 91%|█████████▏| 117/128 [00:30<00:02, 3.99it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:openai._base_client:Retrying request to /chat/completions in 0.750557 seconds
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 92%|█████████▏| 118/128 [00:36<00:13, 1.36s/it][A
|
||
prompt_batches: 100%|██████████| 128/128 [00:36<00:00, 3.53it/s]
|
||
INFO:root:Completed 128 examples in 36.5 seconds.
|
||
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
|
||
Annotation chunk: 14%|█▍ | 1/7 [00:36<03:41, 36.98s/it]INFO:root:Annotating 127 examples with chatgpt_fn
|
||
INFO:root:Using `openai_completions` on 127 prompts using gpt-3.5-turbo-16k-0613.
|
||
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5
|
||
|
||
|
||
prompt_batches: 0%| | 0/127 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 1%| | 1/127 [00:01<03:41, 1.76s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 5%|▍ | 6/127 [00:02<00:37, 3.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 6%|▌ | 7/127 [00:02<00:32, 3.64it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 7%|▋ | 9/127 [00:02<00:25, 4.60it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 9%|▊ | 11/127 [00:03<00:31, 3.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 11%|█ | 14/127 [00:03<00:24, 4.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 12%|█▏ | 15/127 [00:03<00:22, 5.01it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 13%|█▎ | 16/127 [00:04<00:29, 3.72it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 14%|█▍ | 18/127 [00:04<00:21, 4.97it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 15%|█▍ | 19/127 [00:05<00:30, 3.52it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 17%|█▋ | 21/127 [00:05<00:21, 4.85it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 17%|█▋ | 22/127 [00:05<00:19, 5.34it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 18%|█▊ | 23/127 [00:06<00:33, 3.12it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 20%|█▉ | 25/127 [00:06<00:24, 4.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 21%|██▏ | 27/127 [00:06<00:17, 5.64it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 22%|██▏ | 28/127 [00:07<00:35, 2.79it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 26%|██▌ | 33/127 [00:08<00:23, 4.08it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 27%|██▋ | 34/127 [00:08<00:21, 4.38it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 28%|██▊ | 36/127 [00:09<00:18, 4.80it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 30%|██▉ | 38/127 [00:10<00:28, 3.07it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 34%|███▍ | 43/127 [00:11<00:22, 3.69it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 38%|███▊ | 48/127 [00:12<00:18, 4.26it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 39%|███▉ | 50/127 [00:12<00:16, 4.56it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 42%|████▏ | 53/127 [00:13<00:17, 4.30it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 43%|████▎ | 55/127 [00:15<00:29, 2.47it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 50%|████▉ | 63/127 [00:15<00:12, 5.03it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 51%|█████ | 65/127 [00:16<00:12, 4.90it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 52%|█████▏ | 66/127 [00:16<00:14, 4.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 54%|█████▎ | 68/127 [00:16<00:11, 5.03it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 55%|█████▌ | 70/127 [00:17<00:11, 4.84it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 56%|█████▌ | 71/127 [00:17<00:12, 4.52it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 57%|█████▋ | 72/127 [00:17<00:12, 4.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 57%|█████▋ | 73/127 [00:18<00:14, 3.79it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 59%|█████▉ | 75/127 [00:18<00:09, 5.36it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 60%|█████▉ | 76/127 [00:18<00:10, 4.74it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 61%|██████ | 77/127 [00:19<00:13, 3.76it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 61%|██████▏ | 78/127 [00:19<00:14, 3.27it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 64%|██████▍ | 81/127 [00:19<00:08, 5.21it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 65%|██████▍ | 82/127 [00:20<00:10, 4.38it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 65%|██████▌ | 83/127 [00:20<00:12, 3.67it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 68%|██████▊ | 86/127 [00:20<00:08, 4.93it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 69%|██████▊ | 87/127 [00:21<00:10, 3.86it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 69%|██████▉ | 88/127 [00:21<00:09, 4.18it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 70%|███████ | 89/127 [00:21<00:09, 4.18it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 72%|███████▏ | 92/127 [00:22<00:07, 4.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 73%|███████▎ | 93/127 [00:22<00:07, 4.51it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 75%|███████▍ | 95/127 [00:23<00:06, 4.81it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 76%|███████▋ | 97/127 [00:23<00:07, 3.94it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 77%|███████▋ | 98/127 [00:24<00:08, 3.37it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 79%|███████▊ | 100/127 [00:24<00:05, 4.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 80%|████████ | 102/127 [00:24<00:05, 4.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 81%|████████ | 103/127 [00:25<00:05, 4.04it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 82%|████████▏ | 104/127 [00:25<00:05, 4.01it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 83%|████████▎ | 106/127 [00:25<00:03, 5.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 84%|████████▍ | 107/127 [00:25<00:04, 4.25it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 85%|████████▌ | 108/127 [00:26<00:05, 3.37it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 86%|████████▌ | 109/127 [00:26<00:05, 3.44it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 87%|████████▋ | 110/127 [00:27<00:06, 2.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 89%|████████▉ | 113/127 [00:27<00:03, 4.33it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 91%|█████████ | 115/127 [00:28<00:02, 4.42it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 92%|█████████▏| 117/127 [00:28<00:01, 5.12it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 93%|█████████▎| 118/127 [00:28<00:02, 4.10it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 94%|█████████▍| 120/127 [00:29<00:01, 4.44it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 95%|█████████▌| 121/127 [00:29<00:01, 4.88it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 96%|█████████▌| 122/127 [00:29<00:01, 4.81it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 97%|█████████▋| 123/127 [00:30<00:01, 3.42it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 98%|█████████▊| 125/127 [00:30<00:00, 5.04it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 99%|█████████▉| 126/127 [00:30<00:00, 5.49it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 100%|██████████| 127/127 [00:30<00:00, 3.95it/s][A
|
||
prompt_batches: 100%|██████████| 127/127 [00:30<00:00, 4.12it/s]
|
||
INFO:root:Completed 127 examples in 31.2 seconds.
|
||
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
|
||
Annotation chunk: 29%|██▊ | 2/7 [01:08<02:49, 33.83s/it]INFO:root:Annotating 128 examples with chatgpt_fn
|
||
INFO:root:Using `openai_completions` on 128 prompts using gpt-3.5-turbo-16k-0613.
|
||
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5
|
||
|
||
|
||
prompt_batches: 0%| | 0/128 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 1%| | 1/128 [00:01<02:38, 1.25s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 5%|▍ | 6/128 [00:02<00:39, 3.06it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 5%|▌ | 7/128 [00:02<00:34, 3.50it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 6%|▋ | 8/128 [00:02<00:33, 3.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 7%|▋ | 9/128 [00:02<00:28, 4.22it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 9%|▊ | 11/128 [00:03<00:32, 3.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 9%|▉ | 12/128 [00:03<00:27, 4.17it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 10%|█ | 13/128 [00:03<00:33, 3.43it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 12%|█▎ | 16/128 [00:04<00:26, 4.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 14%|█▍ | 18/128 [00:04<00:23, 4.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 15%|█▍ | 19/128 [00:05<00:28, 3.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 16%|█▋ | 21/128 [00:05<00:22, 4.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 18%|█▊ | 23/128 [00:05<00:22, 4.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 19%|█▉ | 24/128 [00:06<00:19, 5.27it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 20%|█▉ | 25/128 [00:06<00:27, 3.70it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 22%|██▏ | 28/128 [00:07<00:22, 4.39it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 23%|██▎ | 30/128 [00:07<00:20, 4.69it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 24%|██▍ | 31/128 [00:08<00:24, 3.89it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 26%|██▌ | 33/128 [00:08<00:21, 4.40it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 27%|██▋ | 34/128 [00:08<00:22, 4.10it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 27%|██▋ | 35/128 [00:09<00:26, 3.53it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 30%|██▉ | 38/128 [00:09<00:17, 5.10it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 30%|███ | 39/128 [00:09<00:23, 3.82it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 31%|███▏ | 40/128 [00:10<00:20, 4.26it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 32%|███▏ | 41/128 [00:10<00:22, 3.89it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 34%|███▍ | 44/128 [00:12<00:45, 1.84it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 41%|████▏ | 53/128 [00:13<00:16, 4.46it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 42%|████▏ | 54/128 [00:13<00:15, 4.65it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 44%|████▍ | 56/128 [00:13<00:13, 5.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 45%|████▍ | 57/128 [00:14<00:14, 4.81it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 45%|████▌ | 58/128 [00:14<00:19, 3.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 47%|████▋ | 60/128 [00:14<00:14, 4.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 48%|████▊ | 62/128 [00:15<00:13, 5.01it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 49%|████▉ | 63/128 [00:16<00:20, 3.14it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 50%|█████ | 64/128 [00:16<00:23, 2.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 53%|█████▎ | 68/128 [00:17<00:14, 4.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 55%|█████▍ | 70/128 [00:17<00:15, 3.85it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 57%|█████▋ | 73/128 [00:18<00:11, 4.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 58%|█████▊ | 74/128 [00:18<00:12, 4.44it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 59%|█████▊ | 75/128 [00:18<00:11, 4.55it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 59%|█████▉ | 76/128 [00:19<00:11, 4.38it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 61%|██████ | 78/128 [00:19<00:10, 4.57it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 62%|██████▏ | 79/128 [00:20<00:14, 3.49it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 63%|██████▎ | 81/128 [00:20<00:10, 4.62it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 64%|██████▍ | 82/128 [00:20<00:09, 4.98it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 65%|██████▍ | 83/128 [00:20<00:09, 4.82it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 66%|██████▌ | 84/128 [00:21<00:12, 3.61it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 67%|██████▋ | 86/128 [00:21<00:08, 4.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 69%|██████▉ | 88/128 [00:21<00:08, 4.70it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 70%|██████▉ | 89/128 [00:22<00:10, 3.66it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 71%|███████ | 91/128 [00:22<00:09, 4.01it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 72%|███████▏ | 92/128 [00:22<00:08, 4.06it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 73%|███████▎ | 94/128 [00:23<00:08, 3.85it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 75%|███████▌ | 96/128 [00:23<00:07, 4.51it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 76%|███████▌ | 97/128 [00:24<00:07, 4.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 77%|███████▋ | 99/128 [00:24<00:06, 4.60it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 78%|███████▊ | 100/128 [00:24<00:05, 5.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 79%|███████▉ | 101/128 [00:25<00:09, 2.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 83%|████████▎ | 106/128 [00:26<00:05, 4.33it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 84%|████████▍ | 108/128 [00:26<00:04, 4.43it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 85%|████████▌ | 109/128 [00:27<00:04, 3.84it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 87%|████████▋ | 111/128 [00:27<00:03, 4.33it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 88%|████████▊ | 112/128 [00:27<00:03, 4.49it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 88%|████████▊ | 113/128 [00:28<00:04, 3.61it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 90%|████████▉ | 115/128 [00:28<00:02, 5.14it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 91%|█████████ | 116/128 [00:28<00:02, 4.61it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 91%|█████████▏| 117/128 [00:28<00:02, 4.85it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 92%|█████████▏| 118/128 [00:29<00:02, 3.64it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 93%|█████████▎| 119/128 [00:29<00:02, 4.18it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 95%|█████████▍| 121/128 [00:29<00:01, 3.80it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 96%|█████████▌| 123/128 [00:30<00:01, 4.32it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 97%|█████████▋| 124/128 [00:30<00:00, 4.51it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 98%|█████████▊| 126/128 [00:30<00:00, 4.54it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 99%|█████████▉| 127/128 [00:31<00:00, 3.39it/s][A
|
||
prompt_batches: 100%|██████████| 128/128 [00:31<00:00, 4.07it/s]
|
||
INFO:root:Completed 128 examples in 31.8 seconds.
|
||
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
|
||
Annotation chunk: 43%|████▎ | 3/7 [01:40<02:12, 33.09s/it]INFO:root:Annotating 128 examples with chatgpt_fn
|
||
INFO:root:Using `openai_completions` on 128 prompts using gpt-3.5-turbo-16k-0613.
|
||
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5
|
||
|
||
|
||
prompt_batches: 0%| | 0/128 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 1%| | 1/128 [00:01<02:23, 1.13s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 2%|▏ | 2/128 [00:01<01:14, 1.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 5%|▍ | 6/128 [00:02<00:33, 3.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 5%|▌ | 7/128 [00:02<00:34, 3.52it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 8%|▊ | 10/128 [00:02<00:19, 6.05it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 9%|▉ | 12/128 [00:03<00:30, 3.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 12%|█▏ | 15/128 [00:03<00:19, 5.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 13%|█▎ | 17/128 [00:04<00:24, 4.45it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 15%|█▍ | 19/128 [00:04<00:24, 4.44it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 16%|█▌ | 20/128 [00:04<00:22, 4.86it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 16%|█▋ | 21/128 [00:05<00:29, 3.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 18%|█▊ | 23/128 [00:05<00:26, 3.94it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 19%|█▉ | 24/128 [00:06<00:24, 4.21it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 20%|██ | 26/128 [00:06<00:21, 4.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 21%|██ | 27/128 [00:06<00:22, 4.54it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 22%|██▏ | 28/128 [00:06<00:23, 4.24it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 23%|██▎ | 30/128 [00:07<00:25, 3.91it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 26%|██▌ | 33/128 [00:08<00:21, 4.49it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 27%|██▋ | 34/128 [00:08<00:20, 4.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 27%|██▋ | 35/128 [00:08<00:25, 3.60it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 30%|██▉ | 38/128 [00:09<00:21, 4.10it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 31%|███▏ | 40/128 [00:09<00:19, 4.60it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 32%|███▏ | 41/128 [00:30<05:38, 3.89s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 86%|████████▌ | 110/128 [00:30<00:03, 4.74it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 93%|█████████▎| 119/128 [00:32<00:01, 4.76it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 98%|█████████▊| 126/128 [00:34<00:00, 4.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
prompt_batches: 100%|██████████| 128/128 [00:34<00:00, 3.70it/s]
|
||
INFO:root:Completed 128 examples in 34.9 seconds.
|
||
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
|
||
Annotation chunk: 57%|█████▋ | 4/7 [02:16<01:41, 34.00s/it]INFO:root:Annotating 127 examples with chatgpt_fn
|
||
INFO:root:Using `openai_completions` on 127 prompts using gpt-3.5-turbo-16k-0613.
|
||
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5
|
||
|
||
|
||
prompt_batches: 0%| | 0/127 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 1%| | 1/127 [00:01<02:42, 1.29s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 5%|▍ | 6/127 [00:02<00:37, 3.21it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 6%|▌ | 7/127 [00:02<00:36, 3.33it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 9%|▊ | 11/127 [00:03<00:30, 3.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 10%|█ | 13/127 [00:03<00:24, 4.74it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 12%|█▏ | 15/127 [00:03<00:18, 5.92it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 13%|█▎ | 17/127 [00:04<00:29, 3.72it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 15%|█▍ | 19/127 [00:04<00:23, 4.65it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 17%|█▋ | 21/127 [00:05<00:30, 3.42it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 18%|█▊ | 23/127 [00:05<00:23, 4.51it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 20%|█▉ | 25/127 [00:06<00:24, 4.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 20%|██ | 26/127 [00:08<00:56, 1.79it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 28%|██▊ | 35/127 [00:09<00:23, 3.87it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 30%|██▉ | 38/127 [00:09<00:18, 4.92it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 31%|███▏ | 40/127 [00:10<00:21, 4.12it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 32%|███▏ | 41/127 [00:10<00:21, 3.96it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 34%|███▍ | 43/127 [00:10<00:16, 4.97it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 35%|███▌ | 45/127 [00:11<00:19, 4.25it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 36%|███▌ | 46/127 [00:12<00:21, 3.73it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 39%|███▉ | 50/127 [00:12<00:16, 4.64it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 40%|████ | 51/127 [00:13<00:17, 4.32it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 41%|████ | 52/127 [00:13<00:16, 4.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 43%|████▎ | 55/127 [00:13<00:14, 4.90it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 44%|████▍ | 56/127 [00:14<00:16, 4.44it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 45%|████▍ | 57/127 [00:14<00:16, 4.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 46%|████▋ | 59/127 [00:14<00:14, 4.76it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 47%|████▋ | 60/127 [00:14<00:14, 4.64it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 48%|████▊ | 61/127 [00:15<00:14, 4.64it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 49%|████▉ | 62/127 [00:15<00:17, 3.78it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 50%|█████ | 64/127 [00:15<00:14, 4.40it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 51%|█████ | 65/127 [00:16<00:15, 3.90it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 53%|█████▎ | 67/127 [00:16<00:14, 4.11it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 54%|█████▍ | 69/127 [00:16<00:11, 4.95it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 55%|█████▌ | 70/127 [00:17<00:13, 4.37it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 56%|█████▌ | 71/127 [00:17<00:11, 4.93it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 57%|█████▋ | 72/127 [00:17<00:12, 4.42it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 57%|█████▋ | 73/127 [00:17<00:11, 4.91it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 58%|█████▊ | 74/127 [00:17<00:09, 5.66it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 59%|█████▉ | 75/127 [00:18<00:15, 3.29it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 60%|█████▉ | 76/127 [00:18<00:14, 3.42it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 61%|██████ | 77/127 [00:19<00:23, 2.16it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 64%|██████▍ | 81/127 [00:20<00:10, 4.22it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 65%|██████▌ | 83/127 [00:22<00:23, 1.87it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 72%|███████▏ | 92/127 [00:23<00:08, 4.28it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 73%|███████▎ | 93/127 [00:23<00:07, 4.26it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 75%|███████▍ | 95/127 [00:23<00:07, 4.51it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 76%|███████▋ | 97/127 [00:24<00:06, 4.46it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 77%|███████▋ | 98/127 [00:24<00:06, 4.36it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 79%|███████▊ | 100/127 [00:25<00:06, 4.06it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 80%|████████ | 102/127 [00:25<00:05, 4.82it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 81%|████████ | 103/127 [00:25<00:04, 4.80it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 82%|████████▏ | 104/127 [00:25<00:05, 4.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 83%|████████▎ | 105/127 [00:26<00:06, 3.15it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 85%|████████▌ | 108/127 [00:26<00:03, 5.14it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 86%|████████▌ | 109/127 [00:27<00:04, 4.09it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 87%|████████▋ | 110/127 [00:27<00:04, 3.92it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 87%|████████▋ | 111/127 [00:27<00:04, 3.70it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 90%|████████▉ | 114/127 [00:28<00:02, 5.00it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 91%|█████████ | 115/127 [00:28<00:03, 3.90it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 91%|█████████▏| 116/127 [00:28<00:02, 4.43it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 92%|█████████▏| 117/127 [00:29<00:02, 3.55it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 93%|█████████▎| 118/127 [00:29<00:02, 4.06it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 94%|█████████▎| 119/127 [00:29<00:01, 4.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 94%|█████████▍| 120/127 [00:30<00:02, 3.49it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 96%|█████████▌| 122/127 [00:30<00:01, 4.76it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 97%|█████████▋| 123/127 [00:30<00:00, 4.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 98%|█████████▊| 125/127 [00:31<00:00, 3.77it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 99%|█████████▉| 126/127 [00:31<00:00, 3.90it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 100%|██████████| 127/127 [00:32<00:00, 2.98it/s][A
|
||
prompt_batches: 100%|██████████| 127/127 [00:32<00:00, 3.97it/s]
|
||
INFO:root:Completed 127 examples in 32.4 seconds.
|
||
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
|
||
Annotation chunk: 71%|███████▏ | 5/7 [02:49<01:07, 33.58s/it]INFO:root:Annotating 128 examples with chatgpt_fn
|
||
INFO:root:Using `openai_completions` on 128 prompts using gpt-3.5-turbo-16k-0613.
|
||
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5
|
||
|
||
|
||
prompt_batches: 0%| | 0/128 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 1%| | 1/128 [00:01<02:51, 1.35s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 2%|▏ | 2/128 [00:01<01:21, 1.55it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 5%|▍ | 6/128 [00:02<00:36, 3.31it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 6%|▋ | 8/128 [00:02<00:25, 4.66it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 8%|▊ | 10/128 [00:02<00:20, 5.82it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 9%|▉ | 12/128 [00:03<00:30, 3.80it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 11%|█ | 14/128 [00:03<00:25, 4.47it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 12%|█▎ | 16/128 [00:04<00:29, 3.80it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 14%|█▍ | 18/128 [00:04<00:22, 4.99it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 15%|█▍ | 19/128 [00:04<00:21, 5.09it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 16%|█▌ | 20/128 [00:04<00:19, 5.61it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 16%|█▋ | 21/128 [00:05<00:32, 3.28it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 18%|█▊ | 23/128 [00:05<00:24, 4.23it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 20%|█▉ | 25/128 [00:06<00:21, 4.82it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 20%|██ | 26/128 [00:06<00:23, 4.27it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 21%|██ | 27/128 [00:06<00:21, 4.66it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 22%|██▏ | 28/128 [00:06<00:22, 4.37it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 23%|██▎ | 29/128 [00:07<00:21, 4.68it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 23%|██▎ | 30/128 [00:07<00:21, 4.54it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 24%|██▍ | 31/128 [00:07<00:21, 4.46it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 25%|██▌ | 32/128 [00:07<00:21, 4.37it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 26%|██▌ | 33/128 [00:07<00:20, 4.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 27%|██▋ | 34/128 [00:08<00:27, 3.44it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 28%|██▊ | 36/128 [00:08<00:18, 5.10it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 29%|██▉ | 37/128 [00:08<00:17, 5.25it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 30%|██▉ | 38/128 [00:09<00:22, 4.02it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 30%|███ | 39/128 [00:09<00:26, 3.41it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 32%|███▏ | 41/128 [00:10<00:21, 4.09it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 34%|███▎ | 43/128 [00:10<00:20, 4.06it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 34%|███▍ | 44/128 [00:10<00:18, 4.61it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 36%|███▌ | 46/128 [00:10<00:15, 5.22it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 37%|███▋ | 47/128 [00:11<00:14, 5.65it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 38%|███▊ | 48/128 [00:11<00:19, 4.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 39%|███▉ | 50/128 [00:11<00:14, 5.45it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 40%|███▉ | 51/128 [00:12<00:17, 4.34it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 41%|████▏ | 53/128 [00:12<00:20, 3.62it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 43%|████▎ | 55/128 [00:12<00:15, 4.70it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 44%|████▍ | 56/128 [00:13<00:15, 4.57it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 45%|████▌ | 58/128 [00:15<00:37, 1.85it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 52%|█████▏ | 67/128 [00:16<00:14, 4.12it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 53%|█████▎ | 68/128 [00:16<00:14, 4.01it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 56%|█████▋ | 72/128 [00:17<00:12, 4.34it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 57%|█████▋ | 73/128 [00:17<00:12, 4.24it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 59%|█████▊ | 75/128 [00:18<00:11, 4.48it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 60%|██████ | 77/128 [00:18<00:12, 4.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 62%|██████▎ | 80/128 [00:19<00:09, 5.16it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 63%|██████▎ | 81/128 [00:19<00:08, 5.27it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 64%|██████▍ | 82/128 [00:19<00:09, 4.71it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 65%|██████▍ | 83/128 [00:19<00:10, 4.38it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 66%|██████▌ | 84/128 [00:20<00:14, 3.05it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 68%|██████▊ | 87/128 [00:21<00:10, 3.94it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 70%|██████▉ | 89/128 [00:21<00:08, 4.83it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 71%|███████ | 91/128 [00:21<00:08, 4.37it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 72%|███████▏ | 92/128 [00:22<00:09, 4.00it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 73%|███████▎ | 94/128 [00:22<00:08, 3.92it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 75%|███████▌ | 96/128 [00:23<00:07, 4.47it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 76%|███████▌ | 97/128 [00:23<00:08, 3.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 78%|███████▊ | 100/128 [00:24<00:06, 4.49it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 79%|███████▉ | 101/128 [00:24<00:05, 4.58it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 80%|███████▉ | 102/128 [00:24<00:07, 3.63it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 82%|████████▏ | 105/128 [00:25<00:04, 4.90it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 83%|████████▎ | 106/128 [00:25<00:04, 4.93it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 84%|████████▎ | 107/128 [00:25<00:04, 4.35it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 85%|████████▌ | 109/128 [00:26<00:04, 4.35it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 86%|████████▌ | 110/128 [00:26<00:03, 4.57it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 87%|████████▋ | 111/128 [00:26<00:03, 4.38it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 88%|████████▊ | 112/128 [00:26<00:03, 4.87it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 88%|████████▊ | 113/128 [00:27<00:04, 3.55it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 90%|████████▉ | 115/128 [00:27<00:02, 5.09it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 91%|█████████ | 116/128 [00:27<00:02, 4.20it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 92%|█████████▏| 118/128 [00:28<00:02, 3.55it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 95%|█████████▍| 121/128 [00:28<00:01, 4.88it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 95%|█████████▌| 122/128 [00:28<00:01, 5.13it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 96%|█████████▌| 123/128 [00:29<00:01, 4.05it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 97%|█████████▋| 124/128 [00:29<00:00, 4.57it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 98%|█████████▊| 125/128 [00:29<00:00, 4.98it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 98%|█████████▊| 126/128 [00:29<00:00, 5.47it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 99%|█████████▉| 127/128 [00:29<00:00, 4.81it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 100%|██████████| 128/128 [00:30<00:00, 3.33it/s][A
|
||
prompt_batches: 100%|██████████| 128/128 [00:30<00:00, 4.19it/s]
|
||
INFO:root:Completed 128 examples in 30.9 seconds.
|
||
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
|
||
Annotation chunk: 86%|████████▌ | 6/7 [03:20<00:32, 32.81s/it]INFO:root:Annotating 37 examples with chatgpt_fn
|
||
INFO:root:Using `openai_completions` on 37 prompts using gpt-3.5-turbo-16k-0613.
|
||
INFO:root:Kwargs to completion: {'n': 1, 'model': 'gpt-3.5-turbo-16k-0613', 'is_chat': True, 'temperature': 0, 'function_call': {'name': 'print_best_model'}, 'functions': [{'name': 'print_best_model', 'description': 'Print the best model given the preferred output.', 'parameters': {'type': 'object', 'properties': {'best_output': {'type': 'string', 'description': "Name of the best output, should be 'Output (a)' or 'Output (b)'"}}}, 'required': ['best_output']}]}. num_procs=5
|
||
|
||
|
||
prompt_batches: 0%| | 0/37 [00:00<?, ?it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 3%|▎ | 1/37 [00:01<00:50, 1.39s/it][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 11%|█ | 4/37 [00:01<00:10, 3.19it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 16%|█▌ | 6/37 [00:02<00:10, 2.86it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 19%|█▉ | 7/37 [00:02<00:10, 2.75it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 30%|██▉ | 11/37 [00:03<00:06, 4.02it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 32%|███▏ | 12/37 [00:03<00:06, 3.59it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 43%|████▎ | 16/37 [00:04<00:04, 4.23it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 46%|████▌ | 17/37 [00:04<00:05, 3.96it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 54%|█████▍ | 20/37 [00:05<00:02, 5.93it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 59%|█████▉ | 22/37 [00:05<00:03, 4.35it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 62%|██████▏ | 23/37 [00:06<00:03, 4.62it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 70%|███████ | 26/37 [00:06<00:02, 4.35it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 73%|███████▎ | 27/37 [00:06<00:02, 4.54it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 78%|███████▊ | 29/37 [00:07<00:01, 4.74it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 84%|████████▍ | 31/37 [00:08<00:01, 3.79it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 92%|█████████▏| 34/37 [00:08<00:00, 5.16it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
INFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 97%|█████████▋| 36/37 [00:09<00:00, 3.73it/s][AINFO:httpx:HTTP Request: POST https://api.openai-proxy.org/v1/chat/completions "HTTP/1.1 200 OK"
|
||
|
||
|
||
prompt_batches: 100%|██████████| 37/37 [00:09<00:00, 3.85it/s][A
|
||
prompt_batches: 100%|██████████| 37/37 [00:09<00:00, 3.90it/s]
|
||
INFO:root:Completed 37 examples in 9.8 seconds.
|
||
INFO:root:Saving all annotations to /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
INFO:root:Loading all annotations from /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/evaluators_configs/chatgpt_fn/annotations_seed0_configs.json.
|
||
|
||
Annotation chunk: 100%|██████████| 7/7 [03:30<00:00, 25.42s/it]
|
||
Annotation chunk: 100%|██████████| 7/7 [03:30<00:00, 30.08s/it]
|
||
/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/metrics.py:26: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '1.5' has dtype incompatible with int64, please explicitly cast to a compatible dtype first.
|
||
series_preferences[series_preferences == 0] = 1.5
|
||
INFO:root:Saving all results to output/chatgpt_fn_--LMCocktail-7B-v1
|
||
INFO:root:Saving result to the precomputed leaderboard at /home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/LLM-infrastructure/alpaca_eval/src/alpaca_eval/leaderboards/data_AlpacaEval/chatgpt_fn_leaderboard.csv
|
||
win_rate standard_error n_total avg_length
|
||
gpt4 73.79 1.54 805 1365
|
||
LMCocktail-7B-v1 73.54 1.55 805 1870
|
||
claude 70.37 1.60 805 1082
|
||
chatgpt 66.09 1.66 805 811
|
||
wizardlm-13b 65.16 1.67 805 985
|
||
vicuna-13b 64.10 1.69 805 1037
|
||
guanaco-65b 62.36 1.71 805 1249
|
||
oasst-rlhf-llama-33b 62.05 1.71 805 1079
|
||
alpaca-farm-ppo-human 60.25 1.72 805 803
|
||
falcon-40b-instruct 56.52 1.74 805 662
|
||
text_davinci_003 50.00 0.00 805 307
|
||
alpaca-7b 45.22 1.74 805 396
|
||
text_davinci_001 28.07 1.56 805 296
|