pai-qwen1_5-4b-doc2qa/output.log

[2024-04-20 23:35:42,048] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[2024-04-20 23:35:43,623] [WARNING] [runner.py:202:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only.
[2024-04-20 23:35:43,624] [INFO] [runner.py:568:main] cmd = /opt/conda/bin/python3.8 -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMCwgMSwgMiwgMywgNCwgNSwgNiwgN119 --master_addr=127.0.0.1 --master_port=29500 --enable_each_rank_log=None train.py --model_name_or_path /workspace/qwen2_4b/ --data_path refgpt_train.json --output_dir qwen2_4b_4epoch_refgpt --num_train_epochs 4 --model_max_length 2048 --per_device_train_batch_size 1 --per_device_eval_batch_size 2 --gradient_accumulation_steps 8 --evaluation_strategy no --save_strategy steps --save_steps 780 --save_total_limit 10 --learning_rate 2e-5 --warmup_ratio 0.05 --logging_steps 2 --lr_scheduler_type cosine --report_to tensorboard --bf16 True --deepspeed ds_config.json --use_lora False
[2024-04-20 23:35:44,554] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_TC=136
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.9.9-1+cuda11.3
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_DEV_PACKAGE_VERSION=2.9.9-1
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_MIN_NCHANNELS=4
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_NET_PLUGIN=none
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_VERSION=2.12.10
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_SOCKET_IFNAME=eth
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_DEBUG=INFO
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_HCA=mlx5
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_GID_INDEX=3
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_QPS_PER_CONNECTION=8
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_TIMEOUT=22
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_SL=5
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_PACKAGE=libnccl2=2.9.9-1+cuda11.3
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_PACKAGE_NAME=libnccl2
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_PACKAGE_VERSION=2.9.9-1
[2024-04-20 23:35:46,123] [INFO] [launch.py:145:main] WORLD INFO DICT: {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]}
[2024-04-20 23:35:46,123] [INFO] [launch.py:151:main] nnodes=1, num_local_procs=8, node_rank=0
[2024-04-20 23:35:46,123] [INFO] [launch.py:162:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]})
[2024-04-20 23:35:46,123] [INFO] [launch.py:163:main] dist_world_size=8
[2024-04-20 23:35:46,123] [INFO] [launch.py:165:main] Setting CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
[2024-04-20 23:35:46,128] [INFO] [launch.py:253:main] process 267 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=0', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,131] [INFO] [launch.py:253:main] process 268 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=1', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,134] [INFO] [launch.py:253:main] process 269 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=2', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,137] [INFO] [launch.py:253:main] process 270 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=3', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,140] [INFO] [launch.py:253:main] process 271 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=4', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,143] [INFO] [launch.py:253:main] process 272 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=5', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,147] [INFO] [launch.py:253:main] process 273 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=6', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,151] [INFO] [launch.py:253:main] process 274 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=7', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:49,984] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:49,990] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:49,999] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,015] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,036] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,050] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,054] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,067] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  using untested triton version (2.3.0), only 1.0.0 is known to be compatible
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  using untested triton version (2.3.0), only 1.0.0 is known to be compatible
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  using untested triton version (2.3.0), only 1.0.0 is known to be compatible
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  using untested triton version (2.3.0), only 1.0.0 is known to be compatible
 [WARNING]  using untested triton version (2.3.0), only 1.0.0 is known to be compatible
 [WARNING]  using untested triton version (2.3.0), only 1.0.0 is known to be compatible
 [WARNING]  using untested triton version (2.3.0), only 1.0.0 is known to be compatible
 [WARNING]  using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[2024-04-20 23:35:52,257] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-04-20 23:35:52,261] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-04-20 23:35:52,261] [INFO] [comm.py:668:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
[2024-04-20 23:35:52,266] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52]  distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 6
[2024-04-20 23:35:52,300] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-04-20 23:35:52,301] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52]  distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 4
[INFO] [2024-04-20 23:35:52]  distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 7
[2024-04-20 23:35:52,309] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52]  distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 2
[2024-04-20 23:35:52,325] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52]  distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 5
[2024-04-20 23:35:52,340] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52]  distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 1
[INFO] [2024-04-20 23:35:53]  distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 3
[INFO] [2024-04-20 23:35:53]  distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 0
[INFO] [2024-04-20 23:35:53]  distributed_c10d.py : Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53]  distributed_c10d.py : Rank 3: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53]  distributed_c10d.py : Rank 5: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53]  distributed_c10d.py : Rank 6: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53]  distributed_c10d.py : Rank 1: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53]  distributed_c10d.py : Rank 4: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53]  distributed_c10d.py : Rank 2: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53]  distributed_c10d.py : Rank 7: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:12<00:12, 12.32s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:12<00:12, 12.54s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:12<00:12, 12.70s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:12<00:12, 12.58s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:12<00:12, 12.57s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:12<00:12, 12.44s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:12<00:12, 12.58s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:12<00:12, 12.87s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.23s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.24s/it]

Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.38s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.43s/it]

Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.36s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.39s/it]

Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.31s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.35s/it]

Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.08s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.13s/it]

Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.15s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.22s/it]

Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.13s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.20s/it]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.08s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.20s/it]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
[INFO] [2024-04-20 23:36:35]  train.py : Formatting inputs...
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Plugin name set by env to libnccl-net-none.so

dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Using network IB
NCCL version 2.12.10+cuda11.3
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO Plugin name set by env to libnccl-net-none.so

dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO Plugin name set by env to libnccl-net-none.so

dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO Plugin name set by env to libnccl-net-none.so

libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO Plugin name set by env to libnccl-net-none.so

dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO Plugin name set by env to libnccl-net-none.so

dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO Plugin name set by env to libnccl-net-none.so

libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO Plugin name set by env to libnccl-net-none.so

dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Setting affinity for GPU 0 to 0fffff
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Setting affinity for GPU 2 to 0fffff
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Setting affinity for GPU 3 to 0fffff
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Setting affinity for GPU 1 to 0fffff
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 00/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 01/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 02/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 03/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 04/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 05/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 06/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 07/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 08/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 09/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 10/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 11/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 12/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 13/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 14/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 15/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 16/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 17/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 18/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 19/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 20/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 21/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 22/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 23/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 00 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 00 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 00 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 00 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 00 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 00 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 01 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 01 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 01 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 01 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 01 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 01 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 02 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 02 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 02 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 02 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 02 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 00 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 02 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 03 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 03 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 03 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 03 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 01 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 03 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 00 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 04 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 04 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 04 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 04 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 02 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 04 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 01 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 05 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 05 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 05 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 05 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 03 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 05 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 06 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 02 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 06 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 06 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 03 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 04 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 06 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 06 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 07 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 07 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 07 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 04 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 05 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 03 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 07 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 07 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 08 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 08 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 05 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 08 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 04 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 06 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 08 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 08 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 09 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 09 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 09 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 06 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 05 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 07 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 09 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 10 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 09 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 10 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 10 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 07 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 08 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 06 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 10 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 11 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 10 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 11 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 11 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 08 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 09 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 07 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 11 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 11 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 12 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 12 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 09 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 12 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 10 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 08 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 12 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 13 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 12 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 13 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 13 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 09 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 10 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 11 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 14 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 14 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 13 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 14 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 10 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 11 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 15 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 14 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 15 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 11 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 12 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 16 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 15 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 16 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 12 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 13 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 17 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 16 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 17 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 14 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 13 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 18 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 17 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 18 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 15 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 14 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 19 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 18 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 19 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 16 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 20 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 13 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 15 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 15 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 19 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 17 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 20 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 21 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 12 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 14 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 16 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 18 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 21 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 20 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 22 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 17 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 19 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 21 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 22 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 23 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 18 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 20 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 23 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 19 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 21 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 20 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 22 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 21 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 23 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 22 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 23 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 22 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 23 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 13 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 16 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 15 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 14 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 17 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 15 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 18 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 16 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 19 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 16 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 17 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 20 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 17 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 18 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 21 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 18 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 19 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 22 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 19 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 20 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 23 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 20 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 21 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 21 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 22 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 22 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 23 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 23 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 00 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 01 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 02 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 03 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 04 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 05 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 06 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 07 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 08 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 09 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 10 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 11 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 12 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 13 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 00 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 14 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 01 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 15 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 02 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 16 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 03 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 17 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 04 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 18 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 05 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 19 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 06 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 20 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 07 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 21 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 08 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 22 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 09 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 23 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 10 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 11 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 12 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 13 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 00 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 00 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 00 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 00 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 14 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 01 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 00 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 01 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 01 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 01 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 01 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 02 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 15 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 02 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 02 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 02 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 02 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 03 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 16 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 03 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 03 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 03 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 03 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 04 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 17 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 04 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 04 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 04 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 04 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 18 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 05 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 05 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 05 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 05 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 19 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 06 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 06 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 06 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 06 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 20 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 07 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 07 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 07 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 07 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 21 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 08 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 08 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 05 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 08 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 22 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 08 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 09 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 09 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 06 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 09 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 23 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 10 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 09 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 10 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 07 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 10 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 10 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 11 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 11 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 08 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 11 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 12 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 12 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 11 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 09 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 12 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 13 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 13 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 13 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 12 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 14 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 14 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 14 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 13 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 15 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 15 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 15 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 14 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 16 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 16 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 16 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 15 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 17 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 17 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 17 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 16 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 18 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 18 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 18 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 17 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 19 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 19 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 18 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 19 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 10 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 20 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 20 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 19 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 21 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 20 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 21 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 20 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 22 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 21 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 22 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 21 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 23 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 22 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 23 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 22 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 23 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 23 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 11 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 12 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 13 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 14 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 15 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 16 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 17 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 18 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 19 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 20 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 21 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 22 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 23 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO comm 0x7f3ffc008fb0 rank 5 nranks 8 cudaDev 5 busId 60 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO comm 0x7f3114008fb0 rank 3 nranks 8 cudaDev 3 busId 40 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO comm 0x7f3794008fb0 rank 1 nranks 8 cudaDev 1 busId 20 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO comm 0x7f4f34008fb0 rank 7 nranks 8 cudaDev 7 busId 80 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO comm 0x7f53a0008fb0 rank 2 nranks 8 cudaDev 2 busId 30 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO comm 0x7fed4c008fb0 rank 0 nranks 8 cudaDev 0 busId 10 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO comm 0x7fa874008fb0 rank 6 nranks 8 cudaDev 6 busId 70 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Launch mode Parallel
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO comm 0x7f4758008fb0 rank 4 nranks 8 cudaDev 4 busId 50 - Init COMPLETE
[INFO] [2024-04-20 23:38:55]  train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55]  train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55]  train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55]  train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55]  train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55]  train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55]  train.py : Data Num: 49920
[INFO] [2024-04-20 23:38:55]  train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55]  train.py : Sample 34992 of the training set: {'input_ids': tensor([104600,     12,  44636, 102430, 100280, 102064,  11095,  15843,     71,
          1174,  15459,  58463,  99449, 100280, 102064, 104995,     16,     18,
            15,  86402, 102064, 107339,  72881,  70769,   3837,  17714,  58463,
         99449,  33108, 109033, 104995,     23,     11,     15,     15,     15,
        102266,  31838,  37029,   1773, 106676,  72881,   9370, 101468, 105570,
         42140,  34204,  92894,  58463,  99449, 100280, 102064, 101468,   9370,
         59743,  33108,   1773,  92894, 101194, 101468, 109824,  29524, 106676,
        106758, 104995,     16,     15,     15, 102266, 105899, 101705,  72881,
          9909,  39120,    644,  48272,  44636, 102430,  72881,   3837,     23,
            15, 102266, 105899,  44956,  99727,  72881,   9909,     42,     84,
           352,  48272, 112964, 106835,  33108, 105906, 112545,     23,     15,
        102266, 105899, 104600,  72881,   1773,  18493, 104152, 104600,     12,
         44636, 102430, 100280, 102064,  15946,   3837, 101043, 104600,  72881,
          5373,  44636, 102430,  72881,  33108, 106676,  72881,  18493,     16,
            24, 101186, 103982,  18830, 108201, 100169,   1773,  26940,  99470,
        104600,  72881,  25067,  99461,  33956,  33477,  61443,  34204,     22,
        101186,   3837,  20412, 101888,  58463,  99449, 113228, 105871,  82832,
          3837, 103957, 112964,  17340, 103993, 109755,   9909,  73743,  99915,
         11095,  25079,  33108,   1425,    292,   4378,   5942,  74276, 104600,
            12,  44636, 102430, 100280, 102064,   9370, 101460, 105761, 104363,
         20412, 101514,  99740,  78685,   3837,  99740,  78685, 116966, 100405,
        104610,  99278, 102357,  70074,   9370, 119301, 119301, 111947, 111947,
          9370, 102124,  90395, 100136, 104250,  70074,  47872, 102478,   1773,
         33975,     25,    220, 108386,  37945,  56007, 107809, 104169,  99526,
         99526, 104600,     12,  44636, 102430, 100280, 102064, 101037,  26850,
         71703,     25,    220, 108386,   3837, 103942,  73670,   1773, 104600,
            12,  44636, 102430, 100280, 102064, 101909,  67071, 104995,     16,
            18,     15,  86402, 102064, 107339,  72881,  70769,   3837,  99558,
        114021,  58463,  99449,  33108, 109033, 100361,   3837, 104678, 104995,
            23,     15,     15,     15, 102266,  18493,  37029, 100001, 102064,
          1773,  90919,   3837, 106676,  72881,  37029, 104346, 102538,   3837,
         99427,  71304,  92894,  58463,  99449, 100280, 102064,   9370,  59743,
         33108,   1773, 110075,  37029, 104346, 106204, 109824, 100630, 101705,
         72881,   9909,  39120,    644,   3837,  94237,     16,     15,     15,
        102266,  37029,   3837,  99558,  18493, 106676, 106758,  64359,  44636,
        102430,  72881,   5373,  44956,  99727,  72881,   9909,     42,     84,
           352,   3837,  94237,     23,     15, 102266,  37029,   7552, 101034,
        104600,  72881,   9909,  18493, 112964, 106835,  33108, 105906, 112545,
         18830,     23,     15, 102266,  37029,  74276,  18493, 100001, 104600,
            12,  44636, 102430, 100280, 102064,  15946,   3837, 104600,  72881,
          5373,  44636, 102430,  72881,  33108, 106676,  72881,  18830, 108201,
        100169,   3837,  73670, 109234,  26939,     16,     24, 101186, 103982,
          1773, 100654, 100146,  91111,  99470, 104600,  72881,  25067, 114292,
        101888,  58463,  99449, 113228, 105871,  82832,   3837, 105985,     22,
        101186, 104435,  65577,   3837, 103957,  99250, 112964,  17340, 109755,
          1773, 116880,   3837, 104600,     12,  44636, 102430, 100280, 102064,
          9370, 101460, 105761, 115093, 103926, 101514,  99740,  78685,   5373,
        116966, 100405, 100136,  16530,  99278, 102357,  70074,   3837,  91572,
        104250,  70074,  47872, 102478,   3407,  33975,     25,  49434,    239,
        104958, 104600,     12,  44636, 102430, 100280, 102064, 118619,  99740,
         78685,  37945,  56007, 100001,  99740,  78685, 104139, 100772,  11319,
        106998,  64471,  33108, 109023,  99604,  26850,  71703,     25,  54851,
          9370,   3837, 104600,     12,  44636, 102430, 100280, 102064, 104877,
        109023,  99883,   3837, 100629,  33126, 104653,  99740,  78685,   1773,
        100001,  99740,  78685,   9370, 116966, 115390,   3837, 102119,  16530,
        111240, 102357,  70074,   3837, 105884, 116966,  13343, 102274,  99792,
        103148,   3837, 106085, 109023, 101047, 111817,  78685,   3837,  29524,
          2073,     89,  33590,   2073,  23815,    854,  49567, 116966,  13343,
        107689, 118216, 102073, 102357,  70074,   1773, 104043,   3837, 104600,
            12,  44636, 102430, 100280, 102064,  18493, 116966,  13343, 102119,
        110921, 106050,  70074,  47872, 102478,   3837, 105064,  33108, 109023,
        103943, 100687, 102665,   3837,  99519, 109023, 101158,  70074,  47872,
        102064,   3837, 101970,  70074,  47872,  73670,  51463, 101970,  99689,
         64559,   1773, 106279,   3837, 104600,     12,  44636, 102430, 100280,
        102064,   9370,  99740,  78685, 100772,  33108, 109023, 101208,   3837,
         99558,  20742,  99601, 100733,  33071,   5373, 116966,   9370, 100405,
         33071, 101034, 104250,  70074,  47872, 102478,  17447,   1773, 151643]), 'labels': tensor([  -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
         33975,     25,    220, 108386,  37945,  56007, 107809, 104169,  99526,
         99526, 104600,     12,  44636, 102430, 100280, 102064, 101037,  26850,
         71703,     25,    220, 108386,   3837, 103942,  73670,   1773, 104600,
            12,  44636, 102430, 100280, 102064, 101909,  67071, 104995,     16,
            18,     15,  86402, 102064, 107339,  72881,  70769,   3837,  99558,
        114021,  58463,  99449,  33108, 109033, 100361,   3837, 104678, 104995,
            23,     15,     15,     15, 102266,  18493,  37029, 100001, 102064,
          1773,  90919,   3837, 106676,  72881,  37029, 104346, 102538,   3837,
         99427,  71304,  92894,  58463,  99449, 100280, 102064,   9370,  59743,
         33108,   1773, 110075,  37029, 104346, 106204, 109824, 100630, 101705,
         72881,   9909,  39120,    644,   3837,  94237,     16,     15,     15,
        102266,  37029,   3837,  99558,  18493, 106676, 106758,  64359,  44636,
        102430,  72881,   5373,  44956,  99727,  72881,   9909,     42,     84,
           352,   3837,  94237,     23,     15, 102266,  37029,   7552, 101034,
        104600,  72881,   9909,  18493, 112964, 106835,  33108, 105906, 112545,
         18830,     23,     15, 102266,  37029,  74276,  18493, 100001, 104600,
            12,  44636, 102430, 100280, 102064,  15946,   3837, 104600,  72881,
          5373,  44636, 102430,  72881,  33108, 106676,  72881,  18830, 108201,
        100169,   3837,  73670, 109234,  26939,     16,     24, 101186, 103982,
          1773, 100654, 100146,  91111,  99470, 104600,  72881,  25067, 114292,
        101888,  58463,  99449, 113228, 105871,  82832,   3837, 105985,     22,
        101186, 104435,  65577,   3837, 103957,  99250, 112964,  17340, 109755,
          1773, 116880,   3837, 104600,     12,  44636, 102430, 100280, 102064,
          9370, 101460, 105761, 115093, 103926, 101514,  99740,  78685,   5373,
        116966, 100405, 100136,  16530,  99278, 102357,  70074,   3837,  91572,
        104250,  70074,  47872, 102478,   3407,  33975,     25,  49434,    239,
        104958, 104600,     12,  44636, 102430, 100280, 102064, 118619,  99740,
         78685,  37945,  56007, 100001,  99740,  78685, 104139, 100772,  11319,
        106998,  64471,  33108, 109023,  99604,  26850,  71703,     25,  54851,
          9370,   3837, 104600,     12,  44636, 102430, 100280, 102064, 104877,
        109023,  99883,   3837, 100629,  33126, 104653,  99740,  78685,   1773,
        100001,  99740,  78685,   9370, 116966, 115390,   3837, 102119,  16530,
        111240, 102357,  70074,   3837, 105884, 116966,  13343, 102274,  99792,
        103148,   3837, 106085, 109023, 101047, 111817,  78685,   3837,  29524,
          2073,     89,  33590,   2073,  23815,    854,  49567, 116966,  13343,
        107689, 118216, 102073, 102357,  70074,   1773, 104043,   3837, 104600,
            12,  44636, 102430, 100280, 102064,  18493, 116966,  13343, 102119,
        110921, 106050,  70074,  47872, 102478,   3837, 105064,  33108, 109023,
        103943, 100687, 102665,   3837,  99519, 109023, 101158,  70074,  47872,
        102064,   3837, 101970,  70074,  47872,  73670,  51463, 101970,  99689,
         64559,   1773, 106279,   3837, 104600,     12,  44636, 102430, 100280,
        102064,   9370,  99740,  78685, 100772,  33108, 109023, 101208,   3837,
         99558,  20742,  99601, 100733,  33071,   5373, 116966,   9370, 100405,
         33071, 101034, 104250,  70074,  47872, 102478,  17447,   1773, 151643])}, torch.Size([603]).
孟-高棉诸语言Mon-Khmer languages南亚诸语言大约130种语言组成的语族，为南亚和东南亚大约8,000万人所使用。越南语的使用者远远多于其他南亚诸语言使用者的总和。其他有很多使用者的语言如越南北部大约100万人使用的芒语（Muong），高棉语，80万人使用的库阿语（Kuay），缅甸南部和泰国部分地区80万人使用的孟语。在所有的孟-高棉诸语言中，只有孟语、高棉语和越南语在19世纪以前有书写传统。《古孟语》已经确证写于7世纪，是关于南亚起源的手稿，后来缅甸人进行了改编（参阅Mon kingdom和Indic writing systems）。孟-高棉诸语言的典型语音特征是大量的母音，母音发音简单而不是带呼吸声的叽叽嘎嘎的表达，并且缺乏声调差异。Human: 你好，请问你能给我讲讲孟-高棉诸语言吗？

Assistant: 你好，当然可以。孟-高棉诸语言是一个由大约130种语言组成的语族，主要分布在南亚和东南亚地区，共有大约8000万人在使用这些语言。其中，越南语使用人数最多，远超其他南亚诸语言的总和。还有一些使用人数较多的语言包括芒语（Muong，约100万人使用，主要在越南北部）、高棉语、库阿语（Kuay，约80万人使用）以及孟语（在缅甸南部和泰国部分地区有80万人使用）。在这些孟-高棉诸语言中，孟语、高棉语和越南语有书写传统，可以追溯到19世纪以前。特别的是，《古孟语》是一部关于南亚起源的手稿，早在7世纪就有记录，后来被缅甸人改编。总的来说，孟-高棉诸语言的典型语音特点是拥有大量的母音、发音简单且不带呼吸声，同时缺乏声调差异。

Human: 我了解到孟-高棉诸语言有很多种母音，请问这些母音有什么特点？比如说是否和汉语不同？

Assistant: 是的，孟-高棉诸语言对比汉语来说，具有更丰富的母音。这些母音的发音比较简单，通常不伴随呼吸声，也就是说发音时声音比较单纯，不像汉语中的浊音，如“z”，“zh”等发音时伴随着喉咙里的呼吸声。此外，孟-高棉诸语言在发音时通常不会有明显的声调差异，这一点和汉语也有明显区别，因为汉语是一种声调语言，不同的声调可以表示不同的词义。总之，孟-高棉诸语言的母音特点和汉语相比，主要表现在丰富性、发音的简单性以及缺乏声调差异上。<|endoftext|>
[INFO] [2024-04-20 23:38:55]  train.py : Sample 46308 of the training set: {'input_ids': tensor([100349, 102028,  85361, 112272,     16,     24,     17,     23,  21216,
            17,     15,     15,     15,   7948,     53,    329,    318,    320,
            47,   3433,     72,   1020,   1579,    859,    701,  28507, 104328,
        100378, 104727,   1773,     16,     24,     19,     15, 104227, 107886,
          3837,  42411,  99992,  30709, 101430, 101193, 102508,  99418, 104534,
          3837, 103922,  99364, 113259, 100378, 103132,   1773, 102888,   3837,
         42411, 104247,  26940, 120036,  99686, 100027,  25067,   9909,  28085,
          3891,    665,     11,    220,     16,     24,     20,     18,   7552,
        104195,   9370, 106575,   1773,  42411, 115999,  62926, 100656, 108598,
         34187, 101251, 105651,  62945,  38035, 100378,  26940, 106020, 100211,
        101989,  25067,   9909,   3036,   4264,   4290,  24511,     11,    220,
            16,     24,     20,     21,  48272,  75882,  34718, 108438, 100648,
        102375, 102461, 107309, 109452, 120411, 110803, 106411, 106681,   1773,
        103957,   3837,  42411,  14777,  87256, 100152,  99487,  67279,  99959,
         16530,  25710,   9370,  99619,   8863,   1773,  42411,  11622,  99497,
        116170, 102627,   9909,   2082,  78728,    794,   3352,   7725,   7552,
        100172,  26940, 104264, 100145,  25067,      7,  97215,    782,  83657,
         19379,     11,    220,     16,     24,     20,     24,  69515,  11622,
        100861, 102261,  93488, 100172,  26940, 108481,  82847, 113724,  25067,
          9909,   3428,  54102,  35782,     11,    220,     16,     24,     21,
            23,  48272,  62926,  11622, 104156, 100841, 105115, 100721, 105658,
        103639, 100172,  26940, 100397,  99244, 108571,  99232,  25067,   9909,
            53,    558,    323,  47775,    361,     11,    220,     16,     24,
            21,     17,  48272,  24562, 105352, 100132, 100648, 102375,   3837,
        100161, 101961, 104428, 100648, 103430,   1773,  33975,     25,    220,
         14880, 106525, 100349, 102028,  13935, 101190, 100222, 101316, 105518,
         26850,  71703,     25,  10236,    121,    245, 102028,  13935, 101190,
        100222, 101316,   9909,  78425,  92112,    318,   3837,  35987,  13072,
            53,    329,    318,    393,   3433,     72,   1020,   1579,    859,
          7552, 109182, 104328, 100378, 104727,   3837, 100648, 111978, 114984,
            17,     15, 101186,     19,     15, 104227,   1773, 102965,   3837,
         42411, 107709, 102508,  99418,  17447, 104247, 104534,   3837, 101889,
        106438, 100378, 100650,   1773, 100648, 104727, 103132, 104847, 100694,
         19108,   3837, 104033,     16,     24,     20,     21, 104300, 100378,
         26940, 106020, 100211, 101989,  25067,   9909,   3036,   4264,   4290,
         24511,  48272,  99652, 108438,  42411, 102375, 102461, 107309,  13935,
        109452, 120411,  99787, 110803, 106411, 106681,   1773, 104043,   3837,
        106473, 115999, 106650,  92894, 102280, 104869,  90395,  57218, 108171,
        100527, 114224,  71817,  38182,  99721,   3407,  33975,     25,    220,
         42411, 104727,  34187, 102224, 100527, 104869,  26850,  71703,     25,
         10236,    121,    245, 102028,  13935, 101190, 100222, 101316, 104727,
        105593,  80942, 103215,   9370, 100378,   1773,  77557,  91111, 106020,
        100211, 101989,  25067,   9909,     16,     24,     20,     21,   7948,
          3837, 107119, 102461, 107309,  13935, 109452, 120411,  64359,  26940,
        104264, 100145,  25067,   9909,     16,     24,     20,     24,   7948,
          3837, 107119,   2082,  78728,    794,   3352,   7725,  64359,  26940,
        100397,  99244, 108571,  99232,  25067,   9909,     16,     24,     21,
            17,   7948,   3837, 107119, 104156, 107318, 105115,  13935,  99462,
         99458, 103639,   7552, 101034,  26940, 108481,  82847, 113724,  25067,
          9909,     16,     24,     21,     23,   7948,   3837, 107119,  98237,
         13935,  23384,  93488,  74276, 100001, 100378,  99902, 106844, 108124,
        104727, 101262,   3837,  97706,  17714,  42411,  57218, 108171, 102280,
        114224, 107298, 104257, 100133,   3407,  33975,     25,    220,  42411,
         99557, 103928, 101968, 102152, 107258, 105815, 100145,  11319,   4710,
         71703,     25,  10236,    121,    245, 102028,  13935, 101190, 100222,
        101316,  57218, 108171, 101968, 102152, 107258, 105815, 100145,   1773,
         45181, 113460,  27369, 100192,   3837,  42411,  57218, 102461, 107309,
         13935, 109452, 120411,   9909,   6828,  12597,    665,  63204,    354,
          7552, 104388,  90395,  18493, 104727,   9370, 100378,  26940, 106020,
        100211, 101989,  25067,  15946,  57218,  99517,  99721,   1773,  91572,
          3837, 106473,  99798,  57218,   2082,  78728,    794,   3352,   7725,
        104388,  62926, 104727,  34187,  99517, 107119,   9370, 100378,  26940,
        104264, 100145,  25067,   1773, 104043,   3837, 106473,  57218,  98237,
         13935,  23384,  93488,  99721, 104421,  34187, 100378,  26940, 108481,
         82847, 113724,  87243, 110133, 102280, 104328, 114224, 104156, 107318,
        105115,  13935,  99462,  99458, 103639,   9909,     34,  27009,    422,
          1952,  56023,   7552, 107258, 103430, 100145,   3837, 111733,  26940,
        100397,  99244, 108571,  99232,  25067,   9370, 104727,   3407,  33975,
            25,    220, 100648, 104040,  33108, 100772, 102021,  26850,  71703,
            25,  51461,    117,  16038, 113460,  27369,   3837, 100349, 102028,
         13935, 101190, 100222, 101316,   9370, 104040,  33108, 100772, 101321,
         42411, 106614, 109362, 101968,  57566,  33108, 102550,   3837, 104033,
        101968,   9370, 110803, 105997,   1773, 100648, 100378, 100663, 101207,
         23031, 101968, 100782,  63836,  62926, 101107, 104258,   9370, 101280,
         33108, 110803,   1773,  77557,  91111, 106020, 100211, 101989,  25067,
        108438, 102461, 107309,  13935, 109452, 120411, 110803, 106411, 106681,
         24968,  92894, 100663,  29524,  26940, 104264, 100145,  25067,   9909,
        107119,   2082,  78728,    794,   3352,   7725,  64359,  26940, 108481,
         82847, 113724,  25067,   9909, 107119,  98237,  13935,  23384,  93488,
          7552,  49567,  74763, 100629, 105790, 105117,   1773,  91572,   3837,
         42411, 104026, 100137,  67279,  99959, 104339,   3837,  57218, 108171,
        100527, 114224,  99721,  96050, 107222, 100378, 102707, 104847,  19108,
          1773, 151643]), 'labels': tensor([  -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,   -100,
          -100,   -100,   -100,   -100,   -100,   -100,  33975,     25,    220,
         14880, 106525, 100349, 102028,  13935, 101190, 100222, 101316, 105518,
         26850,  71703,     25,  10236,    121,    245, 102028,  13935, 101190,
        100222, 101316,   9909,  78425,  92112,    318,   3837,  35987,  13072,
            53,    329,    318,    393,   3433,     72,   1020,   1579,    859,
          7552, 109182, 104328, 100378, 104727,   3837, 100648, 111978, 114984,
            17,     15, 101186,     19,     15, 104227,   1773, 102965,   3837,
         42411, 107709, 102508,  99418,  17447, 104247, 104534,   3837, 101889,
        106438, 100378, 100650,   1773, 100648, 104727, 103132, 104847, 100694,
         19108,   3837, 104033,     16,     24,     20,     21, 104300, 100378,
         26940, 106020, 100211, 101989,  25067,   9909,   3036,   4264,   4290,
         24511,  48272,  99652, 108438,  42411, 102375, 102461, 107309,  13935,
        109452, 120411,  99787, 110803, 106411, 106681,   1773, 104043,   3837,
        106473, 115999, 106650,  92894, 102280, 104869,  90395,  57218, 108171,
        100527, 114224,  71817,  38182,  99721,   3407,  33975,     25,    220,
         42411, 104727,  34187, 102224, 100527, 104869,  26850,  71703,     25,
         10236,    121,    245, 102028,  13935, 101190, 100222, 101316, 104727,
        105593,  80942, 103215,   9370, 100378,   1773,  77557,  91111, 106020,
        100211, 101989,  25067,   9909,     16,     24,     20,     21,   7948,
          3837, 107119, 102461, 107309,  13935, 109452, 120411,  64359,  26940,
        104264, 100145,  25067,   9909,     16,     24,     20,     24,   7948,
          3837, 107119,   2082,  78728,    794,   3352,   7725,  64359,  26940,
        100397,  99244, 108571,  99232,  25067,   9909,     16,     24,     21,
            17,   7948,   3837, 107119, 104156, 107318, 105115,  13935,  99462,
         99458, 103639,   7552, 101034,  26940, 108481,  82847, 113724,  25067,
          9909,     16,     24,     21,     23,   7948,   3837, 107119,  98237,
         13935,  23384,  93488,  74276, 100001, 100378,  99902, 106844, 108124,
        104727, 101262,   3837,  97706,  17714,  42411,  57218, 108171, 102280,
        114224, 107298, 104257, 100133,   3407,  33975,     25,    220,  42411,
         99557, 103928, 101968, 102152, 107258, 105815, 100145,  11319,   4710,
         71703,     25,  10236,    121,    245, 102028,  13935, 101190, 100222,
        101316,  57218, 108171, 101968, 102152, 107258, 105815, 100145,   1773,
         45181, 113460,  27369, 100192,   3837,  42411,  57218, 102461, 107309,
         13935, 109452, 120411,   9909,   6828,  12597,    665,  63204,    354,
          7552, 104388,  90395,  18493, 104727,   9370, 100378,  26940, 106020,
        100211, 101989,  25067,  15946,  57218,  99517,  99721,   1773,  91572,
          3837, 106473,  99798,  57218,   2082,  78728,    794,   3352,   7725,
        104388,  62926, 104727,  34187,  99517, 107119,   9370, 100378,  26940,
        104264, 100145,  25067,   1773, 104043,   3837, 106473,  57218,  98237,
         13935,  23384,  93488,  99721, 104421,  34187, 100378,  26940, 108481,
         82847, 113724,  87243, 110133, 102280, 104328, 114224, 104156, 107318,
        105115,  13935,  99462,  99458, 103639,   9909,     34,  27009,    422,
          1952,  56023,   7552, 107258, 103430, 100145,   3837, 111733,  26940,
        100397,  99244, 108571,  99232,  25067,   9370, 104727,   3407,  33975,
            25,    220, 100648, 104040,  33108, 100772, 102021,  26850,  71703,
            25,  51461,    117,  16038, 113460,  27369,   3837, 100349, 102028,
         13935, 101190, 100222, 101316,   9370, 104040,  33108, 100772, 101321,
         42411, 106614, 109362, 101968,  57566,  33108, 102550,   3837, 104033,
        101968,   9370, 110803, 105997,   1773, 100648, 100378, 100663, 101207,
         23031, 101968, 100782,  63836,  62926, 101107, 104258,   9370, 101280,
         33108, 110803,   1773,  77557,  91111, 106020, 100211, 101989,  25067,
        108438, 102461, 107309,  13935, 109452, 120411, 110803, 106411, 106681,
         24968,  92894, 100663,  29524,  26940, 104264, 100145,  25067,   9909,
        107119,   2082,  78728,    794,   3352,   7725,  64359,  26940, 108481,
         82847, 113724,  25067,   9909, 107119,  98237,  13935,  23384,  93488,
          7552,  49567,  74763, 100629, 105790, 105117,   1773,  91572,   3837,
         42411, 104026, 100137,  67279,  99959, 104339,   3837,  57218, 108171,
        100527, 114224,  99721,  96050, 107222, 100378, 102707, 104847,  19108,
          1773, 151643])}, torch.Size([740]).
罗杰华汀1928～2000年Vadim (Plemiannikov), Roger法国电影导演。1940年代中期，他有一小段时间从事舞台剧演员，之后便转入电影生涯。最早，他担任《茱丽叶》（Juliette, 1953）一片的助理。他执导并共同编写了高度成功的异色电影《上帝创造女人》（And God Created Woman, 1956），该片奠定了他的妻子碧姬芭铎性感象征的地位。后来，他一再利用这个包赢不输的套数。他用史绰柏（Annette Stroyberg）拍《危险关系》(Dangerous Liaisons, 1959)，用珍芳达拍《太空英雌》（Barbarella, 1968），并用凯萨琳丹妮芙拍《乱世姐妹花》（Vice and Virtue, 1962），前两位都是他的妻子，最后一位则是他的情人。Human: 请告诉我罗杰·瓦迪姆是谁？

Assistant: 罗杰·瓦迪姆（Roger Vadim，全名Vadim Plemiannikov）是一位法国电影导演，他的职业生涯始于20世纪40年代。最初，他曾在舞台剧上担任演员，然后转向电影领域。他的导演生涯取得了许多成功，尤其是1956年的电影《上帝创造女人》（And God Created Woman），它奠定了他妻子碧姬·芭铎成为性感象征的地位。此外，他还执导了许多其他著名影片，并与多位知名女星进行过合作。

Human: 他导演了哪些知名影片？

Assistant: 罗杰·瓦迪姆导演了一些广受欢迎的电影。例如，《上帝创造女人》（1956年，主演碧姬·芭铎）、《危险关系》（1959年，主演Annette Stroyberg）、《乱世姐妹花》（1962年，主演凯瑟琳·德纳芙）以及《太空英雌》（1968年，主演简·方达）。这些电影不仅确立了他的导演地位，还为他与多位著名女星的合作提供了平台。

Human: 他跟那些女性明星有过私人关系？ 

Assistant: 罗杰·瓦迪姆与多位女性明星有过私人关系。从所提供的信息来看，他与碧姬·芭铎（Brigitte Bardot）结婚，并在导演的电影《上帝创造女人》中与她合作。同时，他还曾与Annette Stroyberg结婚并导演了她主演的电影《危险关系》。此外，他还与简·方达合作拍摄了电影《太空英雌》，并与著名法国女星凯瑟琳·德纳芙（Catherine Deneuve）有过情人关系，同时也是《乱世姐妹花》的导演。

Human: 他的风格和特点是什么？

Assistant: 根据所提供的信息，罗杰·瓦迪姆的风格和特点在于他善于发掘女性美和魅力，尤其是女性的性感一面。他的电影作品往往以女性为主角并表现她们的美丽和性感。例如，《上帝创造女人》奠定了碧姬·芭铎性感象征的地位；其他作品如《危险关系》（主演Annette Stroyberg）、《太空英雌》（主演简·方达）等也具有相似的特点。同时，他运用这种包赢的方法，与多位知名女星合作，在当时的电影市场上取得了成功。<|endoftext|>
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead: 
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
  warnings.warn(
[WARNING] [2024-04-20 23:38:55]  logging.py : Detected kernel version 4.19.91, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
[INFO] [2024-04-20 23:39:01]  distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 0
[INFO] [2024-04-20 23:39:11]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:39:21]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:39:31]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:39:41]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:39:51]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:01]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:11]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:21]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:31]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:41]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:51]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:01]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:11]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead: 
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead: 
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead: 
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead: 
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead: 
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead: 
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead: 
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
  warnings.warn(
[INFO] [2024-04-20 23:41:21]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:31]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:41]  distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:43]  distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 5
[INFO] [2024-04-20 23:41:43]  distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 1
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 4
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 6
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 7
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 3
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 2
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Rank 2: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Rank 0: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Rank 5: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Rank 4: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Rank 1: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Rank 3: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Rank 6: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44]  distributed_c10d.py : Rank 7: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Setting affinity for GPU 3 to 0fffff
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Setting affinity for GPU 1 to 0fffff
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Setting affinity for GPU 0 to 0fffff
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Setting affinity for GPU 2 to 0fffff
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 00/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 01/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 02/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 03/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 04/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 05/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 06/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 07/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 08/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 09/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 10/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 11/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 12/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 13/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 14/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 15/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 16/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 17/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 18/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 19/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 20/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 21/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 22/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 23/24 :    0   1   2   3   4   5   6   7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 00 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 00 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 00 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 00 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 00 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 01 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 00 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 01 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 01 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 01 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 01 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 02 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 01 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 02 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 02 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 02 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 03 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 02 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 02 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 03 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 03 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 03 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 03 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 04 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 03 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 04 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 04 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 05 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 04 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 04 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 04 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 05 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 05 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 06 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 05 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 05 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 06 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 06 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 07 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 06 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 00 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 06 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 05 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 07 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 07 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 08 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 00 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 07 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 01 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 06 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 08 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 08 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 09 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 01 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 08 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 02 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 07 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 09 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 09 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 07 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 02 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 10 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 09 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 03 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 08 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 10 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 10 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 08 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 03 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 10 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 04 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 09 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 11 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 11 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 11 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 09 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 04 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 11 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 05 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 12 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 12 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 12 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 10 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 10 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 05 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 12 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 06 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 13 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 11 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 11 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 06 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 13 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 13 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 07 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 13 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 14 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 12 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 12 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 07 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 14 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 08 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 14 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 15 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 13 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 08 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 15 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 09 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 15 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 14 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 09 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 16 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 10 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 15 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 16 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 10 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 13 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 14 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 16 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 17 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 11 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 16 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 17 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 11 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 14 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 15 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 17 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 18 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 12 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 17 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 18 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 15 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 12 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 18 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 19 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 13 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 18 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 19 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 16 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 13 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 19 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 20 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 14 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 19 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 20 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 17 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 14 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 20 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 21 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 15 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 16 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 20 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 21 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 18 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 15 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 21 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 22 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 16 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 21 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 22 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 16 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 22 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 23 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 22 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 17 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 23 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 17 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 23 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 18 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 19 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 20 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 21 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 22 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 17 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 23 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 18 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 19 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 20 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 21 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 22 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 23 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 23 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 18 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 19 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 19 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 20 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 20 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 21 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 21 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 22 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 22 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 23 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 23 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 00 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 01 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 02 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 03 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 04 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 05 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 06 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 00 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 00 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 00 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 00 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 07 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 01 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 01 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 01 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 01 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 08 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 02 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 09 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 02 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 03 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 10 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 03 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 11 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 04 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 04 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 12 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 05 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 05 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 02 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 00 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 13 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 06 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 06 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 03 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 14 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 01 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 07 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 07 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 04 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 15 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 02 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 08 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 08 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 05 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 16 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 03 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 09 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 09 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 02 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 06 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 17 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 04 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 10 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 10 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 03 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 07 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 18 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 05 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 11 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 11 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 04 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 19 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 08 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 06 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 12 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 12 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 05 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 09 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 20 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 07 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 13 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 13 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 06 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 10 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 08 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 14 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 14 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 07 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 11 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 09 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 15 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 15 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 08 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 12 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 10 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 16 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 16 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 09 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 13 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 21 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 11 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 17 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 17 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 10 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 14 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 22 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 18 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 18 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 11 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 15 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 23 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 19 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 00 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 20 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 19 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 12 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 16 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 01 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 21 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 17 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 13 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 20 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 02 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 12 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 22 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 21 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 18 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 13 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 23 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 22 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 14 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 03 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 19 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 14 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 23 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 20 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 15 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 21 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 16 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 15 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 22 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 16 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 23 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 17 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 04 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 18 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 05 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 19 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 17 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 06 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 20 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 18 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 07 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 21 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 19 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 08 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 22 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 20 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 09 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 23 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 21 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 10 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 22 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 11 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 23 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 12 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 13 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 14 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 15 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 16 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 17 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 18 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 19 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 20 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 21 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 22 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 23 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO comm 0x7f51f8008fb0 rank 7 nranks 8 cudaDev 7 busId 80 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO comm 0x7f4a10008fb0 rank 4 nranks 8 cudaDev 4 busId 50 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO comm 0x7f5660008fb0 rank 2 nranks 8 cudaDev 2 busId 30 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO comm 0x7f33d4008fb0 rank 3 nranks 8 cudaDev 3 busId 40 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO comm 0x7f3a58008fb0 rank 1 nranks 8 cudaDev 1 busId 20 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO comm 0x7ff06c008fb0 rank 0 nranks 8 cudaDev 0 busId 10 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Launch mode Parallel
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO comm 0x7fab28008fb0 rank 6 nranks 8 cudaDev 6 busId 70 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO comm 0x7f42b0008fb0 rank 5 nranks 8 cudaDev 5 busId 60 - Init COMPLETE
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  labels = [torch.tensor(x) for x in labels]

  0%|          | 0/3120 [00:00<?, ?it/s]train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  labels = [torch.tensor(x) for x in labels]

  0%|          | 1/3120 [00:05<5:09:32,  5.95s/it]
  0%|          | 2/3120 [00:10<4:30:19,  5.20s/it]
                                                  
{'loss': 1.3137, 'learning_rate': 2.564102564102564e-07, 'epoch': 0.0}

  0%|          | 2/3120 [00:10<4:30:19,  5.20s/it]
  0%|          | 3/3120 [00:15<4:25:21,  5.11s/it]
  0%|          | 4/3120 [00:20<4:22:57,  5.06s/it]
                                                  
{'loss': 1.2524, 'learning_rate': 5.128205128205128e-07, 'epoch': 0.01}

  0%|          | 4/3120 [00:20<4:22:57,  5.06s/it]
  0%|          | 5/3120 [00:25<4:18:16,  4.97s/it]
  0%|          | 6/3120 [00:30<4:24:20,  5.09s/it]
                                                  
{'loss': 1.2224, 'learning_rate': 7.692307692307694e-07, 'epoch': 0.01}

  0%|          | 6/3120 [00:30<4:24:20,  5.09s/it]
  0%|          | 7/3120 [00:35<4:14:34,  4.91s/it]
  0%|          | 8/3120 [00:40<4:13:38,  4.89s/it]
                                                  
{'loss': 1.2209, 'learning_rate': 1.0256410256410257e-06, 'epoch': 0.01}

  0%|          | 8/3120 [00:40<4:13:38,  4.89s/it]
  0%|          | 9/3120 [00:45<4:17:13,  4.96s/it]
  0%|          | 10/3120 [00:50<4:15:29,  4.93s/it]
                                                   
{'loss': 1.1776, 'learning_rate': 1.282051282051282e-06, 'epoch': 0.01}

  0%|          | 10/3120 [00:50<4:15:29,  4.93s/it]
  0%|          | 11/3120 [00:54<4:13:45,  4.90s/it]
  0%|          | 12/3120 [01:00<4:19:03,  5.00s/it]
                                                   
{'loss': 1.124, 'learning_rate': 1.5384615384615387e-06, 'epoch': 0.02}

  0%|          | 12/3120 [01:00<4:19:03,  5.00s/it]
  0%|          | 13/3120 [01:05<4:21:25,  5.05s/it]
  0%|          | 14/3120 [01:10<4:18:59,  5.00s/it]
                                                   
{'loss': 1.1545, 'learning_rate': 1.794871794871795e-06, 'epoch': 0.02}

  0%|          | 14/3120 [01:10<4:18:59,  5.00s/it]
  0%|          | 15/3120 [01:15<4:17:56,  4.98s/it]
  1%|          | 16/3120 [01:19<4:13:28,  4.90s/it]
                                                   
{'loss': 1.1266, 'learning_rate': 2.0512820512820513e-06, 'epoch': 0.02}

  1%|          | 16/3120 [01:19<4:13:28,  4.90s/it]
  1%|          | 17/3120 [01:24<4:13:51,  4.91s/it]
  1%|          | 18/3120 [01:29<4:11:40,  4.87s/it]
                                                   
{'loss': 1.1284, 'learning_rate': 2.307692307692308e-06, 'epoch': 0.02}

  1%|          | 18/3120 [01:29<4:11:40,  4.87s/it]
  1%|          | 19/3120 [01:34<4:15:58,  4.95s/it]
  1%|          | 20/3120 [01:39<4:17:45,  4.99s/it]
                                                   
{'loss': 1.0904, 'learning_rate': 2.564102564102564e-06, 'epoch': 0.03}

  1%|          | 20/3120 [01:39<4:17:45,  4.99s/it]
  1%|          | 21/3120 [01:44<4:20:04,  5.04s/it]
  1%|          | 22/3120 [01:49<4:19:15,  5.02s/it]
                                                   
{'loss': 1.0894, 'learning_rate': 2.8205128205128207e-06, 'epoch': 0.03}

  1%|          | 22/3120 [01:49<4:19:15,  5.02s/it]
  1%|          | 23/3120 [01:54<4:13:35,  4.91s/it]
  1%|          | 24/3120 [01:59<4:12:04,  4.89s/it]
                                                   
{'loss': 1.0946, 'learning_rate': 3.0769230769230774e-06, 'epoch': 0.03}

  1%|          | 24/3120 [01:59<4:12:04,  4.89s/it]
  1%|          | 25/3120 [02:04<4:16:22,  4.97s/it]
  1%|          | 26/3120 [02:09<4:17:09,  4.99s/it]
                                                   
{'loss': 1.1264, 'learning_rate': 3.3333333333333333e-06, 'epoch': 0.03}

  1%|          | 26/3120 [02:09<4:17:09,  4.99s/it]
  1%|          | 27/3120 [02:14<4:22:14,  5.09s/it]
  1%|          | 28/3120 [02:20<4:23:03,  5.10s/it]
                                                   
{'loss': 1.0495, 'learning_rate': 3.58974358974359e-06, 'epoch': 0.04}

  1%|          | 28/3120 [02:20<4:23:03,  5.10s/it]
  1%|          | 29/3120 [02:25<4:23:57,  5.12s/it]
  1%|          | 30/3120 [02:30<4:21:40,  5.08s/it]
                                                   
{'loss': 1.0859, 'learning_rate': 3.846153846153847e-06, 'epoch': 0.04}

  1%|          | 30/3120 [02:30<4:21:40,  5.08s/it]
  1%|          | 31/3120 [02:35<4:20:06,  5.05s/it]
  1%|          | 32/3120 [02:40<4:19:37,  5.04s/it]
                                                   
{'loss': 1.0676, 'learning_rate': 4.102564102564103e-06, 'epoch': 0.04}

  1%|          | 32/3120 [02:40<4:19:37,  5.04s/it]
  1%|          | 33/3120 [02:45<4:17:08,  5.00s/it]
  1%|          | 34/3120 [02:49<4:13:46,  4.93s/it]
                                                   
{'loss': 1.0675, 'learning_rate': 4.358974358974359e-06, 'epoch': 0.04}

  1%|          | 34/3120 [02:49<4:13:46,  4.93s/it]
  1%|          | 35/3120 [02:55<4:20:52,  5.07s/it]
  1%|          | 36/3120 [03:00<4:16:29,  4.99s/it]
                                                   
{'loss': 1.0887, 'learning_rate': 4.615384615384616e-06, 'epoch': 0.05}

  1%|          | 36/3120 [03:00<4:16:29,  4.99s/it]
  1%|          | 37/3120 [03:04<4:10:37,  4.88s/it]
  1%|          | 38/3120 [03:10<4:18:19,  5.03s/it]
                                                   
{'loss': 1.0623, 'learning_rate': 4.871794871794872e-06, 'epoch': 0.05}

  1%|          | 38/3120 [03:10<4:18:19,  5.03s/it]
  1%|▏         | 39/3120 [03:15<4:17:54,  5.02s/it]
  1%|▏         | 40/3120 [03:19<4:15:23,  4.98s/it]
                                                   
{'loss': 1.0979, 'learning_rate': 5.128205128205128e-06, 'epoch': 0.05}

  1%|▏         | 40/3120 [03:19<4:15:23,  4.98s/it]
  1%|▏         | 41/3120 [03:25<4:17:23,  5.02s/it]
  1%|▏         | 42/3120 [03:30<4:18:37,  5.04s/it]
                                                   
{'loss': 1.0907, 'learning_rate': 5.384615384615385e-06, 'epoch': 0.05}

  1%|▏         | 42/3120 [03:30<4:18:37,  5.04s/it]
  1%|▏         | 43/3120 [03:35<4:16:56,  5.01s/it]
  1%|▏         | 44/3120 [03:40<4:16:37,  5.01s/it]
                                                   
{'loss': 1.1248, 'learning_rate': 5.641025641025641e-06, 'epoch': 0.06}

  1%|▏         | 44/3120 [03:40<4:16:37,  5.01s/it]
  1%|▏         | 45/3120 [03:45<4:17:21,  5.02s/it]
  1%|▏         | 46/3120 [03:49<4:13:21,  4.95s/it]
                                                   
{'loss': 1.0717, 'learning_rate': 5.897435897435898e-06, 'epoch': 0.06}

  1%|▏         | 46/3120 [03:49<4:13:21,  4.95s/it]
  2%|▏         | 47/3120 [03:54<4:11:45,  4.92s/it]
  2%|▏         | 48/3120 [03:59<4:14:07,  4.96s/it]
                                                   
{'loss': 1.0472, 'learning_rate': 6.153846153846155e-06, 'epoch': 0.06}

  2%|▏         | 48/3120 [03:59<4:14:07,  4.96s/it]
  2%|▏         | 49/3120 [04:05<4:16:39,  5.01s/it]
  2%|▏         | 50/3120 [04:10<4:22:51,  5.14s/it]
                                                   
{'loss': 1.0468, 'learning_rate': 6.410256410256412e-06, 'epoch': 0.06}

  2%|▏         | 50/3120 [04:10<4:22:51,  5.14s/it]
  2%|▏         | 51/3120 [04:15<4:22:01,  5.12s/it]
  2%|▏         | 52/3120 [04:20<4:21:38,  5.12s/it]
                                                   
{'loss': 1.1036, 'learning_rate': 6.666666666666667e-06, 'epoch': 0.07}

  2%|▏         | 52/3120 [04:20<4:21:38,  5.12s/it]
  2%|▏         | 53/3120 [04:25<4:14:09,  4.97s/it]
  2%|▏         | 54/3120 [04:30<4:17:59,  5.05s/it]
                                                   
{'loss': 1.0443, 'learning_rate': 6.923076923076923e-06, 'epoch': 0.07}

  2%|▏         | 54/3120 [04:30<4:17:59,  5.05s/it]
  2%|▏         | 55/3120 [04:35<4:22:28,  5.14s/it]
  2%|▏         | 56/3120 [04:40<4:17:06,  5.03s/it]
                                                   
{'loss': 1.1326, 'learning_rate': 7.17948717948718e-06, 'epoch': 0.07}

  2%|▏         | 56/3120 [04:40<4:17:06,  5.03s/it]
  2%|▏         | 57/3120 [04:45<4:20:17,  5.10s/it]
  2%|▏         | 58/3120 [04:51<4:21:30,  5.12s/it]
                                                   
{'loss': 1.0333, 'learning_rate': 7.435897435897437e-06, 'epoch': 0.07}

  2%|▏         | 58/3120 [04:51<4:21:30,  5.12s/it]
  2%|▏         | 59/3120 [04:56<4:27:37,  5.25s/it]
  2%|▏         | 60/3120 [05:01<4:19:43,  5.09s/it]
                                                   
{'loss': 1.0917, 'learning_rate': 7.692307692307694e-06, 'epoch': 0.08}

  2%|▏         | 60/3120 [05:01<4:19:43,  5.09s/it]
  2%|▏         | 61/3120 [05:06<4:19:53,  5.10s/it]
  2%|▏         | 62/3120 [05:11<4:17:56,  5.06s/it]
                                                   
{'loss': 1.1045, 'learning_rate': 7.948717948717949e-06, 'epoch': 0.08}

  2%|▏         | 62/3120 [05:11<4:17:56,  5.06s/it]
  2%|▏         | 63/3120 [05:16<4:16:42,  5.04s/it]
  2%|▏         | 64/3120 [05:21<4:16:39,  5.04s/it]
                                                   
{'loss': 1.0835, 'learning_rate': 8.205128205128205e-06, 'epoch': 0.08}

  2%|▏         | 64/3120 [05:21<4:16:39,  5.04s/it]
  2%|▏         | 65/3120 [05:26<4:16:02,  5.03s/it]
  2%|▏         | 66/3120 [05:31<4:12:51,  4.97s/it]
                                                   
{'loss': 1.0796, 'learning_rate': 8.461538461538462e-06, 'epoch': 0.08}

  2%|▏         | 66/3120 [05:31<4:12:51,  4.97s/it]
  2%|▏         | 67/3120 [05:36<4:16:50,  5.05s/it]
  2%|▏         | 68/3120 [05:41<4:16:13,  5.04s/it]
                                                   
{'loss': 1.1246, 'learning_rate': 8.717948717948719e-06, 'epoch': 0.09}

  2%|▏         | 68/3120 [05:41<4:16:13,  5.04s/it]
  2%|▏         | 69/3120 [05:46<4:11:33,  4.95s/it]
  2%|▏         | 70/3120 [05:51<4:14:18,  5.00s/it]
                                                   
{'loss': 1.0481, 'learning_rate': 8.974358974358976e-06, 'epoch': 0.09}

  2%|▏         | 70/3120 [05:51<4:14:18,  5.00s/it]
  2%|▏         | 71/3120 [05:56<4:18:51,  5.09s/it]
  2%|▏         | 72/3120 [06:01<4:14:01,  5.00s/it]
                                                   
{'loss': 1.0502, 'learning_rate': 9.230769230769232e-06, 'epoch': 0.09}

  2%|▏         | 72/3120 [06:01<4:14:01,  5.00s/it]
  2%|▏         | 73/3120 [06:06<4:16:21,  5.05s/it]
  2%|▏         | 74/3120 [06:11<4:18:53,  5.10s/it]
                                                   
{'loss': 1.1048, 'learning_rate': 9.487179487179487e-06, 'epoch': 0.09}

  2%|▏         | 74/3120 [06:11<4:18:53,  5.10s/it]
  2%|▏         | 75/3120 [06:16<4:07:41,  4.88s/it]
  2%|▏         | 76/3120 [06:21<4:12:14,  4.97s/it]
                                                   
{'loss': 1.1112, 'learning_rate': 9.743589743589744e-06, 'epoch': 0.1}

  2%|▏         | 76/3120 [06:21<4:12:14,  4.97s/it]
  2%|▏         | 77/3120 [06:26<4:09:02,  4.91s/it]
  2%|▎         | 78/3120 [06:31<4:12:58,  4.99s/it]
                                                   
{'loss': 1.1319, 'learning_rate': 1e-05, 'epoch': 0.1}

  2%|▎         | 78/3120 [06:31<4:12:58,  4.99s/it]
  3%|▎         | 79/3120 [06:35<4:06:21,  4.86s/it]
  3%|▎         | 80/3120 [06:40<4:09:48,  4.93s/it]
                                                   
{'loss': 1.095, 'learning_rate': 1.0256410256410256e-05, 'epoch': 0.1}

  3%|▎         | 80/3120 [06:40<4:09:48,  4.93s/it]
  3%|▎         | 81/3120 [06:45<4:08:17,  4.90s/it]
  3%|▎         | 82/3120 [06:50<4:10:50,  4.95s/it]
                                                   
{'loss': 1.0906, 'learning_rate': 1.0512820512820514e-05, 'epoch': 0.11}

  3%|▎         | 82/3120 [06:50<4:10:50,  4.95s/it]
  3%|▎         | 83/3120 [06:55<4:08:15,  4.90s/it]
  3%|▎         | 84/3120 [07:00<4:04:59,  4.84s/it]
                                                   
{'loss': 1.1147, 'learning_rate': 1.076923076923077e-05, 'epoch': 0.11}

  3%|▎         | 84/3120 [07:00<4:04:59,  4.84s/it]
  3%|▎         | 85/3120 [07:05<4:07:57,  4.90s/it]
  3%|▎         | 86/3120 [07:10<4:06:00,  4.87s/it]
                                                   
{'loss': 1.0714, 'learning_rate': 1.1025641025641028e-05, 'epoch': 0.11}

  3%|▎         | 86/3120 [07:10<4:06:00,  4.87s/it]
  3%|▎         | 87/3120 [07:14<4:04:46,  4.84s/it]
  3%|▎         | 88/3120 [07:20<4:10:26,  4.96s/it]
                                                   
{'loss': 1.1125, 'learning_rate': 1.1282051282051283e-05, 'epoch': 0.11}

  3%|▎         | 88/3120 [07:20<4:10:26,  4.96s/it]
  3%|▎         | 89/3120 [07:25<4:10:23,  4.96s/it]
  3%|▎         | 90/3120 [07:30<4:13:09,  5.01s/it]
                                                   
{'loss': 1.0838, 'learning_rate': 1.1538461538461538e-05, 'epoch': 0.12}

  3%|▎         | 90/3120 [07:30<4:13:09,  5.01s/it]
  3%|▎         | 91/3120 [07:35<4:15:36,  5.06s/it]
  3%|▎         | 92/3120 [07:40<4:19:58,  5.15s/it]
                                                   
{'loss': 1.1119, 'learning_rate': 1.1794871794871796e-05, 'epoch': 0.12}

  3%|▎         | 92/3120 [07:40<4:19:58,  5.15s/it]
  3%|▎         | 93/3120 [07:45<4:19:42,  5.15s/it]
  3%|▎         | 94/3120 [07:50<4:06:00,  4.88s/it]
                                                   
{'loss': 1.1097, 'learning_rate': 1.2051282051282051e-05, 'epoch': 0.12}

  3%|▎         | 94/3120 [07:50<4:06:00,  4.88s/it]
  3%|▎         | 95/3120 [07:55<4:06:02,  4.88s/it]
  3%|▎         | 96/3120 [08:00<4:05:59,  4.88s/it]
                                                   
{'loss': 1.1344, 'learning_rate': 1.230769230769231e-05, 'epoch': 0.12}

  3%|▎         | 96/3120 [08:00<4:05:59,  4.88s/it]
  3%|▎         | 97/3120 [08:05<4:10:01,  4.96s/it]
  3%|▎         | 98/3120 [08:10<4:11:53,  5.00s/it]
                                                   
{'loss': 1.1108, 'learning_rate': 1.2564102564102565e-05, 'epoch': 0.13}

  3%|▎         | 98/3120 [08:10<4:11:53,  5.00s/it]
  3%|▎         | 99/3120 [08:15<4:14:00,  5.04s/it]
  3%|▎         | 100/3120 [08:20<4:13:59,  5.05s/it]
                                                    
{'loss': 1.1308, 'learning_rate': 1.2820512820512823e-05, 'epoch': 0.13}

  3%|▎         | 100/3120 [08:20<4:13:59,  5.05s/it]
  3%|▎         | 101/3120 [08:25<4:10:01,  4.97s/it]
  3%|▎         | 102/3120 [08:30<4:14:31,  5.06s/it]
                                                    
{'loss': 1.1198, 'learning_rate': 1.3076923076923078e-05, 'epoch': 0.13}

  3%|▎         | 102/3120 [08:30<4:14:31,  5.06s/it]
  3%|▎         | 103/3120 [08:35<4:14:00,  5.05s/it]
  3%|▎         | 104/3120 [08:40<4:09:57,  4.97s/it]
                                                    
{'loss': 1.1589, 'learning_rate': 1.3333333333333333e-05, 'epoch': 0.13}

  3%|▎         | 104/3120 [08:40<4:09:57,  4.97s/it]
  3%|▎         | 105/3120 [08:45<4:07:17,  4.92s/it]
  3%|▎         | 106/3120 [08:50<4:08:27,  4.95s/it]
                                                    
{'loss': 1.1473, 'learning_rate': 1.3589743589743592e-05, 'epoch': 0.14}

  3%|▎         | 106/3120 [08:50<4:08:27,  4.95s/it]
  3%|▎         | 107/3120 [08:55<4:07:32,  4.93s/it]
  3%|▎         | 108/3120 [09:00<4:14:11,  5.06s/it]
                                                    
{'loss': 1.0719, 'learning_rate': 1.3846153846153847e-05, 'epoch': 0.14}

  3%|▎         | 108/3120 [09:00<4:14:11,  5.06s/it]
  3%|▎         | 109/3120 [09:05<4:15:52,  5.10s/it]
  4%|▎         | 110/3120 [09:10<4:12:53,  5.04s/it]
                                                    
{'loss': 1.1418, 'learning_rate': 1.4102564102564105e-05, 'epoch': 0.14}

  4%|▎         | 110/3120 [09:10<4:12:53,  5.04s/it]
  4%|▎         | 111/3120 [09:15<4:09:07,  4.97s/it]
  4%|▎         | 112/3120 [09:20<4:08:00,  4.95s/it]
                                                    
{'loss': 1.1056, 'learning_rate': 1.435897435897436e-05, 'epoch': 0.14}

  4%|▎         | 112/3120 [09:20<4:08:00,  4.95s/it]
  4%|▎         | 113/3120 [09:25<4:06:00,  4.91s/it]
  4%|▎         | 114/3120 [09:30<4:10:08,  4.99s/it]
                                                    
{'loss': 1.1027, 'learning_rate': 1.4615384615384615e-05, 'epoch': 0.15}

  4%|▎         | 114/3120 [09:30<4:10:08,  4.99s/it]
  4%|▎         | 115/3120 [09:35<4:07:20,  4.94s/it]
  4%|▎         | 116/3120 [09:39<4:04:49,  4.89s/it]
                                                    
{'loss': 1.1641, 'learning_rate': 1.4871794871794874e-05, 'epoch': 0.15}

  4%|▎         | 116/3120 [09:39<4:04:49,  4.89s/it]
  4%|▍         | 117/3120 [09:44<4:00:32,  4.81s/it]
  4%|▍         | 118/3120 [09:49<4:02:09,  4.84s/it]
                                                    
{'loss': 1.094, 'learning_rate': 1.5128205128205129e-05, 'epoch': 0.15}

  4%|▍         | 118/3120 [09:49<4:02:09,  4.84s/it]
  4%|▍         | 119/3120 [09:54<4:03:05,  4.86s/it]
  4%|▍         | 120/3120 [09:59<4:02:00,  4.84s/it]
                                                    
{'loss': 1.1528, 'learning_rate': 1.5384615384615387e-05, 'epoch': 0.15}

  4%|▍         | 120/3120 [09:59<4:02:00,  4.84s/it]
  4%|▍         | 121/3120 [10:03<3:59:40,  4.80s/it]
  4%|▍         | 122/3120 [10:08<3:59:02,  4.78s/it]
                                                    
{'loss': 1.1395, 'learning_rate': 1.5641025641025644e-05, 'epoch': 0.16}

  4%|▍         | 122/3120 [10:08<3:59:02,  4.78s/it]
  4%|▍         | 123/3120 [10:13<3:59:32,  4.80s/it]
  4%|▍         | 124/3120 [10:18<3:59:33,  4.80s/it]
                                                    
{'loss': 1.1329, 'learning_rate': 1.5897435897435897e-05, 'epoch': 0.16}

  4%|▍         | 124/3120 [10:18<3:59:33,  4.80s/it]
  4%|▍         | 125/3120 [10:22<4:00:21,  4.82s/it]
  4%|▍         | 126/3120 [10:28<4:05:51,  4.93s/it]
                                                    
{'loss': 1.0875, 'learning_rate': 1.6153846153846154e-05, 'epoch': 0.16}

  4%|▍         | 126/3120 [10:28<4:05:51,  4.93s/it]
  4%|▍         | 127/3120 [10:33<4:10:37,  5.02s/it]
  4%|▍         | 128/3120 [10:38<4:07:12,  4.96s/it]
                                                    
{'loss': 1.1137, 'learning_rate': 1.641025641025641e-05, 'epoch': 0.16}

  4%|▍         | 128/3120 [10:38<4:07:12,  4.96s/it]
  4%|▍         | 129/3120 [10:43<4:05:39,  4.93s/it]
  4%|▍         | 130/3120 [10:47<4:00:48,  4.83s/it]
                                                    
{'loss': 1.1722, 'learning_rate': 1.6666666666666667e-05, 'epoch': 0.17}

  4%|▍         | 130/3120 [10:47<4:00:48,  4.83s/it]
  4%|▍         | 131/3120 [10:52<4:01:56,  4.86s/it]
  4%|▍         | 132/3120 [10:57<4:01:44,  4.85s/it]
                                                    
{'loss': 1.0949, 'learning_rate': 1.6923076923076924e-05, 'epoch': 0.17}

  4%|▍         | 132/3120 [10:57<4:01:44,  4.85s/it]
  4%|▍         | 133/3120 [11:02<3:59:58,  4.82s/it]
  4%|▍         | 134/3120 [11:07<4:03:26,  4.89s/it]
                                                    
{'loss': 1.1418, 'learning_rate': 1.717948717948718e-05, 'epoch': 0.17}

  4%|▍         | 134/3120 [11:07<4:03:26,  4.89s/it]
  4%|▍         | 135/3120 [11:11<4:00:45,  4.84s/it]
  4%|▍         | 136/3120 [11:16<4:00:21,  4.83s/it]
                                                    
{'loss': 1.158, 'learning_rate': 1.7435897435897438e-05, 'epoch': 0.17}

  4%|▍         | 136/3120 [11:16<4:00:21,  4.83s/it]
  4%|▍         | 137/3120 [11:21<4:01:38,  4.86s/it]
  4%|▍         | 138/3120 [11:26<4:07:56,  4.99s/it]
                                                    
{'loss': 1.1303, 'learning_rate': 1.7692307692307694e-05, 'epoch': 0.18}

  4%|▍         | 138/3120 [11:26<4:07:56,  4.99s/it]
  4%|▍         | 139/3120 [11:31<4:05:42,  4.95s/it]
  4%|▍         | 140/3120 [11:36<4:04:06,  4.91s/it]
                                                    
{'loss': 1.189, 'learning_rate': 1.794871794871795e-05, 'epoch': 0.18}

  4%|▍         | 140/3120 [11:36<4:04:06,  4.91s/it]
  5%|▍         | 141/3120 [11:41<4:02:23,  4.88s/it]
  5%|▍         | 142/3120 [11:46<3:59:21,  4.82s/it]
                                                    
{'loss': 1.1031, 'learning_rate': 1.8205128205128208e-05, 'epoch': 0.18}

  5%|▍         | 142/3120 [11:46<3:59:21,  4.82s/it]
  5%|▍         | 143/3120 [11:51<4:01:35,  4.87s/it]
  5%|▍         | 144/3120 [11:56<4:04:34,  4.93s/it]
                                                    
{'loss': 1.1348, 'learning_rate': 1.8461538461538465e-05, 'epoch': 0.18}

  5%|▍         | 144/3120 [11:56<4:04:34,  4.93s/it]
  5%|▍         | 145/3120 [12:01<4:05:26,  4.95s/it]
  5%|▍         | 146/3120 [12:06<4:07:53,  5.00s/it]
                                                    
{'loss': 1.1124, 'learning_rate': 1.8717948717948718e-05, 'epoch': 0.19}

  5%|▍         | 146/3120 [12:06<4:07:53,  5.00s/it]
  5%|▍         | 147/3120 [12:11<4:06:47,  4.98s/it]
  5%|▍         | 148/3120 [12:15<4:03:05,  4.91s/it]
                                                    
{'loss': 1.1338, 'learning_rate': 1.8974358974358975e-05, 'epoch': 0.19}

  5%|▍         | 148/3120 [12:15<4:03:05,  4.91s/it]
  5%|▍         | 149/3120 [12:20<4:03:21,  4.91s/it]
  5%|▍         | 150/3120 [12:25<4:05:27,  4.96s/it]
                                                    
{'loss': 1.1585, 'learning_rate': 1.923076923076923e-05, 'epoch': 0.19}

  5%|▍         | 150/3120 [12:25<4:05:27,  4.96s/it]
  5%|▍         | 151/3120 [12:31<4:06:58,  4.99s/it]
  5%|▍         | 152/3120 [12:36<4:10:16,  5.06s/it]
                                                    
{'loss': 1.1598, 'learning_rate': 1.9487179487179488e-05, 'epoch': 0.19}

  5%|▍         | 152/3120 [12:36<4:10:16,  5.06s/it]
  5%|▍         | 153/3120 [12:41<4:07:57,  5.01s/it]
  5%|▍         | 154/3120 [12:46<4:13:48,  5.13s/it]
                                                    
{'loss': 1.186, 'learning_rate': 1.9743589743589745e-05, 'epoch': 0.2}

  5%|▍         | 154/3120 [12:46<4:13:48,  5.13s/it]
  5%|▍         | 155/3120 [12:51<4:10:50,  5.08s/it]
  5%|▌         | 156/3120 [12:56<4:05:23,  4.97s/it]
                                                    
{'loss': 1.1578, 'learning_rate': 2e-05, 'epoch': 0.2}

  5%|▌         | 156/3120 [12:56<4:05:23,  4.97s/it]
  5%|▌         | 157/3120 [13:01<4:04:45,  4.96s/it]
  5%|▌         | 158/3120 [13:06<4:08:18,  5.03s/it]
                                                    
{'loss': 1.2017, 'learning_rate': 1.9999977531546567e-05, 'epoch': 0.2}

  5%|▌         | 158/3120 [13:06<4:08:18,  5.03s/it]
  5%|▌         | 159/3120 [13:11<4:09:51,  5.06s/it]
  5%|▌         | 160/3120 [13:16<4:08:19,  5.03s/it]
                                                    
{'loss': 1.1756, 'learning_rate': 1.999991012628722e-05, 'epoch': 0.21}

  5%|▌         | 160/3120 [13:16<4:08:19,  5.03s/it]
  5%|▌         | 161/3120 [13:21<4:03:53,  4.95s/it]
  5%|▌         | 162/3120 [13:26<4:03:36,  4.94s/it]
                                                    
{'loss': 1.1565, 'learning_rate': 1.9999797784524866e-05, 'epoch': 0.21}

  5%|▌         | 162/3120 [13:26<4:03:36,  4.94s/it]
  5%|▌         | 163/3120 [13:30<4:01:07,  4.89s/it]
  5%|▌         | 164/3120 [13:35<4:00:27,  4.88s/it]
                                                    
{'loss': 1.1721, 'learning_rate': 1.999964050676434e-05, 'epoch': 0.21}

  5%|▌         | 164/3120 [13:35<4:00:27,  4.88s/it]
  5%|▌         | 165/3120 [13:40<3:58:55,  4.85s/it]
  5%|▌         | 166/3120 [13:45<3:56:49,  4.81s/it]
                                                    
{'loss': 1.1388, 'learning_rate': 1.9999438293712382e-05, 'epoch': 0.21}

  5%|▌         | 166/3120 [13:45<3:56:49,  4.81s/it]
  5%|▌         | 167/3120 [13:50<4:01:13,  4.90s/it]
  5%|▌         | 168/3120 [13:54<3:54:17,  4.76s/it]
                                                    
{'loss': 1.1761, 'learning_rate': 1.999919114627769e-05, 'epoch': 0.22}

  5%|▌         | 168/3120 [13:54<3:54:17,  4.76s/it]
  5%|▌         | 169/3120 [13:59<3:58:04,  4.84s/it]
  5%|▌         | 170/3120 [14:04<3:59:23,  4.87s/it]
                                                    
{'loss': 1.1619, 'learning_rate': 1.999889906557086e-05, 'epoch': 0.22}

  5%|▌         | 170/3120 [14:04<3:59:23,  4.87s/it]
  5%|▌         | 171/3120 [14:09<3:55:47,  4.80s/it]
  6%|▌         | 172/3120 [14:14<4:02:30,  4.94s/it]
                                                    
{'loss': 1.1452, 'learning_rate': 1.999856205290442e-05, 'epoch': 0.22}

  6%|▌         | 172/3120 [14:14<4:02:30,  4.94s/it]
  6%|▌         | 173/3120 [14:19<4:01:15,  4.91s/it]
  6%|▌         | 174/3120 [14:24<4:00:19,  4.89s/it]
                                                    
{'loss': 1.1364, 'learning_rate': 1.9998180109792793e-05, 'epoch': 0.22}

  6%|▌         | 174/3120 [14:24<4:00:19,  4.89s/it]
  6%|▌         | 175/3120 [14:29<3:58:43,  4.86s/it]
  6%|▌         | 176/3120 [14:34<4:00:38,  4.90s/it]
                                                    
{'loss': 1.1926, 'learning_rate': 1.999775323795232e-05, 'epoch': 0.23}

  6%|▌         | 176/3120 [14:34<4:00:38,  4.90s/it]
  6%|▌         | 177/3120 [14:38<3:57:19,  4.84s/it]
  6%|▌         | 178/3120 [14:43<3:52:03,  4.73s/it]
                                                    
{'loss': 1.1646, 'learning_rate': 1.9997281439301218e-05, 'epoch': 0.23}

  6%|▌         | 178/3120 [14:43<3:52:03,  4.73s/it]
  6%|▌         | 179/3120 [14:48<4:01:35,  4.93s/it]
  6%|▌         | 180/3120 [14:53<3:59:53,  4.90s/it]
                                                    
{'loss': 1.1484, 'learning_rate': 1.999676471595962e-05, 'epoch': 0.23}

  6%|▌         | 180/3120 [14:53<3:59:53,  4.90s/it]
  6%|▌         | 181/3120 [14:59<4:08:41,  5.08s/it]
  6%|▌         | 182/3120 [15:03<4:05:00,  5.00s/it]
                                                    
{'loss': 1.1563, 'learning_rate': 1.9996203070249516e-05, 'epoch': 0.23}

  6%|▌         | 182/3120 [15:03<4:05:00,  5.00s/it]
  6%|▌         | 183/3120 [15:08<4:00:26,  4.91s/it]
  6%|▌         | 184/3120 [15:14<4:11:50,  5.15s/it]
                                                    
{'loss': 1.1448, 'learning_rate': 1.9995596504694764e-05, 'epoch': 0.24}

  6%|▌         | 184/3120 [15:14<4:11:50,  5.15s/it]
  6%|▌         | 185/3120 [15:19<4:06:19,  5.04s/it]
  6%|▌         | 186/3120 [15:24<4:07:19,  5.06s/it]
                                                    
{'loss': 1.1539, 'learning_rate': 1.9994945022021085e-05, 'epoch': 0.24}

  6%|▌         | 186/3120 [15:24<4:07:19,  5.06s/it]
  6%|▌         | 187/3120 [15:28<4:01:36,  4.94s/it]
  6%|▌         | 188/3120 [15:34<4:06:14,  5.04s/it]
                                                    
{'loss': 1.1047, 'learning_rate': 1.999424862515604e-05, 'epoch': 0.24}

  6%|▌         | 188/3120 [15:34<4:06:14,  5.04s/it]
  6%|▌         | 189/3120 [15:39<4:07:10,  5.06s/it]
  6%|▌         | 190/3120 [15:44<4:03:42,  4.99s/it]
                                                    
{'loss': 1.145, 'learning_rate': 1.999350731722902e-05, 'epoch': 0.24}

  6%|▌         | 190/3120 [15:44<4:03:42,  4.99s/it]
  6%|▌         | 191/3120 [15:48<4:02:51,  4.98s/it]
  6%|▌         | 192/3120 [15:53<3:59:35,  4.91s/it]
                                                    
{'loss': 1.2109, 'learning_rate': 1.9992721101571238e-05, 'epoch': 0.25}

  6%|▌         | 192/3120 [15:53<3:59:35,  4.91s/it]
  6%|▌         | 193/3120 [15:58<4:02:11,  4.96s/it]
  6%|▌         | 194/3120 [16:03<3:56:56,  4.86s/it]
                                                    
{'loss': 1.1766, 'learning_rate': 1.9991889981715696e-05, 'epoch': 0.25}

  6%|▌         | 194/3120 [16:03<3:56:56,  4.86s/it]
  6%|▋         | 195/3120 [16:08<3:58:04,  4.88s/it]
  6%|▋         | 196/3120 [16:13<3:55:08,  4.83s/it]
                                                    
{'loss': 1.1013, 'learning_rate': 1.99910139613972e-05, 'epoch': 0.25}

  6%|▋         | 196/3120 [16:13<3:55:08,  4.83s/it]
  6%|▋         | 197/3120 [16:17<3:53:50,  4.80s/it]
  6%|▋         | 198/3120 [16:22<3:57:21,  4.87s/it]
                                                    
{'loss': 1.1406, 'learning_rate': 1.9990093044552304e-05, 'epoch': 0.25}

  6%|▋         | 198/3120 [16:22<3:57:21,  4.87s/it]
  6%|▋         | 199/3120 [16:27<3:56:30,  4.86s/it]
  6%|▋         | 200/3120 [16:32<3:48:51,  4.70s/it]
                                                    
{'loss': 1.1561, 'learning_rate': 1.998912723531933e-05, 'epoch': 0.26}

  6%|▋         | 200/3120 [16:32<3:48:51,  4.70s/it]
  6%|▋         | 201/3120 [16:37<3:54:14,  4.81s/it]
  6%|▋         | 202/3120 [16:41<3:51:24,  4.76s/it]
                                                    
{'loss': 1.1512, 'learning_rate': 1.9988116538038326e-05, 'epoch': 0.26}

  6%|▋         | 202/3120 [16:41<3:51:24,  4.76s/it]
  7%|▋         | 203/3120 [16:46<3:48:35,  4.70s/it]
  7%|▋         | 204/3120 [16:51<3:54:32,  4.83s/it]
                                                    
{'loss': 1.1941, 'learning_rate': 1.9987060957251047e-05, 'epoch': 0.26}

  7%|▋         | 204/3120 [16:51<3:54:32,  4.83s/it]
  7%|▋         | 205/3120 [16:56<3:56:29,  4.87s/it]
  7%|▋         | 206/3120 [17:00<3:52:53,  4.80s/it]
                                                    
{'loss': 1.1429, 'learning_rate': 1.9985960497700953e-05, 'epoch': 0.26}

  7%|▋         | 206/3120 [17:00<3:52:53,  4.80s/it]
  7%|▋         | 207/3120 [17:05<3:54:42,  4.83s/it]
  7%|▋         | 208/3120 [17:11<4:00:58,  4.97s/it]
                                                    
{'loss': 1.1689, 'learning_rate': 1.9984815164333163e-05, 'epoch': 0.27}

  7%|▋         | 208/3120 [17:11<4:00:58,  4.97s/it]
  7%|▋         | 209/3120 [17:15<3:56:31,  4.88s/it]
  7%|▋         | 210/3120 [17:20<3:55:26,  4.85s/it]
                                                    
{'loss': 1.201, 'learning_rate': 1.998362496229446e-05, 'epoch': 0.27}

  7%|▋         | 210/3120 [17:20<3:55:26,  4.85s/it]
  7%|▋         | 211/3120 [17:25<3:53:42,  4.82s/it]
  7%|▋         | 212/3120 [17:30<4:02:01,  4.99s/it]
                                                    
{'loss': 1.1526, 'learning_rate': 1.998238989693323e-05, 'epoch': 0.27}

  7%|▋         | 212/3120 [17:30<4:02:01,  4.99s/it]
  7%|▋         | 213/3120 [17:35<3:57:45,  4.91s/it]
  7%|▋         | 214/3120 [17:40<4:02:57,  5.02s/it]
                                                    
{'loss': 1.1902, 'learning_rate': 1.998110997379949e-05, 'epoch': 0.27}

  7%|▋         | 214/3120 [17:40<4:02:57,  5.02s/it]
  7%|▋         | 215/3120 [17:45<4:02:56,  5.02s/it]
  7%|▋         | 216/3120 [17:50<4:02:49,  5.02s/it]
                                                    
{'loss': 1.1751, 'learning_rate': 1.997978519864481e-05, 'epoch': 0.28}

  7%|▋         | 216/3120 [17:50<4:02:49,  5.02s/it]
  7%|▋         | 217/3120 [17:55<4:00:55,  4.98s/it]
  7%|▋         | 218/3120 [18:01<4:06:36,  5.10s/it]
                                                    
{'loss': 1.1176, 'learning_rate': 1.997841557742232e-05, 'epoch': 0.28}

  7%|▋         | 218/3120 [18:01<4:06:36,  5.10s/it]
  7%|▋         | 219/3120 [18:06<4:06:39,  5.10s/it]
  7%|▋         | 220/3120 [18:11<4:03:40,  5.04s/it]
                                                    
{'loss': 1.1814, 'learning_rate': 1.9977001116286675e-05, 'epoch': 0.28}

  7%|▋         | 220/3120 [18:11<4:03:40,  5.04s/it]
  7%|▋         | 221/3120 [18:15<3:58:43,  4.94s/it]
  7%|▋         | 222/3120 [18:21<4:03:31,  5.04s/it]
                                                    
{'loss': 1.1603, 'learning_rate': 1.9975541821594028e-05, 'epoch': 0.28}

  7%|▋         | 222/3120 [18:21<4:03:31,  5.04s/it]
  7%|▋         | 223/3120 [18:26<4:02:55,  5.03s/it]
  7%|▋         | 224/3120 [18:31<4:04:32,  5.07s/it]
                                                    
{'loss': 1.1679, 'learning_rate': 1.9974037699901993e-05, 'epoch': 0.29}

  7%|▋         | 224/3120 [18:31<4:04:32,  5.07s/it]
  7%|▋         | 225/3120 [18:36<4:07:09,  5.12s/it]
  7%|▋         | 226/3120 [18:41<4:03:27,  5.05s/it]
                                                    
{'loss': 1.1865, 'learning_rate': 1.9972488757969635e-05, 'epoch': 0.29}

  7%|▋         | 226/3120 [18:41<4:03:27,  5.05s/it]
  7%|▋         | 227/3120 [18:46<4:08:04,  5.14s/it]
  7%|▋         | 228/3120 [18:51<4:07:05,  5.13s/it]
                                                    
{'loss': 1.1097, 'learning_rate': 1.9970895002757413e-05, 'epoch': 0.29}

  7%|▋         | 228/3120 [18:51<4:07:05,  5.13s/it]
  7%|▋         | 229/3120 [18:57<4:08:01,  5.15s/it]
  7%|▋         | 230/3120 [19:01<4:03:18,  5.05s/it]
                                                    
{'loss': 1.1651, 'learning_rate': 1.9969256441427173e-05, 'epoch': 0.29}

  7%|▋         | 230/3120 [19:01<4:03:18,  5.05s/it]
  7%|▋         | 231/3120 [19:07<4:09:06,  5.17s/it]
  7%|▋         | 232/3120 [19:12<4:07:37,  5.14s/it]
                                                    
{'loss': 1.1874, 'learning_rate': 1.9967573081342103e-05, 'epoch': 0.3}

  7%|▋         | 232/3120 [19:12<4:07:37,  5.14s/it]
  7%|▋         | 233/3120 [19:17<4:03:01,  5.05s/it]
  8%|▊         | 234/3120 [19:21<3:58:47,  4.96s/it]
                                                    
{'loss': 1.1893, 'learning_rate': 1.99658449300667e-05, 'epoch': 0.3}

  8%|▊         | 234/3120 [19:21<3:58:47,  4.96s/it]
  8%|▊         | 235/3120 [19:26<3:53:40,  4.86s/it]
  8%|▊         | 236/3120 [19:31<3:59:34,  4.98s/it]
                                                    
{'loss': 1.1387, 'learning_rate': 1.9964071995366744e-05, 'epoch': 0.3}

  8%|▊         | 236/3120 [19:31<3:59:34,  4.98s/it]
  8%|▊         | 237/3120 [19:37<4:01:50,  5.03s/it]
  8%|▊         | 238/3120 [19:42<4:02:13,  5.04s/it]
                                                    
{'loss': 1.155, 'learning_rate': 1.9962254285209256e-05, 'epoch': 0.31}

  8%|▊         | 238/3120 [19:42<4:02:13,  5.04s/it]
  8%|▊         | 239/3120 [19:46<3:55:27,  4.90s/it]
  8%|▊         | 240/3120 [19:51<3:54:53,  4.89s/it]
                                                    
{'loss': 1.1563, 'learning_rate': 1.9960391807762462e-05, 'epoch': 0.31}

  8%|▊         | 240/3120 [19:51<3:54:53,  4.89s/it]
  8%|▊         | 241/3120 [19:56<3:54:06,  4.88s/it]
  8%|▊         | 242/3120 [20:01<3:55:41,  4.91s/it]
                                                    
{'loss': 1.1502, 'learning_rate': 1.9958484571395756e-05, 'epoch': 0.31}

  8%|▊         | 242/3120 [20:01<3:55:41,  4.91s/it]
  8%|▊         | 243/3120 [20:06<3:55:23,  4.91s/it]
  8%|▊         | 244/3120 [20:11<3:55:39,  4.92s/it]
                                                    
{'loss': 1.15, 'learning_rate': 1.9956532584679676e-05, 'epoch': 0.31}

  8%|▊         | 244/3120 [20:11<3:55:39,  4.92s/it]
  8%|▊         | 245/3120 [20:16<4:01:07,  5.03s/it]
  8%|▊         | 246/3120 [20:21<3:58:23,  4.98s/it]
                                                    
{'loss': 1.2085, 'learning_rate': 1.995453585638584e-05, 'epoch': 0.32}

  8%|▊         | 246/3120 [20:21<3:58:23,  4.98s/it]
  8%|▊         | 247/3120 [20:26<4:05:30,  5.13s/it]
  8%|▊         | 248/3120 [20:31<4:04:49,  5.11s/it]
                                                    
{'loss': 1.0918, 'learning_rate': 1.995249439548693e-05, 'epoch': 0.32}

  8%|▊         | 248/3120 [20:31<4:04:49,  5.11s/it]
  8%|▊         | 249/3120 [20:36<3:59:59,  5.02s/it]
  8%|▊         | 250/3120 [20:41<4:00:01,  5.02s/it]
                                                    
{'loss': 1.1681, 'learning_rate': 1.9950408211156637e-05, 'epoch': 0.32}

  8%|▊         | 250/3120 [20:41<4:00:01,  5.02s/it]
  8%|▊         | 251/3120 [20:46<3:56:00,  4.94s/it]
  8%|▊         | 252/3120 [20:51<3:50:55,  4.83s/it]
                                                    
{'loss': 1.1875, 'learning_rate': 1.994827731276963e-05, 'epoch': 0.32}

  8%|▊         | 252/3120 [20:51<3:50:55,  4.83s/it]
  8%|▊         | 253/3120 [20:56<3:57:58,  4.98s/it]
  8%|▊         | 254/3120 [21:01<3:53:08,  4.88s/it]
                                                    
{'loss': 1.1399, 'learning_rate': 1.9946101709901513e-05, 'epoch': 0.33}

  8%|▊         | 254/3120 [21:01<3:53:08,  4.88s/it]
  8%|▊         | 255/3120 [21:05<3:53:19,  4.89s/it]
  8%|▊         | 256/3120 [21:10<3:51:15,  4.84s/it]
                                                    
{'loss': 1.1883, 'learning_rate': 1.994388141232876e-05, 'epoch': 0.33}

  8%|▊         | 256/3120 [21:10<3:51:15,  4.84s/it]
  8%|▊         | 257/3120 [21:15<3:50:13,  4.82s/it]
  8%|▊         | 258/3120 [21:20<3:50:45,  4.84s/it]
                                                    
{'loss': 1.1678, 'learning_rate': 1.9941616430028713e-05, 'epoch': 0.33}

  8%|▊         | 258/3120 [21:20<3:50:45,  4.84s/it]
  8%|▊         | 259/3120 [21:25<3:55:11,  4.93s/it]
  8%|▊         | 260/3120 [21:30<3:51:36,  4.86s/it]
                                                    
{'loss': 1.2118, 'learning_rate': 1.9939306773179498e-05, 'epoch': 0.33}

  8%|▊         | 260/3120 [21:30<3:51:36,  4.86s/it]
  8%|▊         | 261/3120 [21:35<3:53:25,  4.90s/it]
  8%|▊         | 262/3120 [21:39<3:49:54,  4.83s/it]
                                                    
{'loss': 1.1649, 'learning_rate': 1.9936952452159995e-05, 'epoch': 0.34}

  8%|▊         | 262/3120 [21:39<3:49:54,  4.83s/it]
  8%|▊         | 263/3120 [21:44<3:54:00,  4.91s/it]
  8%|▊         | 264/3120 [21:49<3:46:03,  4.75s/it]
                                                    
{'loss': 1.1545, 'learning_rate': 1.9934553477549795e-05, 'epoch': 0.34}

  8%|▊         | 264/3120 [21:49<3:46:03,  4.75s/it]
  8%|▊         | 265/3120 [21:54<3:49:40,  4.83s/it]
  9%|▊         | 266/3120 [21:59<4:00:12,  5.05s/it]
                                                    
{'loss': 1.1819, 'learning_rate': 1.9932109860129154e-05, 'epoch': 0.34}

  9%|▊         | 266/3120 [21:59<4:00:12,  5.05s/it]
  9%|▊         | 267/3120 [22:04<3:58:45,  5.02s/it]
  9%|▊         | 268/3120 [22:10<4:04:41,  5.15s/it]
                                                    
{'loss': 1.1457, 'learning_rate': 1.992962161087893e-05, 'epoch': 0.34}

  9%|▊         | 268/3120 [22:10<4:04:41,  5.15s/it]
  9%|▊         | 269/3120 [22:15<3:59:35,  5.04s/it]
  9%|▊         | 270/3120 [22:19<3:56:48,  4.99s/it]
                                                    
{'loss': 1.1614, 'learning_rate': 1.992708874098054e-05, 'epoch': 0.35}

  9%|▊         | 270/3120 [22:19<3:56:48,  4.99s/it]
  9%|▊         | 271/3120 [22:25<3:58:37,  5.03s/it]
  9%|▊         | 272/3120 [22:29<3:55:14,  4.96s/it]
                                                    
{'loss': 1.1435, 'learning_rate': 1.9924511261815928e-05, 'epoch': 0.35}

  9%|▊         | 272/3120 [22:29<3:55:14,  4.96s/it]
  9%|▉         | 273/3120 [22:34<3:51:34,  4.88s/it]
  9%|▉         | 274/3120 [22:39<3:55:11,  4.96s/it]
                                                    
{'loss': 1.1353, 'learning_rate': 1.992188918496748e-05, 'epoch': 0.35}

  9%|▉         | 274/3120 [22:39<3:55:11,  4.96s/it]
  9%|▉         | 275/3120 [22:44<3:53:17,  4.92s/it]
  9%|▉         | 276/3120 [22:49<3:53:46,  4.93s/it]
                                                    
{'loss': 1.1642, 'learning_rate': 1.9919222522217998e-05, 'epoch': 0.35}

  9%|▉         | 276/3120 [22:49<3:53:46,  4.93s/it]
  9%|▉         | 277/3120 [22:54<3:51:24,  4.88s/it]
  9%|▉         | 278/3120 [22:59<3:57:30,  5.01s/it]
                                                    
{'loss': 1.1312, 'learning_rate': 1.991651128555064e-05, 'epoch': 0.36}

  9%|▉         | 278/3120 [22:59<3:57:30,  5.01s/it]
  9%|▉         | 279/3120 [23:04<3:58:41,  5.04s/it]
  9%|▉         | 280/3120 [23:09<3:55:10,  4.97s/it]
                                                    
{'loss': 1.1559, 'learning_rate': 1.9913755487148874e-05, 'epoch': 0.36}

  9%|▉         | 280/3120 [23:09<3:55:10,  4.97s/it]
  9%|▉         | 281/3120 [23:14<3:51:23,  4.89s/it]
  9%|▉         | 282/3120 [23:18<3:47:11,  4.80s/it]
                                                    
{'loss': 1.1477, 'learning_rate': 1.9910955139396395e-05, 'epoch': 0.36}

  9%|▉         | 282/3120 [23:18<3:47:11,  4.80s/it]
  9%|▉         | 283/3120 [23:23<3:50:07,  4.87s/it]
  9%|▉         | 284/3120 [23:28<3:48:04,  4.83s/it]
                                                    
{'loss': 1.1672, 'learning_rate': 1.9908110254877107e-05, 'epoch': 0.36}

  9%|▉         | 284/3120 [23:28<3:48:04,  4.83s/it]
  9%|▉         | 285/3120 [23:33<3:45:00,  4.76s/it]
  9%|▉         | 286/3120 [23:38<3:47:37,  4.82s/it]
                                                    
{'loss': 1.1704, 'learning_rate': 1.990522084637503e-05, 'epoch': 0.37}

  9%|▉         | 286/3120 [23:38<3:47:37,  4.82s/it]
  9%|▉         | 287/3120 [23:43<3:49:26,  4.86s/it]
  9%|▉         | 288/3120 [23:48<3:54:26,  4.97s/it]
                                                    
{'loss': 1.1242, 'learning_rate': 1.990228692687429e-05, 'epoch': 0.37}

  9%|▉         | 288/3120 [23:48<3:54:26,  4.97s/it]
  9%|▉         | 289/3120 [23:53<3:53:35,  4.95s/it]
  9%|▉         | 290/3120 [23:58<3:53:36,  4.95s/it]
                                                    
{'loss': 1.164, 'learning_rate': 1.9899308509558998e-05, 'epoch': 0.37}

  9%|▉         | 290/3120 [23:58<3:53:36,  4.95s/it]
  9%|▉         | 291/3120 [24:02<3:52:11,  4.92s/it]
  9%|▉         | 292/3120 [24:08<3:57:21,  5.04s/it]
                                                    
{'loss': 1.1559, 'learning_rate': 1.9896285607813245e-05, 'epoch': 0.37}

  9%|▉         | 292/3120 [24:08<3:57:21,  5.04s/it]
  9%|▉         | 293/3120 [24:13<3:54:12,  4.97s/it]
  9%|▉         | 294/3120 [24:18<3:54:22,  4.98s/it]
                                                    
{'loss': 1.1351, 'learning_rate': 1.9893218235221016e-05, 'epoch': 0.38}

  9%|▉         | 294/3120 [24:18<3:54:22,  4.98s/it]
  9%|▉         | 295/3120 [24:22<3:52:35,  4.94s/it]
  9%|▉         | 296/3120 [24:27<3:53:47,  4.97s/it]
                                                    
{'loss': 1.2004, 'learning_rate': 1.989010640556614e-05, 'epoch': 0.38}

  9%|▉         | 296/3120 [24:27<3:53:47,  4.97s/it]
 10%|▉         | 297/3120 [24:32<3:51:01,  4.91s/it]
 10%|▉         | 298/3120 [24:37<3:54:16,  4.98s/it]
                                                    
{'loss': 1.2014, 'learning_rate': 1.988695013283221e-05, 'epoch': 0.38}

 10%|▉         | 298/3120 [24:37<3:54:16,  4.98s/it]
 10%|▉         | 299/3120 [24:43<3:56:40,  5.03s/it]
 10%|▉         | 300/3120 [24:47<3:55:06,  5.00s/it]
                                                    
{'loss': 1.136, 'learning_rate': 1.988374943120254e-05, 'epoch': 0.38}

 10%|▉         | 300/3120 [24:47<3:55:06,  5.00s/it]
 10%|▉         | 301/3120 [24:53<3:56:04,  5.02s/it]
 10%|▉         | 302/3120 [24:57<3:52:04,  4.94s/it]
                                                    
{'loss': 1.1745, 'learning_rate': 1.9880504315060096e-05, 'epoch': 0.39}

 10%|▉         | 302/3120 [24:57<3:52:04,  4.94s/it]
 10%|▉         | 303/3120 [25:02<3:53:14,  4.97s/it]
 10%|▉         | 304/3120 [25:07<3:56:06,  5.03s/it]
                                                    
{'loss': 1.1619, 'learning_rate': 1.9877214798987428e-05, 'epoch': 0.39}

 10%|▉         | 304/3120 [25:08<3:56:06,  5.03s/it]
 10%|▉         | 305/3120 [25:12<3:54:18,  4.99s/it]
 10%|▉         | 306/3120 [25:18<3:55:48,  5.03s/it]
                                                    
{'loss': 1.1502, 'learning_rate': 1.9873880897766597e-05, 'epoch': 0.39}

 10%|▉         | 306/3120 [25:18<3:55:48,  5.03s/it]
 10%|▉         | 307/3120 [25:23<3:56:19,  5.04s/it]
 10%|▉         | 308/3120 [25:27<3:50:46,  4.92s/it]
                                                    
{'loss': 1.1542, 'learning_rate': 1.9870502626379127e-05, 'epoch': 0.39}

 10%|▉         | 308/3120 [25:27<3:50:46,  4.92s/it]
 10%|▉         | 309/3120 [25:32<3:44:14,  4.79s/it]
 10%|▉         | 310/3120 [25:37<3:48:49,  4.89s/it]
                                                    
{'loss': 1.104, 'learning_rate': 1.9867080000005933e-05, 'epoch': 0.4}

 10%|▉         | 310/3120 [25:37<3:48:49,  4.89s/it]
 10%|▉         | 311/3120 [25:41<3:44:37,  4.80s/it]
 10%|█         | 312/3120 [25:46<3:46:24,  4.84s/it]
                                                    
{'loss': 1.1589, 'learning_rate': 1.9863613034027224e-05, 'epoch': 0.4}

 10%|█         | 312/3120 [25:46<3:46:24,  4.84s/it]
 10%|█         | 313/3120 [25:51<3:47:29,  4.86s/it]
 10%|█         | 314/3120 [25:56<3:47:12,  4.86s/it]
                                                    
{'loss': 1.1237, 'learning_rate': 1.9860101744022485e-05, 'epoch': 0.4}

 10%|█         | 314/3120 [25:56<3:47:12,  4.86s/it]
 10%|█         | 315/3120 [26:01<3:47:12,  4.86s/it]
 10%|█         | 316/3120 [26:06<3:47:52,  4.88s/it]
                                                    
{'loss': 1.1635, 'learning_rate': 1.985654614577036e-05, 'epoch': 0.41}

 10%|█         | 316/3120 [26:06<3:47:52,  4.88s/it]
 10%|█         | 317/3120 [26:11<3:51:25,  4.95s/it]
 10%|█         | 318/3120 [26:16<3:51:13,  4.95s/it]
                                                    
{'loss': 1.1501, 'learning_rate': 1.985294625524861e-05, 'epoch': 0.41}

 10%|█         | 318/3120 [26:16<3:51:13,  4.95s/it]
 10%|█         | 319/3120 [26:21<3:54:46,  5.03s/it]
 10%|█         | 320/3120 [26:26<3:53:58,  5.01s/it]
                                                    
{'loss': 1.1633, 'learning_rate': 1.9849302088634034e-05, 'epoch': 0.41}

 10%|█         | 320/3120 [26:26<3:53:58,  5.01s/it]
 10%|█         | 321/3120 [26:31<3:45:21,  4.83s/it]
 10%|█         | 322/3120 [26:35<3:42:28,  4.77s/it]
                                                    
{'loss': 1.1703, 'learning_rate': 1.9845613662302384e-05, 'epoch': 0.41}

 10%|█         | 322/3120 [26:35<3:42:28,  4.77s/it]
 10%|█         | 323/3120 [26:40<3:45:35,  4.84s/it]
 10%|█         | 324/3120 [26:45<3:45:19,  4.84s/it]
                                                    
{'loss': 1.1496, 'learning_rate': 1.9841880992828306e-05, 'epoch': 0.42}

 10%|█         | 324/3120 [26:45<3:45:19,  4.84s/it]
 10%|█         | 325/3120 [26:50<3:49:16,  4.92s/it]
 10%|█         | 326/3120 [26:55<3:53:17,  5.01s/it]
                                                    
{'loss': 1.22, 'learning_rate': 1.9838104096985266e-05, 'epoch': 0.42}

 10%|█         | 326/3120 [26:55<3:53:17,  5.01s/it]
 10%|█         | 327/3120 [27:00<3:48:22,  4.91s/it]
 11%|█         | 328/3120 [27:05<3:55:45,  5.07s/it]
                                                    
{'loss': 1.1372, 'learning_rate': 1.9834282991745465e-05, 'epoch': 0.42}

 11%|█         | 328/3120 [27:05<3:55:45,  5.07s/it]
 11%|█         | 329/3120 [27:10<3:53:11,  5.01s/it]
 11%|█         | 330/3120 [27:15<3:53:41,  5.03s/it]
                                                    
{'loss': 1.1988, 'learning_rate': 1.9830417694279766e-05, 'epoch': 0.42}

 11%|█         | 330/3120 [27:15<3:53:41,  5.03s/it]
 11%|█         | 331/3120 [27:20<3:50:25,  4.96s/it]
 11%|█         | 332/3120 [27:25<3:53:08,  5.02s/it]
                                                    
{'loss': 1.1268, 'learning_rate': 1.9826508221957624e-05, 'epoch': 0.43}

 11%|█         | 332/3120 [27:25<3:53:08,  5.02s/it]
 11%|█         | 333/3120 [27:30<3:50:14,  4.96s/it]
 11%|█         | 334/3120 [27:35<3:48:34,  4.92s/it]
                                                    
{'loss': 1.1662, 'learning_rate': 1.9822554592346996e-05, 'epoch': 0.43}

 11%|█         | 334/3120 [27:35<3:48:34,  4.92s/it]
 11%|█         | 335/3120 [27:40<3:54:41,  5.06s/it]
 11%|█         | 336/3120 [27:45<3:52:41,  5.01s/it]
                                                    
{'loss': 1.1371, 'learning_rate': 1.981855682321427e-05, 'epoch': 0.43}

 11%|█         | 336/3120 [27:45<3:52:41,  5.01s/it]
 11%|█         | 337/3120 [27:50<3:54:27,  5.05s/it]
 11%|█         | 338/3120 [27:55<3:48:20,  4.92s/it]
                                                    
{'loss': 1.1876, 'learning_rate': 1.981451493252418e-05, 'epoch': 0.43}

 11%|█         | 338/3120 [27:55<3:48:20,  4.92s/it]
 11%|█         | 339/3120 [28:00<3:47:04,  4.90s/it]
 11%|█         | 340/3120 [28:05<3:48:21,  4.93s/it]
                                                    
{'loss': 1.1379, 'learning_rate': 1.981042893843974e-05, 'epoch': 0.44}

 11%|█         | 340/3120 [28:05<3:48:21,  4.93s/it]
 11%|█         | 341/3120 [28:10<3:47:54,  4.92s/it]
 11%|█         | 342/3120 [28:15<3:45:05,  4.86s/it]
                                                    
{'loss': 1.1569, 'learning_rate': 1.9806298859322143e-05, 'epoch': 0.44}

 11%|█         | 342/3120 [28:15<3:45:05,  4.86s/it]
 11%|█         | 343/3120 [28:19<3:44:38,  4.85s/it]
 11%|█         | 344/3120 [28:24<3:47:07,  4.91s/it]
                                                    
{'loss': 1.0988, 'learning_rate': 1.980212471373068e-05, 'epoch': 0.44}

 11%|█         | 344/3120 [28:24<3:47:07,  4.91s/it]
 11%|█         | 345/3120 [28:30<3:54:55,  5.08s/it]
 11%|█         | 346/3120 [28:36<4:04:29,  5.29s/it]
                                                    
{'loss': 1.137, 'learning_rate': 1.979790652042268e-05, 'epoch': 0.44}

 11%|█         | 346/3120 [28:36<4:04:29,  5.29s/it]
 11%|█         | 347/3120 [28:41<4:05:22,  5.31s/it]
 11%|█         | 348/3120 [28:46<4:01:38,  5.23s/it]
                                                    
{'loss': 1.1034, 'learning_rate': 1.979364429835339e-05, 'epoch': 0.45}

 11%|█         | 348/3120 [28:46<4:01:38,  5.23s/it]
 11%|█         | 349/3120 [28:51<3:56:09,  5.11s/it]
 11%|█         | 350/3120 [28:56<3:54:55,  5.09s/it]
                                                    
{'loss': 1.1428, 'learning_rate': 1.9789338066675924e-05, 'epoch': 0.45}

 11%|█         | 350/3120 [28:56<3:54:55,  5.09s/it]
 11%|█▏        | 351/3120 [29:01<3:51:47,  5.02s/it]
 11%|█▏        | 352/3120 [29:06<3:48:25,  4.95s/it]
                                                    
{'loss': 1.1757, 'learning_rate': 1.978498784474115e-05, 'epoch': 0.45}

 11%|█▏        | 352/3120 [29:06<3:48:25,  4.95s/it]
 11%|█▏        | 353/3120 [29:10<3:41:18,  4.80s/it]
 11%|█▏        | 354/3120 [29:15<3:41:49,  4.81s/it]
                                                    
{'loss': 1.186, 'learning_rate': 1.978059365209762e-05, 'epoch': 0.45}

 11%|█▏        | 354/3120 [29:15<3:41:49,  4.81s/it]
 11%|█▏        | 355/3120 [29:20<3:41:30,  4.81s/it]
 11%|█▏        | 356/3120 [29:25<3:42:19,  4.83s/it]
                                                    
{'loss': 1.1064, 'learning_rate': 1.9776155508491482e-05, 'epoch': 0.46}

 11%|█▏        | 356/3120 [29:25<3:42:19,  4.83s/it]
 11%|█▏        | 357/3120 [29:29<3:40:11,  4.78s/it]
 11%|█▏        | 358/3120 [29:35<3:48:15,  4.96s/it]
                                                    
{'loss': 1.2007, 'learning_rate': 1.977167343386638e-05, 'epoch': 0.46}

 11%|█▏        | 358/3120 [29:35<3:48:15,  4.96s/it]
 12%|█▏        | 359/3120 [29:40<3:47:45,  4.95s/it]
 12%|█▏        | 360/3120 [29:45<3:52:30,  5.05s/it]
                                                    
{'loss': 1.2118, 'learning_rate': 1.9767147448363366e-05, 'epoch': 0.46}

 12%|█▏        | 360/3120 [29:45<3:52:30,  5.05s/it]
 12%|█▏        | 361/3120 [29:50<3:53:38,  5.08s/it]
 12%|█▏        | 362/3120 [29:55<3:55:44,  5.13s/it]
                                                    
{'loss': 1.134, 'learning_rate': 1.9762577572320826e-05, 'epoch': 0.46}

 12%|█▏        | 362/3120 [29:55<3:55:44,  5.13s/it]
 12%|█▏        | 363/3120 [30:01<3:58:22,  5.19s/it]
 12%|█▏        | 364/3120 [30:05<3:53:50,  5.09s/it]
                                                    
{'loss': 1.1829, 'learning_rate': 1.9757963826274357e-05, 'epoch': 0.47}

 12%|█▏        | 364/3120 [30:05<3:53:50,  5.09s/it]
 12%|█▏        | 365/3120 [30:10<3:51:24,  5.04s/it]
 12%|█▏        | 366/3120 [30:15<3:46:54,  4.94s/it]
                                                    
{'loss': 1.1299, 'learning_rate': 1.975330623095672e-05, 'epoch': 0.47}

 12%|█▏        | 366/3120 [30:15<3:46:54,  4.94s/it]
 12%|█▏        | 367/3120 [30:20<3:49:03,  4.99s/it]
 12%|█▏        | 368/3120 [30:25<3:47:52,  4.97s/it]
                                                    
{'loss': 1.198, 'learning_rate': 1.97486048072977e-05, 'epoch': 0.47}

 12%|█▏        | 368/3120 [30:25<3:47:52,  4.97s/it]
 12%|█▏        | 369/3120 [30:30<3:42:25,  4.85s/it]
 12%|█▏        | 370/3120 [30:35<3:45:49,  4.93s/it]
                                                    
{'loss': 1.1158, 'learning_rate': 1.9743859576424044e-05, 'epoch': 0.47}

 12%|█▏        | 370/3120 [30:35<3:45:49,  4.93s/it]
 12%|█▏        | 371/3120 [30:40<3:55:59,  5.15s/it]
 12%|█▏        | 372/3120 [30:45<3:49:17,  5.01s/it]
                                                    
{'loss': 1.167, 'learning_rate': 1.9739070559659347e-05, 'epoch': 0.48}

 12%|█▏        | 372/3120 [30:45<3:49:17,  5.01s/it]
 12%|█▏        | 373/3120 [30:50<3:53:45,  5.11s/it]
 12%|█▏        | 374/3120 [30:55<3:47:37,  4.97s/it]
                                                    
{'loss': 1.1606, 'learning_rate': 1.9734237778523977e-05, 'epoch': 0.48}

 12%|█▏        | 374/3120 [30:55<3:47:37,  4.97s/it]
 12%|█▏        | 375/3120 [31:00<3:47:23,  4.97s/it]
 12%|█▏        | 376/3120 [31:05<3:47:23,  4.97s/it]
                                                    
{'loss': 1.1466, 'learning_rate': 1.972936125473495e-05, 'epoch': 0.48}

 12%|█▏        | 376/3120 [31:05<3:47:23,  4.97s/it]
 12%|█▏        | 377/3120 [31:10<3:46:32,  4.96s/it]
 12%|█▏        | 378/3120 [31:15<3:47:34,  4.98s/it]
                                                    
{'loss': 1.1417, 'learning_rate': 1.9724441010205865e-05, 'epoch': 0.48}

 12%|█▏        | 378/3120 [31:15<3:47:34,  4.98s/it]
 12%|█▏        | 379/3120 [31:20<3:45:10,  4.93s/it]
 12%|█▏        | 380/3120 [31:24<3:41:26,  4.85s/it]
                                                    
{'loss': 1.192, 'learning_rate': 1.9719477067046768e-05, 'epoch': 0.49}

 12%|█▏        | 380/3120 [31:24<3:41:26,  4.85s/it]
 12%|█▏        | 381/3120 [31:29<3:41:36,  4.85s/it]
 12%|█▏        | 382/3120 [31:35<3:49:46,  5.04s/it]
                                                    
{'loss': 1.1691, 'learning_rate': 1.971446944756409e-05, 'epoch': 0.49}

 12%|█▏        | 382/3120 [31:35<3:49:46,  5.04s/it]
 12%|█▏        | 383/3120 [31:40<3:48:13,  5.00s/it]
 12%|█▏        | 384/3120 [31:45<3:51:29,  5.08s/it]
                                                    
{'loss': 1.1153, 'learning_rate': 1.9709418174260523e-05, 'epoch': 0.49}

 12%|█▏        | 384/3120 [31:45<3:51:29,  5.08s/it]
 12%|█▏        | 385/3120 [31:50<3:54:20,  5.14s/it]
 12%|█▏        | 386/3120 [31:55<3:51:13,  5.07s/it]
                                                    
{'loss': 1.1259, 'learning_rate': 1.9704323269834925e-05, 'epoch': 0.49}

 12%|█▏        | 386/3120 [31:55<3:51:13,  5.07s/it]
 12%|█▏        | 387/3120 [32:00<3:45:23,  4.95s/it]
 12%|█▏        | 388/3120 [32:05<3:46:21,  4.97s/it]
                                                    
{'loss': 1.1723, 'learning_rate': 1.9699184757182225e-05, 'epoch': 0.5}

 12%|█▏        | 388/3120 [32:05<3:46:21,  4.97s/it]
 12%|█▏        | 389/3120 [32:10<3:43:29,  4.91s/it]
 12%|█▎        | 390/3120 [32:14<3:42:30,  4.89s/it]
                                                    
{'loss': 1.1526, 'learning_rate': 1.9694002659393306e-05, 'epoch': 0.5}

 12%|█▎        | 390/3120 [32:14<3:42:30,  4.89s/it]
 13%|█▎        | 391/3120 [32:19<3:41:25,  4.87s/it]
 13%|█▎        | 392/3120 [32:24<3:39:10,  4.82s/it]
                                                    
{'loss': 1.1425, 'learning_rate': 1.9688776999754913e-05, 'epoch': 0.5}

 13%|█▎        | 392/3120 [32:24<3:39:10,  4.82s/it]
 13%|█▎        | 393/3120 [32:29<3:40:43,  4.86s/it]
 13%|█▎        | 394/3120 [32:34<3:41:29,  4.88s/it]
                                                    
{'loss': 1.1615, 'learning_rate': 1.9683507801749547e-05, 'epoch': 0.51}

 13%|█▎        | 394/3120 [32:34<3:41:29,  4.88s/it]
 13%|█▎        | 395/3120 [32:39<3:43:09,  4.91s/it]
 13%|█▎        | 396/3120 [32:44<3:40:50,  4.86s/it]
                                                    
{'loss': 1.144, 'learning_rate': 1.9678195089055347e-05, 'epoch': 0.51}

 13%|█▎        | 396/3120 [32:44<3:40:50,  4.86s/it]
 13%|█▎        | 397/3120 [32:49<3:43:01,  4.91s/it]
 13%|█▎        | 398/3120 [32:54<3:49:43,  5.06s/it]
                                                    
{'loss': 1.0944, 'learning_rate': 1.9672838885546007e-05, 'epoch': 0.51}

 13%|█▎        | 398/3120 [32:54<3:49:43,  5.06s/it]
 13%|█▎        | 399/3120 [32:59<3:49:52,  5.07s/it]
 13%|█▎        | 400/3120 [33:05<3:55:34,  5.20s/it]
                                                    
{'loss': 1.112, 'learning_rate': 1.966743921529065e-05, 'epoch': 0.51}

 13%|█▎        | 400/3120 [33:05<3:55:34,  5.20s/it]
 13%|█▎        | 401/3120 [33:10<3:56:20,  5.22s/it]
 13%|█▎        | 402/3120 [33:14<3:45:05,  4.97s/it]
                                                    
{'loss': 1.1535, 'learning_rate': 1.9661996102553716e-05, 'epoch': 0.52}

 13%|█▎        | 402/3120 [33:14<3:45:05,  4.97s/it]
 13%|█▎        | 403/3120 [33:19<3:43:56,  4.95s/it]
 13%|█▎        | 404/3120 [33:24<3:44:59,  4.97s/it]
                                                    
{'loss': 1.1526, 'learning_rate': 1.965650957179488e-05, 'epoch': 0.52}

 13%|█▎        | 404/3120 [33:24<3:44:59,  4.97s/it]
 13%|█▎        | 405/3120 [33:29<3:49:44,  5.08s/it]
 13%|█▎        | 406/3120 [33:34<3:46:02,  5.00s/it]
                                                    
{'loss': 1.157, 'learning_rate': 1.9650979647668908e-05, 'epoch': 0.52}

 13%|█▎        | 406/3120 [33:34<3:46:02,  5.00s/it]
 13%|█▎        | 407/3120 [33:40<3:50:29,  5.10s/it]
 13%|█▎        | 408/3120 [33:45<3:52:12,  5.14s/it]
                                                    
{'loss': 1.1447, 'learning_rate': 1.9645406355025565e-05, 'epoch': 0.52}

 13%|█▎        | 408/3120 [33:45<3:52:12,  5.14s/it]
 13%|█▎        | 409/3120 [33:50<3:58:35,  5.28s/it]
 13%|█▎        | 410/3120 [33:56<3:55:48,  5.22s/it]
                                                    
{'loss': 1.1192, 'learning_rate': 1.963978971890951e-05, 'epoch': 0.53}

 13%|█▎        | 410/3120 [33:56<3:55:48,  5.22s/it]
 13%|█▎        | 411/3120 [34:00<3:46:27,  5.02s/it]
 13%|█▎        | 412/3120 [34:05<3:46:04,  5.01s/it]
                                                    
{'loss': 1.19, 'learning_rate': 1.963412976456017e-05, 'epoch': 0.53}

 13%|█▎        | 412/3120 [34:05<3:46:04,  5.01s/it]
 13%|█▎        | 413/3120 [34:10<3:42:06,  4.92s/it]
 13%|█▎        | 414/3120 [34:15<3:41:27,  4.91s/it]
                                                    
{'loss': 1.1789, 'learning_rate': 1.9628426517411625e-05, 'epoch': 0.53}

 13%|█▎        | 414/3120 [34:15<3:41:27,  4.91s/it]
 13%|█▎        | 415/3120 [34:20<3:44:51,  4.99s/it]
 13%|█▎        | 416/3120 [34:25<3:42:23,  4.93s/it]
                                                    
{'loss': 1.204, 'learning_rate': 1.9622680003092503e-05, 'epoch': 0.53}

 13%|█▎        | 416/3120 [34:25<3:42:23,  4.93s/it]
 13%|█▎        | 417/3120 [34:30<3:47:52,  5.06s/it]
 13%|█▎        | 418/3120 [34:35<3:49:02,  5.09s/it]
                                                    
{'loss': 1.1377, 'learning_rate': 1.9616890247425867e-05, 'epoch': 0.54}

 13%|█▎        | 418/3120 [34:35<3:49:02,  5.09s/it]
 13%|█▎        | 419/3120 [34:40<3:50:57,  5.13s/it]
 13%|█▎        | 420/3120 [34:46<3:55:38,  5.24s/it]
                                                    
{'loss': 1.1645, 'learning_rate': 1.9611057276429085e-05, 'epoch': 0.54}

 13%|█▎        | 420/3120 [34:46<3:55:38,  5.24s/it]
 13%|█▎        | 421/3120 [34:50<3:46:43,  5.04s/it]
 14%|█▎        | 422/3120 [34:55<3:41:20,  4.92s/it]
                                                    
{'loss': 1.1234, 'learning_rate': 1.9605181116313725e-05, 'epoch': 0.54}

 14%|█▎        | 422/3120 [34:55<3:41:20,  4.92s/it]
 14%|█▎        | 423/3120 [35:00<3:39:34,  4.88s/it]
 14%|█▎        | 424/3120 [35:05<3:39:50,  4.89s/it]
                                                    
{'loss': 1.124, 'learning_rate': 1.9599261793485432e-05, 'epoch': 0.54}

 14%|█▎        | 424/3120 [35:05<3:39:50,  4.89s/it]
 14%|█▎        | 425/3120 [35:10<3:37:09,  4.83s/it]
 14%|█▎        | 426/3120 [35:16<3:52:30,  5.18s/it]
                                                    
{'loss': 1.1944, 'learning_rate': 1.959329933454381e-05, 'epoch': 0.55}

 14%|█▎        | 426/3120 [35:16<3:52:30,  5.18s/it]
 14%|█▎        | 427/3120 [35:20<3:44:33,  5.00s/it]
 14%|█▎        | 428/3120 [35:26<3:50:07,  5.13s/it]
                                                    
{'loss': 1.1968, 'learning_rate': 1.958729376628231e-05, 'epoch': 0.55}

 14%|█▎        | 428/3120 [35:26<3:50:07,  5.13s/it]
 14%|█▍        | 429/3120 [35:30<3:45:50,  5.04s/it]
 14%|█▍        | 430/3120 [35:35<3:43:14,  4.98s/it]
                                                    
{'loss': 1.1025, 'learning_rate': 1.9581245115688093e-05, 'epoch': 0.55}

 14%|█▍        | 430/3120 [35:35<3:43:14,  4.98s/it]
 14%|█▍        | 431/3120 [35:40<3:43:09,  4.98s/it]
 14%|█▍        | 432/3120 [35:46<3:51:15,  5.16s/it]
                                                    
{'loss': 1.1932, 'learning_rate': 1.957515340994193e-05, 'epoch': 0.55}

 14%|█▍        | 432/3120 [35:46<3:51:15,  5.16s/it]
 14%|█▍        | 433/3120 [35:50<3:44:23,  5.01s/it]
 14%|█▍        | 434/3120 [35:55<3:44:01,  5.00s/it]
                                                    
{'loss': 1.1497, 'learning_rate': 1.9569018676418053e-05, 'epoch': 0.56}

 14%|█▍        | 434/3120 [35:55<3:44:01,  5.00s/it]
 14%|█▍        | 435/3120 [36:00<3:40:07,  4.92s/it]
 14%|█▍        | 436/3120 [36:05<3:45:54,  5.05s/it]
                                                    
{'loss': 1.1787, 'learning_rate': 1.956284094268407e-05, 'epoch': 0.56}

 14%|█▍        | 436/3120 [36:05<3:45:54,  5.05s/it]
 14%|█▍        | 437/3120 [36:10<3:45:03,  5.03s/it]
 14%|█▍        | 438/3120 [36:15<3:43:11,  4.99s/it]
                                                    
{'loss': 1.1771, 'learning_rate': 1.9556620236500794e-05, 'epoch': 0.56}

 14%|█▍        | 438/3120 [36:15<3:43:11,  4.99s/it]
 14%|█▍        | 439/3120 [36:20<3:42:50,  4.99s/it]
 14%|█▍        | 440/3120 [36:25<3:41:03,  4.95s/it]
                                                    
{'loss': 1.1826, 'learning_rate': 1.955035658582216e-05, 'epoch': 0.56}

 14%|█▍        | 440/3120 [36:25<3:41:03,  4.95s/it]
 14%|█▍        | 441/3120 [36:31<3:47:00,  5.08s/it]
 14%|█▍        | 442/3120 [36:35<3:43:33,  5.01s/it]
                                                    
{'loss': 1.1726, 'learning_rate': 1.9544050018795076e-05, 'epoch': 0.57}

 14%|█▍        | 442/3120 [36:35<3:43:33,  5.01s/it]
 14%|█▍        | 443/3120 [36:41<3:48:28,  5.12s/it]
 14%|█▍        | 444/3120 [36:45<3:41:45,  4.97s/it]
                                                    
{'loss': 1.1397, 'learning_rate': 1.9537700563759303e-05, 'epoch': 0.57}

 14%|█▍        | 444/3120 [36:45<3:41:45,  4.97s/it]
 14%|█▍        | 445/3120 [36:50<3:36:19,  4.85s/it]
 14%|█▍        | 446/3120 [36:55<3:35:19,  4.83s/it]
                                                    
{'loss': 1.1658, 'learning_rate': 1.9531308249247328e-05, 'epoch': 0.57}

 14%|█▍        | 446/3120 [36:55<3:35:19,  4.83s/it]
 14%|█▍        | 447/3120 [37:00<3:42:38,  5.00s/it]
 14%|█▍        | 448/3120 [37:05<3:38:54,  4.92s/it]
                                                    
{'loss': 1.1805, 'learning_rate': 1.9524873103984234e-05, 'epoch': 0.57}

 14%|█▍        | 448/3120 [37:05<3:38:54,  4.92s/it]
 14%|█▍        | 449/3120 [37:10<3:46:51,  5.10s/it]
 14%|█▍        | 450/3120 [37:16<3:47:07,  5.10s/it]
                                                    
{'loss': 1.1383, 'learning_rate': 1.9518395156887574e-05, 'epoch': 0.58}

 14%|█▍        | 450/3120 [37:16<3:47:07,  5.10s/it]
 14%|█▍        | 451/3120 [37:20<3:38:26,  4.91s/it]
 14%|█▍        | 452/3120 [37:25<3:42:49,  5.01s/it]
                                                    
{'loss': 1.1558, 'learning_rate': 1.9511874437067243e-05, 'epoch': 0.58}

 14%|█▍        | 452/3120 [37:25<3:42:49,  5.01s/it]
 15%|█▍        | 453/3120 [37:31<3:47:08,  5.11s/it]
 15%|█▍        | 454/3120 [37:36<3:46:29,  5.10s/it]
                                                    
{'loss': 1.1584, 'learning_rate': 1.950531097382533e-05, 'epoch': 0.58}

 15%|█▍        | 454/3120 [37:36<3:46:29,  5.10s/it]
 15%|█▍        | 455/3120 [37:40<3:39:39,  4.95s/it]
 15%|█▍        | 456/3120 [37:45<3:39:17,  4.94s/it]
                                                    
{'loss': 1.2029, 'learning_rate': 1.949870479665602e-05, 'epoch': 0.58}

 15%|█▍        | 456/3120 [37:45<3:39:17,  4.94s/it]
 15%|█▍        | 457/3120 [37:50<3:44:08,  5.05s/it]
 15%|█▍        | 458/3120 [37:56<3:45:00,  5.07s/it]
                                                    
{'loss': 1.1418, 'learning_rate': 1.9492055935245418e-05, 'epoch': 0.59}

 15%|█▍        | 458/3120 [37:56<3:45:00,  5.07s/it]
 15%|█▍        | 459/3120 [38:01<3:43:42,  5.04s/it]
 15%|█▍        | 460/3120 [38:05<3:39:07,  4.94s/it]
                                                    
{'loss': 1.176, 'learning_rate': 1.9485364419471454e-05, 'epoch': 0.59}

 15%|█▍        | 460/3120 [38:05<3:39:07,  4.94s/it]
 15%|█▍        | 461/3120 [38:10<3:38:53,  4.94s/it]
 15%|█▍        | 462/3120 [38:15<3:37:10,  4.90s/it]
                                                    
{'loss': 1.1742, 'learning_rate': 1.9478630279403737e-05, 'epoch': 0.59}

 15%|█▍        | 462/3120 [38:15<3:37:10,  4.90s/it]
 15%|█▍        | 463/3120 [38:20<3:37:16,  4.91s/it]
 15%|█▍        | 464/3120 [38:25<3:42:26,  5.02s/it]
                                                    
{'loss': 1.0948, 'learning_rate': 1.9471853545303407e-05, 'epoch': 0.59}

 15%|█▍        | 464/3120 [38:25<3:42:26,  5.02s/it]
 15%|█▍        | 465/3120 [38:30<3:40:25,  4.98s/it]
 15%|█▍        | 466/3120 [38:35<3:36:53,  4.90s/it]
                                                    
{'loss': 1.1567, 'learning_rate': 1.9465034247623005e-05, 'epoch': 0.6}

 15%|█▍        | 466/3120 [38:35<3:36:53,  4.90s/it]
 15%|█▍        | 467/3120 [38:40<3:42:38,  5.04s/it]
 15%|█▌        | 468/3120 [38:45<3:44:09,  5.07s/it]
                                                    
{'loss': 1.1644, 'learning_rate': 1.9458172417006347e-05, 'epoch': 0.6}

 15%|█▌        | 468/3120 [38:45<3:44:09,  5.07s/it]
 15%|█▌        | 469/3120 [38:51<3:47:35,  5.15s/it]
 15%|█▌        | 470/3120 [38:56<3:47:32,  5.15s/it]
                                                    
{'loss': 1.1226, 'learning_rate': 1.9451268084288385e-05, 'epoch': 0.6}

 15%|█▌        | 470/3120 [38:56<3:47:32,  5.15s/it]
 15%|█▌        | 471/3120 [39:01<3:45:19,  5.10s/it]
 15%|█▌        | 472/3120 [39:06<3:40:22,  4.99s/it]
                                                    
{'loss': 1.0994, 'learning_rate': 1.9444321280495045e-05, 'epoch': 0.61}

 15%|█▌        | 472/3120 [39:06<3:40:22,  4.99s/it]
 15%|█▌        | 473/3120 [39:10<3:36:45,  4.91s/it]
 15%|█▌        | 474/3120 [39:15<3:33:14,  4.84s/it]
                                                    
{'loss': 1.1429, 'learning_rate': 1.943733203684312e-05, 'epoch': 0.61}

 15%|█▌        | 474/3120 [39:15<3:33:14,  4.84s/it]
 15%|█▌        | 475/3120 [39:20<3:29:38,  4.76s/it]
 15%|█▌        | 476/3120 [39:24<3:30:29,  4.78s/it]
                                                    
{'loss': 1.1411, 'learning_rate': 1.9430300384740108e-05, 'epoch': 0.61}

 15%|█▌        | 476/3120 [39:24<3:30:29,  4.78s/it]
 15%|█▌        | 477/3120 [39:30<3:36:49,  4.92s/it]
 15%|█▌        | 478/3120 [39:35<3:40:29,  5.01s/it]
                                                    
{'loss': 1.1425, 'learning_rate': 1.942322635578408e-05, 'epoch': 0.61}

 15%|█▌        | 478/3120 [39:35<3:40:29,  5.01s/it]
 15%|█▌        | 479/3120 [39:40<3:42:02,  5.04s/it]
 15%|█▌        | 480/3120 [39:45<3:39:49,  5.00s/it]
                                                    
{'loss': 1.1447, 'learning_rate': 1.9416109981763526e-05, 'epoch': 0.62}

 15%|█▌        | 480/3120 [39:45<3:39:49,  5.00s/it]
 15%|█▌        | 481/3120 [39:50<3:39:40,  4.99s/it]
 15%|█▌        | 482/3120 [39:55<3:35:31,  4.90s/it]
                                                    
{'loss': 1.1677, 'learning_rate': 1.940895129465724e-05, 'epoch': 0.62}

 15%|█▌        | 482/3120 [39:55<3:35:31,  4.90s/it]
 15%|█▌        | 483/3120 [40:00<3:37:43,  4.95s/it]
 16%|█▌        | 484/3120 [40:04<3:35:38,  4.91s/it]
                                                    
{'loss': 1.1862, 'learning_rate': 1.9401750326634144e-05, 'epoch': 0.62}

 16%|█▌        | 484/3120 [40:04<3:35:38,  4.91s/it]
 16%|█▌        | 485/3120 [40:09<3:34:52,  4.89s/it]
 16%|█▌        | 486/3120 [40:14<3:34:53,  4.89s/it]
                                                    
{'loss': 1.1923, 'learning_rate': 1.939450711005316e-05, 'epoch': 0.62}

 16%|█▌        | 486/3120 [40:14<3:34:53,  4.89s/it]
 16%|█▌        | 487/3120 [40:19<3:29:30,  4.77s/it]
 16%|█▌        | 488/3120 [40:24<3:35:26,  4.91s/it]
                                                    
{'loss': 1.202, 'learning_rate': 1.9387221677463064e-05, 'epoch': 0.63}

 16%|█▌        | 488/3120 [40:24<3:35:26,  4.91s/it]
 16%|█▌        | 489/3120 [40:29<3:33:28,  4.87s/it]
 16%|█▌        | 490/3120 [40:34<3:35:54,  4.93s/it]
                                                    
{'loss': 1.1419, 'learning_rate': 1.9379894061602335e-05, 'epoch': 0.63}

 16%|█▌        | 490/3120 [40:34<3:35:54,  4.93s/it]
 16%|█▌        | 491/3120 [40:39<3:34:32,  4.90s/it]
 16%|█▌        | 492/3120 [40:44<3:38:37,  4.99s/it]
                                                    
{'loss': 1.111, 'learning_rate': 1.9372524295399014e-05, 'epoch': 0.63}

 16%|█▌        | 492/3120 [40:44<3:38:37,  4.99s/it]
 16%|█▌        | 493/3120 [40:49<3:38:16,  4.99s/it]
 16%|█▌        | 494/3120 [40:53<3:33:44,  4.88s/it]
                                                    
{'loss': 1.1734, 'learning_rate': 1.936511241197055e-05, 'epoch': 0.63}

 16%|█▌        | 494/3120 [40:53<3:33:44,  4.88s/it]
 16%|█▌        | 495/3120 [40:59<3:39:11,  5.01s/it]
 16%|█▌        | 496/3120 [41:04<3:37:34,  4.98s/it]
                                                    
{'loss': 1.1437, 'learning_rate': 1.9357658444623655e-05, 'epoch': 0.64}

 16%|█▌        | 496/3120 [41:04<3:37:34,  4.98s/it]
 16%|█▌        | 497/3120 [41:09<3:41:06,  5.06s/it]
 16%|█▌        | 498/3120 [41:14<3:37:46,  4.98s/it]
                                                    
{'loss': 1.1303, 'learning_rate': 1.9350162426854152e-05, 'epoch': 0.64}

 16%|█▌        | 498/3120 [41:14<3:37:46,  4.98s/it]
 16%|█▌        | 499/3120 [41:19<3:42:33,  5.09s/it]
 16%|█▌        | 500/3120 [41:23<3:33:55,  4.90s/it]
                                                    
{'loss': 1.1779, 'learning_rate': 1.9342624392346826e-05, 'epoch': 0.64}

 16%|█▌        | 500/3120 [41:23<3:33:55,  4.90s/it]
 16%|█▌        | 501/3120 [41:28<3:29:08,  4.79s/it]
 16%|█▌        | 502/3120 [41:33<3:36:35,  4.96s/it]
                                                    
{'loss': 1.1967, 'learning_rate': 1.933504437497527e-05, 'epoch': 0.64}

 16%|█▌        | 502/3120 [41:33<3:36:35,  4.96s/it]
 16%|█▌        | 503/3120 [41:38<3:32:03,  4.86s/it]
 16%|█▌        | 504/3120 [41:43<3:32:10,  4.87s/it]
                                                    
{'loss': 1.1155, 'learning_rate': 1.9327422408801744e-05, 'epoch': 0.65}

 16%|█▌        | 504/3120 [41:43<3:32:10,  4.87s/it]
 16%|█▌        | 505/3120 [41:48<3:31:44,  4.86s/it]
 16%|█▌        | 506/3120 [41:52<3:28:03,  4.78s/it]
                                                    
{'loss': 1.1579, 'learning_rate': 1.9319758528076998e-05, 'epoch': 0.65}

 16%|█▌        | 506/3120 [41:52<3:28:03,  4.78s/it]
 16%|█▋        | 507/3120 [41:57<3:32:44,  4.89s/it]
 16%|█▋        | 508/3120 [42:02<3:33:02,  4.89s/it]
                                                    
{'loss': 1.144, 'learning_rate': 1.9312052767240153e-05, 'epoch': 0.65}

 16%|█▋        | 508/3120 [42:02<3:33:02,  4.89s/it]
 16%|█▋        | 509/3120 [42:07<3:27:26,  4.77s/it]
 16%|█▋        | 510/3120 [42:11<3:24:44,  4.71s/it]
                                                    
{'loss': 1.1733, 'learning_rate': 1.93043051609185e-05, 'epoch': 0.65}

 16%|█▋        | 510/3120 [42:11<3:24:44,  4.71s/it]
 16%|█▋        | 511/3120 [42:16<3:27:56,  4.78s/it]
 16%|█▋        | 512/3120 [42:21<3:32:46,  4.90s/it]
                                                    
{'loss': 1.1563, 'learning_rate': 1.92965157439274e-05, 'epoch': 0.66}

 16%|█▋        | 512/3120 [42:21<3:32:46,  4.90s/it]
 16%|█▋        | 513/3120 [42:26<3:32:34,  4.89s/it]
 16%|█▋        | 514/3120 [42:31<3:33:25,  4.91s/it]
                                                    
{'loss': 1.1467, 'learning_rate': 1.9288684551270072e-05, 'epoch': 0.66}

 16%|█▋        | 514/3120 [42:31<3:33:25,  4.91s/it]
 17%|█▋        | 515/3120 [42:36<3:32:11,  4.89s/it]
 17%|█▋        | 516/3120 [42:41<3:32:41,  4.90s/it]
                                                    
{'loss': 1.212, 'learning_rate': 1.9280811618137486e-05, 'epoch': 0.66}

 17%|█▋        | 516/3120 [42:41<3:32:41,  4.90s/it]
 17%|█▋        | 517/3120 [42:46<3:35:15,  4.96s/it]
 17%|█▋        | 518/3120 [42:51<3:35:52,  4.98s/it]
                                                    
{'loss': 1.1307, 'learning_rate': 1.9272896979908153e-05, 'epoch': 0.66}

 17%|█▋        | 518/3120 [42:51<3:35:52,  4.98s/it]
 17%|█▋        | 519/3120 [42:56<3:36:23,  4.99s/it]
 17%|█▋        | 520/3120 [43:02<3:40:14,  5.08s/it]
                                                    
{'loss': 1.1368, 'learning_rate': 1.9264940672148018e-05, 'epoch': 0.67}

 17%|█▋        | 520/3120 [43:02<3:40:14,  5.08s/it]
 17%|█▋        | 521/3120 [43:06<3:34:40,  4.96s/it]
 17%|█▋        | 522/3120 [43:11<3:38:02,  5.04s/it]
                                                    
{'loss': 1.1799, 'learning_rate': 1.9256942730610268e-05, 'epoch': 0.67}

 17%|█▋        | 522/3120 [43:11<3:38:02,  5.04s/it]
 17%|█▋        | 523/3120 [43:16<3:32:52,  4.92s/it]
 17%|█▋        | 524/3120 [43:21<3:34:33,  4.96s/it]
                                                    
{'loss': 1.1466, 'learning_rate': 1.9248903191235177e-05, 'epoch': 0.67}

 17%|█▋        | 524/3120 [43:21<3:34:33,  4.96s/it]
 17%|█▋        | 525/3120 [43:26<3:37:45,  5.03s/it]
 17%|█▋        | 526/3120 [43:31<3:35:35,  4.99s/it]
                                                    
{'loss': 1.2009, 'learning_rate': 1.9240822090149944e-05, 'epoch': 0.67}

 17%|█▋        | 526/3120 [43:31<3:35:35,  4.99s/it]
 17%|█▋        | 527/3120 [43:36<3:32:29,  4.92s/it]
 17%|█▋        | 528/3120 [43:41<3:32:33,  4.92s/it]
                                                    
{'loss': 1.1678, 'learning_rate': 1.9232699463668543e-05, 'epoch': 0.68}

 17%|█▋        | 528/3120 [43:41<3:32:33,  4.92s/it]
 17%|█▋        | 529/3120 [43:46<3:32:44,  4.93s/it]
 17%|█▋        | 530/3120 [43:51<3:31:51,  4.91s/it]
                                                    
{'loss': 1.1532, 'learning_rate': 1.9224535348291543e-05, 'epoch': 0.68}

 17%|█▋        | 530/3120 [43:51<3:31:51,  4.91s/it]
 17%|█▋        | 531/3120 [43:56<3:32:53,  4.93s/it]
 17%|█▋        | 532/3120 [44:01<3:36:07,  5.01s/it]
                                                    
{'loss': 1.1784, 'learning_rate': 1.9216329780705955e-05, 'epoch': 0.68}

 17%|█▋        | 532/3120 [44:01<3:36:07,  5.01s/it]
 17%|█▋        | 533/3120 [44:06<3:33:46,  4.96s/it]
 17%|█▋        | 534/3120 [44:10<3:31:13,  4.90s/it]
                                                    
{'loss': 1.1642, 'learning_rate': 1.9208082797785057e-05, 'epoch': 0.68}

 17%|█▋        | 534/3120 [44:10<3:31:13,  4.90s/it]
 17%|█▋        | 535/3120 [44:15<3:29:28,  4.86s/it]
 17%|█▋        | 536/3120 [44:20<3:29:54,  4.87s/it]
                                                    
{'loss': 1.1825, 'learning_rate': 1.9199794436588244e-05, 'epoch': 0.69}

 17%|█▋        | 536/3120 [44:20<3:29:54,  4.87s/it]
 17%|█▋        | 537/3120 [44:25<3:27:54,  4.83s/it]
 17%|█▋        | 538/3120 [44:30<3:31:48,  4.92s/it]
                                                    
{'loss': 1.1526, 'learning_rate': 1.9191464734360844e-05, 'epoch': 0.69}

 17%|█▋        | 538/3120 [44:30<3:31:48,  4.92s/it]
 17%|█▋        | 539/3120 [44:35<3:27:21,  4.82s/it]
 17%|█▋        | 540/3120 [44:39<3:25:42,  4.78s/it]
                                                    
{'loss': 1.1468, 'learning_rate': 1.9183093728533966e-05, 'epoch': 0.69}

 17%|█▋        | 540/3120 [44:39<3:25:42,  4.78s/it]
 17%|█▋        | 541/3120 [44:44<3:27:19,  4.82s/it]
 17%|█▋        | 542/3120 [44:49<3:29:34,  4.88s/it]
                                                    
{'loss': 1.1444, 'learning_rate': 1.917468145672432e-05, 'epoch': 0.69}

 17%|█▋        | 542/3120 [44:49<3:29:34,  4.88s/it]
 17%|█▋        | 543/3120 [44:54<3:32:41,  4.95s/it]
 17%|█▋        | 544/3120 [44:59<3:27:01,  4.82s/it]
                                                    
{'loss': 1.1357, 'learning_rate': 1.916622795673405e-05, 'epoch': 0.7}

 17%|█▋        | 544/3120 [44:59<3:27:01,  4.82s/it]
 17%|█▋        | 545/3120 [45:04<3:32:15,  4.95s/it]
 18%|█▊        | 546/3120 [45:09<3:32:04,  4.94s/it]
                                                    
{'loss': 1.1569, 'learning_rate': 1.9157733266550577e-05, 'epoch': 0.7}

 18%|█▊        | 546/3120 [45:09<3:32:04,  4.94s/it]
 18%|█▊        | 547/3120 [45:14<3:31:09,  4.92s/it]
 18%|█▊        | 548/3120 [45:19<3:36:59,  5.06s/it]
                                                    
{'loss': 1.1081, 'learning_rate': 1.9149197424346405e-05, 'epoch': 0.7}

 18%|█▊        | 548/3120 [45:19<3:36:59,  5.06s/it]
 18%|█▊        | 549/3120 [45:24<3:38:16,  5.09s/it]
 18%|█▊        | 550/3120 [45:30<3:38:24,  5.10s/it]
                                                    
{'loss': 1.1765, 'learning_rate': 1.9140620468478967e-05, 'epoch': 0.71}

 18%|█▊        | 550/3120 [45:30<3:38:24,  5.10s/it]
 18%|█▊        | 551/3120 [45:35<3:40:38,  5.15s/it]
 18%|█▊        | 552/3120 [45:40<3:35:31,  5.04s/it]
                                                    
{'loss': 1.2001, 'learning_rate': 1.913200243749046e-05, 'epoch': 0.71}

 18%|█▊        | 552/3120 [45:40<3:35:31,  5.04s/it]
 18%|█▊        | 553/3120 [45:45<3:35:50,  5.05s/it]
 18%|█▊        | 554/3120 [45:50<3:33:50,  5.00s/it]
                                                    
{'loss': 1.1429, 'learning_rate': 1.9123343370107636e-05, 'epoch': 0.71}

 18%|█▊        | 554/3120 [45:50<3:33:50,  5.00s/it]
 18%|█▊        | 555/3120 [45:54<3:31:56,  4.96s/it]
 18%|█▊        | 556/3120 [45:59<3:32:16,  4.97s/it]
                                                    
{'loss': 1.1898, 'learning_rate': 1.9114643305241678e-05, 'epoch': 0.71}

 18%|█▊        | 556/3120 [45:59<3:32:16,  4.97s/it]
 18%|█▊        | 557/3120 [46:04<3:28:02,  4.87s/it]
 18%|█▊        | 558/3120 [46:09<3:30:30,  4.93s/it]
                                                    
{'loss': 1.1473, 'learning_rate': 1.910590228198798e-05, 'epoch': 0.72}

 18%|█▊        | 558/3120 [46:09<3:30:30,  4.93s/it]
 18%|█▊        | 559/3120 [46:14<3:30:08,  4.92s/it]
 18%|█▊        | 560/3120 [46:19<3:30:07,  4.92s/it]
                                                    
{'loss': 1.1505, 'learning_rate': 1.9097120339625994e-05, 'epoch': 0.72}

 18%|█▊        | 560/3120 [46:19<3:30:07,  4.92s/it]
 18%|█▊        | 561/3120 [46:24<3:35:43,  5.06s/it]
 18%|█▊        | 562/3120 [46:30<3:38:06,  5.12s/it]
                                                    
{'loss': 1.1208, 'learning_rate': 1.9088297517619056e-05, 'epoch': 0.72}

 18%|█▊        | 562/3120 [46:30<3:38:06,  5.12s/it]
 18%|█▊        | 563/3120 [46:34<3:35:15,  5.05s/it]
 18%|█▊        | 564/3120 [46:39<3:32:31,  4.99s/it]
                                                    
{'loss': 1.1064, 'learning_rate': 1.9079433855614203e-05, 'epoch': 0.72}

 18%|█▊        | 564/3120 [46:39<3:32:31,  4.99s/it]
 18%|█▊        | 565/3120 [46:44<3:28:10,  4.89s/it]
 18%|█▊        | 566/3120 [46:49<3:25:21,  4.82s/it]
                                                    
{'loss': 1.1549, 'learning_rate': 1.9070529393441985e-05, 'epoch': 0.73}

 18%|█▊        | 566/3120 [46:49<3:25:21,  4.82s/it]
 18%|█▊        | 567/3120 [46:53<3:22:09,  4.75s/it]
 18%|█▊        | 568/3120 [46:58<3:26:33,  4.86s/it]
                                                    
{'loss': 1.1308, 'learning_rate': 1.9061584171116302e-05, 'epoch': 0.73}

 18%|█▊        | 568/3120 [46:58<3:26:33,  4.86s/it]
 18%|█▊        | 569/3120 [47:03<3:26:40,  4.86s/it]
 18%|█▊        | 570/3120 [47:08<3:30:19,  4.95s/it]
                                                    
{'loss': 1.1827, 'learning_rate': 1.9052598228834217e-05, 'epoch': 0.73}

 18%|█▊        | 570/3120 [47:08<3:30:19,  4.95s/it]
 18%|█▊        | 571/3120 [47:13<3:27:45,  4.89s/it]
 18%|█▊        | 572/3120 [47:18<3:32:27,  5.00s/it]
                                                    
{'loss': 1.1394, 'learning_rate': 1.9043571606975776e-05, 'epoch': 0.73}

 18%|█▊        | 572/3120 [47:18<3:32:27,  5.00s/it]
 18%|█▊        | 573/3120 [47:23<3:31:00,  4.97s/it]
 18%|█▊        | 574/3120 [47:29<3:39:57,  5.18s/it]
                                                    
{'loss': 1.1002, 'learning_rate': 1.9034504346103825e-05, 'epoch': 0.74}

 18%|█▊        | 574/3120 [47:29<3:39:57,  5.18s/it]
 18%|█▊        | 575/3120 [47:34<3:38:07,  5.14s/it]
 18%|█▊        | 576/3120 [47:39<3:35:43,  5.09s/it]
                                                    
{'loss': 1.1308, 'learning_rate': 1.9025396486963827e-05, 'epoch': 0.74}

 18%|█▊        | 576/3120 [47:39<3:35:43,  5.09s/it]
 18%|█▊        | 577/3120 [47:44<3:31:08,  4.98s/it]
 19%|█▊        | 578/3120 [47:49<3:40:06,  5.20s/it]
                                                    
{'loss': 1.1007, 'learning_rate': 1.9016248070483686e-05, 'epoch': 0.74}

 19%|█▊        | 578/3120 [47:49<3:40:06,  5.20s/it]
 19%|█▊        | 579/3120 [47:54<3:35:54,  5.10s/it]
 19%|█▊        | 580/3120 [47:59<3:31:02,  4.99s/it]
                                                    
{'loss': 1.1288, 'learning_rate': 1.900705913777356e-05, 'epoch': 0.74}

 19%|█▊        | 580/3120 [47:59<3:31:02,  4.99s/it]
 19%|█▊        | 581/3120 [48:04<3:32:19,  5.02s/it]
 19%|█▊        | 582/3120 [48:09<3:26:40,  4.89s/it]
                                                    
{'loss': 1.1672, 'learning_rate': 1.8997829730125662e-05, 'epoch': 0.75}

 19%|█▊        | 582/3120 [48:09<3:26:40,  4.89s/it]
 19%|█▊        | 583/3120 [48:14<3:26:30,  4.88s/it]
 19%|█▊        | 584/3120 [48:18<3:25:14,  4.86s/it]
                                                    
{'loss': 1.099, 'learning_rate': 1.89885598890141e-05, 'epoch': 0.75}

 19%|█▊        | 584/3120 [48:18<3:25:14,  4.86s/it]
 19%|█▉        | 585/3120 [48:23<3:23:10,  4.81s/it]
 19%|█▉        | 586/3120 [48:28<3:25:33,  4.87s/it]
                                                    
{'loss': 1.1658, 'learning_rate': 1.8979249656094672e-05, 'epoch': 0.75}

 19%|█▉        | 586/3120 [48:28<3:25:33,  4.87s/it]
 19%|█▉        | 587/3120 [48:32<3:20:36,  4.75s/it]
 19%|█▉        | 588/3120 [48:38<3:23:47,  4.83s/it]
                                                    
{'loss': 1.1251, 'learning_rate': 1.8969899073204687e-05, 'epoch': 0.75}

 19%|█▉        | 588/3120 [48:38<3:23:47,  4.83s/it]
 19%|█▉        | 589/3120 [48:42<3:22:51,  4.81s/it]
 19%|█▉        | 590/3120 [48:47<3:24:58,  4.86s/it]
                                                    
{'loss': 1.1284, 'learning_rate': 1.896050818236277e-05, 'epoch': 0.76}

 19%|█▉        | 590/3120 [48:47<3:24:58,  4.86s/it]
 19%|█▉        | 591/3120 [48:53<3:32:03,  5.03s/it]
 19%|█▉        | 592/3120 [48:58<3:33:29,  5.07s/it]
                                                    
{'loss': 1.1105, 'learning_rate': 1.895107702576868e-05, 'epoch': 0.76}

 19%|█▉        | 592/3120 [48:58<3:33:29,  5.07s/it]
 19%|█▉        | 593/3120 [49:03<3:31:09,  5.01s/it]
 19%|█▉        | 594/3120 [49:08<3:30:33,  5.00s/it]
                                                    
{'loss': 1.1285, 'learning_rate': 1.8941605645803115e-05, 'epoch': 0.76}

 19%|█▉        | 594/3120 [49:08<3:30:33,  5.00s/it]
 19%|█▉        | 595/3120 [49:13<3:29:26,  4.98s/it]
 19%|█▉        | 596/3120 [49:17<3:28:04,  4.95s/it]
                                                    
{'loss': 1.1648, 'learning_rate': 1.8932094085027534e-05, 'epoch': 0.76}

 19%|█▉        | 596/3120 [49:17<3:28:04,  4.95s/it]
 19%|█▉        | 597/3120 [49:22<3:27:13,  4.93s/it]
 19%|█▉        | 598/3120 [49:28<3:34:18,  5.10s/it]
                                                    
{'loss': 1.1517, 'learning_rate': 1.8922542386183942e-05, 'epoch': 0.77}

 19%|█▉        | 598/3120 [49:28<3:34:18,  5.10s/it]
 19%|█▉        | 599/3120 [49:33<3:34:04,  5.10s/it]
 19%|█▉        | 600/3120 [49:38<3:31:06,  5.03s/it]
                                                    
{'loss': 1.1652, 'learning_rate': 1.891295059219472e-05, 'epoch': 0.77}

 19%|█▉        | 600/3120 [49:38<3:31:06,  5.03s/it]
 19%|█▉        | 601/3120 [49:43<3:34:35,  5.11s/it]
 19%|█▉        | 602/3120 [49:48<3:31:51,  5.05s/it]
                                                    
{'loss': 1.1424, 'learning_rate': 1.8903318746162428e-05, 'epoch': 0.77}

 19%|█▉        | 602/3120 [49:48<3:31:51,  5.05s/it]
 19%|█▉        | 603/3120 [49:53<3:33:28,  5.09s/it]
 19%|█▉        | 604/3120 [49:58<3:32:47,  5.07s/it]
                                                    
{'loss': 1.1157, 'learning_rate': 1.88936468913696e-05, 'epoch': 0.77}

 19%|█▉        | 604/3120 [49:58<3:32:47,  5.07s/it]
 19%|█▉        | 605/3120 [50:03<3:31:43,  5.05s/it]
 19%|█▉        | 606/3120 [50:08<3:33:02,  5.08s/it]
                                                    
{'loss': 1.1597, 'learning_rate': 1.888393507127856e-05, 'epoch': 0.78}

 19%|█▉        | 606/3120 [50:08<3:33:02,  5.08s/it]
 19%|█▉        | 607/3120 [50:13<3:30:53,  5.04s/it]
 19%|█▉        | 608/3120 [50:18<3:30:26,  5.03s/it]
                                                    
{'loss': 1.1169, 'learning_rate': 1.8874183329531222e-05, 'epoch': 0.78}

 19%|█▉        | 608/3120 [50:18<3:30:26,  5.03s/it]
 20%|█▉        | 609/3120 [50:23<3:29:36,  5.01s/it]
 20%|█▉        | 610/3120 [50:28<3:30:08,  5.02s/it]
                                                    
{'loss': 1.1528, 'learning_rate': 1.88643917099489e-05, 'epoch': 0.78}

 20%|█▉        | 610/3120 [50:28<3:30:08,  5.02s/it]
 20%|█▉        | 611/3120 [50:33<3:30:37,  5.04s/it]
 20%|█▉        | 612/3120 [50:38<3:27:06,  4.95s/it]
                                                    
{'loss': 1.1508, 'learning_rate': 1.8854560256532098e-05, 'epoch': 0.78}

 20%|█▉        | 612/3120 [50:38<3:27:06,  4.95s/it]
 20%|█▉        | 613/3120 [50:43<3:26:57,  4.95s/it]
 20%|█▉        | 614/3120 [50:48<3:24:37,  4.90s/it]
                                                    
{'loss': 1.138, 'learning_rate': 1.8844689013460335e-05, 'epoch': 0.79}

 20%|█▉        | 614/3120 [50:48<3:24:37,  4.90s/it]
 20%|█▉        | 615/3120 [50:52<3:19:45,  4.78s/it]
 20%|█▉        | 616/3120 [50:58<3:25:35,  4.93s/it]
                                                    
{'loss': 1.1486, 'learning_rate': 1.883477802509192e-05, 'epoch': 0.79}

 20%|█▉        | 616/3120 [50:58<3:25:35,  4.93s/it]
 20%|█▉        | 617/3120 [51:02<3:23:02,  4.87s/it]
 20%|█▉        | 618/3120 [51:07<3:25:17,  4.92s/it]
                                                    
{'loss': 1.1427, 'learning_rate': 1.8824827335963767e-05, 'epoch': 0.79}

 20%|█▉        | 618/3120 [51:07<3:25:17,  4.92s/it]
 20%|█▉        | 619/3120 [51:12<3:23:46,  4.89s/it]
 20%|█▉        | 620/3120 [51:17<3:26:39,  4.96s/it]
                                                    
{'loss': 1.1143, 'learning_rate': 1.88148369907912e-05, 'epoch': 0.79}

 20%|█▉        | 620/3120 [51:17<3:26:39,  4.96s/it]
 20%|█▉        | 621/3120 [51:22<3:27:45,  4.99s/it]
 20%|█▉        | 622/3120 [51:28<3:31:44,  5.09s/it]
                                                    
{'loss': 1.1366, 'learning_rate': 1.8804807034467735e-05, 'epoch': 0.8}

 20%|█▉        | 622/3120 [51:28<3:31:44,  5.09s/it]
 20%|█▉        | 623/3120 [51:33<3:31:43,  5.09s/it]
 20%|██        | 624/3120 [51:37<3:23:05,  4.88s/it]
                                                    
{'loss': 1.1975, 'learning_rate': 1.879473751206489e-05, 'epoch': 0.8}

 20%|██        | 624/3120 [51:37<3:23:05,  4.88s/it]
 20%|██        | 625/3120 [51:42<3:21:03,  4.84s/it]
 20%|██        | 626/3120 [51:47<3:18:54,  4.79s/it]
                                                    
{'loss': 1.1549, 'learning_rate': 1.8784628468831997e-05, 'epoch': 0.8}

 20%|██        | 626/3120 [51:47<3:18:54,  4.79s/it]
 20%|██        | 627/3120 [51:52<3:19:47,  4.81s/it]
 20%|██        | 628/3120 [51:57<3:22:13,  4.87s/it]
                                                    
{'loss': 1.1532, 'learning_rate': 1.877447995019596e-05, 'epoch': 0.81}

 20%|██        | 628/3120 [51:57<3:22:13,  4.87s/it]
 20%|██        | 629/3120 [52:01<3:22:05,  4.87s/it]
 20%|██        | 630/3120 [52:06<3:21:38,  4.86s/it]
                                                    
{'loss': 1.1389, 'learning_rate': 1.876429200176108e-05, 'epoch': 0.81}

 20%|██        | 630/3120 [52:06<3:21:38,  4.86s/it]
 20%|██        | 631/3120 [52:11<3:24:36,  4.93s/it]
 20%|██        | 632/3120 [52:16<3:19:03,  4.80s/it]
                                                    
{'loss': 1.1319, 'learning_rate': 1.875406466930886e-05, 'epoch': 0.81}

 20%|██        | 632/3120 [52:16<3:19:03,  4.80s/it]
 20%|██        | 633/3120 [52:21<3:23:43,  4.91s/it]
 20%|██        | 634/3120 [52:26<3:25:57,  4.97s/it]
                                                    
{'loss': 1.1022, 'learning_rate': 1.874379799879775e-05, 'epoch': 0.81}

 20%|██        | 634/3120 [52:26<3:25:57,  4.97s/it]
 20%|██        | 635/3120 [52:31<3:24:27,  4.94s/it]
 20%|██        | 636/3120 [52:36<3:25:19,  4.96s/it]
                                                    
{'loss': 1.1313, 'learning_rate': 1.8733492036363007e-05, 'epoch': 0.82}

 20%|██        | 636/3120 [52:36<3:25:19,  4.96s/it]
 20%|██        | 637/3120 [52:41<3:29:55,  5.07s/it]
 20%|██        | 638/3120 [52:46<3:25:34,  4.97s/it]
                                                    
{'loss': 1.1646, 'learning_rate': 1.872314682831643e-05, 'epoch': 0.82}

 20%|██        | 638/3120 [52:46<3:25:34,  4.97s/it]
 20%|██        | 639/3120 [52:51<3:23:17,  4.92s/it]
 21%|██        | 640/3120 [52:55<3:18:48,  4.81s/it]
                                                    
{'loss': 1.208, 'learning_rate': 1.8712762421146185e-05, 'epoch': 0.82}

 21%|██        | 640/3120 [52:55<3:18:48,  4.81s/it]
 21%|██        | 641/3120 [53:00<3:17:23,  4.78s/it]
 21%|██        | 642/3120 [53:05<3:13:48,  4.69s/it]
                                                    
{'loss': 1.1441, 'learning_rate': 1.8702338861516587e-05, 'epoch': 0.82}

 21%|██        | 642/3120 [53:05<3:13:48,  4.69s/it]
 21%|██        | 643/3120 [53:09<3:12:54,  4.67s/it]
 21%|██        | 644/3120 [53:14<3:15:41,  4.74s/it]
                                                    
{'loss': 1.1564, 'learning_rate': 1.8691876196267892e-05, 'epoch': 0.83}

 21%|██        | 644/3120 [53:14<3:15:41,  4.74s/it]
 21%|██        | 645/3120 [53:19<3:18:14,  4.81s/it]
 21%|██        | 646/3120 [53:24<3:24:36,  4.96s/it]
                                                    
{'loss': 1.1334, 'learning_rate': 1.8681374472416075e-05, 'epoch': 0.83}

 21%|██        | 646/3120 [53:24<3:24:36,  4.96s/it]
 21%|██        | 647/3120 [53:30<3:29:04,  5.07s/it]
 21%|██        | 648/3120 [53:35<3:29:43,  5.09s/it]
                                                    
{'loss': 1.0704, 'learning_rate': 1.867083373715264e-05, 'epoch': 0.83}

 21%|██        | 648/3120 [53:35<3:29:43,  5.09s/it]
 21%|██        | 649/3120 [53:40<3:26:07,  5.01s/it]
 21%|██        | 650/3120 [53:44<3:21:47,  4.90s/it]
                                                    
{'loss': 1.1261, 'learning_rate': 1.866025403784439e-05, 'epoch': 0.83}

 21%|██        | 650/3120 [53:44<3:21:47,  4.90s/it]
 21%|██        | 651/3120 [53:49<3:18:56,  4.83s/it]
 21%|██        | 652/3120 [53:54<3:19:53,  4.86s/it]
                                                    
{'loss': 1.1484, 'learning_rate': 1.8649635422033218e-05, 'epoch': 0.84}

 21%|██        | 652/3120 [53:54<3:19:53,  4.86s/it]
 21%|██        | 653/3120 [53:59<3:20:16,  4.87s/it]
 21%|██        | 654/3120 [54:04<3:20:38,  4.88s/it]
                                                    
{'loss': 1.1134, 'learning_rate': 1.86389779374359e-05, 'epoch': 0.84}

 21%|██        | 654/3120 [54:04<3:20:38,  4.88s/it]
 21%|██        | 655/3120 [54:09<3:21:55,  4.92s/it]
 21%|██        | 656/3120 [54:14<3:29:36,  5.10s/it]
                                                    
{'loss': 1.1219, 'learning_rate': 1.862828163194388e-05, 'epoch': 0.84}

 21%|██        | 656/3120 [54:14<3:29:36,  5.10s/it]
 21%|██        | 657/3120 [54:19<3:29:13,  5.10s/it]
 21%|██        | 658/3120 [54:24<3:26:00,  5.02s/it]
                                                    
{'loss': 1.1314, 'learning_rate': 1.861754655362304e-05, 'epoch': 0.84}

 21%|██        | 658/3120 [54:24<3:26:00,  5.02s/it]
 21%|██        | 659/3120 [54:29<3:23:57,  4.97s/it]
 21%|██        | 660/3120 [54:34<3:22:17,  4.93s/it]
                                                    
{'loss': 1.1601, 'learning_rate': 1.8606772750713503e-05, 'epoch': 0.85}

 21%|██        | 660/3120 [54:34<3:22:17,  4.93s/it]
 21%|██        | 661/3120 [54:39<3:22:51,  4.95s/it]
 21%|██        | 662/3120 [54:44<3:25:08,  5.01s/it]
                                                    
{'loss': 1.168, 'learning_rate': 1.8595960271629412e-05, 'epoch': 0.85}

 21%|██        | 662/3120 [54:44<3:25:08,  5.01s/it]
 21%|██▏       | 663/3120 [54:50<3:32:33,  5.19s/it]
 21%|██▏       | 664/3120 [54:55<3:29:02,  5.11s/it]
                                                    
{'loss': 1.1485, 'learning_rate': 1.8585109164958698e-05, 'epoch': 0.85}

 21%|██▏       | 664/3120 [54:55<3:29:02,  5.11s/it]
 21%|██▏       | 665/3120 [55:00<3:33:31,  5.22s/it]
 21%|██▏       | 666/3120 [55:05<3:33:30,  5.22s/it]
                                                    
{'loss': 1.1637, 'learning_rate': 1.857421947946288e-05, 'epoch': 0.85}

 21%|██▏       | 666/3120 [55:05<3:33:30,  5.22s/it]
 21%|██▏       | 667/3120 [55:11<3:37:41,  5.32s/it]
 21%|██▏       | 668/3120 [55:16<3:35:23,  5.27s/it]
                                                    
{'loss': 1.1241, 'learning_rate': 1.8563291264076834e-05, 'epoch': 0.86}

 21%|██▏       | 668/3120 [55:16<3:35:23,  5.27s/it]
 21%|██▏       | 669/3120 [55:21<3:34:38,  5.25s/it]
 21%|██▏       | 670/3120 [55:26<3:28:44,  5.11s/it]
                                                    
{'loss': 1.1053, 'learning_rate': 1.8552324567908585e-05, 'epoch': 0.86}

 21%|██▏       | 670/3120 [55:26<3:28:44,  5.11s/it]
 22%|██▏       | 671/3120 [55:31<3:23:31,  4.99s/it]
 22%|██▏       | 672/3120 [55:35<3:19:52,  4.90s/it]
                                                    
{'loss': 1.1382, 'learning_rate': 1.8541319440239066e-05, 'epoch': 0.86}

 22%|██▏       | 672/3120 [55:35<3:19:52,  4.90s/it]
 22%|██▏       | 673/3120 [55:40<3:16:19,  4.81s/it]
 22%|██▏       | 674/3120 [55:45<3:16:49,  4.83s/it]
                                                    
{'loss': 1.1319, 'learning_rate': 1.8530275930521925e-05, 'epoch': 0.86}

 22%|██▏       | 674/3120 [55:45<3:16:49,  4.83s/it]
 22%|██▏       | 675/3120 [55:50<3:21:39,  4.95s/it]
 22%|██▏       | 676/3120 [55:55<3:24:10,  5.01s/it]
                                                    
{'loss': 1.0963, 'learning_rate': 1.851919408838327e-05, 'epoch': 0.87}

 22%|██▏       | 676/3120 [55:55<3:24:10,  5.01s/it]
 22%|██▏       | 677/3120 [56:00<3:19:25,  4.90s/it]
 22%|██▏       | 678/3120 [56:05<3:18:33,  4.88s/it]
                                                    
{'loss': 1.127, 'learning_rate': 1.8508073963621482e-05, 'epoch': 0.87}

 22%|██▏       | 678/3120 [56:05<3:18:33,  4.88s/it]
 22%|██▏       | 679/3120 [56:10<3:20:20,  4.92s/it]
 22%|██▏       | 680/3120 [56:14<3:17:34,  4.86s/it]
                                                    
{'loss': 1.1608, 'learning_rate': 1.8496915606206952e-05, 'epoch': 0.87}

 22%|██▏       | 680/3120 [56:14<3:17:34,  4.86s/it]
 22%|██▏       | 681/3120 [56:19<3:14:16,  4.78s/it]
 22%|██▏       | 682/3120 [56:24<3:18:50,  4.89s/it]
                                                    
{'loss': 1.158, 'learning_rate': 1.8485719066281894e-05, 'epoch': 0.87}

 22%|██▏       | 682/3120 [56:24<3:18:50,  4.89s/it]
 22%|██▏       | 683/3120 [56:29<3:19:59,  4.92s/it]
 22%|██▏       | 684/3120 [56:34<3:19:18,  4.91s/it]
                                                    
{'loss': 1.1696, 'learning_rate': 1.847448439416009e-05, 'epoch': 0.88}

 22%|██▏       | 684/3120 [56:34<3:19:18,  4.91s/it]
 22%|██▏       | 685/3120 [56:39<3:17:35,  4.87s/it]
 22%|██▏       | 686/3120 [56:44<3:22:03,  4.98s/it]
                                                    
{'loss': 1.1744, 'learning_rate': 1.8463211640326686e-05, 'epoch': 0.88}

 22%|██▏       | 686/3120 [56:44<3:22:03,  4.98s/it]
 22%|██▏       | 687/3120 [56:49<3:23:55,  5.03s/it]
 22%|██▏       | 688/3120 [56:54<3:24:43,  5.05s/it]
                                                    
{'loss': 1.1363, 'learning_rate': 1.845190085543795e-05, 'epoch': 0.88}

 22%|██▏       | 688/3120 [56:54<3:24:43,  5.05s/it]
 22%|██▏       | 689/3120 [56:59<3:21:17,  4.97s/it]
 22%|██▏       | 690/3120 [57:04<3:23:39,  5.03s/it]
                                                    
{'loss': 1.1001, 'learning_rate': 1.8440552090321047e-05, 'epoch': 0.88}

 22%|██▏       | 690/3120 [57:04<3:23:39,  5.03s/it]
 22%|██▏       | 691/3120 [57:10<3:26:25,  5.10s/it]
 22%|██▏       | 692/3120 [57:15<3:28:01,  5.14s/it]
                                                    
{'loss': 1.1647, 'learning_rate': 1.842916539597382e-05, 'epoch': 0.89}

 22%|██▏       | 692/3120 [57:15<3:28:01,  5.14s/it]
 22%|██▏       | 693/3120 [57:20<3:28:53,  5.16s/it]
 22%|██▏       | 694/3120 [57:25<3:26:44,  5.11s/it]
                                                    
{'loss': 1.119, 'learning_rate': 1.8417740823564552e-05, 'epoch': 0.89}

 22%|██▏       | 694/3120 [57:25<3:26:44,  5.11s/it]
 22%|██▏       | 695/3120 [57:30<3:21:50,  4.99s/it]
 22%|██▏       | 696/3120 [57:34<3:18:24,  4.91s/it]
                                                    
{'loss': 1.1718, 'learning_rate': 1.8406278424431737e-05, 'epoch': 0.89}

 22%|██▏       | 696/3120 [57:34<3:18:24,  4.91s/it]
 22%|██▏       | 697/3120 [57:40<3:28:54,  5.17s/it]
 22%|██▏       | 698/3120 [57:45<3:27:20,  5.14s/it]
                                                    
{'loss': 1.1101, 'learning_rate': 1.839477825008385e-05, 'epoch': 0.89}

 22%|██▏       | 698/3120 [57:45<3:27:20,  5.14s/it]
 22%|██▏       | 699/3120 [57:50<3:26:24,  5.12s/it]
 22%|██▏       | 700/3120 [57:55<3:26:47,  5.13s/it]
                                                    
{'loss': 1.1245, 'learning_rate': 1.8383240352199118e-05, 'epoch': 0.9}

 22%|██▏       | 700/3120 [57:55<3:26:47,  5.13s/it]
 22%|██▏       | 701/3120 [58:01<3:31:15,  5.24s/it]
 22%|██▎       | 702/3120 [58:05<3:20:19,  4.97s/it]
                                                    
{'loss': 1.1056, 'learning_rate': 1.8371664782625287e-05, 'epoch': 0.9}

 22%|██▎       | 702/3120 [58:05<3:20:19,  4.97s/it]
 23%|██▎       | 703/3120 [58:10<3:16:08,  4.87s/it]
 23%|██▎       | 704/3120 [58:15<3:12:58,  4.79s/it]
                                                    
{'loss': 1.1578, 'learning_rate': 1.8360051593379383e-05, 'epoch': 0.9}

 23%|██▎       | 704/3120 [58:15<3:12:58,  4.79s/it]
 23%|██▎       | 705/3120 [58:20<3:17:26,  4.91s/it]
 23%|██▎       | 706/3120 [58:25<3:20:07,  4.97s/it]
                                                    
{'loss': 1.1112, 'learning_rate': 1.8348400836647492e-05, 'epoch': 0.91}

 23%|██▎       | 706/3120 [58:25<3:20:07,  4.97s/it]
 23%|██▎       | 707/3120 [58:30<3:21:52,  5.02s/it]
 23%|██▎       | 708/3120 [58:35<3:23:51,  5.07s/it]
                                                    
{'loss': 1.1375, 'learning_rate': 1.8336712564784506e-05, 'epoch': 0.91}

 23%|██▎       | 708/3120 [58:35<3:23:51,  5.07s/it]
 23%|██▎       | 709/3120 [58:40<3:23:57,  5.08s/it]
 23%|██▎       | 710/3120 [58:45<3:20:30,  4.99s/it]
                                                    
{'loss': 1.1008, 'learning_rate': 1.83249868303139e-05, 'epoch': 0.91}

 23%|██▎       | 710/3120 [58:45<3:20:30,  4.99s/it]
 23%|██▎       | 711/3120 [58:50<3:21:00,  5.01s/it]
 23%|██▎       | 712/3120 [58:55<3:21:47,  5.03s/it]
                                                    
{'loss': 1.1385, 'learning_rate': 1.8313223685927507e-05, 'epoch': 0.91}

 23%|██▎       | 712/3120 [58:55<3:21:47,  5.03s/it]
 23%|██▎       | 713/3120 [59:00<3:20:23,  5.00s/it]
 23%|██▎       | 714/3120 [59:05<3:20:12,  4.99s/it]
                                                    
{'loss': 1.0728, 'learning_rate': 1.8301423184485253e-05, 'epoch': 0.92}

 23%|██▎       | 714/3120 [59:05<3:20:12,  4.99s/it]
 23%|██▎       | 715/3120 [59:10<3:19:27,  4.98s/it]
 23%|██▎       | 716/3120 [59:14<3:12:10,  4.80s/it]
                                                    
{'loss': 1.1382, 'learning_rate': 1.8289585379014942e-05, 'epoch': 0.92}

 23%|██▎       | 716/3120 [59:14<3:12:10,  4.80s/it]
 23%|██▎       | 717/3120 [59:19<3:14:53,  4.87s/it]
 23%|██▎       | 718/3120 [59:24<3:13:50,  4.84s/it]
                                                    
{'loss': 1.1376, 'learning_rate': 1.827771032271201e-05, 'epoch': 0.92}

 23%|██▎       | 718/3120 [59:24<3:13:50,  4.84s/it]
 23%|██▎       | 719/3120 [59:30<3:19:33,  4.99s/it]
 23%|██▎       | 720/3120 [59:34<3:14:56,  4.87s/it]
                                                    
{'loss': 1.2051, 'learning_rate': 1.8265798068939295e-05, 'epoch': 0.92}

 23%|██▎       | 720/3120 [59:34<3:14:56,  4.87s/it]
 23%|██▎       | 721/3120 [59:38<3:07:12,  4.68s/it]
 23%|██▎       | 722/3120 [59:43<3:11:01,  4.78s/it]
                                                    
{'loss': 1.1694, 'learning_rate': 1.825384867122677e-05, 'epoch': 0.93}

 23%|██▎       | 722/3120 [59:43<3:11:01,  4.78s/it]
 23%|██▎       | 723/3120 [59:48<3:13:05,  4.83s/it]
 23%|██▎       | 724/3120 [59:53<3:12:28,  4.82s/it]
                                                    
{'loss': 1.1092, 'learning_rate': 1.8241862183271338e-05, 'epoch': 0.93}

 23%|██▎       | 724/3120 [59:53<3:12:28,  4.82s/it]
 23%|██▎       | 725/3120 [59:58<3:14:16,  4.87s/it]
 23%|██▎       | 726/3120 [1:00:03<3:13:43,  4.86s/it]
                                                      
{'loss': 1.1343, 'learning_rate': 1.8229838658936566e-05, 'epoch': 0.93}

 23%|██▎       | 726/3120 [1:00:03<3:13:43,  4.86s/it]
 23%|██▎       | 727/3120 [1:00:07<3:08:59,  4.74s/it]
 23%|██▎       | 728/3120 [1:00:13<3:14:03,  4.87s/it]
                                                      
{'loss': 1.1233, 'learning_rate': 1.821777815225245e-05, 'epoch': 0.93}

 23%|██▎       | 728/3120 [1:00:13<3:14:03,  4.87s/it]
 23%|██▎       | 729/3120 [1:00:17<3:11:25,  4.80s/it]
 23%|██▎       | 730/3120 [1:00:22<3:13:08,  4.85s/it]
                                                      
{'loss': 1.0899, 'learning_rate': 1.8205680717415188e-05, 'epoch': 0.94}

 23%|██▎       | 730/3120 [1:00:22<3:13:08,  4.85s/it]
 23%|██▎       | 731/3120 [1:00:27<3:12:28,  4.83s/it]
 23%|██▎       | 732/3120 [1:00:32<3:16:25,  4.94s/it]
                                                      
{'loss': 1.1064, 'learning_rate': 1.81935464087869e-05, 'epoch': 0.94}

 23%|██▎       | 732/3120 [1:00:32<3:16:25,  4.94s/it]
 23%|██▎       | 733/3120 [1:00:37<3:15:54,  4.92s/it]
 24%|██▎       | 734/3120 [1:00:42<3:16:32,  4.94s/it]
                                                      
{'loss': 1.1538, 'learning_rate': 1.8181375280895417e-05, 'epoch': 0.94}

 24%|██▎       | 734/3120 [1:00:42<3:16:32,  4.94s/it]
 24%|██▎       | 735/3120 [1:00:47<3:16:51,  4.95s/it]
 24%|██▎       | 736/3120 [1:00:52<3:20:03,  5.04s/it]
                                                      
{'loss': 1.1558, 'learning_rate': 1.8169167388434024e-05, 'epoch': 0.94}

 24%|██▎       | 736/3120 [1:00:52<3:20:03,  5.04s/it]
 24%|██▎       | 737/3120 [1:00:58<3:25:20,  5.17s/it]
 24%|██▎       | 738/3120 [1:01:03<3:25:44,  5.18s/it]
                                                      
{'loss': 1.1142, 'learning_rate': 1.815692278626122e-05, 'epoch': 0.95}

 24%|██▎       | 738/3120 [1:01:03<3:25:44,  5.18s/it]
 24%|██▎       | 739/3120 [1:01:08<3:23:11,  5.12s/it]
 24%|██▎       | 740/3120 [1:01:13<3:20:56,  5.07s/it]
                                                      
{'loss': 1.1397, 'learning_rate': 1.8144641529400445e-05, 'epoch': 0.95}

 24%|██▎       | 740/3120 [1:01:13<3:20:56,  5.07s/it]
 24%|██▍       | 741/3120 [1:01:18<3:20:49,  5.06s/it]
 24%|██▍       | 742/3120 [1:01:22<3:12:56,  4.87s/it]
                                                      
{'loss': 1.1285, 'learning_rate': 1.8132323673039883e-05, 'epoch': 0.95}

 24%|██▍       | 742/3120 [1:01:22<3:12:56,  4.87s/it]
 24%|██▍       | 743/3120 [1:01:27<3:12:34,  4.86s/it]
 24%|██▍       | 744/3120 [1:01:32<3:13:05,  4.88s/it]
                                                      
{'loss': 1.1637, 'learning_rate': 1.8119969272532164e-05, 'epoch': 0.95}

 24%|██▍       | 744/3120 [1:01:32<3:13:05,  4.88s/it]
 24%|██▍       | 745/3120 [1:01:37<3:11:04,  4.83s/it]
 24%|██▍       | 746/3120 [1:01:42<3:11:49,  4.85s/it]
                                                      
{'loss': 1.1269, 'learning_rate': 1.8107578383394145e-05, 'epoch': 0.96}

 24%|██▍       | 746/3120 [1:01:42<3:11:49,  4.85s/it]
 24%|██▍       | 747/3120 [1:01:46<3:07:24,  4.74s/it]
 24%|██▍       | 748/3120 [1:01:51<3:08:37,  4.77s/it]
                                                      
{'loss': 1.1177, 'learning_rate': 1.8095151061306647e-05, 'epoch': 0.96}

 24%|██▍       | 748/3120 [1:01:51<3:08:37,  4.77s/it]
 24%|██▍       | 749/3120 [1:01:56<3:10:19,  4.82s/it]
 24%|██▍       | 750/3120 [1:02:01<3:12:21,  4.87s/it]
                                                      
{'loss': 1.1103, 'learning_rate': 1.808268736211421e-05, 'epoch': 0.96}

 24%|██▍       | 750/3120 [1:02:01<3:12:21,  4.87s/it]
 24%|██▍       | 751/3120 [1:02:06<3:10:17,  4.82s/it]
 24%|██▍       | 752/3120 [1:02:11<3:11:49,  4.86s/it]
                                                      
{'loss': 1.1156, 'learning_rate': 1.8070187341824848e-05, 'epoch': 0.96}

 24%|██▍       | 752/3120 [1:02:11<3:11:49,  4.86s/it]
 24%|██▍       | 753/3120 [1:02:16<3:12:31,  4.88s/it]
 24%|██▍       | 754/3120 [1:02:20<3:10:11,  4.82s/it]
                                                      
{'loss': 1.0974, 'learning_rate': 1.8057651056609784e-05, 'epoch': 0.97}

 24%|██▍       | 754/3120 [1:02:20<3:10:11,  4.82s/it]
 24%|██▍       | 755/3120 [1:02:25<3:11:29,  4.86s/it]
 24%|██▍       | 756/3120 [1:02:30<3:11:29,  4.86s/it]
                                                      
{'loss': 1.1858, 'learning_rate': 1.8045078562803203e-05, 'epoch': 0.97}

 24%|██▍       | 756/3120 [1:02:30<3:11:29,  4.86s/it]
 24%|██▍       | 757/3120 [1:02:35<3:10:27,  4.84s/it]
 24%|██▍       | 758/3120 [1:02:40<3:12:15,  4.88s/it]
                                                      
{'loss': 1.1328, 'learning_rate': 1.8032469916902004e-05, 'epoch': 0.97}

 24%|██▍       | 758/3120 [1:02:40<3:12:15,  4.88s/it]
 24%|██▍       | 759/3120 [1:02:45<3:13:50,  4.93s/it]
 24%|██▍       | 760/3120 [1:02:50<3:19:52,  5.08s/it]
                                                      
{'loss': 1.1207, 'learning_rate': 1.8019825175565544e-05, 'epoch': 0.97}

 24%|██▍       | 760/3120 [1:02:50<3:19:52,  5.08s/it]
 24%|██▍       | 761/3120 [1:02:55<3:18:00,  5.04s/it]
 24%|██▍       | 762/3120 [1:03:00<3:17:41,  5.03s/it]
                                                      
{'loss': 1.1782, 'learning_rate': 1.800714439561538e-05, 'epoch': 0.98}

 24%|██▍       | 762/3120 [1:03:00<3:17:41,  5.03s/it]
 24%|██▍       | 763/3120 [1:03:05<3:14:00,  4.94s/it]
 24%|██▍       | 764/3120 [1:03:10<3:16:09,  5.00s/it]
                                                      
{'loss': 1.1429, 'learning_rate': 1.7994427634035016e-05, 'epoch': 0.98}

 24%|██▍       | 764/3120 [1:03:10<3:16:09,  5.00s/it]
 25%|██▍       | 765/3120 [1:03:15<3:14:33,  4.96s/it]
 25%|██▍       | 766/3120 [1:03:20<3:15:38,  4.99s/it]
                                                      
{'loss': 1.1238, 'learning_rate': 1.7981674947969635e-05, 'epoch': 0.98}

 25%|██▍       | 766/3120 [1:03:20<3:15:38,  4.99s/it]
 25%|██▍       | 767/3120 [1:03:25<3:11:55,  4.89s/it]
 25%|██▍       | 768/3120 [1:03:30<3:13:23,  4.93s/it]
                                                      
{'loss': 1.1004, 'learning_rate': 1.7968886394725876e-05, 'epoch': 0.98}

 25%|██▍       | 768/3120 [1:03:30<3:13:23,  4.93s/it]
 25%|██▍       | 769/3120 [1:03:35<3:14:48,  4.97s/it]
 25%|██▍       | 770/3120 [1:03:40<3:20:24,  5.12s/it]
                                                      
{'loss': 1.096, 'learning_rate': 1.7956062031771533e-05, 'epoch': 0.99}

 25%|██▍       | 770/3120 [1:03:40<3:20:24,  5.12s/it]
 25%|██▍       | 771/3120 [1:03:45<3:17:49,  5.05s/it]
 25%|██▍       | 772/3120 [1:03:50<3:16:00,  5.01s/it]
                                                      
{'loss': 1.085, 'learning_rate': 1.7943201916735337e-05, 'epoch': 0.99}

 25%|██▍       | 772/3120 [1:03:50<3:16:00,  5.01s/it]
 25%|██▍       | 773/3120 [1:03:55<3:16:32,  5.02s/it]
 25%|██▍       | 774/3120 [1:04:00<3:17:25,  5.05s/it]
                                                      
{'loss': 1.1077, 'learning_rate': 1.793030610740665e-05, 'epoch': 0.99}

 25%|██▍       | 774/3120 [1:04:00<3:17:25,  5.05s/it]
 25%|██▍       | 775/3120 [1:04:05<3:18:42,  5.08s/it]
 25%|██▍       | 776/3120 [1:04:10<3:14:29,  4.98s/it]
                                                      
{'loss': 1.104, 'learning_rate': 1.791737466173527e-05, 'epoch': 0.99}

 25%|██▍       | 776/3120 [1:04:10<3:14:29,  4.98s/it]
 25%|██▍       | 777/3120 [1:04:15<3:13:10,  4.95s/it]
 25%|██▍       | 778/3120 [1:04:20<3:15:11,  5.00s/it]
                                                      
{'loss': 1.1207, 'learning_rate': 1.7904407637831097e-05, 'epoch': 1.0}

 25%|██▍       | 778/3120 [1:04:20<3:15:11,  5.00s/it]
 25%|██▍       | 779/3120 [1:04:25<3:18:01,  5.08s/it]
 25%|██▌       | 780/3120 [1:04:30<3:18:41,  5.09s/it]
                                                      
{'loss': 1.1076, 'learning_rate': 1.789140509396394e-05, 'epoch': 1.0}

 25%|██▌       | 780/3120 [1:04:30<3:18:41,  5.09s/it]/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
  warnings.warn(
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  labels = [torch.tensor(x) for x in labels]

 25%|██▌       | 781/3120 [1:05:05<9:02:56, 13.93s/it]
 25%|██▌       | 782/3120 [1:05:10<7:16:36, 11.20s/it]
                                                      
{'loss': 0.8392, 'learning_rate': 1.7878367088563195e-05, 'epoch': 1.0}

 25%|██▌       | 782/3120 [1:05:10<7:16:36, 11.20s/it]
 25%|██▌       | 783/3120 [1:05:15<6:02:50,  9.32s/it]
 25%|██▌       | 784/3120 [1:05:20<5:13:51,  8.06s/it]
                                                      
{'loss': 0.7721, 'learning_rate': 1.7865293680217636e-05, 'epoch': 1.01}

 25%|██▌       | 784/3120 [1:05:20<5:13:51,  8.06s/it]
 25%|██▌       | 785/3120 [1:05:25<4:41:42,  7.24s/it]
 25%|██▌       | 786/3120 [1:05:30<4:11:52,  6.47s/it]
                                                      
{'loss': 0.7814, 'learning_rate': 1.7852184927675113e-05, 'epoch': 1.01}

 25%|██▌       | 786/3120 [1:05:30<4:11:52,  6.47s/it]
 25%|██▌       | 787/3120 [1:05:35<3:57:16,  6.10s/it]
 25%|██▌       | 788/3120 [1:05:40<3:39:19,  5.64s/it]
                                                      
{'loss': 0.7843, 'learning_rate': 1.7839040889842307e-05, 'epoch': 1.01}

 25%|██▌       | 788/3120 [1:05:40<3:39:19,  5.64s/it]
 25%|██▌       | 789/3120 [1:05:45<3:34:46,  5.53s/it]
 25%|██▌       | 790/3120 [1:05:50<3:26:52,  5.33s/it]
                                                      
{'loss': 0.7827, 'learning_rate': 1.7825861625784457e-05, 'epoch': 1.01}

 25%|██▌       | 790/3120 [1:05:50<3:26:52,  5.33s/it]
 25%|██▌       | 791/3120 [1:05:55<3:23:19,  5.24s/it]
 25%|██▌       | 792/3120 [1:06:00<3:17:29,  5.09s/it]
                                                      
{'loss': 0.8196, 'learning_rate': 1.7812647194725093e-05, 'epoch': 1.02}

 25%|██▌       | 792/3120 [1:06:00<3:17:29,  5.09s/it]
 25%|██▌       | 793/3120 [1:06:04<3:11:17,  4.93s/it]
 25%|██▌       | 794/3120 [1:06:09<3:08:43,  4.87s/it]
                                                      
{'loss': 0.8131, 'learning_rate': 1.779939765604579e-05, 'epoch': 1.02}

 25%|██▌       | 794/3120 [1:06:09<3:08:43,  4.87s/it]
 25%|██▌       | 795/3120 [1:06:15<3:20:20,  5.17s/it]
 26%|██▌       | 796/3120 [1:06:19<3:14:05,  5.01s/it]
                                                      
{'loss': 0.7676, 'learning_rate': 1.7786113069285877e-05, 'epoch': 1.02}

 26%|██▌       | 796/3120 [1:06:19<3:14:05,  5.01s/it]
 26%|██▌       | 797/3120 [1:06:24<3:11:30,  4.95s/it]
 26%|██▌       | 798/3120 [1:06:29<3:14:08,  5.02s/it]
                                                      
{'loss': 0.8016, 'learning_rate': 1.777279349414217e-05, 'epoch': 1.02}

 26%|██▌       | 798/3120 [1:06:29<3:14:08,  5.02s/it]
 26%|██▌       | 799/3120 [1:06:34<3:14:16,  5.02s/it]
 26%|██▌       | 800/3120 [1:06:40<3:16:08,  5.07s/it]
                                                      
{'loss': 0.7868, 'learning_rate': 1.7759438990468726e-05, 'epoch': 1.03}

 26%|██▌       | 800/3120 [1:06:40<3:16:08,  5.07s/it]
 26%|██▌       | 801/3120 [1:06:44<3:13:46,  5.01s/it]
 26%|██▌       | 802/3120 [1:06:50<3:13:57,  5.02s/it]
                                                      
{'loss': 0.8294, 'learning_rate': 1.7746049618276545e-05, 'epoch': 1.03}

 26%|██▌       | 802/3120 [1:06:50<3:13:57,  5.02s/it]
 26%|██▌       | 803/3120 [1:06:55<3:20:13,  5.18s/it]
 26%|██▌       | 804/3120 [1:07:00<3:18:06,  5.13s/it]
                                                      
{'loss': 0.7904, 'learning_rate': 1.7732625437733338e-05, 'epoch': 1.03}

 26%|██▌       | 804/3120 [1:07:00<3:18:06,  5.13s/it]
 26%|██▌       | 805/3120 [1:07:05<3:15:02,  5.05s/it]
 26%|██▌       | 806/3120 [1:07:10<3:10:09,  4.93s/it]
                                                      
{'loss': 0.7852, 'learning_rate': 1.771916650916321e-05, 'epoch': 1.03}

 26%|██▌       | 806/3120 [1:07:10<3:10:09,  4.93s/it]
 26%|██▌       | 807/3120 [1:07:15<3:10:06,  4.93s/it]
 26%|██▌       | 808/3120 [1:07:20<3:11:38,  4.97s/it]
                                                      
{'loss': 0.772, 'learning_rate': 1.7705672893046425e-05, 'epoch': 1.04}

 26%|██▌       | 808/3120 [1:07:20<3:11:38,  4.97s/it]
 26%|██▌       | 809/3120 [1:07:24<3:06:11,  4.83s/it]
 26%|██▌       | 810/3120 [1:07:29<3:08:36,  4.90s/it]
                                                      
{'loss': 0.772, 'learning_rate': 1.7692144650019125e-05, 'epoch': 1.04}

 26%|██▌       | 810/3120 [1:07:29<3:08:36,  4.90s/it]
 26%|██▌       | 811/3120 [1:07:34<3:10:48,  4.96s/it]
 26%|██▌       | 812/3120 [1:07:40<3:16:27,  5.11s/it]
                                                      
{'loss': 0.7656, 'learning_rate': 1.767858184087304e-05, 'epoch': 1.04}

 26%|██▌       | 812/3120 [1:07:40<3:16:27,  5.11s/it]
 26%|██▌       | 813/3120 [1:07:45<3:16:26,  5.11s/it]
 26%|██▌       | 814/3120 [1:07:50<3:12:49,  5.02s/it]
                                                      
{'loss': 0.7976, 'learning_rate': 1.7664984526555247e-05, 'epoch': 1.04}

 26%|██▌       | 814/3120 [1:07:50<3:12:49,  5.02s/it]
 26%|██▌       | 815/3120 [1:07:55<3:16:18,  5.11s/it]
 26%|██▌       | 816/3120 [1:08:00<3:16:10,  5.11s/it]
                                                      
{'loss': 0.8023, 'learning_rate': 1.765135276816787e-05, 'epoch': 1.05}

 26%|██▌       | 816/3120 [1:08:00<3:16:10,  5.11s/it]
 26%|██▌       | 817/3120 [1:08:05<3:16:11,  5.11s/it]
 26%|██▌       | 818/3120 [1:08:10<3:17:05,  5.14s/it]
                                                      
{'loss': 0.7723, 'learning_rate': 1.7637686626967815e-05, 'epoch': 1.05}

 26%|██▌       | 818/3120 [1:08:10<3:17:05,  5.14s/it]
 26%|██▋       | 819/3120 [1:08:15<3:10:22,  4.96s/it]
 26%|██▋       | 820/3120 [1:08:19<3:03:57,  4.80s/it]
                                                      
{'loss': 0.8008, 'learning_rate': 1.7623986164366487e-05, 'epoch': 1.05}

 26%|██▋       | 820/3120 [1:08:19<3:03:57,  4.80s/it]
 26%|██▋       | 821/3120 [1:08:25<3:09:51,  4.96s/it]
 26%|██▋       | 822/3120 [1:08:30<3:10:19,  4.97s/it]
                                                      
{'loss': 0.7881, 'learning_rate': 1.7610251441929532e-05, 'epoch': 1.05}

 26%|██▋       | 822/3120 [1:08:30<3:10:19,  4.97s/it]
 26%|██▋       | 823/3120 [1:08:35<3:09:35,  4.95s/it]
 26%|██▋       | 824/3120 [1:08:40<3:11:11,  5.00s/it]
                                                      
{'loss': 0.7723, 'learning_rate': 1.7596482521376546e-05, 'epoch': 1.06}

 26%|██▋       | 824/3120 [1:08:40<3:11:11,  5.00s/it]
 26%|██▋       | 825/3120 [1:08:45<3:14:58,  5.10s/it]
 26%|██▋       | 826/3120 [1:08:50<3:13:35,  5.06s/it]
                                                      
{'loss': 0.7637, 'learning_rate': 1.7582679464580796e-05, 'epoch': 1.06}

 26%|██▋       | 826/3120 [1:08:50<3:13:35,  5.06s/it]
 27%|██▋       | 827/3120 [1:08:55<3:12:57,  5.05s/it]
 27%|██▋       | 828/3120 [1:09:00<3:14:47,  5.10s/it]
                                                      
{'loss': 0.8181, 'learning_rate': 1.7568842333568952e-05, 'epoch': 1.06}

 27%|██▋       | 828/3120 [1:09:00<3:14:47,  5.10s/it]
 27%|██▋       | 829/3120 [1:09:06<3:17:08,  5.16s/it]
 27%|██▋       | 830/3120 [1:09:11<3:14:30,  5.10s/it]
                                                      
{'loss': 0.8291, 'learning_rate': 1.75549711905208e-05, 'epoch': 1.06}

 27%|██▋       | 830/3120 [1:09:11<3:14:30,  5.10s/it]
 27%|██▋       | 831/3120 [1:09:16<3:15:09,  5.12s/it]
 27%|██▋       | 832/3120 [1:09:21<3:13:58,  5.09s/it]
                                                      
{'loss': 0.7597, 'learning_rate': 1.7541066097768965e-05, 'epoch': 1.07}

 27%|██▋       | 832/3120 [1:09:21<3:13:58,  5.09s/it]
 27%|██▋       | 833/3120 [1:09:25<3:08:41,  4.95s/it]
 27%|██▋       | 834/3120 [1:09:31<3:13:26,  5.08s/it]
                                                      
{'loss': 0.8401, 'learning_rate': 1.7527127117798635e-05, 'epoch': 1.07}

 27%|██▋       | 834/3120 [1:09:31<3:13:26,  5.08s/it]
 27%|██▋       | 835/3120 [1:09:35<3:05:57,  4.88s/it]
 27%|██▋       | 836/3120 [1:09:40<3:03:08,  4.81s/it]
                                                      
{'loss': 0.8012, 'learning_rate': 1.7513154313247273e-05, 'epoch': 1.07}

 27%|██▋       | 836/3120 [1:09:40<3:03:08,  4.81s/it]
 27%|██▋       | 837/3120 [1:09:45<3:09:45,  4.99s/it]
 27%|██▋       | 838/3120 [1:09:50<3:08:45,  4.96s/it]
                                                      
{'loss': 0.8042, 'learning_rate': 1.7499147746904337e-05, 'epoch': 1.07}

 27%|██▋       | 838/3120 [1:09:50<3:08:45,  4.96s/it]
 27%|██▋       | 839/3120 [1:09:55<3:09:30,  4.99s/it]
 27%|██▋       | 840/3120 [1:10:00<3:08:53,  4.97s/it]
                                                      
{'loss': 0.7931, 'learning_rate': 1.7485107481711014e-05, 'epoch': 1.08}

 27%|██▋       | 840/3120 [1:10:00<3:08:53,  4.97s/it]
 27%|██▋       | 841/3120 [1:10:05<3:05:48,  4.89s/it]
 27%|██▋       | 842/3120 [1:10:09<3:03:02,  4.82s/it]
                                                      
{'loss': 0.815, 'learning_rate': 1.7471033580759903e-05, 'epoch': 1.08}

 27%|██▋       | 842/3120 [1:10:09<3:03:02,  4.82s/it]
 27%|██▋       | 843/3120 [1:10:14<3:04:32,  4.86s/it]
 27%|██▋       | 844/3120 [1:10:19<3:03:57,  4.85s/it]
                                                      
{'loss': 0.8042, 'learning_rate': 1.7456926107294765e-05, 'epoch': 1.08}

 27%|██▋       | 844/3120 [1:10:19<3:03:57,  4.85s/it]
 27%|██▋       | 845/3120 [1:10:24<3:05:49,  4.90s/it]
 27%|██▋       | 846/3120 [1:10:29<3:09:45,  5.01s/it]
                                                      
{'loss': 0.7814, 'learning_rate': 1.7442785124710227e-05, 'epoch': 1.08}

 27%|██▋       | 846/3120 [1:10:29<3:09:45,  5.01s/it]
 27%|██▋       | 847/3120 [1:10:35<3:12:41,  5.09s/it]
 27%|██▋       | 848/3120 [1:10:40<3:10:18,  5.03s/it]
                                                      
{'loss': 0.7983, 'learning_rate': 1.742861069655148e-05, 'epoch': 1.09}

 27%|██▋       | 848/3120 [1:10:40<3:10:18,  5.03s/it]
 27%|██▋       | 849/3120 [1:10:44<3:08:26,  4.98s/it]
 27%|██▋       | 850/3120 [1:10:49<3:06:35,  4.93s/it]
                                                      
{'loss': 0.8045, 'learning_rate': 1.741440288651403e-05, 'epoch': 1.09}

 27%|██▋       | 850/3120 [1:10:49<3:06:35,  4.93s/it]
 27%|██▋       | 851/3120 [1:10:55<3:10:04,  5.03s/it]
 27%|██▋       | 852/3120 [1:11:00<3:11:17,  5.06s/it]
                                                      
{'loss': 0.7718, 'learning_rate': 1.7400161758443377e-05, 'epoch': 1.09}

 27%|██▋       | 852/3120 [1:11:00<3:11:17,  5.06s/it]
 27%|██▋       | 853/3120 [1:11:04<3:06:28,  4.94s/it]
 27%|██▋       | 854/3120 [1:11:09<3:06:14,  4.93s/it]
                                                      
{'loss': 0.8006, 'learning_rate': 1.7385887376334743e-05, 'epoch': 1.09}

 27%|██▋       | 854/3120 [1:11:09<3:06:14,  4.93s/it]
 27%|██▋       | 855/3120 [1:11:14<3:08:56,  5.00s/it]
 27%|██▋       | 856/3120 [1:11:19<3:09:04,  5.01s/it]
                                                      
{'loss': 0.7937, 'learning_rate': 1.737157980433279e-05, 'epoch': 1.1}

 27%|██▋       | 856/3120 [1:11:19<3:09:04,  5.01s/it]
 27%|██▋       | 857/3120 [1:11:25<3:10:52,  5.06s/it]
 28%|██▊       | 858/3120 [1:11:29<3:05:04,  4.91s/it]
                                                      
{'loss': 0.7888, 'learning_rate': 1.735723910673132e-05, 'epoch': 1.1}

 28%|██▊       | 858/3120 [1:11:29<3:05:04,  4.91s/it]
 28%|██▊       | 859/3120 [1:11:34<3:01:46,  4.82s/it]
 28%|██▊       | 860/3120 [1:11:38<2:59:40,  4.77s/it]
                                                      
{'loss': 0.8135, 'learning_rate': 1.7342865347972987e-05, 'epoch': 1.1}

 28%|██▊       | 860/3120 [1:11:38<2:59:40,  4.77s/it]
 28%|██▊       | 861/3120 [1:11:43<3:01:45,  4.83s/it]
 28%|██▊       | 862/3120 [1:11:48<3:03:23,  4.87s/it]
                                                      
{'loss': 0.805, 'learning_rate': 1.7328458592649028e-05, 'epoch': 1.11}

 28%|██▊       | 862/3120 [1:11:48<3:03:23,  4.87s/it]
 28%|██▊       | 863/3120 [1:11:53<3:02:52,  4.86s/it]
 28%|██▊       | 864/3120 [1:11:58<3:01:51,  4.84s/it]
                                                      
{'loss': 0.7874, 'learning_rate': 1.7314018905498932e-05, 'epoch': 1.11}

 28%|██▊       | 864/3120 [1:11:58<3:01:51,  4.84s/it]
 28%|██▊       | 865/3120 [1:12:03<3:02:29,  4.86s/it]
 28%|██▊       | 866/3120 [1:12:08<3:08:16,  5.01s/it]
                                                      
{'loss': 0.773, 'learning_rate': 1.7299546351410197e-05, 'epoch': 1.11}

 28%|██▊       | 866/3120 [1:12:08<3:08:16,  5.01s/it]
 28%|██▊       | 867/3120 [1:12:13<3:06:24,  4.96s/it]
 28%|██▊       | 868/3120 [1:12:18<3:03:31,  4.89s/it]
                                                      
{'loss': 0.8026, 'learning_rate': 1.7285040995418003e-05, 'epoch': 1.11}

 28%|██▊       | 868/3120 [1:12:18<3:03:31,  4.89s/it]
 28%|██▊       | 869/3120 [1:12:23<3:03:43,  4.90s/it]
 28%|██▊       | 870/3120 [1:12:28<3:04:18,  4.91s/it]
                                                      
{'loss': 0.8232, 'learning_rate': 1.7270502902704925e-05, 'epoch': 1.12}

 28%|██▊       | 870/3120 [1:12:28<3:04:18,  4.91s/it]
 28%|██▊       | 871/3120 [1:12:33<3:04:03,  4.91s/it]
 28%|██▊       | 872/3120 [1:12:38<3:03:33,  4.90s/it]
                                                      
{'loss': 0.802, 'learning_rate': 1.7255932138600665e-05, 'epoch': 1.12}

 28%|██▊       | 872/3120 [1:12:38<3:03:33,  4.90s/it]
 28%|██▊       | 873/3120 [1:12:43<3:10:29,  5.09s/it]
 28%|██▊       | 874/3120 [1:12:48<3:09:02,  5.05s/it]
                                                      
{'loss': 0.797, 'learning_rate': 1.7241328768581725e-05, 'epoch': 1.12}

 28%|██▊       | 874/3120 [1:12:48<3:09:02,  5.05s/it]
 28%|██▊       | 875/3120 [1:12:53<3:09:33,  5.07s/it]
 28%|██▊       | 876/3120 [1:12:58<3:09:27,  5.07s/it]
                                                      
{'loss': 0.7957, 'learning_rate': 1.7226692858271133e-05, 'epoch': 1.12}

 28%|██▊       | 876/3120 [1:12:58<3:09:27,  5.07s/it]
 28%|██▊       | 877/3120 [1:13:03<3:10:39,  5.10s/it]
 28%|██▊       | 878/3120 [1:13:09<3:12:20,  5.15s/it]
                                                      
{'loss': 0.7776, 'learning_rate': 1.7212024473438145e-05, 'epoch': 1.13}

 28%|██▊       | 878/3120 [1:13:09<3:12:20,  5.15s/it]
 28%|██▊       | 879/3120 [1:13:13<3:05:21,  4.96s/it]
 28%|██▊       | 880/3120 [1:13:19<3:10:06,  5.09s/it]
                                                      
{'loss': 0.7955, 'learning_rate': 1.7197323679997943e-05, 'epoch': 1.13}

 28%|██▊       | 880/3120 [1:13:19<3:10:06,  5.09s/it]
 28%|██▊       | 881/3120 [1:13:23<3:07:16,  5.02s/it]
 28%|██▊       | 882/3120 [1:13:28<3:02:39,  4.90s/it]
                                                      
{'loss': 0.781, 'learning_rate': 1.718259054401135e-05, 'epoch': 1.13}

 28%|██▊       | 882/3120 [1:13:28<3:02:39,  4.90s/it]
 28%|██▊       | 883/3120 [1:13:34<3:10:22,  5.11s/it]
 28%|██▊       | 884/3120 [1:13:38<3:05:23,  4.97s/it]
                                                      
{'loss': 0.79, 'learning_rate': 1.7167825131684516e-05, 'epoch': 1.13}

 28%|██▊       | 884/3120 [1:13:38<3:05:23,  4.97s/it]
 28%|██▊       | 885/3120 [1:13:43<3:02:20,  4.89s/it]
 28%|██▊       | 886/3120 [1:13:47<2:56:59,  4.75s/it]
                                                      
{'loss': 0.8113, 'learning_rate': 1.7153027509368637e-05, 'epoch': 1.14}

 28%|██▊       | 886/3120 [1:13:47<2:56:59,  4.75s/it]
 28%|██▊       | 887/3120 [1:13:53<3:01:41,  4.88s/it]
 28%|██▊       | 888/3120 [1:13:57<2:58:53,  4.81s/it]
                                                      
{'loss': 0.8052, 'learning_rate': 1.7138197743559656e-05, 'epoch': 1.14}

 28%|██▊       | 888/3120 [1:13:57<2:58:53,  4.81s/it]
 28%|██▊       | 889/3120 [1:14:02<3:02:12,  4.90s/it]
 29%|██▊       | 890/3120 [1:14:07<3:04:07,  4.95s/it]
                                                      
{'loss': 0.7755, 'learning_rate': 1.7123335900897946e-05, 'epoch': 1.14}

 29%|██▊       | 890/3120 [1:14:07<3:04:07,  4.95s/it]
 29%|██▊       | 891/3120 [1:14:12<3:02:32,  4.91s/it]
 29%|██▊       | 892/3120 [1:14:17<3:04:20,  4.96s/it]
                                                      
{'loss': 0.818, 'learning_rate': 1.7108442048168038e-05, 'epoch': 1.14}

 29%|██▊       | 892/3120 [1:14:17<3:04:20,  4.96s/it]
 29%|██▊       | 893/3120 [1:14:22<3:02:21,  4.91s/it]
 29%|██▊       | 894/3120 [1:14:27<3:01:20,  4.89s/it]
                                                      
{'loss': 0.8166, 'learning_rate': 1.7093516252298296e-05, 'epoch': 1.15}

 29%|██▊       | 894/3120 [1:14:27<3:01:20,  4.89s/it]
 29%|██▊       | 895/3120 [1:14:32<3:08:37,  5.09s/it]
 29%|██▊       | 896/3120 [1:14:38<3:08:53,  5.10s/it]
                                                      
{'loss': 0.8, 'learning_rate': 1.707855858036063e-05, 'epoch': 1.15}

 29%|██▊       | 896/3120 [1:14:38<3:08:53,  5.10s/it]
 29%|██▉       | 897/3120 [1:14:43<3:09:27,  5.11s/it]
 29%|██▉       | 898/3120 [1:14:48<3:07:37,  5.07s/it]
                                                      
{'loss': 0.804, 'learning_rate': 1.7063569099570196e-05, 'epoch': 1.15}

 29%|██▉       | 898/3120 [1:14:48<3:07:37,  5.07s/it]
 29%|██▉       | 899/3120 [1:14:53<3:06:12,  5.03s/it]
 29%|██▉       | 900/3120 [1:14:57<3:00:25,  4.88s/it]
                                                      
{'loss': 0.8087, 'learning_rate': 1.7048547877285078e-05, 'epoch': 1.15}

 29%|██▉       | 900/3120 [1:14:57<3:00:25,  4.88s/it]
 29%|██▉       | 901/3120 [1:15:02<3:02:32,  4.94s/it]
 29%|██▉       | 902/3120 [1:15:07<3:01:07,  4.90s/it]
                                                      
{'loss': 0.7976, 'learning_rate': 1.7033494981006003e-05, 'epoch': 1.16}

 29%|██▉       | 902/3120 [1:15:07<3:01:07,  4.90s/it]
 29%|██▉       | 903/3120 [1:15:12<3:02:28,  4.94s/it]
 29%|██▉       | 904/3120 [1:15:17<3:01:10,  4.91s/it]
                                                      
{'loss': 0.7594, 'learning_rate': 1.7018410478376033e-05, 'epoch': 1.16}

 29%|██▉       | 904/3120 [1:15:17<3:01:10,  4.91s/it]
 29%|██▉       | 905/3120 [1:15:22<2:59:53,  4.87s/it]
 29%|██▉       | 906/3120 [1:15:27<3:01:54,  4.93s/it]
                                                      
{'loss': 0.7686, 'learning_rate': 1.7003294437180254e-05, 'epoch': 1.16}

 29%|██▉       | 906/3120 [1:15:27<3:01:54,  4.93s/it]
 29%|██▉       | 907/3120 [1:15:32<3:03:13,  4.97s/it]
 29%|██▉       | 908/3120 [1:15:37<3:04:15,  5.00s/it]
                                                      
{'loss': 0.7879, 'learning_rate': 1.6988146925345487e-05, 'epoch': 1.16}

 29%|██▉       | 908/3120 [1:15:37<3:04:15,  5.00s/it]
 29%|██▉       | 909/3120 [1:15:42<3:05:16,  5.03s/it]
 29%|██▉       | 910/3120 [1:15:47<3:02:41,  4.96s/it]
                                                      
{'loss': 0.8057, 'learning_rate': 1.6972968010939953e-05, 'epoch': 1.17}

 29%|██▉       | 910/3120 [1:15:47<3:02:41,  4.96s/it]
 29%|██▉       | 911/3120 [1:15:52<3:03:04,  4.97s/it]
 29%|██▉       | 912/3120 [1:15:57<3:08:34,  5.12s/it]
                                                      
{'loss': 0.8118, 'learning_rate': 1.695775776217301e-05, 'epoch': 1.17}

 29%|██▉       | 912/3120 [1:15:57<3:08:34,  5.12s/it]
 29%|██▉       | 913/3120 [1:16:03<3:10:40,  5.18s/it]
 29%|██▉       | 914/3120 [1:16:07<3:05:58,  5.06s/it]
                                                      
{'loss': 0.7613, 'learning_rate': 1.6942516247394806e-05, 'epoch': 1.17}

 29%|██▉       | 914/3120 [1:16:07<3:05:58,  5.06s/it]
 29%|██▉       | 915/3120 [1:16:12<3:04:51,  5.03s/it]
 29%|██▉       | 916/3120 [1:16:17<3:03:32,  5.00s/it]
                                                      
{'loss': 0.8186, 'learning_rate': 1.6927243535095995e-05, 'epoch': 1.17}

 29%|██▉       | 916/3120 [1:16:17<3:03:32,  5.00s/it]
 29%|██▉       | 917/3120 [1:16:22<3:01:17,  4.94s/it]
 29%|██▉       | 918/3120 [1:16:27<3:01:21,  4.94s/it]
                                                      
{'loss': 0.8214, 'learning_rate': 1.6911939693907422e-05, 'epoch': 1.18}

 29%|██▉       | 918/3120 [1:16:27<3:01:21,  4.94s/it]
 29%|██▉       | 919/3120 [1:16:32<3:02:51,  4.98s/it]
 29%|██▉       | 920/3120 [1:16:37<3:03:17,  5.00s/it]
                                                      
{'loss': 0.7938, 'learning_rate': 1.6896604792599813e-05, 'epoch': 1.18}

 29%|██▉       | 920/3120 [1:16:37<3:03:17,  5.00s/it]
 30%|██▉       | 921/3120 [1:16:42<2:57:37,  4.85s/it]
 30%|██▉       | 922/3120 [1:16:46<2:55:10,  4.78s/it]
                                                      
{'loss': 0.7804, 'learning_rate': 1.6881238900083472e-05, 'epoch': 1.18}

 30%|██▉       | 922/3120 [1:16:46<2:55:10,  4.78s/it]
 30%|██▉       | 923/3120 [1:16:51<2:55:32,  4.79s/it]
 30%|██▉       | 924/3120 [1:16:56<2:55:32,  4.80s/it]
                                                      
{'loss': 0.8361, 'learning_rate': 1.686584208540797e-05, 'epoch': 1.18}

 30%|██▉       | 924/3120 [1:16:56<2:55:32,  4.80s/it]
 30%|██▉       | 925/3120 [1:17:01<2:57:56,  4.86s/it]
 30%|██▉       | 926/3120 [1:17:06<3:03:20,  5.01s/it]
                                                      
{'loss': 0.7873, 'learning_rate': 1.685041441776183e-05, 'epoch': 1.19}

 30%|██▉       | 926/3120 [1:17:06<3:03:20,  5.01s/it]
 30%|██▉       | 927/3120 [1:17:11<3:03:21,  5.02s/it]
 30%|██▉       | 928/3120 [1:17:16<3:03:10,  5.01s/it]
                                                      
{'loss': 0.8094, 'learning_rate': 1.6834955966472214e-05, 'epoch': 1.19}

 30%|██▉       | 928/3120 [1:17:16<3:03:10,  5.01s/it]
 30%|██▉       | 929/3120 [1:17:21<2:58:49,  4.90s/it]
 30%|██▉       | 930/3120 [1:17:26<3:02:55,  5.01s/it]
                                                      
{'loss': 0.8054, 'learning_rate': 1.6819466801004622e-05, 'epoch': 1.19}

 30%|██▉       | 930/3120 [1:17:26<3:02:55,  5.01s/it]
 30%|██▉       | 931/3120 [1:17:31<3:03:16,  5.02s/it]
 30%|██▉       | 932/3120 [1:17:36<3:02:25,  5.00s/it]
                                                      
{'loss': 0.7698, 'learning_rate': 1.6803946990962577e-05, 'epoch': 1.19}

 30%|██▉       | 932/3120 [1:17:36<3:02:25,  5.00s/it]
 30%|██▉       | 933/3120 [1:17:41<3:05:18,  5.08s/it]
 30%|██▉       | 934/3120 [1:17:46<3:01:24,  4.98s/it]
                                                      
{'loss': 0.7648, 'learning_rate': 1.67883966060873e-05, 'epoch': 1.2}

 30%|██▉       | 934/3120 [1:17:46<3:01:24,  4.98s/it]
 30%|██▉       | 935/3120 [1:17:52<3:05:47,  5.10s/it]
 30%|███       | 936/3120 [1:17:57<3:03:52,  5.05s/it]
                                                      
{'loss': 0.7801, 'learning_rate': 1.6772815716257414e-05, 'epoch': 1.2}

 30%|███       | 936/3120 [1:17:57<3:03:52,  5.05s/it]
 30%|███       | 937/3120 [1:18:02<3:04:47,  5.08s/it]
 30%|███       | 938/3120 [1:18:07<3:04:54,  5.08s/it]
                                                      
{'loss': 0.8163, 'learning_rate': 1.6757204391488617e-05, 'epoch': 1.2}

 30%|███       | 938/3120 [1:18:07<3:04:54,  5.08s/it]
 30%|███       | 939/3120 [1:18:12<3:03:07,  5.04s/it]
 30%|███       | 940/3120 [1:18:17<3:03:23,  5.05s/it]
                                                      
{'loss': 0.8108, 'learning_rate': 1.6741562701933366e-05, 'epoch': 1.21}

 30%|███       | 940/3120 [1:18:17<3:03:23,  5.05s/it]
 30%|███       | 941/3120 [1:18:22<3:00:43,  4.98s/it]
 30%|███       | 942/3120 [1:18:26<2:56:19,  4.86s/it]
                                                      
{'loss': 0.8093, 'learning_rate': 1.672589071788059e-05, 'epoch': 1.21}

 30%|███       | 942/3120 [1:18:26<2:56:19,  4.86s/it]
 30%|███       | 943/3120 [1:18:31<2:53:53,  4.79s/it]
 30%|███       | 944/3120 [1:18:35<2:52:45,  4.76s/it]
                                                      
{'loss': 0.8412, 'learning_rate': 1.671018850975533e-05, 'epoch': 1.21}

 30%|███       | 944/3120 [1:18:35<2:52:45,  4.76s/it]
 30%|███       | 945/3120 [1:18:40<2:49:59,  4.69s/it]
 30%|███       | 946/3120 [1:18:44<2:44:31,  4.54s/it]
                                                      
{'loss': 0.7669, 'learning_rate': 1.6694456148118453e-05, 'epoch': 1.21}

 30%|███       | 946/3120 [1:18:44<2:44:31,  4.54s/it]
 30%|███       | 947/3120 [1:18:49<2:49:20,  4.68s/it]
 30%|███       | 948/3120 [1:18:54<2:56:14,  4.87s/it]
                                                      
{'loss': 0.7766, 'learning_rate': 1.6678693703666327e-05, 'epoch': 1.22}

 30%|███       | 948/3120 [1:18:55<2:56:14,  4.87s/it]
 30%|███       | 949/3120 [1:18:59<2:56:09,  4.87s/it]
 30%|███       | 950/3120 [1:19:04<2:57:04,  4.90s/it]
                                                      
{'loss': 0.8172, 'learning_rate': 1.66629012472305e-05, 'epoch': 1.22}

 30%|███       | 950/3120 [1:19:04<2:57:04,  4.90s/it]
 30%|███       | 951/3120 [1:19:09<2:55:19,  4.85s/it]
 31%|███       | 952/3120 [1:19:14<2:57:20,  4.91s/it]
                                                      
{'loss': 0.7943, 'learning_rate': 1.664707884977739e-05, 'epoch': 1.22}

 31%|███       | 952/3120 [1:19:14<2:57:20,  4.91s/it]
 31%|███       | 953/3120 [1:19:19<2:55:30,  4.86s/it]
 31%|███       | 954/3120 [1:19:23<2:52:52,  4.79s/it]
                                                      
{'loss': 0.7944, 'learning_rate': 1.6631226582407954e-05, 'epoch': 1.22}

 31%|███       | 954/3120 [1:19:23<2:52:52,  4.79s/it]
 31%|███       | 955/3120 [1:19:28<2:52:51,  4.79s/it]
 31%|███       | 956/3120 [1:19:33<2:53:49,  4.82s/it]
                                                      
{'loss': 0.7759, 'learning_rate': 1.661534451635738e-05, 'epoch': 1.23}

 31%|███       | 956/3120 [1:19:33<2:53:49,  4.82s/it]
 31%|███       | 957/3120 [1:19:38<2:55:20,  4.86s/it]
 31%|███       | 958/3120 [1:19:43<2:55:47,  4.88s/it]
                                                      
{'loss': 0.7853, 'learning_rate': 1.6599432722994754e-05, 'epoch': 1.23}

 31%|███       | 958/3120 [1:19:43<2:55:47,  4.88s/it]
 31%|███       | 959/3120 [1:19:48<2:57:01,  4.91s/it]
 31%|███       | 960/3120 [1:19:53<2:56:34,  4.90s/it]
                                                      
{'loss': 0.7879, 'learning_rate': 1.6583491273822763e-05, 'epoch': 1.23}

 31%|███       | 960/3120 [1:19:53<2:56:34,  4.90s/it]
 31%|███       | 961/3120 [1:19:58<3:01:02,  5.03s/it]
 31%|███       | 962/3120 [1:20:03<2:57:52,  4.95s/it]
                                                      
{'loss': 0.7834, 'learning_rate': 1.6567520240477344e-05, 'epoch': 1.23}

 31%|███       | 962/3120 [1:20:03<2:57:52,  4.95s/it]
 31%|███       | 963/3120 [1:20:08<2:57:05,  4.93s/it]
 31%|███       | 964/3120 [1:20:13<2:58:03,  4.96s/it]
                                                      
{'loss': 0.7992, 'learning_rate': 1.655151969472738e-05, 'epoch': 1.24}

 31%|███       | 964/3120 [1:20:13<2:58:03,  4.96s/it]
 31%|███       | 965/3120 [1:20:18<2:56:22,  4.91s/it]
 31%|███       | 966/3120 [1:20:23<2:55:57,  4.90s/it]
                                                      
{'loss': 0.801, 'learning_rate': 1.653548970847438e-05, 'epoch': 1.24}

 31%|███       | 966/3120 [1:20:23<2:55:57,  4.90s/it]
 31%|███       | 967/3120 [1:20:28<2:58:00,  4.96s/it]
 31%|███       | 968/3120 [1:20:33<2:58:10,  4.97s/it]
                                                      
{'loss': 0.7843, 'learning_rate': 1.6519430353752138e-05, 'epoch': 1.24}

 31%|███       | 968/3120 [1:20:33<2:58:10,  4.97s/it]
 31%|███       | 969/3120 [1:20:37<2:54:40,  4.87s/it]
 31%|███       | 970/3120 [1:20:42<2:57:12,  4.95s/it]
                                                      
{'loss': 0.7752, 'learning_rate': 1.6503341702726428e-05, 'epoch': 1.24}

 31%|███       | 970/3120 [1:20:42<2:57:12,  4.95s/it]
 31%|███       | 971/3120 [1:20:47<2:56:48,  4.94s/it]
 31%|███       | 972/3120 [1:20:52<2:53:15,  4.84s/it]
                                                      
{'loss': 0.8022, 'learning_rate': 1.6487223827694673e-05, 'epoch': 1.25}

 31%|███       | 972/3120 [1:20:52<2:53:15,  4.84s/it]
 31%|███       | 973/3120 [1:20:57<2:51:48,  4.80s/it]
 31%|███       | 974/3120 [1:21:01<2:50:07,  4.76s/it]
                                                      
{'loss': 0.8492, 'learning_rate': 1.6471076801085618e-05, 'epoch': 1.25}

 31%|███       | 974/3120 [1:21:01<2:50:07,  4.76s/it]
 31%|███▏      | 975/3120 [1:21:07<2:54:31,  4.88s/it]
 31%|███▏      | 976/3120 [1:21:11<2:52:57,  4.84s/it]
                                                      
{'loss': 0.7996, 'learning_rate': 1.6454900695459e-05, 'epoch': 1.25}

 31%|███▏      | 976/3120 [1:21:11<2:52:57,  4.84s/it]
 31%|███▏      | 977/3120 [1:21:16<2:55:08,  4.90s/it]
 31%|███▏      | 978/3120 [1:21:21<2:57:08,  4.96s/it]
                                                      
{'loss': 0.7611, 'learning_rate': 1.643869558350524e-05, 'epoch': 1.25}

 31%|███▏      | 978/3120 [1:21:21<2:57:08,  4.96s/it]
 31%|███▏      | 979/3120 [1:21:26<2:53:04,  4.85s/it]
 31%|███▏      | 980/3120 [1:21:31<2:51:34,  4.81s/it]
                                                      
{'loss': 0.7562, 'learning_rate': 1.6422461538045104e-05, 'epoch': 1.26}

 31%|███▏      | 980/3120 [1:21:31<2:51:34,  4.81s/it]
 31%|███▏      | 981/3120 [1:21:36<2:56:33,  4.95s/it]
 31%|███▏      | 982/3120 [1:21:41<2:52:37,  4.84s/it]
                                                      
{'loss': 0.7883, 'learning_rate': 1.6406198632029358e-05, 'epoch': 1.26}

 31%|███▏      | 982/3120 [1:21:41<2:52:37,  4.84s/it]
 32%|███▏      | 983/3120 [1:21:46<2:54:19,  4.89s/it]
 32%|███▏      | 984/3120 [1:21:50<2:50:38,  4.79s/it]
                                                      
{'loss': 0.7932, 'learning_rate': 1.638990693853848e-05, 'epoch': 1.26}

 32%|███▏      | 984/3120 [1:21:50<2:50:38,  4.79s/it]
 32%|███▏      | 985/3120 [1:21:55<2:50:02,  4.78s/it]
 32%|███▏      | 986/3120 [1:22:00<2:50:29,  4.79s/it]
                                                      
{'loss': 0.811, 'learning_rate': 1.63735865307823e-05, 'epoch': 1.26}

 32%|███▏      | 986/3120 [1:22:00<2:50:29,  4.79s/it]
 32%|███▏      | 987/3120 [1:22:05<2:53:15,  4.87s/it]
 32%|███▏      | 988/3120 [1:22:10<2:53:02,  4.87s/it]
                                                      
{'loss': 0.7592, 'learning_rate': 1.6357237482099682e-05, 'epoch': 1.27}

 32%|███▏      | 988/3120 [1:22:10<2:53:02,  4.87s/it]
 32%|███▏      | 989/3120 [1:22:15<2:53:52,  4.90s/it]
 32%|███▏      | 990/3120 [1:22:20<2:57:47,  5.01s/it]
                                                      
{'loss': 0.7816, 'learning_rate': 1.6340859865958193e-05, 'epoch': 1.27}

 32%|███▏      | 990/3120 [1:22:20<2:57:47,  5.01s/it]
 32%|███▏      | 991/3120 [1:22:25<2:56:23,  4.97s/it]
 32%|███▏      | 992/3120 [1:22:29<2:53:13,  4.88s/it]
                                                      
{'loss': 0.788, 'learning_rate': 1.6324453755953772e-05, 'epoch': 1.27}

 32%|███▏      | 992/3120 [1:22:29<2:53:13,  4.88s/it]
 32%|███▏      | 993/3120 [1:22:34<2:52:54,  4.88s/it]
 32%|███▏      | 994/3120 [1:22:39<2:51:12,  4.83s/it]
                                                      
{'loss': 0.8024, 'learning_rate': 1.630801922581041e-05, 'epoch': 1.27}

 32%|███▏      | 994/3120 [1:22:39<2:51:12,  4.83s/it]
 32%|███▏      | 995/3120 [1:22:44<2:51:37,  4.85s/it]
 32%|███▏      | 996/3120 [1:22:49<2:52:19,  4.87s/it]
                                                      
{'loss': 0.7902, 'learning_rate': 1.6291556349379794e-05, 'epoch': 1.28}

 32%|███▏      | 996/3120 [1:22:49<2:52:19,  4.87s/it]
 32%|███▏      | 997/3120 [1:22:54<2:56:21,  4.98s/it]
 32%|███▏      | 998/3120 [1:22:59<2:59:27,  5.07s/it]
                                                      
{'loss': 0.791, 'learning_rate': 1.6275065200641004e-05, 'epoch': 1.28}

 32%|███▏      | 998/3120 [1:22:59<2:59:27,  5.07s/it]
 32%|███▏      | 999/3120 [1:23:05<3:00:09,  5.10s/it]
 32%|███▏      | 1000/3120 [1:23:09<2:58:18,  5.05s/it]
                                                       
{'loss': 0.7769, 'learning_rate': 1.6258545853700157e-05, 'epoch': 1.28}

 32%|███▏      | 1000/3120 [1:23:09<2:58:18,  5.05s/it]
 32%|███▏      | 1001/3120 [1:23:14<2:51:51,  4.87s/it]
 32%|███▏      | 1002/3120 [1:23:19<2:52:22,  4.88s/it]
                                                       
{'loss': 0.8325, 'learning_rate': 1.6241998382790095e-05, 'epoch': 1.28}

 32%|███▏      | 1002/3120 [1:23:19<2:52:22,  4.88s/it]
 32%|███▏      | 1003/3120 [1:23:23<2:48:49,  4.78s/it]
 32%|███▏      | 1004/3120 [1:23:28<2:51:45,  4.87s/it]
                                                       
{'loss': 0.7926, 'learning_rate': 1.622542286227003e-05, 'epoch': 1.29}

 32%|███▏      | 1004/3120 [1:23:28<2:51:45,  4.87s/it]
 32%|███▏      | 1005/3120 [1:23:33<2:53:03,  4.91s/it]
 32%|███▏      | 1006/3120 [1:23:38<2:48:17,  4.78s/it]
                                                       
{'loss': 0.8153, 'learning_rate': 1.620881936662522e-05, 'epoch': 1.29}

 32%|███▏      | 1006/3120 [1:23:38<2:48:17,  4.78s/it]
 32%|███▏      | 1007/3120 [1:23:43<2:52:01,  4.88s/it]
 32%|███▏      | 1008/3120 [1:23:48<2:53:01,  4.92s/it]
                                                       
{'loss': 0.7851, 'learning_rate': 1.6192187970466646e-05, 'epoch': 1.29}

 32%|███▏      | 1008/3120 [1:23:48<2:53:01,  4.92s/it]
 32%|███▏      | 1009/3120 [1:23:53<2:52:24,  4.90s/it]
 32%|███▏      | 1010/3120 [1:23:58<2:50:19,  4.84s/it]
                                                       
{'loss': 0.7744, 'learning_rate': 1.617552874853065e-05, 'epoch': 1.29}

 32%|███▏      | 1010/3120 [1:23:58<2:50:19,  4.84s/it]
 32%|███▏      | 1011/3120 [1:24:02<2:49:01,  4.81s/it]
 32%|███▏      | 1012/3120 [1:24:07<2:47:54,  4.78s/it]
                                                       
{'loss': 0.7978, 'learning_rate': 1.615884177567863e-05, 'epoch': 1.3}

 32%|███▏      | 1012/3120 [1:24:07<2:47:54,  4.78s/it]
 32%|███▏      | 1013/3120 [1:24:12<2:47:33,  4.77s/it]
 32%|███▎      | 1014/3120 [1:24:17<2:50:20,  4.85s/it]
                                                       
{'loss': 0.7764, 'learning_rate': 1.6142127126896682e-05, 'epoch': 1.3}

 32%|███▎      | 1014/3120 [1:24:17<2:50:20,  4.85s/it]
 33%|███▎      | 1015/3120 [1:24:22<2:51:51,  4.90s/it]
 33%|███▎      | 1016/3120 [1:24:27<2:51:52,  4.90s/it]
                                                       
{'loss': 0.7693, 'learning_rate': 1.6125384877295255e-05, 'epoch': 1.3}

 33%|███▎      | 1016/3120 [1:24:27<2:51:52,  4.90s/it]
 33%|███▎      | 1017/3120 [1:24:32<2:56:31,  5.04s/it]
 33%|███▎      | 1018/3120 [1:24:37<2:58:28,  5.09s/it]
                                                       
{'loss': 0.8497, 'learning_rate': 1.6108615102108854e-05, 'epoch': 1.31}

 33%|███▎      | 1018/3120 [1:24:37<2:58:28,  5.09s/it]
 33%|███▎      | 1019/3120 [1:24:42<2:57:24,  5.07s/it]
 33%|███▎      | 1020/3120 [1:24:47<2:56:19,  5.04s/it]
                                                       
{'loss': 0.7874, 'learning_rate': 1.6091817876695655e-05, 'epoch': 1.31}

 33%|███▎      | 1020/3120 [1:24:47<2:56:19,  5.04s/it]
 33%|███▎      | 1021/3120 [1:24:53<2:58:32,  5.10s/it]
 33%|███▎      | 1022/3120 [1:24:57<2:53:50,  4.97s/it]
                                                       
{'loss': 0.7844, 'learning_rate': 1.6074993276537198e-05, 'epoch': 1.31}

 33%|███▎      | 1022/3120 [1:24:57<2:53:50,  4.97s/it]
 33%|███▎      | 1023/3120 [1:25:02<2:52:56,  4.95s/it]
 33%|███▎      | 1024/3120 [1:25:07<2:51:04,  4.90s/it]
                                                       
{'loss': 0.8288, 'learning_rate': 1.6058141377238026e-05, 'epoch': 1.31}

 33%|███▎      | 1024/3120 [1:25:07<2:51:04,  4.90s/it]
 33%|███▎      | 1025/3120 [1:25:12<2:48:32,  4.83s/it]
 33%|███▎      | 1026/3120 [1:25:17<2:52:26,  4.94s/it]
                                                       
{'loss': 0.8386, 'learning_rate': 1.6041262254525362e-05, 'epoch': 1.32}

 33%|███▎      | 1026/3120 [1:25:17<2:52:26,  4.94s/it]
 33%|███▎      | 1027/3120 [1:25:21<2:48:03,  4.82s/it]
 33%|███▎      | 1028/3120 [1:25:26<2:51:16,  4.91s/it]
                                                       
{'loss': 0.7879, 'learning_rate': 1.602435598424877e-05, 'epoch': 1.32}

 33%|███▎      | 1028/3120 [1:25:26<2:51:16,  4.91s/it]
 33%|███▎      | 1029/3120 [1:25:31<2:48:59,  4.85s/it]
 33%|███▎      | 1030/3120 [1:25:36<2:48:11,  4.83s/it]
                                                       
{'loss': 0.8181, 'learning_rate': 1.600742264237979e-05, 'epoch': 1.32}

 33%|███▎      | 1030/3120 [1:25:36<2:48:11,  4.83s/it]
 33%|███▎      | 1031/3120 [1:25:41<2:46:41,  4.79s/it]
 33%|███▎      | 1032/3120 [1:25:45<2:46:02,  4.77s/it]
                                                       
{'loss': 0.7813, 'learning_rate': 1.599046230501163e-05, 'epoch': 1.32}

 33%|███▎      | 1032/3120 [1:25:45<2:46:02,  4.77s/it]
 33%|███▎      | 1033/3120 [1:25:50<2:44:59,  4.74s/it]
 33%|███▎      | 1034/3120 [1:25:55<2:50:26,  4.90s/it]
                                                       
{'loss': 0.8164, 'learning_rate': 1.5973475048358795e-05, 'epoch': 1.33}

 33%|███▎      | 1034/3120 [1:25:55<2:50:26,  4.90s/it]
 33%|███▎      | 1035/3120 [1:26:00<2:52:44,  4.97s/it]
 33%|███▎      | 1036/3120 [1:26:05<2:49:11,  4.87s/it]
                                                       
{'loss': 0.8212, 'learning_rate': 1.5956460948756765e-05, 'epoch': 1.33}

 33%|███▎      | 1036/3120 [1:26:05<2:49:11,  4.87s/it]
 33%|███▎      | 1037/3120 [1:26:10<2:45:02,  4.75s/it]
 33%|███▎      | 1038/3120 [1:26:15<2:47:20,  4.82s/it]
                                                       
{'loss': 0.7928, 'learning_rate': 1.593942008266164e-05, 'epoch': 1.33}

 33%|███▎      | 1038/3120 [1:26:15<2:47:20,  4.82s/it]
 33%|███▎      | 1039/3120 [1:26:20<2:49:46,  4.89s/it]
 33%|███▎      | 1040/3120 [1:26:24<2:49:22,  4.89s/it]
                                                       
{'loss': 0.7925, 'learning_rate': 1.5922352526649803e-05, 'epoch': 1.33}

 33%|███▎      | 1040/3120 [1:26:24<2:49:22,  4.89s/it]
 33%|███▎      | 1041/3120 [1:26:30<2:52:27,  4.98s/it]
 33%|███▎      | 1042/3120 [1:26:35<2:52:39,  4.99s/it]
                                                       
{'loss': 0.7599, 'learning_rate': 1.5905258357417567e-05, 'epoch': 1.34}

 33%|███▎      | 1042/3120 [1:26:35<2:52:39,  4.99s/it]
 33%|███▎      | 1043/3120 [1:26:39<2:49:57,  4.91s/it]
 33%|███▎      | 1044/3120 [1:26:44<2:49:50,  4.91s/it]
                                                       
{'loss': 0.7971, 'learning_rate': 1.5888137651780847e-05, 'epoch': 1.34}

 33%|███▎      | 1044/3120 [1:26:44<2:49:50,  4.91s/it]
 33%|███▎      | 1045/3120 [1:26:49<2:50:40,  4.94s/it]
 34%|███▎      | 1046/3120 [1:26:54<2:53:16,  5.01s/it]
                                                       
{'loss': 0.8046, 'learning_rate': 1.5870990486674794e-05, 'epoch': 1.34}

 34%|███▎      | 1046/3120 [1:26:54<2:53:16,  5.01s/it]
 34%|███▎      | 1047/3120 [1:26:59<2:50:50,  4.94s/it]
 34%|███▎      | 1048/3120 [1:27:04<2:50:56,  4.95s/it]
                                                       
{'loss': 0.7811, 'learning_rate': 1.585381693915346e-05, 'epoch': 1.34}

 34%|███▎      | 1048/3120 [1:27:04<2:50:56,  4.95s/it]
 34%|███▎      | 1049/3120 [1:27:09<2:51:24,  4.97s/it]
 34%|███▎      | 1050/3120 [1:27:14<2:52:59,  5.01s/it]
                                                       
{'loss': 0.7737, 'learning_rate': 1.583661708638947e-05, 'epoch': 1.35}

 34%|███▎      | 1050/3120 [1:27:14<2:52:59,  5.01s/it]
 34%|███▎      | 1051/3120 [1:27:19<2:52:23,  5.00s/it]
 34%|███▎      | 1052/3120 [1:27:24<2:50:42,  4.95s/it]
                                                       
{'loss': 0.7909, 'learning_rate': 1.581939100567363e-05, 'epoch': 1.35}

 34%|███▎      | 1052/3120 [1:27:24<2:50:42,  4.95s/it]
 34%|███▍      | 1053/3120 [1:27:29<2:50:25,  4.95s/it]
 34%|███▍      | 1054/3120 [1:27:34<2:48:21,  4.89s/it]
                                                       
{'loss': 0.8154, 'learning_rate': 1.580213877441462e-05, 'epoch': 1.35}

 34%|███▍      | 1054/3120 [1:27:34<2:48:21,  4.89s/it]
 34%|███▍      | 1055/3120 [1:27:39<2:48:19,  4.89s/it]
 34%|███▍      | 1056/3120 [1:27:44<2:52:48,  5.02s/it]
                                                       
{'loss': 0.831, 'learning_rate': 1.5784860470138633e-05, 'epoch': 1.35}

 34%|███▍      | 1056/3120 [1:27:44<2:52:48,  5.02s/it]
 34%|███▍      | 1057/3120 [1:27:49<2:51:13,  4.98s/it]
 34%|███▍      | 1058/3120 [1:27:54<2:50:11,  4.95s/it]
                                                       
{'loss': 0.7842, 'learning_rate': 1.5767556170489027e-05, 'epoch': 1.36}

 34%|███▍      | 1058/3120 [1:27:54<2:50:11,  4.95s/it]
 34%|███▍      | 1059/3120 [1:27:59<2:53:32,  5.05s/it]
 34%|███▍      | 1060/3120 [1:28:04<2:53:52,  5.06s/it]
                                                       
{'loss': 0.7851, 'learning_rate': 1.5750225953225968e-05, 'epoch': 1.36}

 34%|███▍      | 1060/3120 [1:28:04<2:53:52,  5.06s/it]
 34%|███▍      | 1061/3120 [1:28:09<2:51:17,  4.99s/it]
 34%|███▍      | 1062/3120 [1:28:15<2:56:55,  5.16s/it]
                                                       
{'loss': 0.8102, 'learning_rate': 1.573286989622609e-05, 'epoch': 1.36}

 34%|███▍      | 1062/3120 [1:28:15<2:56:55,  5.16s/it]
 34%|███▍      | 1063/3120 [1:28:20<2:54:52,  5.10s/it]
 34%|███▍      | 1064/3120 [1:28:25<2:53:19,  5.06s/it]
                                                       
{'loss': 0.842, 'learning_rate': 1.5715488077482152e-05, 'epoch': 1.36}

 34%|███▍      | 1064/3120 [1:28:25<2:53:19,  5.06s/it]
 34%|███▍      | 1065/3120 [1:28:29<2:47:29,  4.89s/it]
 34%|███▍      | 1066/3120 [1:28:34<2:46:24,  4.86s/it]
                                                       
{'loss': 0.7943, 'learning_rate': 1.5698080575102662e-05, 'epoch': 1.37}

 34%|███▍      | 1066/3120 [1:28:34<2:46:24,  4.86s/it]
 34%|███▍      | 1067/3120 [1:28:39<2:45:38,  4.84s/it]
 34%|███▍      | 1068/3120 [1:28:44<2:49:01,  4.94s/it]
                                                       
{'loss': 0.7573, 'learning_rate': 1.568064746731156e-05, 'epoch': 1.37}

 34%|███▍      | 1068/3120 [1:28:44<2:49:01,  4.94s/it]
 34%|███▍      | 1069/3120 [1:28:49<2:52:00,  5.03s/it]
 34%|███▍      | 1070/3120 [1:28:54<2:49:56,  4.97s/it]
                                                       
{'loss': 0.837, 'learning_rate': 1.5663188832447833e-05, 'epoch': 1.37}

 34%|███▍      | 1070/3120 [1:28:54<2:49:56,  4.97s/it]
 34%|███▍      | 1071/3120 [1:28:59<2:49:05,  4.95s/it]
 34%|███▍      | 1072/3120 [1:29:04<2:49:37,  4.97s/it]
                                                       
{'loss': 0.8127, 'learning_rate': 1.5645704748965193e-05, 'epoch': 1.37}

 34%|███▍      | 1072/3120 [1:29:04<2:49:37,  4.97s/it]
 34%|███▍      | 1073/3120 [1:29:09<2:51:25,  5.02s/it]
 34%|███▍      | 1074/3120 [1:29:14<2:48:13,  4.93s/it]
                                                       
{'loss': 0.7727, 'learning_rate': 1.5628195295431696e-05, 'epoch': 1.38}

 34%|███▍      | 1074/3120 [1:29:14<2:48:13,  4.93s/it]
 34%|███▍      | 1075/3120 [1:29:19<2:50:02,  4.99s/it]
 34%|███▍      | 1076/3120 [1:29:24<2:49:35,  4.98s/it]
                                                       
{'loss': 0.7898, 'learning_rate': 1.5610660550529413e-05, 'epoch': 1.38}

 34%|███▍      | 1076/3120 [1:29:24<2:49:35,  4.98s/it]
 35%|███▍      | 1077/3120 [1:29:28<2:47:24,  4.92s/it]
 35%|███▍      | 1078/3120 [1:29:33<2:43:40,  4.81s/it]
                                                       
{'loss': 0.7841, 'learning_rate': 1.5593100593054063e-05, 'epoch': 1.38}

 35%|███▍      | 1078/3120 [1:29:33<2:43:40,  4.81s/it]
 35%|███▍      | 1079/3120 [1:29:38<2:43:45,  4.81s/it]
 35%|███▍      | 1080/3120 [1:29:43<2:43:54,  4.82s/it]
                                                       
{'loss': 0.8299, 'learning_rate': 1.557551550191467e-05, 'epoch': 1.38}

 35%|███▍      | 1080/3120 [1:29:43<2:43:54,  4.82s/it]
 35%|███▍      | 1081/3120 [1:29:48<2:46:22,  4.90s/it]
 35%|███▍      | 1082/3120 [1:29:53<2:51:08,  5.04s/it]
                                                       
{'loss': 0.8236, 'learning_rate': 1.5557905356133182e-05, 'epoch': 1.39}

 35%|███▍      | 1082/3120 [1:29:53<2:51:08,  5.04s/it]
 35%|███▍      | 1083/3120 [1:29:58<2:47:08,  4.92s/it]
 35%|███▍      | 1084/3120 [1:30:03<2:50:48,  5.03s/it]
                                                       
{'loss': 0.8314, 'learning_rate': 1.554027023484416e-05, 'epoch': 1.39}

 35%|███▍      | 1084/3120 [1:30:03<2:50:48,  5.03s/it]
 35%|███▍      | 1085/3120 [1:30:08<2:46:48,  4.92s/it]
 35%|███▍      | 1086/3120 [1:30:13<2:47:39,  4.95s/it]
                                                       
{'loss': 0.8049, 'learning_rate': 1.5522610217294377e-05, 'epoch': 1.39}

 35%|███▍      | 1086/3120 [1:30:13<2:47:39,  4.95s/it]
 35%|███▍      | 1087/3120 [1:30:18<2:48:39,  4.98s/it]
 35%|███▍      | 1088/3120 [1:30:23<2:46:39,  4.92s/it]
                                                       
{'loss': 0.7687, 'learning_rate': 1.550492538284249e-05, 'epoch': 1.39}

 35%|███▍      | 1088/3120 [1:30:23<2:46:39,  4.92s/it]
 35%|███▍      | 1089/3120 [1:30:28<2:50:15,  5.03s/it]
 35%|███▍      | 1090/3120 [1:30:33<2:49:07,  5.00s/it]
                                                       
{'loss': 0.8071, 'learning_rate': 1.5487215810958675e-05, 'epoch': 1.4}

 35%|███▍      | 1090/3120 [1:30:33<2:49:07,  5.00s/it]
 35%|███▍      | 1091/3120 [1:30:38<2:50:41,  5.05s/it]
 35%|███▌      | 1092/3120 [1:30:43<2:50:29,  5.04s/it]
                                                       
{'loss': 0.7917, 'learning_rate': 1.5469481581224274e-05, 'epoch': 1.4}

 35%|███▌      | 1092/3120 [1:30:43<2:50:29,  5.04s/it]
 35%|███▌      | 1093/3120 [1:30:48<2:46:25,  4.93s/it]
 35%|███▌      | 1094/3120 [1:30:53<2:49:04,  5.01s/it]
                                                       
{'loss': 0.7914, 'learning_rate': 1.545172277333142e-05, 'epoch': 1.4}

 35%|███▌      | 1094/3120 [1:30:53<2:49:04,  5.01s/it]
 35%|███▌      | 1095/3120 [1:30:58<2:52:42,  5.12s/it]
 35%|███▌      | 1096/3120 [1:31:03<2:52:42,  5.12s/it]
                                                       
{'loss': 0.8045, 'learning_rate': 1.5433939467082713e-05, 'epoch': 1.41}

 35%|███▌      | 1096/3120 [1:31:03<2:52:42,  5.12s/it]
 35%|███▌      | 1097/3120 [1:31:09<2:56:14,  5.23s/it]
 35%|███▌      | 1098/3120 [1:31:14<2:57:07,  5.26s/it]
                                                       
{'loss': 0.7788, 'learning_rate': 1.5416131742390827e-05, 'epoch': 1.41}

 35%|███▌      | 1098/3120 [1:31:14<2:57:07,  5.26s/it]
 35%|███▌      | 1099/3120 [1:31:19<2:54:39,  5.19s/it]
 35%|███▌      | 1100/3120 [1:31:24<2:53:25,  5.15s/it]
                                                       
{'loss': 0.851, 'learning_rate': 1.5398299679278172e-05, 'epoch': 1.41}

 35%|███▌      | 1100/3120 [1:31:24<2:53:25,  5.15s/it]
 35%|███▌      | 1101/3120 [1:31:29<2:48:43,  5.01s/it]
 35%|███▌      | 1102/3120 [1:31:34<2:52:14,  5.12s/it]
                                                       
{'loss': 0.7832, 'learning_rate': 1.5380443357876516e-05, 'epoch': 1.41}

 35%|███▌      | 1102/3120 [1:31:34<2:52:14,  5.12s/it]
 35%|███▌      | 1103/3120 [1:31:40<2:53:33,  5.16s/it]
 35%|███▌      | 1104/3120 [1:31:45<2:53:55,  5.18s/it]
                                                       
{'loss': 0.7591, 'learning_rate': 1.5362562858426655e-05, 'epoch': 1.42}

 35%|███▌      | 1104/3120 [1:31:45<2:53:55,  5.18s/it]
 35%|███▌      | 1105/3120 [1:31:50<2:50:52,  5.09s/it]
 35%|███▌      | 1106/3120 [1:31:55<2:49:19,  5.04s/it]
                                                       
{'loss': 0.767, 'learning_rate': 1.5344658261278013e-05, 'epoch': 1.42}

 35%|███▌      | 1106/3120 [1:31:55<2:49:19,  5.04s/it]
 35%|███▌      | 1107/3120 [1:32:00<2:49:31,  5.05s/it]
 36%|███▌      | 1108/3120 [1:32:05<2:51:20,  5.11s/it]
                                                       
{'loss': 0.8645, 'learning_rate': 1.5326729646888314e-05, 'epoch': 1.42}

 36%|███▌      | 1108/3120 [1:32:05<2:51:20,  5.11s/it]
 36%|███▌      | 1109/3120 [1:32:10<2:51:29,  5.12s/it]
 36%|███▌      | 1110/3120 [1:32:15<2:49:43,  5.07s/it]
                                                       
{'loss': 0.7996, 'learning_rate': 1.530877709582321e-05, 'epoch': 1.42}

 36%|███▌      | 1110/3120 [1:32:15<2:49:43,  5.07s/it]
 36%|███▌      | 1111/3120 [1:32:20<2:44:29,  4.91s/it]
 36%|███▌      | 1112/3120 [1:32:25<2:44:48,  4.92s/it]
                                                       
{'loss': 0.8467, 'learning_rate': 1.5290800688755906e-05, 'epoch': 1.43}

 36%|███▌      | 1112/3120 [1:32:25<2:44:48,  4.92s/it]
 36%|███▌      | 1113/3120 [1:32:29<2:44:25,  4.92s/it]
 36%|███▌      | 1114/3120 [1:32:35<2:48:41,  5.05s/it]
                                                       
{'loss': 0.8371, 'learning_rate': 1.527280050646682e-05, 'epoch': 1.43}

 36%|███▌      | 1114/3120 [1:32:35<2:48:41,  5.05s/it]
 36%|███▌      | 1115/3120 [1:32:40<2:49:12,  5.06s/it]
 36%|███▌      | 1116/3120 [1:32:45<2:52:57,  5.18s/it]
                                                       
{'loss': 0.8251, 'learning_rate': 1.5254776629843204e-05, 'epoch': 1.43}

 36%|███▌      | 1116/3120 [1:32:45<2:52:57,  5.18s/it]
 36%|███▌      | 1117/3120 [1:32:50<2:50:20,  5.10s/it]
 36%|███▌      | 1118/3120 [1:32:55<2:47:30,  5.02s/it]
                                                       
{'loss': 0.8115, 'learning_rate': 1.523672913987878e-05, 'epoch': 1.43}

 36%|███▌      | 1118/3120 [1:32:55<2:47:30,  5.02s/it]
 36%|███▌      | 1119/3120 [1:33:00<2:47:02,  5.01s/it]
 36%|███▌      | 1120/3120 [1:33:05<2:47:37,  5.03s/it]
                                                       
{'loss': 0.8108, 'learning_rate': 1.5218658117673389e-05, 'epoch': 1.44}

 36%|███▌      | 1120/3120 [1:33:05<2:47:37,  5.03s/it]
 36%|███▌      | 1121/3120 [1:33:10<2:49:21,  5.08s/it]
 36%|███▌      | 1122/3120 [1:33:15<2:49:21,  5.09s/it]
                                                       
{'loss': 0.7774, 'learning_rate': 1.5200563644432614e-05, 'epoch': 1.44}

 36%|███▌      | 1122/3120 [1:33:15<2:49:21,  5.09s/it]
 36%|███▌      | 1123/3120 [1:33:21<2:50:20,  5.12s/it]
 36%|███▌      | 1124/3120 [1:33:26<2:50:41,  5.13s/it]
                                                       
{'loss': 0.7728, 'learning_rate': 1.518244580146742e-05, 'epoch': 1.44}

 36%|███▌      | 1124/3120 [1:33:26<2:50:41,  5.13s/it]
 36%|███▌      | 1125/3120 [1:33:31<2:49:59,  5.11s/it]
 36%|███▌      | 1126/3120 [1:33:36<2:49:12,  5.09s/it]
                                                       
{'loss': 0.7847, 'learning_rate': 1.5164304670193792e-05, 'epoch': 1.44}

 36%|███▌      | 1126/3120 [1:33:36<2:49:12,  5.09s/it]
 36%|███▌      | 1127/3120 [1:33:41<2:46:59,  5.03s/it]
 36%|███▌      | 1128/3120 [1:33:46<2:48:14,  5.07s/it]
                                                       
{'loss': 0.8291, 'learning_rate': 1.5146140332132359e-05, 'epoch': 1.45}

 36%|███▌      | 1128/3120 [1:33:46<2:48:14,  5.07s/it]
 36%|███▌      | 1129/3120 [1:33:51<2:47:16,  5.04s/it]
 36%|███▌      | 1130/3120 [1:33:56<2:47:35,  5.05s/it]
                                                       
{'loss': 0.7675, 'learning_rate': 1.5127952868908043e-05, 'epoch': 1.45}

 36%|███▌      | 1130/3120 [1:33:56<2:47:35,  5.05s/it]
 36%|███▋      | 1131/3120 [1:34:01<2:45:55,  5.01s/it]
 36%|███▋      | 1132/3120 [1:34:06<2:47:59,  5.07s/it]
                                                       
{'loss': 0.8281, 'learning_rate': 1.5109742362249673e-05, 'epoch': 1.45}

 36%|███▋      | 1132/3120 [1:34:06<2:47:59,  5.07s/it]
 36%|███▋      | 1133/3120 [1:34:11<2:50:51,  5.16s/it]
 36%|███▋      | 1134/3120 [1:34:17<2:51:27,  5.18s/it]
                                                       
{'loss': 0.8432, 'learning_rate': 1.5091508893989633e-05, 'epoch': 1.45}

 36%|███▋      | 1134/3120 [1:34:17<2:51:27,  5.18s/it]
 36%|███▋      | 1135/3120 [1:34:22<2:48:14,  5.09s/it]
 36%|███▋      | 1136/3120 [1:34:27<2:49:49,  5.14s/it]
                                                       
{'loss': 0.7984, 'learning_rate': 1.5073252546063493e-05, 'epoch': 1.46}

 36%|███▋      | 1136/3120 [1:34:27<2:49:49,  5.14s/it]
 36%|███▋      | 1137/3120 [1:34:32<2:48:05,  5.09s/it]
 36%|███▋      | 1138/3120 [1:34:37<2:47:20,  5.07s/it]
                                                       
{'loss': 0.8033, 'learning_rate': 1.5054973400509631e-05, 'epoch': 1.46}

 36%|███▋      | 1138/3120 [1:34:37<2:47:20,  5.07s/it]
 37%|███▋      | 1139/3120 [1:34:41<2:41:07,  4.88s/it]
 37%|███▋      | 1140/3120 [1:34:46<2:40:44,  4.87s/it]
                                                       
{'loss': 0.8412, 'learning_rate': 1.5036671539468879e-05, 'epoch': 1.46}

 37%|███▋      | 1140/3120 [1:34:46<2:40:44,  4.87s/it]
 37%|███▋      | 1141/3120 [1:34:51<2:41:48,  4.91s/it]
 37%|███▋      | 1142/3120 [1:34:56<2:46:39,  5.06s/it]
                                                       
{'loss': 0.7544, 'learning_rate': 1.5018347045184132e-05, 'epoch': 1.46}

 37%|███▋      | 1142/3120 [1:34:56<2:46:39,  5.06s/it]
 37%|███▋      | 1143/3120 [1:35:01<2:42:45,  4.94s/it]
 37%|███▋      | 1144/3120 [1:35:06<2:43:51,  4.98s/it]
                                                       
{'loss': 0.7962, 'learning_rate': 1.5000000000000002e-05, 'epoch': 1.47}

 37%|███▋      | 1144/3120 [1:35:06<2:43:51,  4.98s/it]
 37%|███▋      | 1145/3120 [1:35:11<2:40:33,  4.88s/it]
 37%|███▋      | 1146/3120 [1:35:16<2:43:25,  4.97s/it]
                                                       
{'loss': 0.8351, 'learning_rate': 1.4981630486362435e-05, 'epoch': 1.47}

 37%|███▋      | 1146/3120 [1:35:16<2:43:25,  4.97s/it]
 37%|███▋      | 1147/3120 [1:35:21<2:41:56,  4.92s/it]
 37%|███▋      | 1148/3120 [1:35:26<2:41:40,  4.92s/it]
                                                       
{'loss': 0.7913, 'learning_rate': 1.4963238586818346e-05, 'epoch': 1.47}

 37%|███▋      | 1148/3120 [1:35:26<2:41:40,  4.92s/it]
 37%|███▋      | 1149/3120 [1:35:31<2:39:58,  4.87s/it]
 37%|███▋      | 1150/3120 [1:35:35<2:39:58,  4.87s/it]
                                                       
{'loss': 0.8417, 'learning_rate': 1.4944824384015238e-05, 'epoch': 1.47}

 37%|███▋      | 1150/3120 [1:35:35<2:39:58,  4.87s/it]
 37%|███▋      | 1151/3120 [1:35:40<2:37:45,  4.81s/it]
 37%|███▋      | 1152/3120 [1:35:46<2:44:29,  5.01s/it]
                                                       
{'loss': 0.766, 'learning_rate': 1.4926387960700843e-05, 'epoch': 1.48}

 37%|███▋      | 1152/3120 [1:35:46<2:44:29,  5.01s/it]
 37%|███▋      | 1153/3120 [1:35:50<2:39:29,  4.87s/it]
 37%|███▋      | 1154/3120 [1:35:55<2:43:04,  4.98s/it]
                                                       
{'loss': 0.7899, 'learning_rate': 1.4907929399722752e-05, 'epoch': 1.48}

 37%|███▋      | 1154/3120 [1:35:55<2:43:04,  4.98s/it]
 37%|███▋      | 1155/3120 [1:36:00<2:40:41,  4.91s/it]
 37%|███▋      | 1156/3120 [1:36:05<2:40:59,  4.92s/it]
                                                       
{'loss': 0.7746, 'learning_rate': 1.488944878402802e-05, 'epoch': 1.48}

 37%|███▋      | 1156/3120 [1:36:05<2:40:59,  4.92s/it]
 37%|███▋      | 1157/3120 [1:36:10<2:40:10,  4.90s/it]
 37%|███▋      | 1158/3120 [1:36:15<2:38:41,  4.85s/it]
                                                       
{'loss': 0.8103, 'learning_rate': 1.4870946196662822e-05, 'epoch': 1.48}

 37%|███▋      | 1158/3120 [1:36:15<2:38:41,  4.85s/it]
 37%|███▋      | 1159/3120 [1:36:20<2:42:01,  4.96s/it]
 37%|███▋      | 1160/3120 [1:36:25<2:44:55,  5.05s/it]
                                                       
{'loss': 0.8422, 'learning_rate': 1.4852421720772064e-05, 'epoch': 1.49}

 37%|███▋      | 1160/3120 [1:36:25<2:44:55,  5.05s/it]
 37%|███▋      | 1161/3120 [1:36:30<2:41:12,  4.94s/it]
 37%|███▋      | 1162/3120 [1:36:35<2:48:01,  5.15s/it]
                                                       
{'loss': 0.8136, 'learning_rate': 1.4833875439599004e-05, 'epoch': 1.49}

 37%|███▋      | 1162/3120 [1:36:35<2:48:01,  5.15s/it]
 37%|███▋      | 1163/3120 [1:36:41<2:49:04,  5.18s/it]
 37%|███▋      | 1164/3120 [1:36:46<2:46:58,  5.12s/it]
                                                       
{'loss': 0.7957, 'learning_rate': 1.4815307436484898e-05, 'epoch': 1.49}

 37%|███▋      | 1164/3120 [1:36:46<2:46:58,  5.12s/it]
 37%|███▋      | 1165/3120 [1:36:50<2:44:24,  5.05s/it]
 37%|███▋      | 1166/3120 [1:36:55<2:41:29,  4.96s/it]
                                                       
{'loss': 0.8081, 'learning_rate': 1.4796717794868606e-05, 'epoch': 1.49}

 37%|███▋      | 1166/3120 [1:36:55<2:41:29,  4.96s/it]
 37%|███▋      | 1167/3120 [1:37:00<2:42:02,  4.98s/it]
 37%|███▋      | 1168/3120 [1:37:05<2:39:07,  4.89s/it]
                                                       
{'loss': 0.8154, 'learning_rate': 1.4778106598286235e-05, 'epoch': 1.5}

 37%|███▋      | 1168/3120 [1:37:05<2:39:07,  4.89s/it]
 37%|███▋      | 1169/3120 [1:37:10<2:41:22,  4.96s/it]
 38%|███▊      | 1170/3120 [1:37:16<2:46:02,  5.11s/it]
                                                       
{'loss': 0.8193, 'learning_rate': 1.4759473930370738e-05, 'epoch': 1.5}

 38%|███▊      | 1170/3120 [1:37:16<2:46:02,  5.11s/it]
 38%|███▊      | 1171/3120 [1:37:21<2:44:59,  5.08s/it]
 38%|███▊      | 1172/3120 [1:37:25<2:42:37,  5.01s/it]
                                                       
{'loss': 0.7687, 'learning_rate': 1.4740819874851562e-05, 'epoch': 1.5}

 38%|███▊      | 1172/3120 [1:37:25<2:42:37,  5.01s/it]
 38%|███▊      | 1173/3120 [1:37:31<2:44:45,  5.08s/it]
 38%|███▊      | 1174/3120 [1:37:36<2:48:43,  5.20s/it]
                                                       
{'loss': 0.7366, 'learning_rate': 1.4722144515554263e-05, 'epoch': 1.51}

 38%|███▊      | 1174/3120 [1:37:36<2:48:43,  5.20s/it]
 38%|███▊      | 1175/3120 [1:37:41<2:45:35,  5.11s/it]
 38%|███▊      | 1176/3120 [1:37:46<2:44:00,  5.06s/it]
                                                       
{'loss': 0.7733, 'learning_rate': 1.4703447936400135e-05, 'epoch': 1.51}

 38%|███▊      | 1176/3120 [1:37:46<2:44:00,  5.06s/it]
 38%|███▊      | 1177/3120 [1:37:51<2:43:55,  5.06s/it]
 38%|███▊      | 1178/3120 [1:37:56<2:44:58,  5.10s/it]
                                                       
{'loss': 0.8318, 'learning_rate': 1.4684730221405814e-05, 'epoch': 1.51}

 38%|███▊      | 1178/3120 [1:37:56<2:44:58,  5.10s/it]
 38%|███▊      | 1179/3120 [1:38:01<2:44:23,  5.08s/it]
 38%|███▊      | 1180/3120 [1:38:06<2:40:36,  4.97s/it]
                                                       
{'loss': 0.8014, 'learning_rate': 1.4665991454682924e-05, 'epoch': 1.51}

 38%|███▊      | 1180/3120 [1:38:06<2:40:36,  4.97s/it]
 38%|███▊      | 1181/3120 [1:38:11<2:42:35,  5.03s/it]
 38%|███▊      | 1182/3120 [1:38:16<2:39:08,  4.93s/it]
                                                       
{'loss': 0.7706, 'learning_rate': 1.4647231720437687e-05, 'epoch': 1.52}

 38%|███▊      | 1182/3120 [1:38:16<2:39:08,  4.93s/it]
 38%|███▊      | 1183/3120 [1:38:21<2:39:01,  4.93s/it]
 38%|███▊      | 1184/3120 [1:38:25<2:37:19,  4.88s/it]
                                                       
{'loss': 0.8441, 'learning_rate': 1.4628451102970546e-05, 'epoch': 1.52}

 38%|███▊      | 1184/3120 [1:38:25<2:37:19,  4.88s/it]
 38%|███▊      | 1185/3120 [1:38:30<2:38:16,  4.91s/it]
 38%|███▊      | 1186/3120 [1:38:35<2:34:52,  4.80s/it]
                                                       
{'loss': 0.7787, 'learning_rate': 1.4609649686675786e-05, 'epoch': 1.52}

 38%|███▊      | 1186/3120 [1:38:35<2:34:52,  4.80s/it]
 38%|███▊      | 1187/3120 [1:38:40<2:35:08,  4.82s/it]
 38%|███▊      | 1188/3120 [1:38:45<2:34:13,  4.79s/it]
                                                       
{'loss': 0.8178, 'learning_rate': 1.4590827556041158e-05, 'epoch': 1.52}

 38%|███▊      | 1188/3120 [1:38:45<2:34:13,  4.79s/it]
 38%|███▊      | 1189/3120 [1:38:49<2:34:19,  4.80s/it]
 38%|███▊      | 1190/3120 [1:38:54<2:33:31,  4.77s/it]
                                                       
{'loss': 0.823, 'learning_rate': 1.4571984795647494e-05, 'epoch': 1.53}

 38%|███▊      | 1190/3120 [1:38:54<2:33:31,  4.77s/it]
 38%|███▊      | 1191/3120 [1:38:59<2:35:54,  4.85s/it]
 38%|███▊      | 1192/3120 [1:39:04<2:37:15,  4.89s/it]
                                                       
{'loss': 0.8072, 'learning_rate': 1.4553121490168335e-05, 'epoch': 1.53}

 38%|███▊      | 1192/3120 [1:39:04<2:37:15,  4.89s/it]
 38%|███▊      | 1193/3120 [1:39:09<2:35:23,  4.84s/it]
 38%|███▊      | 1194/3120 [1:39:14<2:39:15,  4.96s/it]
                                                       
{'loss': 0.8222, 'learning_rate': 1.4534237724369534e-05, 'epoch': 1.53}

 38%|███▊      | 1194/3120 [1:39:14<2:39:15,  4.96s/it]
 38%|███▊      | 1195/3120 [1:39:19<2:41:08,  5.02s/it]
 38%|███▊      | 1196/3120 [1:39:25<2:43:29,  5.10s/it]
                                                       
{'loss': 0.776, 'learning_rate': 1.4515333583108896e-05, 'epoch': 1.53}

 38%|███▊      | 1196/3120 [1:39:25<2:43:29,  5.10s/it]
 38%|███▊      | 1197/3120 [1:39:29<2:40:25,  5.01s/it]
 38%|███▊      | 1198/3120 [1:39:34<2:40:58,  5.03s/it]
                                                       
{'loss': 0.7757, 'learning_rate': 1.4496409151335786e-05, 'epoch': 1.54}

 38%|███▊      | 1198/3120 [1:39:34<2:40:58,  5.03s/it]
 38%|███▊      | 1199/3120 [1:39:39<2:39:17,  4.98s/it]
 38%|███▊      | 1200/3120 [1:39:44<2:38:22,  4.95s/it]
                                                       
{'loss': 0.8204, 'learning_rate': 1.4477464514090745e-05, 'epoch': 1.54}

 38%|███▊      | 1200/3120 [1:39:44<2:38:22,  4.95s/it]
 38%|███▊      | 1201/3120 [1:39:49<2:38:34,  4.96s/it]
 39%|███▊      | 1202/3120 [1:39:54<2:38:13,  4.95s/it]
                                                       
{'loss': 0.7843, 'learning_rate': 1.4458499756505116e-05, 'epoch': 1.54}

 39%|███▊      | 1202/3120 [1:39:54<2:38:13,  4.95s/it]
 39%|███▊      | 1203/3120 [1:39:59<2:38:40,  4.97s/it]
 39%|███▊      | 1204/3120 [1:40:04<2:41:27,  5.06s/it]
                                                       
{'loss': 0.8112, 'learning_rate': 1.443951496380065e-05, 'epoch': 1.54}

 39%|███▊      | 1204/3120 [1:40:04<2:41:27,  5.06s/it]
 39%|███▊      | 1205/3120 [1:40:09<2:39:08,  4.99s/it]
 39%|███▊      | 1206/3120 [1:40:14<2:40:48,  5.04s/it]
                                                       
{'loss': 0.8246, 'learning_rate': 1.4420510221289137e-05, 'epoch': 1.55}

 39%|███▊      | 1206/3120 [1:40:14<2:40:48,  5.04s/it]
 39%|███▊      | 1207/3120 [1:40:19<2:40:42,  5.04s/it]
 39%|███▊      | 1208/3120 [1:40:24<2:35:20,  4.87s/it]
                                                       
{'loss': 0.7993, 'learning_rate': 1.4401485614372009e-05, 'epoch': 1.55}

 39%|███▊      | 1208/3120 [1:40:24<2:35:20,  4.87s/it]
 39%|███▉      | 1209/3120 [1:40:29<2:37:15,  4.94s/it]
 39%|███▉      | 1210/3120 [1:40:34<2:41:34,  5.08s/it]
                                                       
{'loss': 0.8087, 'learning_rate': 1.4382441228539962e-05, 'epoch': 1.55}

 39%|███▉      | 1210/3120 [1:40:34<2:41:34,  5.08s/it]
 39%|███▉      | 1211/3120 [1:40:40<2:44:20,  5.17s/it]
 39%|███▉      | 1212/3120 [1:40:45<2:42:53,  5.12s/it]
                                                       
{'loss': 0.7602, 'learning_rate': 1.4363377149372584e-05, 'epoch': 1.55}

 39%|███▉      | 1212/3120 [1:40:45<2:42:53,  5.12s/it]
 39%|███▉      | 1213/3120 [1:40:49<2:38:20,  4.98s/it]
 39%|███▉      | 1214/3120 [1:40:54<2:37:31,  4.96s/it]
                                                       
{'loss': 0.7935, 'learning_rate': 1.434429346253794e-05, 'epoch': 1.56}

 39%|███▉      | 1214/3120 [1:40:54<2:37:31,  4.96s/it]
 39%|███▉      | 1215/3120 [1:40:59<2:37:49,  4.97s/it]
 39%|███▉      | 1216/3120 [1:41:04<2:37:26,  4.96s/it]
                                                       
{'loss': 0.8377, 'learning_rate': 1.4325190253792222e-05, 'epoch': 1.56}

 39%|███▉      | 1216/3120 [1:41:04<2:37:26,  4.96s/it]
 39%|███▉      | 1217/3120 [1:41:09<2:37:36,  4.97s/it]
 39%|███▉      | 1218/3120 [1:41:14<2:37:38,  4.97s/it]
                                                       
{'loss': 0.8346, 'learning_rate': 1.430606760897934e-05, 'epoch': 1.56}

 39%|███▉      | 1218/3120 [1:41:14<2:37:38,  4.97s/it]
 39%|███▉      | 1219/3120 [1:41:20<2:41:52,  5.11s/it]
 39%|███▉      | 1220/3120 [1:41:25<2:41:11,  5.09s/it]
                                                       
{'loss': 0.7851, 'learning_rate': 1.4286925614030542e-05, 'epoch': 1.56}

 39%|███▉      | 1220/3120 [1:41:25<2:41:11,  5.09s/it]
 39%|███▉      | 1221/3120 [1:41:30<2:41:11,  5.09s/it]
 39%|███▉      | 1222/3120 [1:41:35<2:38:56,  5.02s/it]
                                                       
{'loss': 0.787, 'learning_rate': 1.4267764354964038e-05, 'epoch': 1.57}

 39%|███▉      | 1222/3120 [1:41:35<2:38:56,  5.02s/it]
 39%|███▉      | 1223/3120 [1:41:40<2:39:42,  5.05s/it]
 39%|███▉      | 1224/3120 [1:41:44<2:35:35,  4.92s/it]
                                                       
{'loss': 0.8389, 'learning_rate': 1.4248583917884595e-05, 'epoch': 1.57}

 39%|███▉      | 1224/3120 [1:41:44<2:35:35,  4.92s/it]
 39%|███▉      | 1225/3120 [1:41:49<2:37:02,  4.97s/it]
 39%|███▉      | 1226/3120 [1:41:54<2:37:05,  4.98s/it]
                                                       
{'loss': 0.7794, 'learning_rate': 1.4229384388983167e-05, 'epoch': 1.57}

 39%|███▉      | 1226/3120 [1:41:54<2:37:05,  4.98s/it]
 39%|███▉      | 1227/3120 [1:42:00<2:39:07,  5.04s/it]
 39%|███▉      | 1228/3120 [1:42:05<2:40:50,  5.10s/it]
                                                       
{'loss': 0.8116, 'learning_rate': 1.4210165854536495e-05, 'epoch': 1.57}

 39%|███▉      | 1228/3120 [1:42:05<2:40:50,  5.10s/it]
 39%|███▉      | 1229/3120 [1:42:10<2:36:48,  4.98s/it]
 39%|███▉      | 1230/3120 [1:42:14<2:31:55,  4.82s/it]
                                                       
{'loss': 0.8146, 'learning_rate': 1.4190928400906731e-05, 'epoch': 1.58}

 39%|███▉      | 1230/3120 [1:42:14<2:31:55,  4.82s/it]
 39%|███▉      | 1231/3120 [1:42:19<2:28:34,  4.72s/it]
 39%|███▉      | 1232/3120 [1:42:24<2:36:26,  4.97s/it]
                                                       
{'loss': 0.8344, 'learning_rate': 1.4171672114541042e-05, 'epoch': 1.58}

 39%|███▉      | 1232/3120 [1:42:24<2:36:26,  4.97s/it]
 40%|███▉      | 1233/3120 [1:42:29<2:33:12,  4.87s/it]
 40%|███▉      | 1234/3120 [1:42:34<2:32:23,  4.85s/it]
                                                       
{'loss': 0.8259, 'learning_rate': 1.415239708197122e-05, 'epoch': 1.58}

 40%|███▉      | 1234/3120 [1:42:34<2:32:23,  4.85s/it]
 40%|███▉      | 1235/3120 [1:42:39<2:34:20,  4.91s/it]
 40%|███▉      | 1236/3120 [1:42:44<2:38:05,  5.03s/it]
                                                       
{'loss': 0.7878, 'learning_rate': 1.4133103389813302e-05, 'epoch': 1.58}

 40%|███▉      | 1236/3120 [1:42:44<2:38:05,  5.03s/it]
 40%|███▉      | 1237/3120 [1:42:49<2:35:30,  4.95s/it]
 40%|███▉      | 1238/3120 [1:42:54<2:36:00,  4.97s/it]
                                                       
{'loss': 0.7825, 'learning_rate': 1.411379112476717e-05, 'epoch': 1.59}

 40%|███▉      | 1238/3120 [1:42:54<2:36:00,  4.97s/it]
 40%|███▉      | 1239/3120 [1:42:59<2:35:39,  4.96s/it]
 40%|███▉      | 1240/3120 [1:43:04<2:36:07,  4.98s/it]
                                                       
{'loss': 0.7542, 'learning_rate': 1.409446037361617e-05, 'epoch': 1.59}

 40%|███▉      | 1240/3120 [1:43:04<2:36:07,  4.98s/it]
 40%|███▉      | 1241/3120 [1:43:08<2:33:36,  4.90s/it]
 40%|███▉      | 1242/3120 [1:43:13<2:32:11,  4.86s/it]
                                                       
{'loss': 0.8001, 'learning_rate': 1.4075111223226721e-05, 'epoch': 1.59}

 40%|███▉      | 1242/3120 [1:43:13<2:32:11,  4.86s/it]
 40%|███▉      | 1243/3120 [1:43:18<2:32:04,  4.86s/it]
 40%|███▉      | 1244/3120 [1:43:23<2:30:25,  4.81s/it]
                                                       
{'loss': 0.8038, 'learning_rate': 1.4055743760547918e-05, 'epoch': 1.59}

 40%|███▉      | 1244/3120 [1:43:23<2:30:25,  4.81s/it]
 40%|███▉      | 1245/3120 [1:43:28<2:32:29,  4.88s/it]
 40%|███▉      | 1246/3120 [1:43:32<2:31:16,  4.84s/it]
                                                       
{'loss': 0.7916, 'learning_rate': 1.4036358072611147e-05, 'epoch': 1.6}

 40%|███▉      | 1246/3120 [1:43:32<2:31:16,  4.84s/it]
 40%|███▉      | 1247/3120 [1:43:38<2:33:58,  4.93s/it]
 40%|████      | 1248/3120 [1:43:43<2:34:58,  4.97s/it]
                                                       
{'loss': 0.8181, 'learning_rate': 1.4016954246529697e-05, 'epoch': 1.6}

 40%|████      | 1248/3120 [1:43:43<2:34:58,  4.97s/it]
 40%|████      | 1249/3120 [1:43:48<2:35:12,  4.98s/it]
 40%|████      | 1250/3120 [1:43:52<2:30:41,  4.84s/it]
                                                       
{'loss': 0.8473, 'learning_rate': 1.3997532369498357e-05, 'epoch': 1.6}

 40%|████      | 1250/3120 [1:43:52<2:30:41,  4.84s/it]
 40%|████      | 1251/3120 [1:43:57<2:30:34,  4.83s/it]
 40%|████      | 1252/3120 [1:44:02<2:33:39,  4.94s/it]
                                                       
{'loss': 0.7634, 'learning_rate': 1.3978092528793032e-05, 'epoch': 1.61}

 40%|████      | 1252/3120 [1:44:02<2:33:39,  4.94s/it]
 40%|████      | 1253/3120 [1:44:07<2:35:53,  5.01s/it]
 40%|████      | 1254/3120 [1:44:12<2:34:30,  4.97s/it]
                                                       
{'loss': 0.8079, 'learning_rate': 1.3958634811770361e-05, 'epoch': 1.61}

 40%|████      | 1254/3120 [1:44:12<2:34:30,  4.97s/it]
 40%|████      | 1255/3120 [1:44:17<2:33:56,  4.95s/it]
 40%|████      | 1256/3120 [1:44:22<2:33:28,  4.94s/it]
                                                       
{'loss': 0.8156, 'learning_rate': 1.39391593058673e-05, 'epoch': 1.61}

 40%|████      | 1256/3120 [1:44:22<2:33:28,  4.94s/it]
 40%|████      | 1257/3120 [1:44:27<2:35:03,  4.99s/it]
 40%|████      | 1258/3120 [1:44:32<2:37:53,  5.09s/it]
                                                       
{'loss': 0.7625, 'learning_rate': 1.3919666098600753e-05, 'epoch': 1.61}

 40%|████      | 1258/3120 [1:44:33<2:37:53,  5.09s/it]
 40%|████      | 1259/3120 [1:44:37<2:35:57,  5.03s/it]
 40%|████      | 1260/3120 [1:44:42<2:32:00,  4.90s/it]
                                                       
{'loss': 0.803, 'learning_rate': 1.3900155277567157e-05, 'epoch': 1.62}

 40%|████      | 1260/3120 [1:44:42<2:32:00,  4.90s/it]
 40%|████      | 1261/3120 [1:44:47<2:35:54,  5.03s/it]
 40%|████      | 1262/3120 [1:44:52<2:35:48,  5.03s/it]
                                                       
{'loss': 0.8364, 'learning_rate': 1.3880626930442115e-05, 'epoch': 1.62}

 40%|████      | 1262/3120 [1:44:52<2:35:48,  5.03s/it]
 40%|████      | 1263/3120 [1:44:57<2:32:48,  4.94s/it]
 41%|████      | 1264/3120 [1:45:02<2:34:52,  5.01s/it]
                                                       
{'loss': 0.7946, 'learning_rate': 1.3861081144979975e-05, 'epoch': 1.62}

 41%|████      | 1264/3120 [1:45:02<2:34:52,  5.01s/it]
 41%|████      | 1265/3120 [1:45:07<2:34:16,  4.99s/it]
 41%|████      | 1266/3120 [1:45:13<2:38:44,  5.14s/it]
                                                       
{'loss': 0.8737, 'learning_rate': 1.3841518009013446e-05, 'epoch': 1.62}

 41%|████      | 1266/3120 [1:45:13<2:38:44,  5.14s/it]
 41%|████      | 1267/3120 [1:45:17<2:32:51,  4.95s/it]
 41%|████      | 1268/3120 [1:45:22<2:32:40,  4.95s/it]
                                                       
{'loss': 0.8138, 'learning_rate': 1.382193761045322e-05, 'epoch': 1.63}

 41%|████      | 1268/3120 [1:45:22<2:32:40,  4.95s/it]
 41%|████      | 1269/3120 [1:45:27<2:32:11,  4.93s/it]
 41%|████      | 1270/3120 [1:45:32<2:30:52,  4.89s/it]
                                                       
{'loss': 0.7597, 'learning_rate': 1.380234003728754e-05, 'epoch': 1.63}

 41%|████      | 1270/3120 [1:45:32<2:30:52,  4.89s/it]
 41%|████      | 1271/3120 [1:45:37<2:33:18,  4.97s/it]
 41%|████      | 1272/3120 [1:45:42<2:33:47,  4.99s/it]
                                                       
{'loss': 0.8276, 'learning_rate': 1.378272537758185e-05, 'epoch': 1.63}

 41%|████      | 1272/3120 [1:45:42<2:33:47,  4.99s/it]
 41%|████      | 1273/3120 [1:45:47<2:30:14,  4.88s/it]
 41%|████      | 1274/3120 [1:45:51<2:29:12,  4.85s/it]
                                                       
{'loss': 0.8108, 'learning_rate': 1.3763093719478357e-05, 'epoch': 1.63}

 41%|████      | 1274/3120 [1:45:51<2:29:12,  4.85s/it]
 41%|████      | 1275/3120 [1:45:56<2:29:50,  4.87s/it]
 41%|████      | 1276/3120 [1:46:01<2:30:21,  4.89s/it]
                                                       
{'loss': 0.8137, 'learning_rate': 1.3743445151195658e-05, 'epoch': 1.64}

 41%|████      | 1276/3120 [1:46:01<2:30:21,  4.89s/it]
 41%|████      | 1277/3120 [1:46:06<2:29:51,  4.88s/it]
 41%|████      | 1278/3120 [1:46:11<2:29:47,  4.88s/it]
                                                       
{'loss': 0.8055, 'learning_rate': 1.3723779761028349e-05, 'epoch': 1.64}

 41%|████      | 1278/3120 [1:46:11<2:29:47,  4.88s/it]
 41%|████      | 1279/3120 [1:46:16<2:29:57,  4.89s/it]
 41%|████      | 1280/3120 [1:46:21<2:30:34,  4.91s/it]
                                                       
{'loss': 0.7624, 'learning_rate': 1.37040976373466e-05, 'epoch': 1.64}

 41%|████      | 1280/3120 [1:46:21<2:30:34,  4.91s/it]
 41%|████      | 1281/3120 [1:46:26<2:31:27,  4.94s/it]
 41%|████      | 1282/3120 [1:46:31<2:31:46,  4.95s/it]
                                                       
{'loss': 0.862, 'learning_rate': 1.36843988685958e-05, 'epoch': 1.64}

 41%|████      | 1282/3120 [1:46:31<2:31:46,  4.95s/it]
 41%|████      | 1283/3120 [1:46:36<2:30:12,  4.91s/it]
 41%|████      | 1284/3120 [1:46:41<2:30:37,  4.92s/it]
                                                       
{'loss': 0.7821, 'learning_rate': 1.3664683543296114e-05, 'epoch': 1.65}

 41%|████      | 1284/3120 [1:46:41<2:30:37,  4.92s/it]
 41%|████      | 1285/3120 [1:46:45<2:27:03,  4.81s/it]
 41%|████      | 1286/3120 [1:46:50<2:30:09,  4.91s/it]
                                                       
{'loss': 0.758, 'learning_rate': 1.3644951750042115e-05, 'epoch': 1.65}

 41%|████      | 1286/3120 [1:46:50<2:30:09,  4.91s/it]
 41%|████▏     | 1287/3120 [1:46:55<2:27:58,  4.84s/it]
 41%|████▏     | 1288/3120 [1:47:00<2:28:29,  4.86s/it]
                                                       
{'loss': 0.8178, 'learning_rate': 1.3625203577502384e-05, 'epoch': 1.65}

 41%|████▏     | 1288/3120 [1:47:00<2:28:29,  4.86s/it]
 41%|████▏     | 1289/3120 [1:47:04<2:24:28,  4.73s/it]
 41%|████▏     | 1290/3120 [1:47:10<2:28:12,  4.86s/it]
                                                       
{'loss': 0.8348, 'learning_rate': 1.3605439114419095e-05, 'epoch': 1.65}

 41%|████▏     | 1290/3120 [1:47:10<2:28:12,  4.86s/it]
 41%|████▏     | 1291/3120 [1:47:14<2:28:04,  4.86s/it]
 41%|████▏     | 1292/3120 [1:47:20<2:33:47,  5.05s/it]
                                                       
{'loss': 0.8298, 'learning_rate': 1.3585658449607632e-05, 'epoch': 1.66}

 41%|████▏     | 1292/3120 [1:47:20<2:33:47,  5.05s/it]
 41%|████▏     | 1293/3120 [1:47:25<2:36:57,  5.15s/it]
 41%|████▏     | 1294/3120 [1:47:31<2:38:32,  5.21s/it]
                                                       
{'loss': 0.8194, 'learning_rate': 1.3565861671956189e-05, 'epoch': 1.66}

 41%|████▏     | 1294/3120 [1:47:31<2:38:32,  5.21s/it]
 42%|████▏     | 1295/3120 [1:47:36<2:38:27,  5.21s/it]
 42%|████▏     | 1296/3120 [1:47:41<2:36:38,  5.15s/it]
                                                       
{'loss': 0.8231, 'learning_rate': 1.3546048870425356e-05, 'epoch': 1.66}

 42%|████▏     | 1296/3120 [1:47:41<2:36:38,  5.15s/it]
 42%|████▏     | 1297/3120 [1:47:46<2:37:37,  5.19s/it]
 42%|████▏     | 1298/3120 [1:47:51<2:33:17,  5.05s/it]
                                                       
{'loss': 0.8115, 'learning_rate': 1.352622013404774e-05, 'epoch': 1.66}

 42%|████▏     | 1298/3120 [1:47:51<2:33:17,  5.05s/it]
 42%|████▏     | 1299/3120 [1:47:56<2:31:22,  4.99s/it]
 42%|████▏     | 1300/3120 [1:48:01<2:32:34,  5.03s/it]
                                                       
{'loss': 0.7935, 'learning_rate': 1.3506375551927546e-05, 'epoch': 1.67}

 42%|████▏     | 1300/3120 [1:48:01<2:32:34,  5.03s/it]
 42%|████▏     | 1301/3120 [1:48:06<2:35:16,  5.12s/it]
 42%|████▏     | 1302/3120 [1:48:11<2:36:40,  5.17s/it]
                                                       
{'loss': 0.7573, 'learning_rate': 1.3486515213240188e-05, 'epoch': 1.67}

 42%|████▏     | 1302/3120 [1:48:11<2:36:40,  5.17s/it]
 42%|████▏     | 1303/3120 [1:48:17<2:37:59,  5.22s/it]
 42%|████▏     | 1304/3120 [1:48:22<2:37:35,  5.21s/it]
                                                       
{'loss': 0.819, 'learning_rate': 1.3466639207231882e-05, 'epoch': 1.67}

 42%|████▏     | 1304/3120 [1:48:22<2:37:35,  5.21s/it]
 42%|████▏     | 1305/3120 [1:48:27<2:36:31,  5.17s/it]
 42%|████▏     | 1306/3120 [1:48:32<2:32:49,  5.06s/it]
                                                       
{'loss': 0.8032, 'learning_rate': 1.3446747623219256e-05, 'epoch': 1.67}

 42%|████▏     | 1306/3120 [1:48:32<2:32:49,  5.06s/it]
 42%|████▏     | 1307/3120 [1:48:37<2:33:30,  5.08s/it]
 42%|████▏     | 1308/3120 [1:48:42<2:31:27,  5.02s/it]
                                                       
{'loss': 0.7717, 'learning_rate': 1.3426840550588933e-05, 'epoch': 1.68}

 42%|████▏     | 1308/3120 [1:48:42<2:31:27,  5.02s/it]
 42%|████▏     | 1309/3120 [1:48:47<2:34:53,  5.13s/it]
 42%|████▏     | 1310/3120 [1:48:53<2:37:13,  5.21s/it]
                                                       
{'loss': 0.7919, 'learning_rate': 1.3406918078797142e-05, 'epoch': 1.68}

 42%|████▏     | 1310/3120 [1:48:53<2:37:13,  5.21s/it]
 42%|████▏     | 1311/3120 [1:48:58<2:39:54,  5.30s/it]
 42%|████▏     | 1312/3120 [1:49:03<2:39:13,  5.28s/it]
                                                       
{'loss': 0.7638, 'learning_rate': 1.3386980297369308e-05, 'epoch': 1.68}

 42%|████▏     | 1312/3120 [1:49:03<2:39:13,  5.28s/it]
 42%|████▏     | 1313/3120 [1:49:08<2:34:17,  5.12s/it]
 42%|████▏     | 1314/3120 [1:49:13<2:34:53,  5.15s/it]
                                                       
{'loss': 0.805, 'learning_rate': 1.3367027295899652e-05, 'epoch': 1.68}

 42%|████▏     | 1314/3120 [1:49:13<2:34:53,  5.15s/it]
 42%|████▏     | 1315/3120 [1:49:18<2:32:36,  5.07s/it]
 42%|████▏     | 1316/3120 [1:49:23<2:29:55,  4.99s/it]
                                                       
{'loss': 0.8378, 'learning_rate': 1.3347059164050796e-05, 'epoch': 1.69}

 42%|████▏     | 1316/3120 [1:49:23<2:29:55,  4.99s/it]
 42%|████▏     | 1317/3120 [1:49:28<2:27:51,  4.92s/it]
 42%|████▏     | 1318/3120 [1:49:33<2:29:09,  4.97s/it]
                                                       
{'loss': 0.8601, 'learning_rate': 1.332707599155334e-05, 'epoch': 1.69}

 42%|████▏     | 1318/3120 [1:49:33<2:29:09,  4.97s/it]
 42%|████▏     | 1319/3120 [1:49:38<2:30:18,  5.01s/it]
 42%|████▏     | 1320/3120 [1:49:43<2:29:23,  4.98s/it]
                                                       
{'loss': 0.827, 'learning_rate': 1.3307077868205487e-05, 'epoch': 1.69}

 42%|████▏     | 1320/3120 [1:49:43<2:29:23,  4.98s/it]
 42%|████▏     | 1321/3120 [1:49:48<2:28:07,  4.94s/it]
 42%|████▏     | 1322/3120 [1:49:53<2:34:28,  5.16s/it]
                                                       
{'loss': 0.8289, 'learning_rate': 1.3287064883872614e-05, 'epoch': 1.69}

 42%|████▏     | 1322/3120 [1:49:53<2:34:28,  5.16s/it]
 42%|████▏     | 1323/3120 [1:49:58<2:31:07,  5.05s/it]
 42%|████▏     | 1324/3120 [1:50:03<2:29:07,  4.98s/it]
                                                       
{'loss': 0.8072, 'learning_rate': 1.3267037128486883e-05, 'epoch': 1.7}

 42%|████▏     | 1324/3120 [1:50:03<2:29:07,  4.98s/it]
 42%|████▏     | 1325/3120 [1:50:08<2:27:08,  4.92s/it]
 42%|████▎     | 1326/3120 [1:50:13<2:27:43,  4.94s/it]
                                                       
{'loss': 0.7881, 'learning_rate': 1.3246994692046837e-05, 'epoch': 1.7}

 42%|████▎     | 1326/3120 [1:50:13<2:27:43,  4.94s/it]
 43%|████▎     | 1327/3120 [1:50:18<2:28:21,  4.96s/it]
 43%|████▎     | 1328/3120 [1:50:23<2:27:12,  4.93s/it]
                                                       
{'loss': 0.7684, 'learning_rate': 1.3226937664616977e-05, 'epoch': 1.7}

 43%|████▎     | 1328/3120 [1:50:23<2:27:12,  4.93s/it]
 43%|████▎     | 1329/3120 [1:50:28<2:26:58,  4.92s/it]
 43%|████▎     | 1330/3120 [1:50:32<2:22:57,  4.79s/it]
                                                       
{'loss': 0.7952, 'learning_rate': 1.3206866136327387e-05, 'epoch': 1.71}

 43%|████▎     | 1330/3120 [1:50:32<2:22:57,  4.79s/it]
 43%|████▎     | 1331/3120 [1:50:36<2:19:37,  4.68s/it]
 43%|████▎     | 1332/3120 [1:50:42<2:25:00,  4.87s/it]
                                                       
{'loss': 0.8064, 'learning_rate': 1.3186780197373306e-05, 'epoch': 1.71}

 43%|████▎     | 1332/3120 [1:50:42<2:25:00,  4.87s/it]
 43%|████▎     | 1333/3120 [1:50:46<2:21:10,  4.74s/it]
 43%|████▎     | 1334/3120 [1:50:51<2:25:01,  4.87s/it]
                                                       
{'loss': 0.8248, 'learning_rate': 1.3166679938014728e-05, 'epoch': 1.71}

 43%|████▎     | 1334/3120 [1:50:51<2:25:01,  4.87s/it]
 43%|████▎     | 1335/3120 [1:50:56<2:23:49,  4.83s/it]
 43%|████▎     | 1336/3120 [1:51:02<2:30:01,  5.05s/it]
                                                       
{'loss': 0.7767, 'learning_rate': 1.3146565448576002e-05, 'epoch': 1.71}

 43%|████▎     | 1336/3120 [1:51:02<2:30:01,  5.05s/it]
 43%|████▎     | 1337/3120 [1:51:06<2:25:07,  4.88s/it]
 43%|████▎     | 1338/3120 [1:51:11<2:25:50,  4.91s/it]
                                                       
{'loss': 0.7899, 'learning_rate': 1.3126436819445423e-05, 'epoch': 1.72}

 43%|████▎     | 1338/3120 [1:51:11<2:25:50,  4.91s/it]
 43%|████▎     | 1339/3120 [1:51:16<2:25:36,  4.91s/it]
 43%|████▎     | 1340/3120 [1:51:21<2:26:03,  4.92s/it]
                                                       
{'loss': 0.7839, 'learning_rate': 1.3106294141074825e-05, 'epoch': 1.72}

 43%|████▎     | 1340/3120 [1:51:21<2:26:03,  4.92s/it]
 43%|████▎     | 1341/3120 [1:51:26<2:22:50,  4.82s/it]
 43%|████▎     | 1342/3120 [1:51:31<2:27:54,  4.99s/it]
                                                       
{'loss': 0.7953, 'learning_rate': 1.308613750397917e-05, 'epoch': 1.72}

 43%|████▎     | 1342/3120 [1:51:31<2:27:54,  4.99s/it]
 43%|████▎     | 1343/3120 [1:51:36<2:25:26,  4.91s/it]
 43%|████▎     | 1344/3120 [1:51:41<2:25:21,  4.91s/it]
                                                       
{'loss': 0.7935, 'learning_rate': 1.3065966998736155e-05, 'epoch': 1.72}

 43%|████▎     | 1344/3120 [1:51:41<2:25:21,  4.91s/it]
 43%|████▎     | 1345/3120 [1:51:45<2:23:44,  4.86s/it]
 43%|████▎     | 1346/3120 [1:51:50<2:22:37,  4.82s/it]
                                                       
{'loss': 0.8131, 'learning_rate': 1.304578271598579e-05, 'epoch': 1.73}

 43%|████▎     | 1346/3120 [1:51:50<2:22:37,  4.82s/it]
 43%|████▎     | 1347/3120 [1:51:55<2:25:14,  4.92s/it]
 43%|████▎     | 1348/3120 [1:52:00<2:25:03,  4.91s/it]
                                                       
{'loss': 0.7642, 'learning_rate': 1.302558474643e-05, 'epoch': 1.73}

 43%|████▎     | 1348/3120 [1:52:00<2:25:03,  4.91s/it]
 43%|████▎     | 1349/3120 [1:52:05<2:28:17,  5.02s/it]
 43%|████▎     | 1350/3120 [1:52:10<2:27:46,  5.01s/it]
                                                       
{'loss': 0.7821, 'learning_rate': 1.300537318083221e-05, 'epoch': 1.73}

 43%|████▎     | 1350/3120 [1:52:10<2:27:46,  5.01s/it]
 43%|████▎     | 1351/3120 [1:52:15<2:26:57,  4.98s/it]
 43%|████▎     | 1352/3120 [1:52:20<2:26:32,  4.97s/it]
                                                       
{'loss': 0.8327, 'learning_rate': 1.2985148110016947e-05, 'epoch': 1.73}

 43%|████▎     | 1352/3120 [1:52:20<2:26:32,  4.97s/it]
 43%|████▎     | 1353/3120 [1:52:25<2:26:38,  4.98s/it]
 43%|████▎     | 1354/3120 [1:52:30<2:25:25,  4.94s/it]
                                                       
{'loss': 0.7973, 'learning_rate': 1.2964909624869422e-05, 'epoch': 1.74}

 43%|████▎     | 1354/3120 [1:52:30<2:25:25,  4.94s/it]
 43%|████▎     | 1355/3120 [1:52:35<2:27:40,  5.02s/it]
 43%|████▎     | 1356/3120 [1:52:40<2:25:24,  4.95s/it]
                                                       
{'loss': 0.7698, 'learning_rate': 1.2944657816335124e-05, 'epoch': 1.74}

 43%|████▎     | 1356/3120 [1:52:40<2:25:24,  4.95s/it]
 43%|████▎     | 1357/3120 [1:52:45<2:24:09,  4.91s/it]
 44%|████▎     | 1358/3120 [1:52:49<2:21:32,  4.82s/it]
                                                       
{'loss': 0.8402, 'learning_rate': 1.2924392775419421e-05, 'epoch': 1.74}

 44%|████▎     | 1358/3120 [1:52:49<2:21:32,  4.82s/it]
 44%|████▎     | 1359/3120 [1:52:54<2:22:15,  4.85s/it]
 44%|████▎     | 1360/3120 [1:52:59<2:22:58,  4.87s/it]
                                                       
{'loss': 0.7686, 'learning_rate': 1.2904114593187136e-05, 'epoch': 1.74}

 44%|████▎     | 1360/3120 [1:52:59<2:22:58,  4.87s/it]
 44%|████▎     | 1361/3120 [1:53:04<2:20:50,  4.80s/it]
 44%|████▎     | 1362/3120 [1:53:09<2:23:10,  4.89s/it]
                                                       
{'loss': 0.8225, 'learning_rate': 1.2883823360762149e-05, 'epoch': 1.75}

 44%|████▎     | 1362/3120 [1:53:09<2:23:10,  4.89s/it]
 44%|████▎     | 1363/3120 [1:53:14<2:19:48,  4.77s/it]
 44%|████▎     | 1364/3120 [1:53:19<2:21:28,  4.83s/it]
                                                       
{'loss': 0.7842, 'learning_rate': 1.2863519169326984e-05, 'epoch': 1.75}

 44%|████▎     | 1364/3120 [1:53:19<2:21:28,  4.83s/it]
 44%|████▍     | 1365/3120 [1:53:24<2:23:03,  4.89s/it]
 44%|████▍     | 1366/3120 [1:53:29<2:25:38,  4.98s/it]
                                                       
{'loss': 0.8096, 'learning_rate': 1.284320211012239e-05, 'epoch': 1.75}

 44%|████▍     | 1366/3120 [1:53:29<2:25:38,  4.98s/it]
 44%|████▍     | 1367/3120 [1:53:34<2:24:43,  4.95s/it]
 44%|████▍     | 1368/3120 [1:53:38<2:22:49,  4.89s/it]
                                                       
{'loss': 0.7605, 'learning_rate': 1.2822872274446958e-05, 'epoch': 1.75}

 44%|████▍     | 1368/3120 [1:53:38<2:22:49,  4.89s/it]
 44%|████▍     | 1369/3120 [1:53:44<2:29:52,  5.14s/it]
 44%|████▍     | 1370/3120 [1:53:49<2:28:41,  5.10s/it]
                                                       
{'loss': 0.7653, 'learning_rate': 1.280252975365667e-05, 'epoch': 1.76}

 44%|████▍     | 1370/3120 [1:53:49<2:28:41,  5.10s/it]
 44%|████▍     | 1371/3120 [1:53:54<2:26:39,  5.03s/it]
 44%|████▍     | 1372/3120 [1:53:58<2:20:02,  4.81s/it]
                                                       
{'loss': 0.8186, 'learning_rate': 1.2782174639164528e-05, 'epoch': 1.76}

 44%|████▍     | 1372/3120 [1:53:58<2:20:02,  4.81s/it]
 44%|████▍     | 1373/3120 [1:54:03<2:19:37,  4.80s/it]
 44%|████▍     | 1374/3120 [1:54:08<2:24:46,  4.97s/it]
                                                       
{'loss': 0.8052, 'learning_rate': 1.276180702244012e-05, 'epoch': 1.76}

 44%|████▍     | 1374/3120 [1:54:08<2:24:46,  4.97s/it]
 44%|████▍     | 1375/3120 [1:54:14<2:26:56,  5.05s/it]
 44%|████▍     | 1376/3120 [1:54:19<2:29:37,  5.15s/it]
                                                       
{'loss': 0.8106, 'learning_rate': 1.2741426995009214e-05, 'epoch': 1.76}

 44%|████▍     | 1376/3120 [1:54:19<2:29:37,  5.15s/it]
 44%|████▍     | 1377/3120 [1:54:24<2:27:28,  5.08s/it]
 44%|████▍     | 1378/3120 [1:54:29<2:26:14,  5.04s/it]
                                                       
{'loss': 0.8028, 'learning_rate': 1.2721034648453353e-05, 'epoch': 1.77}

 44%|████▍     | 1378/3120 [1:54:29<2:26:14,  5.04s/it]
 44%|████▍     | 1379/3120 [1:54:34<2:29:54,  5.17s/it]
 44%|████▍     | 1380/3120 [1:54:39<2:28:45,  5.13s/it]
                                                       
{'loss': 0.7903, 'learning_rate': 1.2700630074409427e-05, 'epoch': 1.77}

 44%|████▍     | 1380/3120 [1:54:39<2:28:45,  5.13s/it]
 44%|████▍     | 1381/3120 [1:54:44<2:25:25,  5.02s/it]
 44%|████▍     | 1382/3120 [1:54:49<2:23:16,  4.95s/it]
                                                       
{'loss': 0.7951, 'learning_rate': 1.268021336456929e-05, 'epoch': 1.77}

 44%|████▍     | 1382/3120 [1:54:49<2:23:16,  4.95s/it]
 44%|████▍     | 1383/3120 [1:54:54<2:25:27,  5.02s/it]
 44%|████▍     | 1384/3120 [1:54:59<2:24:29,  4.99s/it]
                                                       
{'loss': 0.7863, 'learning_rate': 1.2659784610679318e-05, 'epoch': 1.77}

 44%|████▍     | 1384/3120 [1:54:59<2:24:29,  4.99s/it]
 44%|████▍     | 1385/3120 [1:55:04<2:25:32,  5.03s/it]
 44%|████▍     | 1386/3120 [1:55:09<2:27:18,  5.10s/it]
                                                       
{'loss': 0.8114, 'learning_rate': 1.2639343904540008e-05, 'epoch': 1.78}

 44%|████▍     | 1386/3120 [1:55:09<2:27:18,  5.10s/it]
 44%|████▍     | 1387/3120 [1:55:15<2:31:10,  5.23s/it]
 44%|████▍     | 1388/3120 [1:55:20<2:26:32,  5.08s/it]
                                                       
{'loss': 0.7654, 'learning_rate': 1.2618891338005574e-05, 'epoch': 1.78}

 44%|████▍     | 1388/3120 [1:55:20<2:26:32,  5.08s/it]
 45%|████▍     | 1389/3120 [1:55:24<2:20:51,  4.88s/it]
 45%|████▍     | 1390/3120 [1:55:29<2:21:35,  4.91s/it]
                                                       
{'loss': 0.8028, 'learning_rate': 1.2598427002983523e-05, 'epoch': 1.78}

 45%|████▍     | 1390/3120 [1:55:29<2:21:35,  4.91s/it]
 45%|████▍     | 1391/3120 [1:55:34<2:24:17,  5.01s/it]
 45%|████▍     | 1392/3120 [1:55:39<2:23:48,  4.99s/it]
                                                       
{'loss': 0.8131, 'learning_rate': 1.2577950991434249e-05, 'epoch': 1.78}

 45%|████▍     | 1392/3120 [1:55:39<2:23:48,  4.99s/it]
 45%|████▍     | 1393/3120 [1:55:44<2:20:07,  4.87s/it]
 45%|████▍     | 1394/3120 [1:55:48<2:17:28,  4.78s/it]
                                                       
{'loss': 0.8281, 'learning_rate': 1.2557463395370609e-05, 'epoch': 1.79}

 45%|████▍     | 1394/3120 [1:55:48<2:17:28,  4.78s/it]
 45%|████▍     | 1395/3120 [1:55:53<2:19:34,  4.85s/it]
 45%|████▍     | 1396/3120 [1:55:58<2:20:36,  4.89s/it]
                                                       
{'loss': 0.779, 'learning_rate': 1.2536964306857526e-05, 'epoch': 1.79}

 45%|████▍     | 1396/3120 [1:55:58<2:20:36,  4.89s/it]
 45%|████▍     | 1397/3120 [1:56:03<2:17:09,  4.78s/it]
 45%|████▍     | 1398/3120 [1:56:08<2:22:21,  4.96s/it]
                                                       
{'loss': 0.7945, 'learning_rate': 1.2516453818011567e-05, 'epoch': 1.79}

 45%|████▍     | 1398/3120 [1:56:08<2:22:21,  4.96s/it]
 45%|████▍     | 1399/3120 [1:56:14<2:24:29,  5.04s/it]
 45%|████▍     | 1400/3120 [1:56:19<2:25:21,  5.07s/it]
                                                       
{'loss': 0.7965, 'learning_rate': 1.2495932021000516e-05, 'epoch': 1.79}

 45%|████▍     | 1400/3120 [1:56:19<2:25:21,  5.07s/it]
 45%|████▍     | 1401/3120 [1:56:24<2:24:16,  5.04s/it]
 45%|████▍     | 1402/3120 [1:56:29<2:22:41,  4.98s/it]
                                                       
{'loss': 0.7763, 'learning_rate': 1.247539900804299e-05, 'epoch': 1.8}

 45%|████▍     | 1402/3120 [1:56:29<2:22:41,  4.98s/it]
 45%|████▍     | 1403/3120 [1:56:33<2:20:26,  4.91s/it]
 45%|████▌     | 1404/3120 [1:56:39<2:23:41,  5.02s/it]
                                                       
{'loss': 0.8241, 'learning_rate': 1.2454854871407993e-05, 'epoch': 1.8}

 45%|████▌     | 1404/3120 [1:56:39<2:23:41,  5.02s/it]
 45%|████▌     | 1405/3120 [1:56:44<2:24:06,  5.04s/it]
 45%|████▌     | 1406/3120 [1:56:48<2:22:11,  4.98s/it]
                                                       
{'loss': 0.8058, 'learning_rate': 1.2434299703414523e-05, 'epoch': 1.8}

 45%|████▌     | 1406/3120 [1:56:48<2:22:11,  4.98s/it]
 45%|████▌     | 1407/3120 [1:56:53<2:19:22,  4.88s/it]
 45%|████▌     | 1408/3120 [1:56:58<2:17:23,  4.82s/it]
                                                       
{'loss': 0.8203, 'learning_rate': 1.2413733596431141e-05, 'epoch': 1.81}

 45%|████▌     | 1408/3120 [1:56:58<2:17:23,  4.82s/it]
 45%|████▌     | 1409/3120 [1:57:03<2:19:15,  4.88s/it]
 45%|████▌     | 1410/3120 [1:57:07<2:15:04,  4.74s/it]
                                                       
{'loss': 0.7938, 'learning_rate': 1.2393156642875579e-05, 'epoch': 1.81}

 45%|████▌     | 1410/3120 [1:57:07<2:15:04,  4.74s/it]
 45%|████▌     | 1411/3120 [1:57:12<2:15:49,  4.77s/it]
 45%|████▌     | 1412/3120 [1:57:17<2:15:01,  4.74s/it]
                                                       
{'loss': 0.7953, 'learning_rate': 1.2372568935214298e-05, 'epoch': 1.81}

 45%|████▌     | 1412/3120 [1:57:17<2:15:01,  4.74s/it]
 45%|████▌     | 1413/3120 [1:57:22<2:16:44,  4.81s/it]
 45%|████▌     | 1414/3120 [1:57:27<2:19:21,  4.90s/it]
                                                       
{'loss': 0.782, 'learning_rate': 1.2351970565962083e-05, 'epoch': 1.81}

 45%|████▌     | 1414/3120 [1:57:27<2:19:21,  4.90s/it]
 45%|████▌     | 1415/3120 [1:57:32<2:20:32,  4.95s/it]
 45%|████▌     | 1416/3120 [1:57:37<2:20:12,  4.94s/it]
                                                       
{'loss': 0.8193, 'learning_rate': 1.2331361627681645e-05, 'epoch': 1.82}

 45%|████▌     | 1416/3120 [1:57:37<2:20:12,  4.94s/it]
 45%|████▌     | 1417/3120 [1:57:42<2:19:43,  4.92s/it]
 45%|████▌     | 1418/3120 [1:57:47<2:20:43,  4.96s/it]
                                                       
{'loss': 0.8184, 'learning_rate': 1.2310742212983167e-05, 'epoch': 1.82}

 45%|████▌     | 1418/3120 [1:57:47<2:20:43,  4.96s/it]
 45%|████▌     | 1419/3120 [1:57:52<2:21:34,  4.99s/it]
 46%|████▌     | 1420/3120 [1:57:57<2:22:06,  5.02s/it]
                                                       
{'loss': 0.8209, 'learning_rate': 1.2290112414523927e-05, 'epoch': 1.82}

 46%|████▌     | 1420/3120 [1:57:57<2:22:06,  5.02s/it]
 46%|████▌     | 1421/3120 [1:58:02<2:23:35,  5.07s/it]
 46%|████▌     | 1422/3120 [1:58:07<2:22:29,  5.04s/it]
                                                       
{'loss': 0.7982, 'learning_rate': 1.2269472325007858e-05, 'epoch': 1.82}

 46%|████▌     | 1422/3120 [1:58:07<2:22:29,  5.04s/it]
 46%|████▌     | 1423/3120 [1:58:12<2:24:13,  5.10s/it]
 46%|████▌     | 1424/3120 [1:58:17<2:23:04,  5.06s/it]
                                                       
{'loss': 0.779, 'learning_rate': 1.2248822037185137e-05, 'epoch': 1.83}

 46%|████▌     | 1424/3120 [1:58:17<2:23:04,  5.06s/it]
 46%|████▌     | 1425/3120 [1:58:22<2:22:20,  5.04s/it]
 46%|████▌     | 1426/3120 [1:58:27<2:19:28,  4.94s/it]
                                                       
{'loss': 0.7905, 'learning_rate': 1.222816164385177e-05, 'epoch': 1.83}

 46%|████▌     | 1426/3120 [1:58:27<2:19:28,  4.94s/it]
 46%|████▌     | 1427/3120 [1:58:32<2:19:19,  4.94s/it]
 46%|████▌     | 1428/3120 [1:58:37<2:19:05,  4.93s/it]
                                                       
{'loss': 0.743, 'learning_rate': 1.2207491237849174e-05, 'epoch': 1.83}

 46%|████▌     | 1428/3120 [1:58:37<2:19:05,  4.93s/it]
 46%|████▌     | 1429/3120 [1:58:42<2:20:20,  4.98s/it]
 46%|████▌     | 1430/3120 [1:58:47<2:22:04,  5.04s/it]
                                                       
{'loss': 0.8098, 'learning_rate': 1.218681091206376e-05, 'epoch': 1.83}

 46%|████▌     | 1430/3120 [1:58:47<2:22:04,  5.04s/it]
 46%|████▌     | 1431/3120 [1:58:52<2:21:43,  5.03s/it]
 46%|████▌     | 1432/3120 [1:58:57<2:21:48,  5.04s/it]
                                                       
{'loss': 0.7813, 'learning_rate': 1.2166120759426515e-05, 'epoch': 1.84}

 46%|████▌     | 1432/3120 [1:58:57<2:21:48,  5.04s/it]
 46%|████▌     | 1433/3120 [1:59:03<2:24:48,  5.15s/it]
 46%|████▌     | 1434/3120 [1:59:08<2:24:04,  5.13s/it]
                                                       
{'loss': 0.7944, 'learning_rate': 1.2145420872912586e-05, 'epoch': 1.84}

 46%|████▌     | 1434/3120 [1:59:08<2:24:04,  5.13s/it]
 46%|████▌     | 1435/3120 [1:59:13<2:24:00,  5.13s/it]
 46%|████▌     | 1436/3120 [1:59:17<2:19:36,  4.97s/it]
                                                       
{'loss': 0.8159, 'learning_rate': 1.2124711345540861e-05, 'epoch': 1.84}

 46%|████▌     | 1436/3120 [1:59:17<2:19:36,  4.97s/it]
 46%|████▌     | 1437/3120 [1:59:22<2:20:41,  5.02s/it]
 46%|████▌     | 1438/3120 [1:59:27<2:19:36,  4.98s/it]
                                                       
{'loss': 0.7834, 'learning_rate': 1.2103992270373548e-05, 'epoch': 1.84}

 46%|████▌     | 1438/3120 [1:59:27<2:19:36,  4.98s/it]
 46%|████▌     | 1439/3120 [1:59:32<2:18:37,  4.95s/it]
 46%|████▌     | 1440/3120 [1:59:37<2:19:04,  4.97s/it]
                                                       
{'loss': 0.8172, 'learning_rate': 1.2083263740515764e-05, 'epoch': 1.85}

 46%|████▌     | 1440/3120 [1:59:37<2:19:04,  4.97s/it]
 46%|████▌     | 1441/3120 [1:59:42<2:17:58,  4.93s/it]
 46%|████▌     | 1442/3120 [1:59:47<2:17:59,  4.93s/it]
                                                       
{'loss': 0.7996, 'learning_rate': 1.2062525849115107e-05, 'epoch': 1.85}

 46%|████▌     | 1442/3120 [1:59:47<2:17:59,  4.93s/it]
 46%|████▋     | 1443/3120 [1:59:52<2:19:03,  4.98s/it]
 46%|████▋     | 1444/3120 [1:59:57<2:19:26,  4.99s/it]
                                                       
{'loss': 0.8369, 'learning_rate': 1.2041778689361254e-05, 'epoch': 1.85}

 46%|████▋     | 1444/3120 [1:59:57<2:19:26,  4.99s/it]
 46%|████▋     | 1445/3120 [2:00:02<2:19:33,  5.00s/it]
 46%|████▋     | 1446/3120 [2:00:07<2:18:17,  4.96s/it]
                                                       
{'loss': 0.767, 'learning_rate': 1.2021022354485514e-05, 'epoch': 1.85}

 46%|████▋     | 1446/3120 [2:00:07<2:18:17,  4.96s/it]
 46%|████▋     | 1447/3120 [2:00:12<2:19:46,  5.01s/it]
 46%|████▋     | 1448/3120 [2:00:17<2:19:34,  5.01s/it]
                                                       
{'loss': 0.8078, 'learning_rate': 1.2000256937760446e-05, 'epoch': 1.86}

 46%|████▋     | 1448/3120 [2:00:17<2:19:34,  5.01s/it]
 46%|████▋     | 1449/3120 [2:00:22<2:19:02,  4.99s/it]
 46%|████▋     | 1450/3120 [2:00:27<2:20:37,  5.05s/it]
                                                       
{'loss': 0.7994, 'learning_rate': 1.1979482532499402e-05, 'epoch': 1.86}

 46%|████▋     | 1450/3120 [2:00:27<2:20:37,  5.05s/it]
 47%|████▋     | 1451/3120 [2:00:32<2:17:38,  4.95s/it]
 47%|████▋     | 1452/3120 [2:00:37<2:17:22,  4.94s/it]
                                                       
{'loss': 0.7912, 'learning_rate': 1.1958699232056135e-05, 'epoch': 1.86}

 47%|████▋     | 1452/3120 [2:00:37<2:17:22,  4.94s/it]
 47%|████▋     | 1453/3120 [2:00:42<2:14:39,  4.85s/it]
 47%|████▋     | 1454/3120 [2:00:46<2:14:54,  4.86s/it]
                                                       
{'loss': 0.81, 'learning_rate': 1.1937907129824371e-05, 'epoch': 1.86}

 47%|████▋     | 1454/3120 [2:00:46<2:14:54,  4.86s/it]
 47%|████▋     | 1455/3120 [2:00:51<2:14:40,  4.85s/it]
 47%|████▋     | 1456/3120 [2:00:57<2:19:08,  5.02s/it]
                                                       
{'loss': 0.8002, 'learning_rate': 1.1917106319237386e-05, 'epoch': 1.87}

 47%|████▋     | 1456/3120 [2:00:57<2:19:08,  5.02s/it]
 47%|████▋     | 1457/3120 [2:01:02<2:18:50,  5.01s/it]
 47%|████▋     | 1458/3120 [2:01:07<2:20:31,  5.07s/it]
                                                       
{'loss': 0.7966, 'learning_rate': 1.1896296893767588e-05, 'epoch': 1.87}

 47%|████▋     | 1458/3120 [2:01:07<2:20:31,  5.07s/it]
 47%|████▋     | 1459/3120 [2:01:12<2:18:02,  4.99s/it]
 47%|████▋     | 1460/3120 [2:01:16<2:15:50,  4.91s/it]
                                                       
{'loss': 0.8234, 'learning_rate': 1.1875478946926094e-05, 'epoch': 1.87}

 47%|████▋     | 1460/3120 [2:01:16<2:15:50,  4.91s/it]
 47%|████▋     | 1461/3120 [2:01:22<2:18:54,  5.02s/it]
 47%|████▋     | 1462/3120 [2:01:26<2:15:22,  4.90s/it]
                                                       
{'loss': 0.7378, 'learning_rate': 1.1854652572262324e-05, 'epoch': 1.87}

 47%|████▋     | 1462/3120 [2:01:26<2:15:22,  4.90s/it]
 47%|████▋     | 1463/3120 [2:01:31<2:17:09,  4.97s/it]
 47%|████▋     | 1464/3120 [2:01:36<2:14:50,  4.89s/it]
                                                       
{'loss': 0.7878, 'learning_rate': 1.1833817863363563e-05, 'epoch': 1.88}

 47%|████▋     | 1464/3120 [2:01:36<2:14:50,  4.89s/it]
 47%|████▋     | 1465/3120 [2:01:41<2:14:45,  4.89s/it]
 47%|████▋     | 1466/3120 [2:01:46<2:14:14,  4.87s/it]
                                                       
{'loss': 0.7829, 'learning_rate': 1.1812974913854545e-05, 'epoch': 1.88}

 47%|████▋     | 1466/3120 [2:01:46<2:14:14,  4.87s/it]
 47%|████▋     | 1467/3120 [2:01:51<2:14:03,  4.87s/it]
 47%|████▋     | 1468/3120 [2:01:56<2:14:25,  4.88s/it]
                                                       
{'loss': 0.7498, 'learning_rate': 1.1792123817397041e-05, 'epoch': 1.88}

 47%|████▋     | 1468/3120 [2:01:56<2:14:25,  4.88s/it]
 47%|████▋     | 1469/3120 [2:02:01<2:15:31,  4.93s/it]
 47%|████▋     | 1470/3120 [2:02:06<2:16:22,  4.96s/it]
                                                       
{'loss': 0.7399, 'learning_rate': 1.1771264667689428e-05, 'epoch': 1.88}

 47%|████▋     | 1470/3120 [2:02:06<2:16:22,  4.96s/it]
 47%|████▋     | 1471/3120 [2:02:11<2:20:59,  5.13s/it]
 47%|████▋     | 1472/3120 [2:02:17<2:22:15,  5.18s/it]
                                                       
{'loss': 0.8065, 'learning_rate': 1.1750397558466273e-05, 'epoch': 1.89}

 47%|████▋     | 1472/3120 [2:02:17<2:22:15,  5.18s/it]
 47%|████▋     | 1473/3120 [2:02:22<2:21:47,  5.17s/it]
 47%|████▋     | 1474/3120 [2:02:27<2:20:43,  5.13s/it]
                                                       
{'loss': 0.7602, 'learning_rate': 1.1729522583497913e-05, 'epoch': 1.89}

 47%|████▋     | 1474/3120 [2:02:27<2:20:43,  5.13s/it]
 47%|████▋     | 1475/3120 [2:02:31<2:17:22,  5.01s/it]
 47%|████▋     | 1476/3120 [2:02:36<2:16:18,  4.97s/it]
                                                       
{'loss': 0.7641, 'learning_rate': 1.1708639836590024e-05, 'epoch': 1.89}

 47%|████▋     | 1476/3120 [2:02:36<2:16:18,  4.97s/it]
 47%|████▋     | 1477/3120 [2:02:41<2:14:41,  4.92s/it]
 47%|████▋     | 1478/3120 [2:02:46<2:13:53,  4.89s/it]
                                                       
{'loss': 0.8267, 'learning_rate': 1.1687749411583213e-05, 'epoch': 1.89}

 47%|████▋     | 1478/3120 [2:02:46<2:13:53,  4.89s/it]
 47%|████▋     | 1479/3120 [2:02:51<2:15:19,  4.95s/it]
 47%|████▋     | 1480/3120 [2:02:56<2:12:37,  4.85s/it]
                                                       
{'loss': 0.7885, 'learning_rate': 1.1666851402352587e-05, 'epoch': 1.9}

 47%|████▋     | 1480/3120 [2:02:56<2:12:37,  4.85s/it]
 47%|████▋     | 1481/3120 [2:03:01<2:14:12,  4.91s/it]
 48%|████▊     | 1482/3120 [2:03:05<2:12:46,  4.86s/it]
                                                       
{'loss': 0.7623, 'learning_rate': 1.164594590280734e-05, 'epoch': 1.9}

 48%|████▊     | 1482/3120 [2:03:05<2:12:46,  4.86s/it]
 48%|████▊     | 1483/3120 [2:03:10<2:11:34,  4.82s/it]
 48%|████▊     | 1484/3120 [2:03:15<2:13:54,  4.91s/it]
                                                       
{'loss': 0.7529, 'learning_rate': 1.1625033006890316e-05, 'epoch': 1.9}

 48%|████▊     | 1484/3120 [2:03:15<2:13:54,  4.91s/it]
 48%|████▊     | 1485/3120 [2:03:20<2:13:20,  4.89s/it]
 48%|████▊     | 1486/3120 [2:03:25<2:09:18,  4.75s/it]
                                                       
{'loss': 0.7962, 'learning_rate': 1.1604112808577603e-05, 'epoch': 1.91}

 48%|████▊     | 1486/3120 [2:03:25<2:09:18,  4.75s/it]
 48%|████▊     | 1487/3120 [2:03:29<2:10:22,  4.79s/it]
 48%|████▊     | 1488/3120 [2:03:34<2:12:09,  4.86s/it]
                                                       
{'loss': 0.7621, 'learning_rate': 1.15831854018781e-05, 'epoch': 1.91}

 48%|████▊     | 1488/3120 [2:03:34<2:12:09,  4.86s/it]
 48%|████▊     | 1489/3120 [2:03:39<2:13:09,  4.90s/it]
 48%|████▊     | 1490/3120 [2:03:45<2:15:46,  5.00s/it]
                                                       
{'loss': 0.7941, 'learning_rate': 1.1562250880833103e-05, 'epoch': 1.91}

 48%|████▊     | 1490/3120 [2:03:45<2:15:46,  5.00s/it]
 48%|████▊     | 1491/3120 [2:03:50<2:16:40,  5.03s/it]
 48%|████▊     | 1492/3120 [2:03:55<2:14:52,  4.97s/it]
                                                       
{'loss': 0.8247, 'learning_rate': 1.154130933951587e-05, 'epoch': 1.91}

 48%|████▊     | 1492/3120 [2:03:55<2:14:52,  4.97s/it]
 48%|████▊     | 1493/3120 [2:04:00<2:16:59,  5.05s/it]
 48%|████▊     | 1494/3120 [2:04:06<2:22:04,  5.24s/it]
                                                       
{'loss': 0.8127, 'learning_rate': 1.1520360872031208e-05, 'epoch': 1.92}

 48%|████▊     | 1494/3120 [2:04:06<2:22:04,  5.24s/it]
 48%|████▊     | 1495/3120 [2:04:11<2:25:48,  5.38s/it]
 48%|████▊     | 1496/3120 [2:04:16<2:21:58,  5.25s/it]
                                                       
{'loss': 0.7675, 'learning_rate': 1.1499405572515059e-05, 'epoch': 1.92}

 48%|████▊     | 1496/3120 [2:04:16<2:21:58,  5.25s/it]
 48%|████▊     | 1497/3120 [2:04:21<2:18:51,  5.13s/it]
 48%|████▊     | 1498/3120 [2:04:26<2:14:37,  4.98s/it]
                                                       
{'loss': 0.7718, 'learning_rate': 1.147844353513405e-05, 'epoch': 1.92}

 48%|████▊     | 1498/3120 [2:04:26<2:14:37,  4.98s/it]
 48%|████▊     | 1499/3120 [2:04:30<2:12:03,  4.89s/it]
 48%|████▊     | 1500/3120 [2:04:35<2:09:46,  4.81s/it]
                                                       
{'loss': 0.839, 'learning_rate': 1.1457474854085095e-05, 'epoch': 1.92}

 48%|████▊     | 1500/3120 [2:04:35<2:09:46,  4.81s/it]
 48%|████▊     | 1501/3120 [2:04:40<2:12:11,  4.90s/it]
 48%|████▊     | 1502/3120 [2:04:45<2:14:46,  5.00s/it]
                                                       
{'loss': 0.7915, 'learning_rate': 1.1436499623594963e-05, 'epoch': 1.93}

 48%|████▊     | 1502/3120 [2:04:45<2:14:46,  5.00s/it]
 48%|████▊     | 1503/3120 [2:04:50<2:12:03,  4.90s/it]
 48%|████▊     | 1504/3120 [2:04:55<2:13:34,  4.96s/it]
                                                       
{'loss': 0.8186, 'learning_rate': 1.1415517937919846e-05, 'epoch': 1.93}

 48%|████▊     | 1504/3120 [2:04:55<2:13:34,  4.96s/it]
 48%|████▊     | 1505/3120 [2:05:00<2:10:50,  4.86s/it]
 48%|████▊     | 1506/3120 [2:05:05<2:12:24,  4.92s/it]
                                                       
{'loss': 0.8163, 'learning_rate': 1.139452989134496e-05, 'epoch': 1.93}

 48%|████▊     | 1506/3120 [2:05:05<2:12:24,  4.92s/it]
 48%|████▊     | 1507/3120 [2:05:10<2:11:14,  4.88s/it]
 48%|████▊     | 1508/3120 [2:05:14<2:10:04,  4.84s/it]
                                                       
{'loss': 0.8156, 'learning_rate': 1.1373535578184083e-05, 'epoch': 1.93}

 48%|████▊     | 1508/3120 [2:05:14<2:10:04,  4.84s/it]
 48%|████▊     | 1509/3120 [2:05:19<2:11:31,  4.90s/it]
 48%|████▊     | 1510/3120 [2:05:25<2:16:13,  5.08s/it]
                                                       
{'loss': 0.8194, 'learning_rate': 1.1352535092779174e-05, 'epoch': 1.94}

 48%|████▊     | 1510/3120 [2:05:25<2:16:13,  5.08s/it]
 48%|████▊     | 1511/3120 [2:05:30<2:12:46,  4.95s/it]
 48%|████▊     | 1512/3120 [2:05:35<2:13:24,  4.98s/it]
                                                       
{'loss': 0.7787, 'learning_rate': 1.1331528529499909e-05, 'epoch': 1.94}

 48%|████▊     | 1512/3120 [2:05:35<2:13:24,  4.98s/it]
 48%|████▊     | 1513/3120 [2:05:40<2:15:36,  5.06s/it]
 49%|████▊     | 1514/3120 [2:05:45<2:14:33,  5.03s/it]
                                                       
{'loss': 0.7798, 'learning_rate': 1.1310515982743293e-05, 'epoch': 1.94}

 49%|████▊     | 1514/3120 [2:05:45<2:14:33,  5.03s/it]
 49%|████▊     | 1515/3120 [2:05:49<2:11:46,  4.93s/it]
 49%|████▊     | 1516/3120 [2:05:55<2:16:59,  5.12s/it]
                                                       
{'loss': 0.7784, 'learning_rate': 1.1289497546933212e-05, 'epoch': 1.94}

 49%|████▊     | 1516/3120 [2:05:55<2:16:59,  5.12s/it]
 49%|████▊     | 1517/3120 [2:06:00<2:16:48,  5.12s/it]
 49%|████▊     | 1518/3120 [2:06:05<2:16:25,  5.11s/it]
                                                       
{'loss': 0.7827, 'learning_rate': 1.1268473316520007e-05, 'epoch': 1.95}

 49%|████▊     | 1518/3120 [2:06:05<2:16:25,  5.11s/it]
 49%|████▊     | 1519/3120 [2:06:10<2:15:13,  5.07s/it]
 49%|████▊     | 1520/3120 [2:06:15<2:15:05,  5.07s/it]
                                                       
{'loss': 0.7978, 'learning_rate': 1.124744338598008e-05, 'epoch': 1.95}

 49%|████▊     | 1520/3120 [2:06:15<2:15:05,  5.07s/it]
 49%|████▉     | 1521/3120 [2:06:20<2:11:04,  4.92s/it]
 49%|████▉     | 1522/3120 [2:06:25<2:09:49,  4.87s/it]
                                                       
{'loss': 0.7453, 'learning_rate': 1.1226407849815422e-05, 'epoch': 1.95}

 49%|████▉     | 1522/3120 [2:06:25<2:09:49,  4.87s/it]
 49%|████▉     | 1523/3120 [2:06:30<2:11:57,  4.96s/it]
 49%|████▉     | 1524/3120 [2:06:35<2:11:41,  4.95s/it]
                                                       
{'loss': 0.8078, 'learning_rate': 1.1205366802553231e-05, 'epoch': 1.95}

 49%|████▉     | 1524/3120 [2:06:35<2:11:41,  4.95s/it]
 49%|████▉     | 1525/3120 [2:06:40<2:11:15,  4.94s/it]
 49%|████▉     | 1526/3120 [2:06:45<2:12:03,  4.97s/it]
                                                       
{'loss': 0.7637, 'learning_rate': 1.1184320338745467e-05, 'epoch': 1.96}

 49%|████▉     | 1526/3120 [2:06:45<2:12:03,  4.97s/it]
 49%|████▉     | 1527/3120 [2:06:49<2:07:40,  4.81s/it]
 49%|████▉     | 1528/3120 [2:06:54<2:06:46,  4.78s/it]
                                                       
{'loss': 0.7807, 'learning_rate': 1.1163268552968422e-05, 'epoch': 1.96}

 49%|████▉     | 1528/3120 [2:06:54<2:06:46,  4.78s/it]
 49%|████▉     | 1529/3120 [2:06:58<2:05:15,  4.72s/it]
 49%|████▉     | 1530/3120 [2:07:03<2:07:26,  4.81s/it]
                                                       
{'loss': 0.8054, 'learning_rate': 1.1142211539822318e-05, 'epoch': 1.96}

 49%|████▉     | 1530/3120 [2:07:03<2:07:26,  4.81s/it]
 49%|████▉     | 1531/3120 [2:07:08<2:08:01,  4.83s/it]
 49%|████▉     | 1532/3120 [2:07:14<2:12:34,  5.01s/it]
                                                       
{'loss': 0.826, 'learning_rate': 1.112114939393085e-05, 'epoch': 1.96}

 49%|████▉     | 1532/3120 [2:07:14<2:12:34,  5.01s/it]
 49%|████▉     | 1533/3120 [2:07:19<2:11:59,  4.99s/it]
 49%|████▉     | 1534/3120 [2:07:24<2:13:33,  5.05s/it]
                                                       
{'loss': 0.8095, 'learning_rate': 1.1100082209940795e-05, 'epoch': 1.97}

 49%|████▉     | 1534/3120 [2:07:24<2:13:33,  5.05s/it]
 49%|████▉     | 1535/3120 [2:07:29<2:15:08,  5.12s/it]
 49%|████▉     | 1536/3120 [2:07:34<2:13:45,  5.07s/it]
                                                       
{'loss': 0.7496, 'learning_rate': 1.1079010082521557e-05, 'epoch': 1.97}

 49%|████▉     | 1536/3120 [2:07:34<2:13:45,  5.07s/it]
 49%|████▉     | 1537/3120 [2:07:39<2:13:48,  5.07s/it]
 49%|████▉     | 1538/3120 [2:07:45<2:16:03,  5.16s/it]
                                                       
{'loss': 0.8398, 'learning_rate': 1.1057933106364758e-05, 'epoch': 1.97}

 49%|████▉     | 1538/3120 [2:07:45<2:16:03,  5.16s/it]
 49%|████▉     | 1539/3120 [2:07:50<2:16:39,  5.19s/it]
 49%|████▉     | 1540/3120 [2:07:55<2:14:05,  5.09s/it]
                                                       
{'loss': 0.7913, 'learning_rate': 1.1036851376183812e-05, 'epoch': 1.97}

 49%|████▉     | 1540/3120 [2:07:55<2:14:05,  5.09s/it]
 49%|████▉     | 1541/3120 [2:08:00<2:16:04,  5.17s/it]
 49%|████▉     | 1542/3120 [2:08:05<2:13:24,  5.07s/it]
                                                       
{'loss': 0.8017, 'learning_rate': 1.101576498671349e-05, 'epoch': 1.98}

 49%|████▉     | 1542/3120 [2:08:05<2:13:24,  5.07s/it]
 49%|████▉     | 1543/3120 [2:08:10<2:10:25,  4.96s/it]
 49%|████▉     | 1544/3120 [2:08:14<2:10:06,  4.95s/it]
                                                       
{'loss': 0.7392, 'learning_rate': 1.0994674032709514e-05, 'epoch': 1.98}

 49%|████▉     | 1544/3120 [2:08:14<2:10:06,  4.95s/it]
 50%|████▉     | 1545/3120 [2:08:20<2:14:21,  5.12s/it]
 50%|████▉     | 1546/3120 [2:08:25<2:12:07,  5.04s/it]
                                                       
{'loss': 0.7795, 'learning_rate': 1.0973578608948094e-05, 'epoch': 1.98}

 50%|████▉     | 1546/3120 [2:08:25<2:12:07,  5.04s/it]
 50%|████▉     | 1547/3120 [2:08:30<2:10:14,  4.97s/it]
 50%|████▉     | 1548/3120 [2:08:35<2:12:12,  5.05s/it]
                                                       
{'loss': 0.7834, 'learning_rate': 1.095247881022555e-05, 'epoch': 1.98}

 50%|████▉     | 1548/3120 [2:08:35<2:12:12,  5.05s/it]
 50%|████▉     | 1549/3120 [2:08:40<2:11:47,  5.03s/it]
 50%|████▉     | 1550/3120 [2:08:45<2:09:22,  4.94s/it]
                                                       
{'loss': 0.7992, 'learning_rate': 1.0931374731357843e-05, 'epoch': 1.99}

 50%|████▉     | 1550/3120 [2:08:45<2:09:22,  4.94s/it]
 50%|████▉     | 1551/3120 [2:08:50<2:12:19,  5.06s/it]
 50%|████▉     | 1552/3120 [2:08:55<2:10:55,  5.01s/it]
                                                       
{'loss': 0.8043, 'learning_rate': 1.091026646718018e-05, 'epoch': 1.99}

 50%|████▉     | 1552/3120 [2:08:55<2:10:55,  5.01s/it]
 50%|████▉     | 1553/3120 [2:09:00<2:13:18,  5.10s/it]
 50%|████▉     | 1554/3120 [2:09:05<2:11:42,  5.05s/it]
                                                       
{'loss': 0.7868, 'learning_rate': 1.0889154112546569e-05, 'epoch': 1.99}

 50%|████▉     | 1554/3120 [2:09:05<2:11:42,  5.05s/it]
 50%|████▉     | 1555/3120 [2:09:10<2:10:07,  4.99s/it]
 50%|████▉     | 1556/3120 [2:09:15<2:12:04,  5.07s/it]
                                                       
{'loss': 0.8144, 'learning_rate': 1.0868037762329405e-05, 'epoch': 1.99}

 50%|████▉     | 1556/3120 [2:09:15<2:12:04,  5.07s/it]
 50%|████▉     | 1557/3120 [2:09:20<2:11:16,  5.04s/it]
 50%|████▉     | 1558/3120 [2:09:25<2:09:53,  4.99s/it]
                                                       
{'loss': 0.7629, 'learning_rate': 1.0846917511419031e-05, 'epoch': 2.0}

 50%|████▉     | 1558/3120 [2:09:25<2:09:53,  4.99s/it]
 50%|████▉     | 1559/3120 [2:09:30<2:07:18,  4.89s/it]
 50%|█████     | 1560/3120 [2:09:35<2:10:47,  5.03s/it]
                                                       
{'loss': 0.8088, 'learning_rate': 1.0825793454723325e-05, 'epoch': 2.0}

 50%|█████     | 1560/3120 [2:09:35<2:10:47,  5.03s/it]/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
  warnings.warn(
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  labels = [torch.tensor(x) for x in labels]

 50%|█████     | 1561/3120 [2:10:13<6:27:48, 14.93s/it]
 50%|█████     | 1562/3120 [2:10:18<5:10:30, 11.96s/it]
                                                       
{'loss': 0.4308, 'learning_rate': 1.0804665687167262e-05, 'epoch': 2.0}

 50%|█████     | 1562/3120 [2:10:18<5:10:30, 11.96s/it]
 50%|█████     | 1563/3120 [2:10:23<4:13:27,  9.77s/it]
 50%|█████     | 1564/3120 [2:10:28<3:36:45,  8.36s/it]
                                                       
{'loss': 0.4056, 'learning_rate': 1.0783534303692493e-05, 'epoch': 2.01}

 50%|█████     | 1564/3120 [2:10:28<3:36:45,  8.36s/it]
 50%|█████     | 1565/3120 [2:10:33<3:13:50,  7.48s/it]
 50%|█████     | 1566/3120 [2:10:38<2:54:01,  6.72s/it]
                                                       
{'loss': 0.4122, 'learning_rate': 1.0762399399256917e-05, 'epoch': 2.01}

 50%|█████     | 1566/3120 [2:10:38<2:54:01,  6.72s/it]
 50%|█████     | 1567/3120 [2:10:43<2:40:22,  6.20s/it]
 50%|█████     | 1568/3120 [2:10:48<2:29:58,  5.80s/it]
                                                       
{'loss': 0.4125, 'learning_rate': 1.0741261068834266e-05, 'epoch': 2.01}

 50%|█████     | 1568/3120 [2:10:48<2:29:58,  5.80s/it]
 50%|█████     | 1569/3120 [2:10:53<2:25:31,  5.63s/it]
 50%|█████     | 1570/3120 [2:10:58<2:20:07,  5.42s/it]
                                                       
{'loss': 0.3983, 'learning_rate': 1.0720119407413647e-05, 'epoch': 2.01}

 50%|█████     | 1570/3120 [2:10:58<2:20:07,  5.42s/it]
 50%|█████     | 1571/3120 [2:11:04<2:19:59,  5.42s/it]
 50%|█████     | 1572/3120 [2:11:08<2:15:35,  5.26s/it]
                                                       
{'loss': 0.3843, 'learning_rate': 1.0698974509999159e-05, 'epoch': 2.02}

 50%|█████     | 1572/3120 [2:11:08<2:15:35,  5.26s/it]
 50%|█████     | 1573/3120 [2:11:13<2:13:03,  5.16s/it]
 50%|█████     | 1574/3120 [2:11:19<2:17:05,  5.32s/it]
                                                       
{'loss': 0.4024, 'learning_rate': 1.0677826471609423e-05, 'epoch': 2.02}

 50%|█████     | 1574/3120 [2:11:19<2:17:05,  5.32s/it]
 50%|█████     | 1575/3120 [2:11:24<2:13:36,  5.19s/it]
 51%|█████     | 1576/3120 [2:11:29<2:12:30,  5.15s/it]
                                                       
{'loss': 0.3865, 'learning_rate': 1.0656675387277183e-05, 'epoch': 2.02}

 51%|█████     | 1576/3120 [2:11:29<2:12:30,  5.15s/it]
 51%|█████     | 1577/3120 [2:11:34<2:11:09,  5.10s/it]
 51%|█████     | 1578/3120 [2:11:39<2:09:40,  5.05s/it]
                                                       
{'loss': 0.3953, 'learning_rate': 1.0635521352048873e-05, 'epoch': 2.02}

 51%|█████     | 1578/3120 [2:11:39<2:09:40,  5.05s/it]
 51%|█████     | 1579/3120 [2:11:44<2:08:00,  4.98s/it]
 51%|█████     | 1580/3120 [2:11:49<2:06:15,  4.92s/it]
                                                       
{'loss': 0.3734, 'learning_rate': 1.0614364460984178e-05, 'epoch': 2.03}

 51%|█████     | 1580/3120 [2:11:49<2:06:15,  4.92s/it]
 51%|█████     | 1581/3120 [2:11:54<2:07:59,  4.99s/it]
 51%|█████     | 1582/3120 [2:11:59<2:08:15,  5.00s/it]
                                                       
{'loss': 0.3556, 'learning_rate': 1.0593204809155628e-05, 'epoch': 2.03}

 51%|█████     | 1582/3120 [2:11:59<2:08:15,  5.00s/it]
 51%|█████     | 1583/3120 [2:12:04<2:07:25,  4.97s/it]
 51%|█████     | 1584/3120 [2:12:09<2:07:49,  4.99s/it]
                                                       
{'loss': 0.3826, 'learning_rate': 1.057204249164815e-05, 'epoch': 2.03}

 51%|█████     | 1584/3120 [2:12:09<2:07:49,  4.99s/it]
 51%|█████     | 1585/3120 [2:12:13<2:05:09,  4.89s/it]
 51%|█████     | 1586/3120 [2:12:18<2:06:26,  4.95s/it]
                                                       
{'loss': 0.3836, 'learning_rate': 1.0550877603558656e-05, 'epoch': 2.03}

 51%|█████     | 1586/3120 [2:12:18<2:06:26,  4.95s/it]
 51%|█████     | 1587/3120 [2:12:23<2:04:33,  4.88s/it]
 51%|█████     | 1588/3120 [2:12:28<2:05:24,  4.91s/it]
                                                       
{'loss': 0.4046, 'learning_rate': 1.0529710239995606e-05, 'epoch': 2.04}

 51%|█████     | 1588/3120 [2:12:28<2:05:24,  4.91s/it]
 51%|█████     | 1589/3120 [2:12:33<2:04:10,  4.87s/it]
 51%|█████     | 1590/3120 [2:12:38<2:04:05,  4.87s/it]
                                                       
{'loss': 0.3772, 'learning_rate': 1.0508540496078582e-05, 'epoch': 2.04}

 51%|█████     | 1590/3120 [2:12:38<2:04:05,  4.87s/it]
 51%|█████     | 1591/3120 [2:12:43<2:04:45,  4.90s/it]
 51%|█████     | 1592/3120 [2:12:47<2:03:33,  4.85s/it]
                                                       
{'loss': 0.3808, 'learning_rate': 1.0487368466937866e-05, 'epoch': 2.04}

 51%|█████     | 1592/3120 [2:12:47<2:03:33,  4.85s/it]
 51%|█████     | 1593/3120 [2:12:53<2:05:14,  4.92s/it]
 51%|█████     | 1594/3120 [2:12:57<2:02:24,  4.81s/it]
                                                       
{'loss': 0.367, 'learning_rate': 1.0466194247714008e-05, 'epoch': 2.04}

 51%|█████     | 1594/3120 [2:12:57<2:02:24,  4.81s/it]
 51%|█████     | 1595/3120 [2:13:02<2:03:07,  4.84s/it]
 51%|█████     | 1596/3120 [2:13:07<2:04:26,  4.90s/it]
                                                       
{'loss': 0.3947, 'learning_rate': 1.0445017933557404e-05, 'epoch': 2.05}

 51%|█████     | 1596/3120 [2:13:07<2:04:26,  4.90s/it]
 51%|█████     | 1597/3120 [2:13:12<2:05:33,  4.95s/it]
 51%|█████     | 1598/3120 [2:13:17<2:02:20,  4.82s/it]
                                                       
{'loss': 0.3663, 'learning_rate': 1.0423839619627854e-05, 'epoch': 2.05}

 51%|█████     | 1598/3120 [2:13:17<2:02:20,  4.82s/it]
 51%|█████▏    | 1599/3120 [2:13:22<2:02:59,  4.85s/it]
 51%|█████▏    | 1600/3120 [2:13:27<2:03:31,  4.88s/it]
                                                       
{'loss': 0.4022, 'learning_rate': 1.0402659401094154e-05, 'epoch': 2.05}

 51%|█████▏    | 1600/3120 [2:13:27<2:03:31,  4.88s/it]
 51%|█████▏    | 1601/3120 [2:13:31<2:04:12,  4.91s/it]
 51%|█████▏    | 1602/3120 [2:13:36<2:03:58,  4.90s/it]
                                                       
{'loss': 0.3824, 'learning_rate': 1.0381477373133652e-05, 'epoch': 2.05}

 51%|█████▏    | 1602/3120 [2:13:36<2:03:58,  4.90s/it]
 51%|█████▏    | 1603/3120 [2:13:42<2:06:58,  5.02s/it]
 51%|█████▏    | 1604/3120 [2:13:47<2:08:23,  5.08s/it]
                                                       
{'loss': 0.3954, 'learning_rate': 1.036029363093183e-05, 'epoch': 2.06}

 51%|█████▏    | 1604/3120 [2:13:47<2:08:23,  5.08s/it]
 51%|█████▏    | 1605/3120 [2:13:52<2:09:09,  5.11s/it]
 51%|█████▏    | 1606/3120 [2:13:57<2:04:04,  4.92s/it]
                                                       
{'loss': 0.4046, 'learning_rate': 1.0339108269681874e-05, 'epoch': 2.06}

 51%|█████▏    | 1606/3120 [2:13:57<2:04:04,  4.92s/it]
 52%|█████▏    | 1607/3120 [2:14:01<2:03:21,  4.89s/it]
 52%|█████▏    | 1608/3120 [2:14:06<2:01:29,  4.82s/it]
                                                       
{'loss': 0.4107, 'learning_rate': 1.0317921384584245e-05, 'epoch': 2.06}

 52%|█████▏    | 1608/3120 [2:14:06<2:01:29,  4.82s/it]
 52%|█████▏    | 1609/3120 [2:14:11<2:04:33,  4.95s/it]
 52%|█████▏    | 1610/3120 [2:14:16<2:02:38,  4.87s/it]
                                                       
{'loss': 0.3856, 'learning_rate': 1.0296733070846252e-05, 'epoch': 2.06}

 52%|█████▏    | 1610/3120 [2:14:16<2:02:38,  4.87s/it]
 52%|█████▏    | 1611/3120 [2:14:21<2:03:14,  4.90s/it]
 52%|█████▏    | 1612/3120 [2:14:26<2:07:45,  5.08s/it]
                                                       
{'loss': 0.375, 'learning_rate': 1.0275543423681622e-05, 'epoch': 2.07}

 52%|█████▏    | 1612/3120 [2:14:26<2:07:45,  5.08s/it]
 52%|█████▏    | 1613/3120 [2:14:31<2:03:45,  4.93s/it]
 52%|█████▏    | 1614/3120 [2:14:36<2:03:02,  4.90s/it]
                                                       
{'loss': 0.4128, 'learning_rate': 1.0254352538310075e-05, 'epoch': 2.07}

 52%|█████▏    | 1614/3120 [2:14:36<2:03:02,  4.90s/it]
 52%|█████▏    | 1615/3120 [2:14:41<2:06:09,  5.03s/it]
 52%|█████▏    | 1616/3120 [2:14:46<2:02:54,  4.90s/it]
                                                       
{'loss': 0.3887, 'learning_rate': 1.0233160509956893e-05, 'epoch': 2.07}

 52%|█████▏    | 1616/3120 [2:14:46<2:02:54,  4.90s/it]
 52%|█████▏    | 1617/3120 [2:14:51<2:04:33,  4.97s/it]
 52%|█████▏    | 1618/3120 [2:14:56<2:04:38,  4.98s/it]
                                                       
{'loss': 0.3704, 'learning_rate': 1.0211967433852501e-05, 'epoch': 2.07}

 52%|█████▏    | 1618/3120 [2:14:56<2:04:38,  4.98s/it]
 52%|█████▏    | 1619/3120 [2:15:01<2:05:12,  5.01s/it]
 52%|█████▏    | 1620/3120 [2:15:05<2:00:37,  4.82s/it]
                                                       
{'loss': 0.382, 'learning_rate': 1.0190773405232024e-05, 'epoch': 2.08}

 52%|█████▏    | 1620/3120 [2:15:05<2:00:37,  4.82s/it]
 52%|█████▏    | 1621/3120 [2:15:10<2:00:33,  4.83s/it]
 52%|█████▏    | 1622/3120 [2:15:15<2:01:40,  4.87s/it]
                                                       
{'loss': 0.3955, 'learning_rate': 1.0169578519334873e-05, 'epoch': 2.08}

 52%|█████▏    | 1622/3120 [2:15:15<2:01:40,  4.87s/it]
 52%|█████▏    | 1623/3120 [2:15:21<2:04:47,  5.00s/it]
 52%|█████▏    | 1624/3120 [2:15:26<2:08:06,  5.14s/it]
                                                       
{'loss': 0.3985, 'learning_rate': 1.014838287140431e-05, 'epoch': 2.08}

 52%|█████▏    | 1624/3120 [2:15:26<2:08:06,  5.14s/it]
 52%|█████▏    | 1625/3120 [2:15:31<2:05:40,  5.04s/it]
 52%|█████▏    | 1626/3120 [2:15:36<2:04:15,  4.99s/it]
                                                       
{'loss': 0.3869, 'learning_rate': 1.012718655668702e-05, 'epoch': 2.08}

 52%|█████▏    | 1626/3120 [2:15:36<2:04:15,  4.99s/it]
 52%|█████▏    | 1627/3120 [2:15:40<2:02:56,  4.94s/it]
 52%|█████▏    | 1628/3120 [2:15:46<2:05:35,  5.05s/it]
                                                       
{'loss': 0.393, 'learning_rate': 1.010598967043268e-05, 'epoch': 2.09}

 52%|█████▏    | 1628/3120 [2:15:46<2:05:35,  5.05s/it]
 52%|█████▏    | 1629/3120 [2:15:51<2:05:15,  5.04s/it]
 52%|█████▏    | 1630/3120 [2:15:56<2:05:58,  5.07s/it]
                                                       
{'loss': 0.3714, 'learning_rate': 1.0084792307893551e-05, 'epoch': 2.09}

 52%|█████▏    | 1630/3120 [2:15:56<2:05:58,  5.07s/it]
 52%|█████▏    | 1631/3120 [2:16:01<2:04:49,  5.03s/it]
 52%|█████▏    | 1632/3120 [2:16:06<2:05:20,  5.05s/it]
                                                       
{'loss': 0.3715, 'learning_rate': 1.0063594564324014e-05, 'epoch': 2.09}

 52%|█████▏    | 1632/3120 [2:16:06<2:05:20,  5.05s/it]
 52%|█████▏    | 1633/3120 [2:16:11<2:04:18,  5.02s/it]
 52%|█████▏    | 1634/3120 [2:16:16<2:07:30,  5.15s/it]
                                                       
{'loss': 0.3817, 'learning_rate': 1.0042396534980176e-05, 'epoch': 2.09}

 52%|█████▏    | 1634/3120 [2:16:16<2:07:30,  5.15s/it]
 52%|█████▏    | 1635/3120 [2:16:21<2:01:30,  4.91s/it]
 52%|█████▏    | 1636/3120 [2:16:25<2:00:12,  4.86s/it]
                                                       
{'loss': 0.394, 'learning_rate': 1.0021198315119426e-05, 'epoch': 2.1}

 52%|█████▏    | 1636/3120 [2:16:25<2:00:12,  4.86s/it]
 52%|█████▏    | 1637/3120 [2:16:31<2:02:46,  4.97s/it]
 52%|█████▎    | 1638/3120 [2:16:36<2:01:59,  4.94s/it]
                                                       
{'loss': 0.3823, 'learning_rate': 1e-05, 'epoch': 2.1}

 52%|█████▎    | 1638/3120 [2:16:36<2:01:59,  4.94s/it]
 53%|█████▎    | 1639/3120 [2:16:41<2:02:15,  4.95s/it]
 53%|█████▎    | 1640/3120 [2:16:45<2:01:37,  4.93s/it]
                                                       
{'loss': 0.3811, 'learning_rate': 9.97880168488058e-06, 'epoch': 2.1}

 53%|█████▎    | 1640/3120 [2:16:45<2:01:37,  4.93s/it]
 53%|█████▎    | 1641/3120 [2:16:50<2:00:52,  4.90s/it]
 53%|█████▎    | 1642/3120 [2:16:55<1:57:14,  4.76s/it]
                                                       
{'loss': 0.4121, 'learning_rate': 9.957603465019827e-06, 'epoch': 2.11}

 53%|█████▎    | 1642/3120 [2:16:55<1:57:14,  4.76s/it]
 53%|█████▎    | 1643/3120 [2:17:00<1:58:41,  4.82s/it]
 53%|█████▎    | 1644/3120 [2:17:04<1:57:36,  4.78s/it]
                                                       
{'loss': 0.3859, 'learning_rate': 9.936405435675991e-06, 'epoch': 2.11}

 53%|█████▎    | 1644/3120 [2:17:04<1:57:36,  4.78s/it]
 53%|█████▎    | 1645/3120 [2:17:10<2:02:31,  4.98s/it]
 53%|█████▎    | 1646/3120 [2:17:15<2:02:01,  4.97s/it]
                                                       
{'loss': 0.3806, 'learning_rate': 9.91520769210645e-06, 'epoch': 2.11}

 53%|█████▎    | 1646/3120 [2:17:15<2:02:01,  4.97s/it]
 53%|█████▎    | 1647/3120 [2:17:20<2:00:34,  4.91s/it]
 53%|█████▎    | 1648/3120 [2:17:25<2:01:18,  4.94s/it]
                                                       
{'loss': 0.385, 'learning_rate': 9.894010329567322e-06, 'epoch': 2.11}

 53%|█████▎    | 1648/3120 [2:17:25<2:01:18,  4.94s/it]
 53%|█████▎    | 1649/3120 [2:17:29<1:59:39,  4.88s/it]
 53%|█████▎    | 1650/3120 [2:17:35<2:02:33,  5.00s/it]
                                                       
{'loss': 0.3755, 'learning_rate': 9.872813443312984e-06, 'epoch': 2.12}

 53%|█████▎    | 1650/3120 [2:17:35<2:02:33,  5.00s/it]
 53%|█████▎    | 1651/3120 [2:17:40<2:02:23,  5.00s/it]
 53%|█████▎    | 1652/3120 [2:17:45<2:02:46,  5.02s/it]
                                                       
{'loss': 0.4136, 'learning_rate': 9.851617128595694e-06, 'epoch': 2.12}

 53%|█████▎    | 1652/3120 [2:17:45<2:02:46,  5.02s/it]
 53%|█████▎    | 1653/3120 [2:17:50<2:02:34,  5.01s/it]
 53%|█████▎    | 1654/3120 [2:17:55<2:02:02,  5.00s/it]
                                                       
{'loss': 0.3898, 'learning_rate': 9.830421480665128e-06, 'epoch': 2.12}

 53%|█████▎    | 1654/3120 [2:17:55<2:02:02,  5.00s/it]
 53%|█████▎    | 1655/3120 [2:17:59<1:57:53,  4.83s/it]
 53%|█████▎    | 1656/3120 [2:18:04<1:59:57,  4.92s/it]
                                                       
{'loss': 0.4021, 'learning_rate': 9.809226594767979e-06, 'epoch': 2.12}

 53%|█████▎    | 1656/3120 [2:18:04<1:59:57,  4.92s/it]
 53%|█████▎    | 1657/3120 [2:18:09<2:00:58,  4.96s/it]
 53%|█████▎    | 1658/3120 [2:18:14<1:59:13,  4.89s/it]
                                                       
{'loss': 0.3928, 'learning_rate': 9.788032566147506e-06, 'epoch': 2.13}

 53%|█████▎    | 1658/3120 [2:18:14<1:59:13,  4.89s/it]
 53%|█████▎    | 1659/3120 [2:18:19<1:59:01,  4.89s/it]
 53%|█████▎    | 1660/3120 [2:18:23<1:54:39,  4.71s/it]
                                                       
{'loss': 0.3639, 'learning_rate': 9.766839490043108e-06, 'epoch': 2.13}

 53%|█████▎    | 1660/3120 [2:18:23<1:54:39,  4.71s/it]
 53%|█████▎    | 1661/3120 [2:18:28<1:54:26,  4.71s/it]
 53%|█████▎    | 1662/3120 [2:18:33<1:55:32,  4.76s/it]
                                                       
{'loss': 0.3843, 'learning_rate': 9.745647461689932e-06, 'epoch': 2.13}

 53%|█████▎    | 1662/3120 [2:18:33<1:55:32,  4.76s/it]
 53%|█████▎    | 1663/3120 [2:18:38<2:00:58,  4.98s/it]
 53%|█████▎    | 1664/3120 [2:18:43<2:00:44,  4.98s/it]
                                                       
{'loss': 0.3798, 'learning_rate': 9.724456576318383e-06, 'epoch': 2.13}

 53%|█████▎    | 1664/3120 [2:18:43<2:00:44,  4.98s/it]
 53%|█████▎    | 1665/3120 [2:18:48<1:59:44,  4.94s/it]
 53%|█████▎    | 1666/3120 [2:18:53<2:00:55,  4.99s/it]
                                                       
{'loss': 0.3915, 'learning_rate': 9.703266929153751e-06, 'epoch': 2.14}

 53%|█████▎    | 1666/3120 [2:18:53<2:00:55,  4.99s/it]
 53%|█████▎    | 1667/3120 [2:18:59<2:04:25,  5.14s/it]
 53%|█████▎    | 1668/3120 [2:19:03<2:02:09,  5.05s/it]
                                                       
{'loss': 0.3685, 'learning_rate': 9.682078615415755e-06, 'epoch': 2.14}

 53%|█████▎    | 1668/3120 [2:19:03<2:02:09,  5.05s/it]
 53%|█████▎    | 1669/3120 [2:19:09<2:02:26,  5.06s/it]
 54%|█████▎    | 1670/3120 [2:19:14<2:02:41,  5.08s/it]
                                                       
{'loss': 0.4122, 'learning_rate': 9.660891730318129e-06, 'epoch': 2.14}

 54%|█████▎    | 1670/3120 [2:19:14<2:02:41,  5.08s/it]
 54%|█████▎    | 1671/3120 [2:19:18<2:00:28,  4.99s/it]
 54%|█████▎    | 1672/3120 [2:19:24<2:02:57,  5.10s/it]
                                                       
{'loss': 0.3752, 'learning_rate': 9.63970636906817e-06, 'epoch': 2.14}

 54%|█████▎    | 1672/3120 [2:19:24<2:02:57,  5.10s/it]
 54%|█████▎    | 1673/3120 [2:19:29<2:01:26,  5.04s/it]
 54%|█████▎    | 1674/3120 [2:19:33<1:59:14,  4.95s/it]
                                                       
{'loss': 0.3927, 'learning_rate': 9.618522626866351e-06, 'epoch': 2.15}

 54%|█████▎    | 1674/3120 [2:19:33<1:59:14,  4.95s/it]
 54%|█████▎    | 1675/3120 [2:19:38<1:59:50,  4.98s/it]
 54%|█████▎    | 1676/3120 [2:19:43<1:59:49,  4.98s/it]
                                                       
{'loss': 0.4052, 'learning_rate': 9.597340598905851e-06, 'epoch': 2.15}

 54%|█████▎    | 1676/3120 [2:19:43<1:59:49,  4.98s/it]
 54%|█████▍    | 1677/3120 [2:19:48<1:57:30,  4.89s/it]
 54%|█████▍    | 1678/3120 [2:19:53<1:58:10,  4.92s/it]
                                                       
{'loss': 0.3777, 'learning_rate': 9.57616038037215e-06, 'epoch': 2.15}

 54%|█████▍    | 1678/3120 [2:19:53<1:58:10,  4.92s/it]
 54%|█████▍    | 1679/3120 [2:19:59<2:02:22,  5.10s/it]
 54%|█████▍    | 1680/3120 [2:20:04<2:01:51,  5.08s/it]
                                                       
{'loss': 0.4091, 'learning_rate': 9.554982066442601e-06, 'epoch': 2.15}

 54%|█████▍    | 1680/3120 [2:20:04<2:01:51,  5.08s/it]
 54%|█████▍    | 1681/3120 [2:20:08<1:59:38,  4.99s/it]
 54%|█████▍    | 1682/3120 [2:20:14<2:00:39,  5.03s/it]
                                                       
{'loss': 0.3733, 'learning_rate': 9.533805752285992e-06, 'epoch': 2.16}

 54%|█████▍    | 1682/3120 [2:20:14<2:00:39,  5.03s/it]
 54%|█████▍    | 1683/3120 [2:20:18<1:58:08,  4.93s/it]
 54%|█████▍    | 1684/3120 [2:20:23<1:59:19,  4.99s/it]
                                                       
{'loss': 0.3721, 'learning_rate': 9.512631533062138e-06, 'epoch': 2.16}

 54%|█████▍    | 1684/3120 [2:20:23<1:59:19,  4.99s/it]
 54%|█████▍    | 1685/3120 [2:20:28<1:58:46,  4.97s/it]
 54%|█████▍    | 1686/3120 [2:20:33<1:58:00,  4.94s/it]
                                                       
{'loss': 0.3936, 'learning_rate': 9.49145950392142e-06, 'epoch': 2.16}

 54%|█████▍    | 1686/3120 [2:20:33<1:58:00,  4.94s/it]
 54%|█████▍    | 1687/3120 [2:20:38<1:57:18,  4.91s/it]
 54%|█████▍    | 1688/3120 [2:20:43<1:54:31,  4.80s/it]
                                                       
{'loss': 0.3851, 'learning_rate': 9.470289760004398e-06, 'epoch': 2.16}

 54%|█████▍    | 1688/3120 [2:20:43<1:54:31,  4.80s/it]
 54%|█████▍    | 1689/3120 [2:20:47<1:52:56,  4.74s/it]
 54%|█████▍    | 1690/3120 [2:20:52<1:51:39,  4.69s/it]
                                                       
{'loss': 0.3796, 'learning_rate': 9.449122396441344e-06, 'epoch': 2.17}

 54%|█████▍    | 1690/3120 [2:20:52<1:51:39,  4.69s/it]
 54%|█████▍    | 1691/3120 [2:20:57<1:55:51,  4.86s/it]
 54%|█████▍    | 1692/3120 [2:21:02<1:56:29,  4.89s/it]
                                                       
{'loss': 0.3848, 'learning_rate': 9.427957508351852e-06, 'epoch': 2.17}

 54%|█████▍    | 1692/3120 [2:21:02<1:56:29,  4.89s/it]
 54%|█████▍    | 1693/3120 [2:21:06<1:53:10,  4.76s/it]
 54%|█████▍    | 1694/3120 [2:21:11<1:53:37,  4.78s/it]
                                                       
{'loss': 0.3836, 'learning_rate': 9.406795190844376e-06, 'epoch': 2.17}

 54%|█████▍    | 1694/3120 [2:21:11<1:53:37,  4.78s/it]
 54%|█████▍    | 1695/3120 [2:21:16<1:53:07,  4.76s/it]
 54%|█████▍    | 1696/3120 [2:21:21<1:55:43,  4.88s/it]
                                                       
{'loss': 0.3959, 'learning_rate': 9.385635539015824e-06, 'epoch': 2.17}

 54%|█████▍    | 1696/3120 [2:21:21<1:55:43,  4.88s/it]
 54%|█████▍    | 1697/3120 [2:21:26<1:54:21,  4.82s/it]
 54%|█████▍    | 1698/3120 [2:21:31<1:55:43,  4.88s/it]
                                                       
{'loss': 0.4006, 'learning_rate': 9.364478647951132e-06, 'epoch': 2.18}

 54%|█████▍    | 1698/3120 [2:21:31<1:55:43,  4.88s/it]
 54%|█████▍    | 1699/3120 [2:21:36<1:55:28,  4.88s/it]
 54%|█████▍    | 1700/3120 [2:21:40<1:54:11,  4.83s/it]
                                                       
{'loss': 0.3844, 'learning_rate': 9.343324612722819e-06, 'epoch': 2.18}

 54%|█████▍    | 1700/3120 [2:21:40<1:54:11,  4.83s/it]
 55%|█████▍    | 1701/3120 [2:21:46<1:56:28,  4.92s/it]
 55%|█████▍    | 1702/3120 [2:21:51<1:58:26,  5.01s/it]
                                                       
{'loss': 0.4128, 'learning_rate': 9.32217352839058e-06, 'epoch': 2.18}

 55%|█████▍    | 1702/3120 [2:21:51<1:58:26,  5.01s/it]
 55%|█████▍    | 1703/3120 [2:21:55<1:55:57,  4.91s/it]
 55%|█████▍    | 1704/3120 [2:22:00<1:53:55,  4.83s/it]
                                                       
{'loss': 0.4061, 'learning_rate': 9.301025490000843e-06, 'epoch': 2.18}

 55%|█████▍    | 1704/3120 [2:22:00<1:53:55,  4.83s/it]
 55%|█████▍    | 1705/3120 [2:22:05<1:55:17,  4.89s/it]
 55%|█████▍    | 1706/3120 [2:22:10<1:55:10,  4.89s/it]
                                                       
{'loss': 0.3827, 'learning_rate': 9.279880592586355e-06, 'epoch': 2.19}

 55%|█████▍    | 1706/3120 [2:22:10<1:55:10,  4.89s/it]
 55%|█████▍    | 1707/3120 [2:22:15<1:57:06,  4.97s/it]
 55%|█████▍    | 1708/3120 [2:22:20<1:59:09,  5.06s/it]
                                                       
{'loss': 0.4085, 'learning_rate': 9.25873893116574e-06, 'epoch': 2.19}

 55%|█████▍    | 1708/3120 [2:22:20<1:59:09,  5.06s/it]
 55%|█████▍    | 1709/3120 [2:22:25<1:59:14,  5.07s/it]
 55%|█████▍    | 1710/3120 [2:22:31<2:00:35,  5.13s/it]
                                                       
{'loss': 0.3793, 'learning_rate': 9.237600600743086e-06, 'epoch': 2.19}

 55%|█████▍    | 1710/3120 [2:22:31<2:00:35,  5.13s/it]
 55%|█████▍    | 1711/3120 [2:22:35<1:56:43,  4.97s/it]
 55%|█████▍    | 1712/3120 [2:22:40<1:56:31,  4.97s/it]
                                                       
{'loss': 0.3724, 'learning_rate': 9.216465696307513e-06, 'epoch': 2.19}

 55%|█████▍    | 1712/3120 [2:22:40<1:56:31,  4.97s/it]
 55%|█████▍    | 1713/3120 [2:22:46<1:58:57,  5.07s/it]
 55%|█████▍    | 1714/3120 [2:22:51<2:00:25,  5.14s/it]
                                                       
{'loss': 0.3611, 'learning_rate': 9.195334312832742e-06, 'epoch': 2.2}

 55%|█████▍    | 1714/3120 [2:22:51<2:00:25,  5.14s/it]
 55%|█████▍    | 1715/3120 [2:22:56<2:00:18,  5.14s/it]
 55%|█████▌    | 1716/3120 [2:23:01<1:57:42,  5.03s/it]
                                                       
{'loss': 0.3698, 'learning_rate': 9.174206545276678e-06, 'epoch': 2.2}

 55%|█████▌    | 1716/3120 [2:23:01<1:57:42,  5.03s/it]
 55%|█████▌    | 1717/3120 [2:23:06<1:56:18,  4.97s/it]
 55%|█████▌    | 1718/3120 [2:23:11<1:56:28,  4.98s/it]
                                                       
{'loss': 0.3819, 'learning_rate': 9.153082488580969e-06, 'epoch': 2.2}

 55%|█████▌    | 1718/3120 [2:23:11<1:56:28,  4.98s/it]
 55%|█████▌    | 1719/3120 [2:23:16<1:55:48,  4.96s/it]
 55%|█████▌    | 1720/3120 [2:23:20<1:55:10,  4.94s/it]
                                                       
{'loss': 0.3903, 'learning_rate': 9.131962237670599e-06, 'epoch': 2.21}

 55%|█████▌    | 1720/3120 [2:23:20<1:55:10,  4.94s/it]
 55%|█████▌    | 1721/3120 [2:23:26<1:55:55,  4.97s/it]
 55%|█████▌    | 1722/3120 [2:23:30<1:53:30,  4.87s/it]
                                                       
{'loss': 0.3894, 'learning_rate': 9.11084588745343e-06, 'epoch': 2.21}

 55%|█████▌    | 1722/3120 [2:23:30<1:53:30,  4.87s/it]
 55%|█████▌    | 1723/3120 [2:23:35<1:54:20,  4.91s/it]
 55%|█████▌    | 1724/3120 [2:23:40<1:56:11,  4.99s/it]
                                                       
{'loss': 0.3942, 'learning_rate': 9.089733532819825e-06, 'epoch': 2.21}

 55%|█████▌    | 1724/3120 [2:23:40<1:56:11,  4.99s/it]
 55%|█████▌    | 1725/3120 [2:23:45<1:54:23,  4.92s/it]
 55%|█████▌    | 1726/3120 [2:23:50<1:53:49,  4.90s/it]
                                                       
{'loss': 0.3799, 'learning_rate': 9.068625268642162e-06, 'epoch': 2.21}

 55%|█████▌    | 1726/3120 [2:23:50<1:53:49,  4.90s/it]
 55%|█████▌    | 1727/3120 [2:23:55<1:58:10,  5.09s/it]
 55%|█████▌    | 1728/3120 [2:24:00<1:57:31,  5.07s/it]
                                                       
{'loss': 0.3966, 'learning_rate': 9.047521189774456e-06, 'epoch': 2.22}

 55%|█████▌    | 1728/3120 [2:24:00<1:57:31,  5.07s/it]
 55%|█████▌    | 1729/3120 [2:24:05<1:53:49,  4.91s/it]
 55%|█████▌    | 1730/3120 [2:24:10<1:56:22,  5.02s/it]
                                                       
{'loss': 0.3808, 'learning_rate': 9.026421391051908e-06, 'epoch': 2.22}

 55%|█████▌    | 1730/3120 [2:24:10<1:56:22,  5.02s/it]
 55%|█████▌    | 1731/3120 [2:24:16<1:58:24,  5.12s/it]
 56%|█████▌    | 1732/3120 [2:24:21<1:58:13,  5.11s/it]
                                                       
{'loss': 0.3848, 'learning_rate': 9.005325967290489e-06, 'epoch': 2.22}

 56%|█████▌    | 1732/3120 [2:24:21<1:58:13,  5.11s/it]
 56%|█████▌    | 1733/3120 [2:24:26<1:55:56,  5.02s/it]
 56%|█████▌    | 1734/3120 [2:24:30<1:54:15,  4.95s/it]
                                                       
{'loss': 0.4027, 'learning_rate': 8.984235013286512e-06, 'epoch': 2.22}

 56%|█████▌    | 1734/3120 [2:24:30<1:54:15,  4.95s/it]
 56%|█████▌    | 1735/3120 [2:24:35<1:53:20,  4.91s/it]
 56%|█████▌    | 1736/3120 [2:24:40<1:53:13,  4.91s/it]
                                                       
{'loss': 0.3943, 'learning_rate': 8.963148623816191e-06, 'epoch': 2.23}

 56%|█████▌    | 1736/3120 [2:24:40<1:53:13,  4.91s/it]
 56%|█████▌    | 1737/3120 [2:24:45<1:54:59,  4.99s/it]
 56%|█████▌    | 1738/3120 [2:24:50<1:53:25,  4.92s/it]
                                                       
{'loss': 0.3921, 'learning_rate': 8.942066893635245e-06, 'epoch': 2.23}

 56%|█████▌    | 1738/3120 [2:24:50<1:53:25,  4.92s/it]
 56%|█████▌    | 1739/3120 [2:24:55<1:54:33,  4.98s/it]
 56%|█████▌    | 1740/3120 [2:25:00<1:55:25,  5.02s/it]
                                                       
{'loss': 0.3841, 'learning_rate': 8.920989917478446e-06, 'epoch': 2.23}

 56%|█████▌    | 1740/3120 [2:25:00<1:55:25,  5.02s/it]
 56%|█████▌    | 1741/3120 [2:25:05<1:53:17,  4.93s/it]
 56%|█████▌    | 1742/3120 [2:25:10<1:52:19,  4.89s/it]
                                                       
{'loss': 0.3639, 'learning_rate': 8.899917790059208e-06, 'epoch': 2.23}

 56%|█████▌    | 1742/3120 [2:25:10<1:52:19,  4.89s/it]
 56%|█████▌    | 1743/3120 [2:25:14<1:50:59,  4.84s/it]
 56%|█████▌    | 1744/3120 [2:25:19<1:51:37,  4.87s/it]
                                                       
{'loss': 0.3924, 'learning_rate': 8.878850606069152e-06, 'epoch': 2.24}

 56%|█████▌    | 1744/3120 [2:25:19<1:51:37,  4.87s/it]
 56%|█████▌    | 1745/3120 [2:25:24<1:51:29,  4.87s/it]
 56%|█████▌    | 1746/3120 [2:25:29<1:49:14,  4.77s/it]
                                                       
{'loss': 0.4187, 'learning_rate': 8.857788460177685e-06, 'epoch': 2.24}

 56%|█████▌    | 1746/3120 [2:25:29<1:49:14,  4.77s/it]
 56%|█████▌    | 1747/3120 [2:25:34<1:48:54,  4.76s/it]
 56%|█████▌    | 1748/3120 [2:25:38<1:49:25,  4.79s/it]
                                                       
{'loss': 0.3605, 'learning_rate': 8.836731447031581e-06, 'epoch': 2.24}

 56%|█████▌    | 1748/3120 [2:25:38<1:49:25,  4.79s/it]
 56%|█████▌    | 1749/3120 [2:25:44<1:54:55,  5.03s/it]
 56%|█████▌    | 1750/3120 [2:25:49<1:54:25,  5.01s/it]
                                                       
{'loss': 0.3867, 'learning_rate': 8.815679661254538e-06, 'epoch': 2.24}

 56%|█████▌    | 1750/3120 [2:25:49<1:54:25,  5.01s/it]
 56%|█████▌    | 1751/3120 [2:25:54<1:54:28,  5.02s/it]
 56%|█████▌    | 1752/3120 [2:25:59<1:55:14,  5.05s/it]
                                                       
{'loss': 0.391, 'learning_rate': 8.79463319744677e-06, 'epoch': 2.25}

 56%|█████▌    | 1752/3120 [2:25:59<1:55:14,  5.05s/it]
 56%|█████▌    | 1753/3120 [2:26:04<1:54:00,  5.00s/it]
 56%|█████▌    | 1754/3120 [2:26:09<1:55:41,  5.08s/it]
                                                       
{'loss': 0.3901, 'learning_rate': 8.77359215018458e-06, 'epoch': 2.25}

 56%|█████▌    | 1754/3120 [2:26:09<1:55:41,  5.08s/it]
 56%|█████▋    | 1755/3120 [2:26:14<1:53:26,  4.99s/it]
 56%|█████▋    | 1756/3120 [2:26:19<1:52:42,  4.96s/it]
                                                       
{'loss': 0.3709, 'learning_rate': 8.752556614019924e-06, 'epoch': 2.25}

 56%|█████▋    | 1756/3120 [2:26:19<1:52:42,  4.96s/it]
 56%|█████▋    | 1757/3120 [2:26:24<1:53:23,  4.99s/it]
 56%|█████▋    | 1758/3120 [2:26:29<1:53:56,  5.02s/it]
                                                       
{'loss': 0.4087, 'learning_rate': 8.731526683479991e-06, 'epoch': 2.25}

 56%|█████▋    | 1758/3120 [2:26:29<1:53:56,  5.02s/it]
 56%|█████▋    | 1759/3120 [2:26:34<1:56:10,  5.12s/it]
 56%|█████▋    | 1760/3120 [2:26:39<1:54:55,  5.07s/it]
                                                       
{'loss': 0.3871, 'learning_rate': 8.710502453066791e-06, 'epoch': 2.26}

 56%|█████▋    | 1760/3120 [2:26:39<1:54:55,  5.07s/it]
 56%|█████▋    | 1761/3120 [2:26:45<1:58:38,  5.24s/it]
 56%|█████▋    | 1762/3120 [2:26:50<1:59:24,  5.28s/it]
                                                       
{'loss': 0.4008, 'learning_rate': 8.689484017256712e-06, 'epoch': 2.26}

 56%|█████▋    | 1762/3120 [2:26:50<1:59:24,  5.28s/it]
 57%|█████▋    | 1763/3120 [2:26:55<1:55:39,  5.11s/it]
 57%|█████▋    | 1764/3120 [2:27:00<1:57:09,  5.18s/it]
                                                       
{'loss': 0.3916, 'learning_rate': 8.668471470500094e-06, 'epoch': 2.26}

 57%|█████▋    | 1764/3120 [2:27:00<1:57:09,  5.18s/it]
 57%|█████▋    | 1765/3120 [2:27:06<1:59:15,  5.28s/it]
 57%|█████▋    | 1766/3120 [2:27:11<1:57:43,  5.22s/it]
                                                       
{'loss': 0.3945, 'learning_rate': 8.647464907220833e-06, 'epoch': 2.26}

 57%|█████▋    | 1766/3120 [2:27:11<1:57:43,  5.22s/it]
 57%|█████▋    | 1767/3120 [2:27:16<1:52:27,  4.99s/it]
 57%|█████▋    | 1768/3120 [2:27:21<1:54:25,  5.08s/it]
                                                       
{'loss': 0.3975, 'learning_rate': 8.626464421815919e-06, 'epoch': 2.27}

 57%|█████▋    | 1768/3120 [2:27:21<1:54:25,  5.08s/it]
 57%|█████▋    | 1769/3120 [2:27:25<1:50:45,  4.92s/it]
 57%|█████▋    | 1770/3120 [2:27:30<1:51:31,  4.96s/it]
                                                       
{'loss': 0.3856, 'learning_rate': 8.605470108655046e-06, 'epoch': 2.27}

 57%|█████▋    | 1770/3120 [2:27:30<1:51:31,  4.96s/it]
 57%|█████▋    | 1771/3120 [2:27:35<1:50:55,  4.93s/it]
 57%|█████▋    | 1772/3120 [2:27:40<1:48:42,  4.84s/it]
                                                       
{'loss': 0.3875, 'learning_rate': 8.584482062080154e-06, 'epoch': 2.27}

 57%|█████▋    | 1772/3120 [2:27:40<1:48:42,  4.84s/it]
 57%|█████▋    | 1773/3120 [2:27:45<1:51:06,  4.95s/it]
 57%|█████▋    | 1774/3120 [2:27:50<1:51:38,  4.98s/it]
                                                       
{'loss': 0.3793, 'learning_rate': 8.56350037640504e-06, 'epoch': 2.27}

 57%|█████▋    | 1774/3120 [2:27:50<1:51:38,  4.98s/it]
 57%|█████▋    | 1775/3120 [2:27:55<1:52:46,  5.03s/it]
 57%|█████▋    | 1776/3120 [2:28:00<1:49:44,  4.90s/it]
                                                       
{'loss': 0.3688, 'learning_rate': 8.542525145914907e-06, 'epoch': 2.28}

 57%|█████▋    | 1776/3120 [2:28:00<1:49:44,  4.90s/it]
 57%|█████▋    | 1777/3120 [2:28:05<1:49:12,  4.88s/it]
 57%|█████▋    | 1778/3120 [2:28:10<1:49:03,  4.88s/it]
                                                       
{'loss': 0.3756, 'learning_rate': 8.521556464865955e-06, 'epoch': 2.28}

 57%|█████▋    | 1778/3120 [2:28:10<1:49:03,  4.88s/it]
 57%|█████▋    | 1779/3120 [2:28:14<1:48:10,  4.84s/it]
 57%|█████▋    | 1780/3120 [2:28:19<1:49:59,  4.93s/it]
                                                       
{'loss': 0.3982, 'learning_rate': 8.500594427484946e-06, 'epoch': 2.28}

 57%|█████▋    | 1780/3120 [2:28:19<1:49:59,  4.93s/it]
 57%|█████▋    | 1781/3120 [2:28:25<1:51:46,  5.01s/it]
 57%|█████▋    | 1782/3120 [2:28:30<1:50:35,  4.96s/it]
                                                       
{'loss': 0.3684, 'learning_rate': 8.479639127968793e-06, 'epoch': 2.28}

 57%|█████▋    | 1782/3120 [2:28:30<1:50:35,  4.96s/it]
 57%|█████▋    | 1783/3120 [2:28:34<1:49:30,  4.91s/it]
 57%|█████▋    | 1784/3120 [2:28:39<1:48:36,  4.88s/it]
                                                       
{'loss': 0.3618, 'learning_rate': 8.458690660484134e-06, 'epoch': 2.29}

 57%|█████▋    | 1784/3120 [2:28:39<1:48:36,  4.88s/it]
 57%|█████▋    | 1785/3120 [2:28:44<1:50:37,  4.97s/it]
 57%|█████▋    | 1786/3120 [2:28:50<1:54:04,  5.13s/it]
                                                       
{'loss': 0.3696, 'learning_rate': 8.4377491191669e-06, 'epoch': 2.29}

 57%|█████▋    | 1786/3120 [2:28:50<1:54:04,  5.13s/it]
 57%|█████▋    | 1787/3120 [2:28:55<1:54:54,  5.17s/it]
 57%|█████▋    | 1788/3120 [2:29:00<1:54:33,  5.16s/it]
                                                       
{'loss': 0.3897, 'learning_rate': 8.416814598121901e-06, 'epoch': 2.29}

 57%|█████▋    | 1788/3120 [2:29:00<1:54:33,  5.16s/it]
 57%|█████▋    | 1789/3120 [2:29:05<1:55:02,  5.19s/it]
 57%|█████▋    | 1790/3120 [2:29:11<1:54:17,  5.16s/it]
                                                       
{'loss': 0.3597, 'learning_rate': 8.395887191422397e-06, 'epoch': 2.29}

 57%|█████▋    | 1790/3120 [2:29:11<1:54:17,  5.16s/it]
 57%|█████▋    | 1791/3120 [2:29:16<1:55:20,  5.21s/it]
 57%|█████▋    | 1792/3120 [2:29:21<1:54:17,  5.16s/it]
                                                       
{'loss': 0.3656, 'learning_rate': 8.374966993109689e-06, 'epoch': 2.3}

 57%|█████▋    | 1792/3120 [2:29:21<1:54:17,  5.16s/it]
 57%|█████▋    | 1793/3120 [2:29:25<1:50:11,  4.98s/it]
 57%|█████▊    | 1794/3120 [2:29:30<1:49:21,  4.95s/it]
                                                       
{'loss': 0.3702, 'learning_rate': 8.35405409719266e-06, 'epoch': 2.3}

 57%|█████▊    | 1794/3120 [2:29:30<1:49:21,  4.95s/it]
 58%|█████▊    | 1795/3120 [2:29:36<1:52:14,  5.08s/it]
 58%|█████▊    | 1796/3120 [2:29:40<1:48:20,  4.91s/it]
                                                       
{'loss': 0.3971, 'learning_rate': 8.333148597647414e-06, 'epoch': 2.3}

 58%|█████▊    | 1796/3120 [2:29:40<1:48:20,  4.91s/it]
 58%|█████▊    | 1797/3120 [2:29:45<1:49:09,  4.95s/it]
 58%|█████▊    | 1798/3120 [2:29:50<1:48:37,  4.93s/it]
                                                       
{'loss': 0.3746, 'learning_rate': 8.312250588416792e-06, 'epoch': 2.31}

 58%|█████▊    | 1798/3120 [2:29:50<1:48:37,  4.93s/it]
 58%|█████▊    | 1799/3120 [2:29:56<1:52:01,  5.09s/it]
 58%|█████▊    | 1800/3120 [2:30:01<1:54:43,  5.21s/it]
                                                       
{'loss': 0.3684, 'learning_rate': 8.291360163409978e-06, 'epoch': 2.31}

 58%|█████▊    | 1800/3120 [2:30:01<1:54:43,  5.21s/it]
 58%|█████▊    | 1801/3120 [2:30:06<1:55:13,  5.24s/it]
 58%|█████▊    | 1802/3120 [2:30:12<1:55:45,  5.27s/it]
                                                       
{'loss': 0.3889, 'learning_rate': 8.27047741650209e-06, 'epoch': 2.31}

 58%|█████▊    | 1802/3120 [2:30:12<1:55:45,  5.27s/it]
 58%|█████▊    | 1803/3120 [2:30:17<1:54:47,  5.23s/it]
 58%|█████▊    | 1804/3120 [2:30:22<1:53:19,  5.17s/it]
                                                       
{'loss': 0.4051, 'learning_rate': 8.249602441533727e-06, 'epoch': 2.31}

 58%|█████▊    | 1804/3120 [2:30:22<1:53:19,  5.17s/it]
 58%|█████▊    | 1805/3120 [2:30:27<1:54:06,  5.21s/it]
 58%|█████▊    | 1806/3120 [2:30:32<1:51:50,  5.11s/it]
                                                       
{'loss': 0.39, 'learning_rate': 8.228735332310575e-06, 'epoch': 2.32}

 58%|█████▊    | 1806/3120 [2:30:32<1:51:50,  5.11s/it]
 58%|█████▊    | 1807/3120 [2:30:38<1:54:24,  5.23s/it]
 58%|█████▊    | 1808/3120 [2:30:42<1:51:28,  5.10s/it]
                                                       
{'loss': 0.3554, 'learning_rate': 8.207876182602959e-06, 'epoch': 2.32}

 58%|█████▊    | 1808/3120 [2:30:42<1:51:28,  5.10s/it]
 58%|█████▊    | 1809/3120 [2:30:47<1:49:28,  5.01s/it]
 58%|█████▊    | 1810/3120 [2:30:52<1:47:31,  4.92s/it]
                                                       
{'loss': 0.3847, 'learning_rate': 8.187025086145458e-06, 'epoch': 2.32}

 58%|█████▊    | 1810/3120 [2:30:52<1:47:31,  4.92s/it]
 58%|█████▊    | 1811/3120 [2:30:57<1:48:31,  4.97s/it]
 58%|█████▊    | 1812/3120 [2:31:02<1:48:03,  4.96s/it]
                                                       
{'loss': 0.3708, 'learning_rate': 8.16618213663644e-06, 'epoch': 2.32}

 58%|█████▊    | 1812/3120 [2:31:02<1:48:03,  4.96s/it]
 58%|█████▊    | 1813/3120 [2:31:07<1:47:39,  4.94s/it]
 58%|█████▊    | 1814/3120 [2:31:12<1:48:17,  4.97s/it]
                                                       
{'loss': 0.3767, 'learning_rate': 8.14534742773768e-06, 'epoch': 2.33}

 58%|█████▊    | 1814/3120 [2:31:12<1:48:17,  4.97s/it]
 58%|█████▊    | 1815/3120 [2:31:17<1:46:21,  4.89s/it]
 58%|█████▊    | 1816/3120 [2:31:22<1:47:31,  4.95s/it]
                                                       
{'loss': 0.3775, 'learning_rate': 8.12452105307391e-06, 'epoch': 2.33}

 58%|█████▊    | 1816/3120 [2:31:22<1:47:31,  4.95s/it]
 58%|█████▊    | 1817/3120 [2:31:27<1:48:45,  5.01s/it]
 58%|█████▊    | 1818/3120 [2:31:32<1:46:27,  4.91s/it]
                                                       
{'loss': 0.4184, 'learning_rate': 8.103703106232416e-06, 'epoch': 2.33}

 58%|█████▊    | 1818/3120 [2:31:32<1:46:27,  4.91s/it]
 58%|█████▊    | 1819/3120 [2:31:36<1:46:46,  4.92s/it]
 58%|█████▊    | 1820/3120 [2:31:41<1:46:15,  4.90s/it]
                                                       
{'loss': 0.3827, 'learning_rate': 8.082893680762619e-06, 'epoch': 2.33}

 58%|█████▊    | 1820/3120 [2:31:41<1:46:15,  4.90s/it]
 58%|█████▊    | 1821/3120 [2:31:47<1:48:07,  4.99s/it]
 58%|█████▊    | 1822/3120 [2:31:51<1:45:26,  4.87s/it]
                                                       
{'loss': 0.3907, 'learning_rate': 8.062092870175629e-06, 'epoch': 2.34}

 58%|█████▊    | 1822/3120 [2:31:51<1:45:26,  4.87s/it]
 58%|█████▊    | 1823/3120 [2:31:57<1:51:06,  5.14s/it]
 58%|█████▊    | 1824/3120 [2:32:02<1:50:59,  5.14s/it]
                                                       
{'loss': 0.3817, 'learning_rate': 8.041300767943867e-06, 'epoch': 2.34}

 58%|█████▊    | 1824/3120 [2:32:02<1:50:59,  5.14s/it]
 58%|█████▊    | 1825/3120 [2:32:07<1:47:22,  4.98s/it]
 59%|█████▊    | 1826/3120 [2:32:11<1:44:01,  4.82s/it]
                                                       
{'loss': 0.3809, 'learning_rate': 8.0205174675006e-06, 'epoch': 2.34}

 59%|█████▊    | 1826/3120 [2:32:11<1:44:01,  4.82s/it]
 59%|█████▊    | 1827/3120 [2:32:16<1:46:41,  4.95s/it]
 59%|█████▊    | 1828/3120 [2:32:21<1:46:03,  4.93s/it]
                                                       
{'loss': 0.3952, 'learning_rate': 7.999743062239557e-06, 'epoch': 2.34}

 59%|█████▊    | 1828/3120 [2:32:21<1:46:03,  4.93s/it]
 59%|█████▊    | 1829/3120 [2:32:26<1:48:04,  5.02s/it]
 59%|█████▊    | 1830/3120 [2:32:31<1:47:50,  5.02s/it]
                                                       
{'loss': 0.3844, 'learning_rate': 7.978977645514488e-06, 'epoch': 2.35}

 59%|█████▊    | 1830/3120 [2:32:31<1:47:50,  5.02s/it]
 59%|█████▊    | 1831/3120 [2:32:37<1:48:19,  5.04s/it]
 59%|█████▊    | 1832/3120 [2:32:42<1:50:09,  5.13s/it]
                                                       
{'loss': 0.385, 'learning_rate': 7.958221310638749e-06, 'epoch': 2.35}

 59%|█████▊    | 1832/3120 [2:32:42<1:50:09,  5.13s/it]
 59%|█████▉    | 1833/3120 [2:32:47<1:51:24,  5.19s/it]
 59%|█████▉    | 1834/3120 [2:32:52<1:49:49,  5.12s/it]
                                                       
{'loss': 0.3926, 'learning_rate': 7.937474150884898e-06, 'epoch': 2.35}

 59%|█████▉    | 1834/3120 [2:32:52<1:49:49,  5.12s/it]
 59%|█████▉    | 1835/3120 [2:32:57<1:47:54,  5.04s/it]
 59%|█████▉    | 1836/3120 [2:33:02<1:48:04,  5.05s/it]
                                                       
{'loss': 0.3856, 'learning_rate': 7.916736259484239e-06, 'epoch': 2.35}

 59%|█████▉    | 1836/3120 [2:33:02<1:48:04,  5.05s/it]
 59%|█████▉    | 1837/3120 [2:33:07<1:49:04,  5.10s/it]
 59%|█████▉    | 1838/3120 [2:33:12<1:48:14,  5.07s/it]
                                                       
{'loss': 0.3861, 'learning_rate': 7.896007729626457e-06, 'epoch': 2.36}

 59%|█████▉    | 1838/3120 [2:33:12<1:48:14,  5.07s/it]
 59%|█████▉    | 1839/3120 [2:33:17<1:48:19,  5.07s/it]
 59%|█████▉    | 1840/3120 [2:33:22<1:45:21,  4.94s/it]
                                                       
{'loss': 0.3853, 'learning_rate': 7.875288654459144e-06, 'epoch': 2.36}

 59%|█████▉    | 1840/3120 [2:33:22<1:45:21,  4.94s/it]
 59%|█████▉    | 1841/3120 [2:33:27<1:47:39,  5.05s/it]
 59%|█████▉    | 1842/3120 [2:33:32<1:47:57,  5.07s/it]
                                                       
{'loss': 0.3897, 'learning_rate': 7.854579127087418e-06, 'epoch': 2.36}

 59%|█████▉    | 1842/3120 [2:33:32<1:47:57,  5.07s/it]
 59%|█████▉    | 1843/3120 [2:33:37<1:46:31,  5.00s/it]
 59%|█████▉    | 1844/3120 [2:33:43<1:47:44,  5.07s/it]
                                                       
{'loss': 0.3908, 'learning_rate': 7.833879240573487e-06, 'epoch': 2.36}

 59%|█████▉    | 1844/3120 [2:33:43<1:47:44,  5.07s/it]
 59%|█████▉    | 1845/3120 [2:33:48<1:47:14,  5.05s/it]
 59%|█████▉    | 1846/3120 [2:33:53<1:48:17,  5.10s/it]
                                                       
{'loss': 0.4056, 'learning_rate': 7.813189087936243e-06, 'epoch': 2.37}

 59%|█████▉    | 1846/3120 [2:33:53<1:48:17,  5.10s/it]
 59%|█████▉    | 1847/3120 [2:33:58<1:46:30,  5.02s/it]
 59%|█████▉    | 1848/3120 [2:34:03<1:46:25,  5.02s/it]
                                                       
{'loss': 0.378, 'learning_rate': 7.792508762150833e-06, 'epoch': 2.37}

 59%|█████▉    | 1848/3120 [2:34:03<1:46:25,  5.02s/it]
 59%|█████▉    | 1849/3120 [2:34:08<1:47:35,  5.08s/it]
 59%|█████▉    | 1850/3120 [2:34:13<1:46:27,  5.03s/it]
                                                       
{'loss': 0.3971, 'learning_rate': 7.771838356148231e-06, 'epoch': 2.37}

 59%|█████▉    | 1850/3120 [2:34:13<1:46:27,  5.03s/it]
 59%|█████▉    | 1851/3120 [2:34:17<1:40:53,  4.77s/it]
 59%|█████▉    | 1852/3120 [2:34:22<1:41:20,  4.80s/it]
                                                       
{'loss': 0.3823, 'learning_rate': 7.751177962814867e-06, 'epoch': 2.37}

 59%|█████▉    | 1852/3120 [2:34:22<1:41:20,  4.80s/it]
 59%|█████▉    | 1853/3120 [2:34:27<1:42:43,  4.86s/it]
 59%|█████▉    | 1854/3120 [2:34:32<1:42:03,  4.84s/it]
                                                       
{'loss': 0.3894, 'learning_rate': 7.730527674992143e-06, 'epoch': 2.38}

 59%|█████▉    | 1854/3120 [2:34:32<1:42:03,  4.84s/it]
 59%|█████▉    | 1855/3120 [2:34:36<1:41:39,  4.82s/it]
 59%|█████▉    | 1856/3120 [2:34:41<1:41:15,  4.81s/it]
                                                       
{'loss': 0.3829, 'learning_rate': 7.709887585476075e-06, 'epoch': 2.38}

 59%|█████▉    | 1856/3120 [2:34:41<1:41:15,  4.81s/it]
 60%|█████▉    | 1857/3120 [2:34:46<1:40:34,  4.78s/it]
 60%|█████▉    | 1858/3120 [2:34:51<1:41:19,  4.82s/it]
                                                       
{'loss': 0.3749, 'learning_rate': 7.689257787016834e-06, 'epoch': 2.38}

 60%|█████▉    | 1858/3120 [2:34:51<1:41:19,  4.82s/it]
 60%|█████▉    | 1859/3120 [2:34:56<1:41:56,  4.85s/it]
 60%|█████▉    | 1860/3120 [2:35:00<1:38:58,  4.71s/it]
                                                       
{'loss': 0.3979, 'learning_rate': 7.668638372318359e-06, 'epoch': 2.38}

 60%|█████▉    | 1860/3120 [2:35:00<1:38:58,  4.71s/it]
 60%|█████▉    | 1861/3120 [2:35:05<1:41:02,  4.82s/it]
 60%|█████▉    | 1862/3120 [2:35:10<1:42:28,  4.89s/it]
                                                       
{'loss': 0.3682, 'learning_rate': 7.648029434037915e-06, 'epoch': 2.39}

 60%|█████▉    | 1862/3120 [2:35:10<1:42:28,  4.89s/it]
 60%|█████▉    | 1863/3120 [2:35:15<1:42:48,  4.91s/it]
 60%|█████▉    | 1864/3120 [2:35:20<1:42:57,  4.92s/it]
                                                       
{'loss': 0.3408, 'learning_rate': 7.627431064785705e-06, 'epoch': 2.39}

 60%|█████▉    | 1864/3120 [2:35:20<1:42:57,  4.92s/it]
 60%|█████▉    | 1865/3120 [2:35:25<1:41:39,  4.86s/it]
 60%|█████▉    | 1866/3120 [2:35:29<1:40:36,  4.81s/it]
                                                       
{'loss': 0.3749, 'learning_rate': 7.606843357124426e-06, 'epoch': 2.39}

 60%|█████▉    | 1866/3120 [2:35:29<1:40:36,  4.81s/it]
 60%|█████▉    | 1867/3120 [2:35:34<1:39:35,  4.77s/it]
 60%|█████▉    | 1868/3120 [2:35:39<1:41:58,  4.89s/it]
                                                       
{'loss': 0.3857, 'learning_rate': 7.5862664035688604e-06, 'epoch': 2.39}

 60%|█████▉    | 1868/3120 [2:35:39<1:41:58,  4.89s/it]
 60%|█████▉    | 1869/3120 [2:35:44<1:42:42,  4.93s/it]
 60%|█████▉    | 1870/3120 [2:35:49<1:43:13,  4.95s/it]
                                                       
{'loss': 0.391, 'learning_rate': 7.565700296585483e-06, 'epoch': 2.4}

 60%|█████▉    | 1870/3120 [2:35:49<1:43:13,  4.95s/it]
 60%|█████▉    | 1871/3120 [2:35:54<1:41:15,  4.86s/it]
 60%|██████    | 1872/3120 [2:35:59<1:42:40,  4.94s/it]
                                                       
{'loss': 0.3709, 'learning_rate': 7.545145128592009e-06, 'epoch': 2.4}

 60%|██████    | 1872/3120 [2:35:59<1:42:40,  4.94s/it]
 60%|██████    | 1873/3120 [2:36:04<1:41:14,  4.87s/it]
 60%|██████    | 1874/3120 [2:36:09<1:42:42,  4.95s/it]
                                                       
{'loss': 0.3874, 'learning_rate': 7.524600991957013e-06, 'epoch': 2.4}

 60%|██████    | 1874/3120 [2:36:09<1:42:42,  4.95s/it]
 60%|██████    | 1875/3120 [2:36:14<1:44:57,  5.06s/it]
 60%|██████    | 1876/3120 [2:36:19<1:45:05,  5.07s/it]
                                                       
{'loss': 0.3881, 'learning_rate': 7.504067978999484e-06, 'epoch': 2.41}

 60%|██████    | 1876/3120 [2:36:19<1:45:05,  5.07s/it]
 60%|██████    | 1877/3120 [2:36:24<1:43:22,  4.99s/it]
 60%|██████    | 1878/3120 [2:36:29<1:44:47,  5.06s/it]
                                                       
{'loss': 0.3869, 'learning_rate': 7.483546181988437e-06, 'epoch': 2.41}

 60%|██████    | 1878/3120 [2:36:29<1:44:47,  5.06s/it]
 60%|██████    | 1879/3120 [2:36:34<1:44:00,  5.03s/it]
 60%|██████    | 1880/3120 [2:36:40<1:44:41,  5.07s/it]
                                                       
{'loss': 0.3638, 'learning_rate': 7.463035693142473e-06, 'epoch': 2.41}

 60%|██████    | 1880/3120 [2:36:40<1:44:41,  5.07s/it]
 60%|██████    | 1881/3120 [2:36:44<1:43:16,  5.00s/it]
 60%|██████    | 1882/3120 [2:36:49<1:43:47,  5.03s/it]
                                                       
{'loss': 0.3865, 'learning_rate': 7.442536604629396e-06, 'epoch': 2.41}

 60%|██████    | 1882/3120 [2:36:49<1:43:47,  5.03s/it]
 60%|██████    | 1883/3120 [2:36:54<1:41:36,  4.93s/it]
 60%|██████    | 1884/3120 [2:36:59<1:38:50,  4.80s/it]
                                                       
{'loss': 0.4102, 'learning_rate': 7.422049008565757e-06, 'epoch': 2.42}

 60%|██████    | 1884/3120 [2:36:59<1:38:50,  4.80s/it]
 60%|██████    | 1885/3120 [2:37:03<1:38:42,  4.80s/it]
 60%|██████    | 1886/3120 [2:37:08<1:37:40,  4.75s/it]
                                                       
{'loss': 0.4113, 'learning_rate': 7.40157299701648e-06, 'epoch': 2.42}

 60%|██████    | 1886/3120 [2:37:08<1:37:40,  4.75s/it]
 60%|██████    | 1887/3120 [2:37:13<1:39:17,  4.83s/it]
 61%|██████    | 1888/3120 [2:37:19<1:43:15,  5.03s/it]
                                                       
{'loss': 0.4165, 'learning_rate': 7.38110866199443e-06, 'epoch': 2.42}

 61%|██████    | 1888/3120 [2:37:19<1:43:15,  5.03s/it]
 61%|██████    | 1889/3120 [2:37:23<1:38:59,  4.83s/it]
 61%|██████    | 1890/3120 [2:37:28<1:38:03,  4.78s/it]
                                                       
{'loss': 0.3775, 'learning_rate': 7.360656095459995e-06, 'epoch': 2.42}

 61%|██████    | 1890/3120 [2:37:28<1:38:03,  4.78s/it]
 61%|██████    | 1891/3120 [2:37:33<1:40:24,  4.90s/it]
 61%|██████    | 1892/3120 [2:37:38<1:41:45,  4.97s/it]
                                                       
{'loss': 0.3756, 'learning_rate': 7.340215389320686e-06, 'epoch': 2.43}

 61%|██████    | 1892/3120 [2:37:38<1:41:45,  4.97s/it]
 61%|██████    | 1893/3120 [2:37:43<1:41:53,  4.98s/it]
 61%|██████    | 1894/3120 [2:37:48<1:41:42,  4.98s/it]
                                                       
{'loss': 0.3758, 'learning_rate': 7.31978663543071e-06, 'epoch': 2.43}

 61%|██████    | 1894/3120 [2:37:48<1:41:42,  4.98s/it]
 61%|██████    | 1895/3120 [2:37:53<1:44:35,  5.12s/it]
 61%|██████    | 1896/3120 [2:37:58<1:43:31,  5.07s/it]
                                                       
{'loss': 0.3839, 'learning_rate': 7.299369925590575e-06, 'epoch': 2.43}

 61%|██████    | 1896/3120 [2:37:58<1:43:31,  5.07s/it]
 61%|██████    | 1897/3120 [2:38:03<1:43:41,  5.09s/it]
 61%|██████    | 1898/3120 [2:38:08<1:41:34,  4.99s/it]
                                                       
{'loss': 0.3921, 'learning_rate': 7.278965351546648e-06, 'epoch': 2.43}

 61%|██████    | 1898/3120 [2:38:08<1:41:34,  4.99s/it]
 61%|██████    | 1899/3120 [2:38:13<1:40:01,  4.92s/it]
 61%|██████    | 1900/3120 [2:38:18<1:42:00,  5.02s/it]
                                                       
{'loss': 0.4005, 'learning_rate': 7.258573004990789e-06, 'epoch': 2.44}

 61%|██████    | 1900/3120 [2:38:18<1:42:00,  5.02s/it]
 61%|██████    | 1901/3120 [2:38:23<1:41:58,  5.02s/it]
 61%|██████    | 1902/3120 [2:38:28<1:41:04,  4.98s/it]
                                                       
{'loss': 0.3916, 'learning_rate': 7.2381929775598835e-06, 'epoch': 2.44}

 61%|██████    | 1902/3120 [2:38:28<1:41:04,  4.98s/it]
 61%|██████    | 1903/3120 [2:38:33<1:40:56,  4.98s/it]
 61%|██████    | 1904/3120 [2:38:38<1:41:13,  4.99s/it]
                                                       
{'loss': 0.3971, 'learning_rate': 7.217825360835475e-06, 'epoch': 2.44}

 61%|██████    | 1904/3120 [2:38:38<1:41:13,  4.99s/it]
 61%|██████    | 1905/3120 [2:38:43<1:39:26,  4.91s/it]
 61%|██████    | 1906/3120 [2:38:48<1:39:12,  4.90s/it]
                                                       
{'loss': 0.3681, 'learning_rate': 7.197470246343333e-06, 'epoch': 2.44}

 61%|██████    | 1906/3120 [2:38:48<1:39:12,  4.90s/it]
 61%|██████    | 1907/3120 [2:38:52<1:38:13,  4.86s/it]
 61%|██████    | 1908/3120 [2:38:57<1:38:58,  4.90s/it]
                                                       
{'loss': 0.3765, 'learning_rate': 7.1771277255530456e-06, 'epoch': 2.45}

 61%|██████    | 1908/3120 [2:38:57<1:38:58,  4.90s/it]
 61%|██████    | 1909/3120 [2:39:03<1:40:25,  4.98s/it]
 61%|██████    | 1910/3120 [2:39:08<1:40:26,  4.98s/it]
                                                       
{'loss': 0.381, 'learning_rate': 7.156797889877613e-06, 'epoch': 2.45}

 61%|██████    | 1910/3120 [2:39:08<1:40:26,  4.98s/it]
 61%|██████▏   | 1911/3120 [2:39:13<1:41:11,  5.02s/it]
 61%|██████▏   | 1912/3120 [2:39:18<1:40:06,  4.97s/it]
                                                       
{'loss': 0.3638, 'learning_rate': 7.136480830673018e-06, 'epoch': 2.45}

 61%|██████▏   | 1912/3120 [2:39:18<1:40:06,  4.97s/it]
 61%|██████▏   | 1913/3120 [2:39:23<1:40:23,  4.99s/it]
 61%|██████▏   | 1914/3120 [2:39:27<1:38:04,  4.88s/it]
                                                       
{'loss': 0.3783, 'learning_rate': 7.116176639237853e-06, 'epoch': 2.45}

 61%|██████▏   | 1914/3120 [2:39:27<1:38:04,  4.88s/it]
 61%|██████▏   | 1915/3120 [2:39:32<1:36:28,  4.80s/it]
 61%|██████▏   | 1916/3120 [2:39:37<1:36:57,  4.83s/it]
                                                       
{'loss': 0.3704, 'learning_rate': 7.095885406812866e-06, 'epoch': 2.46}

 61%|██████▏   | 1916/3120 [2:39:37<1:36:57,  4.83s/it]
 61%|██████▏   | 1917/3120 [2:39:42<1:38:24,  4.91s/it]
 61%|██████▏   | 1918/3120 [2:39:47<1:37:09,  4.85s/it]
                                                       
{'loss': 0.3908, 'learning_rate': 7.075607224580581e-06, 'epoch': 2.46}

 61%|██████▏   | 1918/3120 [2:39:47<1:37:09,  4.85s/it]
 62%|██████▏   | 1919/3120 [2:39:52<1:38:04,  4.90s/it]
 62%|██████▏   | 1920/3120 [2:39:57<1:38:44,  4.94s/it]
                                                       
{'loss': 0.3999, 'learning_rate': 7.05534218366488e-06, 'epoch': 2.46}

 62%|██████▏   | 1920/3120 [2:39:57<1:38:44,  4.94s/it]
 62%|██████▏   | 1921/3120 [2:40:01<1:37:05,  4.86s/it]
 62%|██████▏   | 1922/3120 [2:40:06<1:36:08,  4.81s/it]
                                                       
{'loss': 0.3768, 'learning_rate': 7.035090375130581e-06, 'epoch': 2.46}

 62%|██████▏   | 1922/3120 [2:40:06<1:36:08,  4.81s/it]
 62%|██████▏   | 1923/3120 [2:40:11<1:37:09,  4.87s/it]
 62%|██████▏   | 1924/3120 [2:40:16<1:38:04,  4.92s/it]
                                                       
{'loss': 0.3601, 'learning_rate': 7.014851889983058e-06, 'epoch': 2.47}

 62%|██████▏   | 1924/3120 [2:40:16<1:38:04,  4.92s/it]
 62%|██████▏   | 1925/3120 [2:40:21<1:38:31,  4.95s/it]
 62%|██████▏   | 1926/3120 [2:40:26<1:38:41,  4.96s/it]
                                                       
{'loss': 0.3643, 'learning_rate': 6.994626819167789e-06, 'epoch': 2.47}

 62%|██████▏   | 1926/3120 [2:40:26<1:38:41,  4.96s/it]
 62%|██████▏   | 1927/3120 [2:40:31<1:40:43,  5.07s/it]
 62%|██████▏   | 1928/3120 [2:40:36<1:40:27,  5.06s/it]
                                                       
{'loss': 0.3719, 'learning_rate': 6.974415253570003e-06, 'epoch': 2.47}

 62%|██████▏   | 1928/3120 [2:40:36<1:40:27,  5.06s/it]
 62%|██████▏   | 1929/3120 [2:40:42<1:42:19,  5.16s/it]
 62%|██████▏   | 1930/3120 [2:40:47<1:39:53,  5.04s/it]
                                                       
{'loss': 0.3831, 'learning_rate': 6.9542172840142105e-06, 'epoch': 2.47}

 62%|██████▏   | 1930/3120 [2:40:47<1:39:53,  5.04s/it]
 62%|██████▏   | 1931/3120 [2:40:52<1:39:31,  5.02s/it]
 62%|██████▏   | 1932/3120 [2:40:56<1:38:08,  4.96s/it]
                                                       
{'loss': 0.402, 'learning_rate': 6.934033001263847e-06, 'epoch': 2.48}

 62%|██████▏   | 1932/3120 [2:40:56<1:38:08,  4.96s/it]
 62%|██████▏   | 1933/3120 [2:41:02<1:39:54,  5.05s/it]
 62%|██████▏   | 1934/3120 [2:41:07<1:39:59,  5.06s/it]
                                                       
{'loss': 0.3967, 'learning_rate': 6.913862496020831e-06, 'epoch': 2.48}

 62%|██████▏   | 1934/3120 [2:41:07<1:39:59,  5.06s/it]
 62%|██████▏   | 1935/3120 [2:41:12<1:40:23,  5.08s/it]
 62%|██████▏   | 1936/3120 [2:41:17<1:40:15,  5.08s/it]
                                                       
{'loss': 0.3879, 'learning_rate': 6.893705858925179e-06, 'epoch': 2.48}

 62%|██████▏   | 1936/3120 [2:41:17<1:40:15,  5.08s/it]
 62%|██████▏   | 1937/3120 [2:41:22<1:40:16,  5.09s/it]
 62%|██████▏   | 1938/3120 [2:41:27<1:41:10,  5.14s/it]
                                                       
{'loss': 0.4073, 'learning_rate': 6.873563180554583e-06, 'epoch': 2.48}

 62%|██████▏   | 1938/3120 [2:41:27<1:41:10,  5.14s/it]
 62%|██████▏   | 1939/3120 [2:41:32<1:39:20,  5.05s/it]
 62%|██████▏   | 1940/3120 [2:41:37<1:37:34,  4.96s/it]
                                                       
{'loss': 0.3656, 'learning_rate': 6.853434551424001e-06, 'epoch': 2.49}

 62%|██████▏   | 1940/3120 [2:41:37<1:37:34,  4.96s/it]
 62%|██████▏   | 1941/3120 [2:41:42<1:39:45,  5.08s/it]
 62%|██████▏   | 1942/3120 [2:41:47<1:40:14,  5.11s/it]
                                                       
{'loss': 0.3959, 'learning_rate': 6.833320061985278e-06, 'epoch': 2.49}

 62%|██████▏   | 1942/3120 [2:41:47<1:40:14,  5.11s/it]
 62%|██████▏   | 1943/3120 [2:41:52<1:38:52,  5.04s/it]
 62%|██████▏   | 1944/3120 [2:41:57<1:37:16,  4.96s/it]
                                                       
{'loss': 0.3569, 'learning_rate': 6.813219802626698e-06, 'epoch': 2.49}

 62%|██████▏   | 1944/3120 [2:41:57<1:37:16,  4.96s/it]
 62%|██████▏   | 1945/3120 [2:42:02<1:38:35,  5.03s/it]
 62%|██████▏   | 1946/3120 [2:42:07<1:36:11,  4.92s/it]
                                                       
{'loss': 0.3647, 'learning_rate': 6.793133863672616e-06, 'epoch': 2.49}

 62%|██████▏   | 1946/3120 [2:42:07<1:36:11,  4.92s/it]
 62%|██████▏   | 1947/3120 [2:42:12<1:37:12,  4.97s/it]
 62%|██████▏   | 1948/3120 [2:42:17<1:39:05,  5.07s/it]
                                                       
{'loss': 0.3795, 'learning_rate': 6.773062335383024e-06, 'epoch': 2.5}

 62%|██████▏   | 1948/3120 [2:42:17<1:39:05,  5.07s/it]
 62%|██████▏   | 1949/3120 [2:42:22<1:37:23,  4.99s/it]
 62%|██████▎   | 1950/3120 [2:42:27<1:35:47,  4.91s/it]
                                                       
{'loss': 0.3523, 'learning_rate': 6.7530053079531664e-06, 'epoch': 2.5}

 62%|██████▎   | 1950/3120 [2:42:27<1:35:47,  4.91s/it]
 63%|██████▎   | 1951/3120 [2:42:32<1:34:58,  4.87s/it]
 63%|██████▎   | 1952/3120 [2:42:36<1:34:28,  4.85s/it]
                                                       
{'loss': 0.3683, 'learning_rate': 6.73296287151312e-06, 'epoch': 2.5}

 63%|██████▎   | 1952/3120 [2:42:36<1:34:28,  4.85s/it]
 63%|██████▎   | 1953/3120 [2:42:42<1:36:21,  4.95s/it]
 63%|██████▎   | 1954/3120 [2:42:47<1:38:41,  5.08s/it]
                                                       
{'loss': 0.3993, 'learning_rate': 6.712935116127389e-06, 'epoch': 2.51}

 63%|██████▎   | 1954/3120 [2:42:47<1:38:41,  5.08s/it]
 63%|██████▎   | 1955/3120 [2:42:52<1:37:34,  5.03s/it]
 63%|██████▎   | 1956/3120 [2:42:56<1:34:47,  4.89s/it]
                                                       
{'loss': 0.3766, 'learning_rate': 6.692922131794517e-06, 'epoch': 2.51}

 63%|██████▎   | 1956/3120 [2:42:56<1:34:47,  4.89s/it]
 63%|██████▎   | 1957/3120 [2:43:02<1:36:58,  5.00s/it]
 63%|██████▎   | 1958/3120 [2:43:06<1:34:30,  4.88s/it]
                                                       
{'loss': 0.3865, 'learning_rate': 6.672924008446663e-06, 'epoch': 2.51}

 63%|██████▎   | 1958/3120 [2:43:06<1:34:30,  4.88s/it]
 63%|██████▎   | 1959/3120 [2:43:12<1:36:28,  4.99s/it]
 63%|██████▎   | 1960/3120 [2:43:16<1:35:41,  4.95s/it]
                                                       
{'loss': 0.3695, 'learning_rate': 6.652940835949208e-06, 'epoch': 2.51}

 63%|██████▎   | 1960/3120 [2:43:16<1:35:41,  4.95s/it]
 63%|██████▎   | 1961/3120 [2:43:21<1:34:04,  4.87s/it]
 63%|██████▎   | 1962/3120 [2:43:26<1:35:59,  4.97s/it]
                                                       
{'loss': 0.3803, 'learning_rate': 6.632972704100349e-06, 'epoch': 2.52}

 63%|██████▎   | 1962/3120 [2:43:26<1:35:59,  4.97s/it]
 63%|██████▎   | 1963/3120 [2:43:31<1:35:37,  4.96s/it]
 63%|██████▎   | 1964/3120 [2:43:36<1:36:05,  4.99s/it]
                                                       
{'loss': 0.3885, 'learning_rate': 6.6130197026306945e-06, 'epoch': 2.52}

 63%|██████▎   | 1964/3120 [2:43:36<1:36:05,  4.99s/it]
 63%|██████▎   | 1965/3120 [2:43:42<1:38:22,  5.11s/it]
 63%|██████▎   | 1966/3120 [2:43:47<1:39:54,  5.19s/it]
                                                       
{'loss': 0.38, 'learning_rate': 6.59308192120286e-06, 'epoch': 2.52}

 63%|██████▎   | 1966/3120 [2:43:47<1:39:54,  5.19s/it]
 63%|██████▎   | 1967/3120 [2:43:52<1:40:42,  5.24s/it]
 63%|██████▎   | 1968/3120 [2:43:58<1:41:57,  5.31s/it]
                                                       
{'loss': 0.3645, 'learning_rate': 6.573159449411071e-06, 'epoch': 2.52}

 63%|██████▎   | 1968/3120 [2:43:58<1:41:57,  5.31s/it]
 63%|██████▎   | 1969/3120 [2:44:03<1:38:05,  5.11s/it]
 63%|██████▎   | 1970/3120 [2:44:08<1:39:16,  5.18s/it]
                                                       
{'loss': 0.3649, 'learning_rate': 6.553252376780749e-06, 'epoch': 2.53}

 63%|██████▎   | 1970/3120 [2:44:08<1:39:16,  5.18s/it]
 63%|██████▎   | 1971/3120 [2:44:13<1:37:42,  5.10s/it]
 63%|██████▎   | 1972/3120 [2:44:18<1:38:21,  5.14s/it]
                                                       
{'loss': 0.3767, 'learning_rate': 6.533360792768122e-06, 'epoch': 2.53}

 63%|██████▎   | 1972/3120 [2:44:18<1:38:21,  5.14s/it]
 63%|██████▎   | 1973/3120 [2:44:23<1:37:34,  5.10s/it]
 63%|██████▎   | 1974/3120 [2:44:28<1:35:58,  5.02s/it]
                                                       
{'loss': 0.3762, 'learning_rate': 6.513484786759818e-06, 'epoch': 2.53}

 63%|██████▎   | 1974/3120 [2:44:28<1:35:58,  5.02s/it]
 63%|██████▎   | 1975/3120 [2:44:33<1:35:18,  4.99s/it]
 63%|██████▎   | 1976/3120 [2:44:37<1:33:09,  4.89s/it]
                                                       
{'loss': 0.3885, 'learning_rate': 6.4936244480724575e-06, 'epoch': 2.53}

 63%|██████▎   | 1976/3120 [2:44:37<1:33:09,  4.89s/it]
 63%|██████▎   | 1977/3120 [2:44:43<1:34:12,  4.95s/it]
 63%|██████▎   | 1978/3120 [2:44:48<1:35:55,  5.04s/it]
                                                       
{'loss': 0.3901, 'learning_rate': 6.473779865952263e-06, 'epoch': 2.54}

 63%|██████▎   | 1978/3120 [2:44:48<1:35:55,  5.04s/it]
 63%|██████▎   | 1979/3120 [2:44:53<1:35:59,  5.05s/it]
 63%|██████▎   | 1980/3120 [2:44:58<1:34:16,  4.96s/it]
                                                       
{'loss': 0.3846, 'learning_rate': 6.453951129574644e-06, 'epoch': 2.54}

 63%|██████▎   | 1980/3120 [2:44:58<1:34:16,  4.96s/it]
 63%|██████▎   | 1981/3120 [2:45:03<1:34:18,  4.97s/it]
 64%|██████▎   | 1982/3120 [2:45:08<1:34:04,  4.96s/it]
                                                       
{'loss': 0.3772, 'learning_rate': 6.434138328043815e-06, 'epoch': 2.54}

 64%|██████▎   | 1982/3120 [2:45:08<1:34:04,  4.96s/it]
 64%|██████▎   | 1983/3120 [2:45:12<1:34:00,  4.96s/it]
 64%|██████▎   | 1984/3120 [2:45:17<1:33:46,  4.95s/it]
                                                       
{'loss': 0.3927, 'learning_rate': 6.4143415503923676e-06, 'epoch': 2.54}

 64%|██████▎   | 1984/3120 [2:45:17<1:33:46,  4.95s/it]
 64%|██████▎   | 1985/3120 [2:45:23<1:35:01,  5.02s/it]
 64%|██████▎   | 1986/3120 [2:45:28<1:35:22,  5.05s/it]
                                                       
{'loss': 0.3657, 'learning_rate': 6.39456088558091e-06, 'epoch': 2.55}

 64%|██████▎   | 1986/3120 [2:45:28<1:35:22,  5.05s/it]
 64%|██████▎   | 1987/3120 [2:45:33<1:34:18,  4.99s/it]
 64%|██████▎   | 1988/3120 [2:45:38<1:34:30,  5.01s/it]
                                                       
{'loss': 0.377, 'learning_rate': 6.374796422497622e-06, 'epoch': 2.55}

 64%|██████▎   | 1988/3120 [2:45:38<1:34:30,  5.01s/it]
 64%|██████▍   | 1989/3120 [2:45:43<1:37:49,  5.19s/it]
 64%|██████▍   | 1990/3120 [2:45:48<1:37:28,  5.18s/it]
                                                       
{'loss': 0.3522, 'learning_rate': 6.355048249957887e-06, 'epoch': 2.55}

 64%|██████▍   | 1990/3120 [2:45:48<1:37:28,  5.18s/it]
 64%|██████▍   | 1991/3120 [2:45:53<1:36:10,  5.11s/it]
 64%|██████▍   | 1992/3120 [2:45:58<1:34:17,  5.02s/it]
                                                       
{'loss': 0.378, 'learning_rate': 6.335316456703891e-06, 'epoch': 2.55}

 64%|██████▍   | 1992/3120 [2:45:58<1:34:17,  5.02s/it]
 64%|██████▍   | 1993/3120 [2:46:03<1:33:44,  4.99s/it]
 64%|██████▍   | 1994/3120 [2:46:08<1:33:46,  5.00s/it]
                                                       
{'loss': 0.386, 'learning_rate': 6.315601131404201e-06, 'epoch': 2.56}

 64%|██████▍   | 1994/3120 [2:46:08<1:33:46,  5.00s/it]
 64%|██████▍   | 1995/3120 [2:46:13<1:34:41,  5.05s/it]
 64%|██████▍   | 1996/3120 [2:46:18<1:32:36,  4.94s/it]
                                                       
{'loss': 0.3726, 'learning_rate': 6.295902362653401e-06, 'epoch': 2.56}

 64%|██████▍   | 1996/3120 [2:46:18<1:32:36,  4.94s/it]
 64%|██████▍   | 1997/3120 [2:46:23<1:32:01,  4.92s/it]
 64%|██████▍   | 1998/3120 [2:46:28<1:34:34,  5.06s/it]
                                                       
{'loss': 0.385, 'learning_rate': 6.276220238971653e-06, 'epoch': 2.56}

 64%|██████▍   | 1998/3120 [2:46:28<1:34:34,  5.06s/it]
 64%|██████▍   | 1999/3120 [2:46:33<1:35:35,  5.12s/it]
 64%|██████▍   | 2000/3120 [2:46:38<1:34:12,  5.05s/it]
                                                       
{'loss': 0.36, 'learning_rate': 6.256554848804343e-06, 'epoch': 2.56}

 64%|██████▍   | 2000/3120 [2:46:38<1:34:12,  5.05s/it]
 64%|██████▍   | 2001/3120 [2:46:43<1:30:56,  4.88s/it]
 64%|██████▍   | 2002/3120 [2:46:48<1:33:01,  4.99s/it]
                                                       
{'loss': 0.3847, 'learning_rate': 6.236906280521646e-06, 'epoch': 2.57}

 64%|██████▍   | 2002/3120 [2:46:48<1:33:01,  4.99s/it]
 64%|██████▍   | 2003/3120 [2:46:53<1:32:22,  4.96s/it]
 64%|██████▍   | 2004/3120 [2:46:58<1:32:01,  4.95s/it]
                                                       
{'loss': 0.3653, 'learning_rate': 6.2172746224181524e-06, 'epoch': 2.57}

 64%|██████▍   | 2004/3120 [2:46:58<1:32:01,  4.95s/it]
 64%|██████▍   | 2005/3120 [2:47:03<1:30:54,  4.89s/it]
 64%|██████▍   | 2006/3120 [2:47:08<1:31:59,  4.95s/it]
                                                       
{'loss': 0.3703, 'learning_rate': 6.197659962712461e-06, 'epoch': 2.57}

 64%|██████▍   | 2006/3120 [2:47:08<1:31:59,  4.95s/it]
 64%|██████▍   | 2007/3120 [2:47:13<1:33:44,  5.05s/it]
 64%|██████▍   | 2008/3120 [2:47:18<1:32:22,  4.98s/it]
                                                       
{'loss': 0.3693, 'learning_rate': 6.178062389546784e-06, 'epoch': 2.57}

 64%|██████▍   | 2008/3120 [2:47:18<1:32:22,  4.98s/it]
 64%|██████▍   | 2009/3120 [2:47:23<1:32:22,  4.99s/it]
 64%|██████▍   | 2010/3120 [2:47:28<1:32:42,  5.01s/it]
                                                       
{'loss': 0.3625, 'learning_rate': 6.158481990986558e-06, 'epoch': 2.58}

 64%|██████▍   | 2010/3120 [2:47:28<1:32:42,  5.01s/it]
 64%|██████▍   | 2011/3120 [2:47:33<1:32:16,  4.99s/it]
 64%|██████▍   | 2012/3120 [2:47:38<1:32:03,  4.98s/it]
                                                       
{'loss': 0.3673, 'learning_rate': 6.138918855020028e-06, 'epoch': 2.58}

 64%|██████▍   | 2012/3120 [2:47:38<1:32:03,  4.98s/it]
 65%|██████▍   | 2013/3120 [2:47:43<1:31:49,  4.98s/it]
 65%|██████▍   | 2014/3120 [2:47:48<1:33:36,  5.08s/it]
                                                       
{'loss': 0.3656, 'learning_rate': 6.119373069557889e-06, 'epoch': 2.58}

 65%|██████▍   | 2014/3120 [2:47:48<1:33:36,  5.08s/it]
 65%|██████▍   | 2015/3120 [2:47:53<1:33:11,  5.06s/it]
 65%|██████▍   | 2016/3120 [2:47:58<1:32:09,  5.01s/it]
                                                       
{'loss': 0.3595, 'learning_rate': 6.099844722432844e-06, 'epoch': 2.58}

 65%|██████▍   | 2016/3120 [2:47:58<1:32:09,  5.01s/it]
 65%|██████▍   | 2017/3120 [2:48:03<1:30:05,  4.90s/it]
 65%|██████▍   | 2018/3120 [2:48:08<1:33:18,  5.08s/it]
                                                       
{'loss': 0.3788, 'learning_rate': 6.080333901399252e-06, 'epoch': 2.59}

 65%|██████▍   | 2018/3120 [2:48:08<1:33:18,  5.08s/it]
 65%|██████▍   | 2019/3120 [2:48:13<1:31:40,  5.00s/it]
 65%|██████▍   | 2020/3120 [2:48:18<1:31:43,  5.00s/it]
                                                       
{'loss': 0.3646, 'learning_rate': 6.060840694132701e-06, 'epoch': 2.59}

 65%|██████▍   | 2020/3120 [2:48:18<1:31:43,  5.00s/it]
 65%|██████▍   | 2021/3120 [2:48:23<1:32:41,  5.06s/it]
 65%|██████▍   | 2022/3120 [2:48:28<1:31:11,  4.98s/it]
                                                       
{'loss': 0.3802, 'learning_rate': 6.041365188229641e-06, 'epoch': 2.59}

 65%|██████▍   | 2022/3120 [2:48:28<1:31:11,  4.98s/it]
 65%|██████▍   | 2023/3120 [2:48:33<1:31:53,  5.03s/it]
 65%|██████▍   | 2024/3120 [2:48:38<1:31:35,  5.01s/it]
                                                       
{'loss': 0.3416, 'learning_rate': 6.021907471206971e-06, 'epoch': 2.59}

 65%|██████▍   | 2024/3120 [2:48:38<1:31:35,  5.01s/it]
 65%|██████▍   | 2025/3120 [2:48:43<1:30:43,  4.97s/it]
 65%|██████▍   | 2026/3120 [2:48:48<1:29:37,  4.92s/it]
                                                       
{'loss': 0.3886, 'learning_rate': 6.002467630501646e-06, 'epoch': 2.6}

 65%|██████▍   | 2026/3120 [2:48:48<1:29:37,  4.92s/it]
 65%|██████▍   | 2027/3120 [2:48:52<1:28:30,  4.86s/it]
 65%|██████▌   | 2028/3120 [2:48:57<1:28:21,  4.86s/it]
                                                       
{'loss': 0.3918, 'learning_rate': 5.983045753470308e-06, 'epoch': 2.6}

 65%|██████▌   | 2028/3120 [2:48:57<1:28:21,  4.86s/it]
 65%|██████▌   | 2029/3120 [2:49:02<1:28:30,  4.87s/it]
 65%|██████▌   | 2030/3120 [2:49:07<1:26:30,  4.76s/it]
                                                       
{'loss': 0.3844, 'learning_rate': 5.963641927388854e-06, 'epoch': 2.6}

 65%|██████▌   | 2030/3120 [2:49:07<1:26:30,  4.76s/it]
 65%|██████▌   | 2031/3120 [2:49:12<1:28:18,  4.87s/it]
 65%|██████▌   | 2032/3120 [2:49:17<1:28:24,  4.88s/it]
                                                       
{'loss': 0.3844, 'learning_rate': 5.944256239452085e-06, 'epoch': 2.61}

 65%|██████▌   | 2032/3120 [2:49:17<1:28:24,  4.88s/it]
 65%|██████▌   | 2033/3120 [2:49:21<1:27:23,  4.82s/it]
 65%|██████▌   | 2034/3120 [2:49:27<1:30:03,  4.98s/it]
                                                       
{'loss': 0.3726, 'learning_rate': 5.924888776773281e-06, 'epoch': 2.61}

 65%|██████▌   | 2034/3120 [2:49:27<1:30:03,  4.98s/it]
 65%|██████▌   | 2035/3120 [2:49:32<1:31:02,  5.03s/it]
 65%|██████▌   | 2036/3120 [2:49:37<1:31:44,  5.08s/it]
                                                       
{'loss': 0.3803, 'learning_rate': 5.905539626383831e-06, 'epoch': 2.61}

 65%|██████▌   | 2036/3120 [2:49:37<1:31:44,  5.08s/it]
 65%|██████▌   | 2037/3120 [2:49:43<1:34:09,  5.22s/it]
 65%|██████▌   | 2038/3120 [2:49:47<1:30:44,  5.03s/it]
                                                       
{'loss': 0.3422, 'learning_rate': 5.886208875232833e-06, 'epoch': 2.61}

 65%|██████▌   | 2038/3120 [2:49:47<1:30:44,  5.03s/it]
 65%|██████▌   | 2039/3120 [2:49:52<1:31:38,  5.09s/it]
 65%|██████▌   | 2040/3120 [2:49:57<1:29:40,  4.98s/it]
                                                       
{'loss': 0.3681, 'learning_rate': 5.866896610186701e-06, 'epoch': 2.62}

 65%|██████▌   | 2040/3120 [2:49:57<1:29:40,  4.98s/it]
 65%|██████▌   | 2041/3120 [2:50:02<1:28:09,  4.90s/it]
 65%|██████▌   | 2042/3120 [2:50:07<1:27:11,  4.85s/it]
                                                       
{'loss': 0.3803, 'learning_rate': 5.847602918028785e-06, 'epoch': 2.62}

 65%|██████▌   | 2042/3120 [2:50:07<1:27:11,  4.85s/it]
 65%|██████▌   | 2043/3120 [2:50:12<1:29:12,  4.97s/it]
 66%|██████▌   | 2044/3120 [2:50:17<1:29:41,  5.00s/it]
                                                       
{'loss': 0.366, 'learning_rate': 5.82832788545896e-06, 'epoch': 2.62}

 66%|██████▌   | 2044/3120 [2:50:17<1:29:41,  5.00s/it]
 66%|██████▌   | 2045/3120 [2:50:22<1:29:21,  4.99s/it]
 66%|██████▌   | 2046/3120 [2:50:27<1:29:45,  5.01s/it]
                                                       
{'loss': 0.3866, 'learning_rate': 5.809071599093272e-06, 'epoch': 2.62}

 66%|██████▌   | 2046/3120 [2:50:27<1:29:45,  5.01s/it]
 66%|██████▌   | 2047/3120 [2:50:32<1:31:50,  5.14s/it]
 66%|██████▌   | 2048/3120 [2:50:37<1:29:32,  5.01s/it]
                                                       
{'loss': 0.3796, 'learning_rate': 5.789834145463506e-06, 'epoch': 2.63}

 66%|██████▌   | 2048/3120 [2:50:37<1:29:32,  5.01s/it]
 66%|██████▌   | 2049/3120 [2:50:42<1:30:06,  5.05s/it]
 66%|██████▌   | 2050/3120 [2:50:47<1:30:27,  5.07s/it]
                                                       
{'loss': 0.4013, 'learning_rate': 5.7706156110168384e-06, 'epoch': 2.63}

 66%|██████▌   | 2050/3120 [2:50:47<1:30:27,  5.07s/it]
 66%|██████▌   | 2051/3120 [2:50:53<1:31:22,  5.13s/it]
 66%|██████▌   | 2052/3120 [2:50:58<1:31:46,  5.16s/it]
                                                       
{'loss': 0.3908, 'learning_rate': 5.7514160821154085e-06, 'epoch': 2.63}

 66%|██████▌   | 2052/3120 [2:50:58<1:31:46,  5.16s/it]
 66%|██████▌   | 2053/3120 [2:51:03<1:33:30,  5.26s/it]
 66%|██████▌   | 2054/3120 [2:51:09<1:33:20,  5.25s/it]
                                                       
{'loss': 0.3768, 'learning_rate': 5.732235645035964e-06, 'epoch': 2.63}

 66%|██████▌   | 2054/3120 [2:51:09<1:33:20,  5.25s/it]
 66%|██████▌   | 2055/3120 [2:51:14<1:32:26,  5.21s/it]
 66%|██████▌   | 2056/3120 [2:51:19<1:31:03,  5.14s/it]
                                                       
{'loss': 0.3917, 'learning_rate': 5.713074385969457e-06, 'epoch': 2.64}

 66%|██████▌   | 2056/3120 [2:51:19<1:31:03,  5.14s/it]
 66%|██████▌   | 2057/3120 [2:51:23<1:29:02,  5.03s/it]
 66%|██████▌   | 2058/3120 [2:51:28<1:28:59,  5.03s/it]
                                                       
{'loss': 0.3823, 'learning_rate': 5.693932391020664e-06, 'epoch': 2.64}

 66%|██████▌   | 2058/3120 [2:51:28<1:28:59,  5.03s/it]
 66%|██████▌   | 2059/3120 [2:51:34<1:30:02,  5.09s/it]
 66%|██████▌   | 2060/3120 [2:51:38<1:27:51,  4.97s/it]
                                                       
{'loss': 0.3765, 'learning_rate': 5.67480974620778e-06, 'epoch': 2.64}

 66%|██████▌   | 2060/3120 [2:51:38<1:27:51,  4.97s/it]
 66%|██████▌   | 2061/3120 [2:51:43<1:27:51,  4.98s/it]
 66%|██████▌   | 2062/3120 [2:51:48<1:27:50,  4.98s/it]
                                                       
{'loss': 0.3858, 'learning_rate': 5.65570653746206e-06, 'epoch': 2.64}

 66%|██████▌   | 2062/3120 [2:51:48<1:27:50,  4.98s/it]
 66%|██████▌   | 2063/3120 [2:51:54<1:29:44,  5.09s/it]
 66%|██████▌   | 2064/3120 [2:51:58<1:26:19,  4.90s/it]
                                                       
{'loss': 0.3673, 'learning_rate': 5.63662285062742e-06, 'epoch': 2.65}

 66%|██████▌   | 2064/3120 [2:51:58<1:26:19,  4.90s/it]
 66%|██████▌   | 2065/3120 [2:52:03<1:26:56,  4.94s/it]
 66%|██████▌   | 2066/3120 [2:52:09<1:29:23,  5.09s/it]
                                                       
{'loss': 0.3838, 'learning_rate': 5.617558771460039e-06, 'epoch': 2.65}

 66%|██████▌   | 2066/3120 [2:52:09<1:29:23,  5.09s/it]
 66%|██████▋   | 2067/3120 [2:52:14<1:28:33,  5.05s/it]
 66%|██████▋   | 2068/3120 [2:52:18<1:26:17,  4.92s/it]
                                                       
{'loss': 0.3876, 'learning_rate': 5.598514385627997e-06, 'epoch': 2.65}

 66%|██████▋   | 2068/3120 [2:52:18<1:26:17,  4.92s/it]
 66%|██████▋   | 2069/3120 [2:52:23<1:26:36,  4.94s/it]
 66%|██████▋   | 2070/3120 [2:52:28<1:26:21,  4.93s/it]
                                                       
{'loss': 0.3825, 'learning_rate': 5.579489778710867e-06, 'epoch': 2.65}

 66%|██████▋   | 2070/3120 [2:52:28<1:26:21,  4.93s/it]
 66%|██████▋   | 2071/3120 [2:52:33<1:25:19,  4.88s/it]
 66%|██████▋   | 2072/3120 [2:52:38<1:25:58,  4.92s/it]
                                                       
{'loss': 0.3756, 'learning_rate': 5.56048503619935e-06, 'epoch': 2.66}

 66%|██████▋   | 2072/3120 [2:52:38<1:25:58,  4.92s/it]
 66%|██████▋   | 2073/3120 [2:52:43<1:27:45,  5.03s/it]
 66%|██████▋   | 2074/3120 [2:52:48<1:28:01,  5.05s/it]
                                                       
{'loss': 0.3673, 'learning_rate': 5.541500243494889e-06, 'epoch': 2.66}

 66%|██████▋   | 2074/3120 [2:52:48<1:28:01,  5.05s/it]
 67%|██████▋   | 2075/3120 [2:52:53<1:24:38,  4.86s/it]
 67%|██████▋   | 2076/3120 [2:52:57<1:23:53,  4.82s/it]
                                                       
{'loss': 0.3816, 'learning_rate': 5.522535485909258e-06, 'epoch': 2.66}

 67%|██████▋   | 2076/3120 [2:52:57<1:23:53,  4.82s/it]
 67%|██████▋   | 2077/3120 [2:53:02<1:23:47,  4.82s/it]
 67%|██████▋   | 2078/3120 [2:53:08<1:27:12,  5.02s/it]
                                                       
{'loss': 0.3637, 'learning_rate': 5.5035908486642195e-06, 'epoch': 2.66}

 67%|██████▋   | 2078/3120 [2:53:08<1:27:12,  5.02s/it]
 67%|██████▋   | 2079/3120 [2:53:13<1:26:32,  4.99s/it]
 67%|██████▋   | 2080/3120 [2:53:18<1:25:57,  4.96s/it]
                                                       
{'loss': 0.3577, 'learning_rate': 5.484666416891109e-06, 'epoch': 2.67}

 67%|██████▋   | 2080/3120 [2:53:18<1:25:57,  4.96s/it]
 67%|██████▋   | 2081/3120 [2:53:22<1:23:45,  4.84s/it]
 67%|██████▋   | 2082/3120 [2:53:27<1:25:03,  4.92s/it]
                                                       
{'loss': 0.3672, 'learning_rate': 5.465762275630471e-06, 'epoch': 2.67}

 67%|██████▋   | 2082/3120 [2:53:27<1:25:03,  4.92s/it]
 67%|██████▋   | 2083/3120 [2:53:32<1:26:38,  5.01s/it]
 67%|██████▋   | 2084/3120 [2:53:38<1:27:58,  5.09s/it]
                                                       
{'loss': 0.3735, 'learning_rate': 5.446878509831668e-06, 'epoch': 2.67}

 67%|██████▋   | 2084/3120 [2:53:38<1:27:58,  5.09s/it]
 67%|██████▋   | 2085/3120 [2:53:42<1:24:44,  4.91s/it]
 67%|██████▋   | 2086/3120 [2:53:47<1:25:02,  4.93s/it]
                                                       
{'loss': 0.3645, 'learning_rate': 5.428015204352508e-06, 'epoch': 2.67}

 67%|██████▋   | 2086/3120 [2:53:47<1:25:02,  4.93s/it]
 67%|██████▋   | 2087/3120 [2:53:53<1:28:07,  5.12s/it]
 67%|██████▋   | 2088/3120 [2:53:58<1:28:29,  5.15s/it]
                                                       
{'loss': 0.3621, 'learning_rate': 5.409172443958844e-06, 'epoch': 2.68}

 67%|██████▋   | 2088/3120 [2:53:58<1:28:29,  5.15s/it]
 67%|██████▋   | 2089/3120 [2:54:03<1:26:33,  5.04s/it]
 67%|██████▋   | 2090/3120 [2:54:08<1:25:04,  4.96s/it]
                                                       
{'loss': 0.3423, 'learning_rate': 5.390350313324214e-06, 'epoch': 2.68}

 67%|██████▋   | 2090/3120 [2:54:08<1:25:04,  4.96s/it]
 67%|██████▋   | 2091/3120 [2:54:12<1:23:37,  4.88s/it]
 67%|██████▋   | 2092/3120 [2:54:17<1:25:05,  4.97s/it]
                                                       
{'loss': 0.3821, 'learning_rate': 5.371548897029457e-06, 'epoch': 2.68}

 67%|██████▋   | 2092/3120 [2:54:17<1:25:05,  4.97s/it]
 67%|██████▋   | 2093/3120 [2:54:22<1:24:36,  4.94s/it]
 67%|██████▋   | 2094/3120 [2:54:28<1:26:14,  5.04s/it]
                                                       
{'loss': 0.353, 'learning_rate': 5.352768279562315e-06, 'epoch': 2.68}

 67%|██████▋   | 2094/3120 [2:54:28<1:26:14,  5.04s/it]
 67%|██████▋   | 2095/3120 [2:54:33<1:26:47,  5.08s/it]
 67%|██████▋   | 2096/3120 [2:54:38<1:25:52,  5.03s/it]
                                                       
{'loss': 0.378, 'learning_rate': 5.334008545317082e-06, 'epoch': 2.69}

 67%|██████▋   | 2096/3120 [2:54:38<1:25:52,  5.03s/it]
 67%|██████▋   | 2097/3120 [2:54:43<1:25:48,  5.03s/it]
 67%|██████▋   | 2098/3120 [2:54:48<1:25:50,  5.04s/it]
                                                       
{'loss': 0.352, 'learning_rate': 5.315269778594189e-06, 'epoch': 2.69}

 67%|██████▋   | 2098/3120 [2:54:48<1:25:50,  5.04s/it]
 67%|██████▋   | 2099/3120 [2:54:53<1:27:59,  5.17s/it]
 67%|██████▋   | 2100/3120 [2:54:58<1:25:15,  5.02s/it]
                                                       
{'loss': 0.3678, 'learning_rate': 5.2965520635998676e-06, 'epoch': 2.69}

 67%|██████▋   | 2100/3120 [2:54:58<1:25:15,  5.02s/it]
 67%|██████▋   | 2101/3120 [2:55:03<1:24:08,  4.95s/it]
 67%|██████▋   | 2102/3120 [2:55:07<1:22:39,  4.87s/it]
                                                       
{'loss': 0.3559, 'learning_rate': 5.277855484445736e-06, 'epoch': 2.69}

 67%|██████▋   | 2102/3120 [2:55:07<1:22:39,  4.87s/it]
 67%|██████▋   | 2103/3120 [2:55:12<1:20:58,  4.78s/it]
 67%|██████▋   | 2104/3120 [2:55:17<1:22:13,  4.86s/it]
                                                       
{'loss': 0.3828, 'learning_rate': 5.259180125148442e-06, 'epoch': 2.7}

 67%|██████▋   | 2104/3120 [2:55:17<1:22:13,  4.86s/it]
 67%|██████▋   | 2105/3120 [2:55:22<1:22:12,  4.86s/it]
 68%|██████▊   | 2106/3120 [2:55:27<1:23:10,  4.92s/it]
                                                       
{'loss': 0.3932, 'learning_rate': 5.240526069629265e-06, 'epoch': 2.7}

 68%|██████▊   | 2106/3120 [2:55:27<1:23:10,  4.92s/it]
 68%|██████▊   | 2107/3120 [2:55:32<1:23:49,  4.96s/it]
 68%|██████▊   | 2108/3120 [2:55:37<1:24:43,  5.02s/it]
                                                       
{'loss': 0.3874, 'learning_rate': 5.22189340171377e-06, 'epoch': 2.7}

 68%|██████▊   | 2108/3120 [2:55:37<1:24:43,  5.02s/it]
 68%|██████▊   | 2109/3120 [2:55:42<1:24:46,  5.03s/it]
 68%|██████▊   | 2110/3120 [2:55:48<1:26:18,  5.13s/it]
                                                       
{'loss': 0.3908, 'learning_rate': 5.203282205131395e-06, 'epoch': 2.71}

 68%|██████▊   | 2110/3120 [2:55:48<1:26:18,  5.13s/it]
 68%|██████▊   | 2111/3120 [2:55:53<1:27:54,  5.23s/it]
 68%|██████▊   | 2112/3120 [2:55:58<1:26:47,  5.17s/it]
                                                       
{'loss': 0.3731, 'learning_rate': 5.184692563515104e-06, 'epoch': 2.71}

 68%|██████▊   | 2112/3120 [2:55:58<1:26:47,  5.17s/it]
 68%|██████▊   | 2113/3120 [2:56:03<1:25:32,  5.10s/it]
 68%|██████▊   | 2114/3120 [2:56:08<1:24:14,  5.02s/it]
                                                       
{'loss': 0.3504, 'learning_rate': 5.166124560401002e-06, 'epoch': 2.71}

 68%|██████▊   | 2114/3120 [2:56:08<1:24:14,  5.02s/it]
 68%|██████▊   | 2115/3120 [2:56:13<1:23:47,  5.00s/it]
 68%|██████▊   | 2116/3120 [2:56:18<1:26:35,  5.17s/it]
                                                       
{'loss': 0.3724, 'learning_rate': 5.147578279227943e-06, 'epoch': 2.71}

 68%|██████▊   | 2116/3120 [2:56:18<1:26:35,  5.17s/it]
 68%|██████▊   | 2117/3120 [2:56:23<1:24:40,  5.07s/it]
 68%|██████▊   | 2118/3120 [2:56:28<1:23:10,  4.98s/it]
                                                       
{'loss': 0.3595, 'learning_rate': 5.129053803337181e-06, 'epoch': 2.72}

 68%|██████▊   | 2118/3120 [2:56:28<1:23:10,  4.98s/it]
 68%|██████▊   | 2119/3120 [2:56:33<1:24:34,  5.07s/it]
 68%|██████▊   | 2120/3120 [2:56:38<1:22:24,  4.94s/it]
                                                       
{'loss': 0.3831, 'learning_rate': 5.110551215971981e-06, 'epoch': 2.72}

 68%|██████▊   | 2120/3120 [2:56:38<1:22:24,  4.94s/it]
 68%|██████▊   | 2121/3120 [2:56:43<1:23:58,  5.04s/it]
 68%|██████▊   | 2122/3120 [2:56:48<1:23:04,  4.99s/it]
                                                       
{'loss': 0.3842, 'learning_rate': 5.0920706002772525e-06, 'epoch': 2.72}

 68%|██████▊   | 2122/3120 [2:56:48<1:23:04,  4.99s/it]
 68%|██████▊   | 2123/3120 [2:56:53<1:24:05,  5.06s/it]
 68%|██████▊   | 2124/3120 [2:56:58<1:21:13,  4.89s/it]
                                                       
{'loss': 0.3923, 'learning_rate': 5.073612039299157e-06, 'epoch': 2.72}

 68%|██████▊   | 2124/3120 [2:56:58<1:21:13,  4.89s/it]
 68%|██████▊   | 2125/3120 [2:57:03<1:20:43,  4.87s/it]
 68%|██████▊   | 2126/3120 [2:57:07<1:19:15,  4.78s/it]
                                                       
{'loss': 0.3539, 'learning_rate': 5.055175615984766e-06, 'epoch': 2.73}

 68%|██████▊   | 2126/3120 [2:57:07<1:19:15,  4.78s/it]
 68%|██████▊   | 2127/3120 [2:57:12<1:21:25,  4.92s/it]
 68%|██████▊   | 2128/3120 [2:57:17<1:19:25,  4.80s/it]
                                                       
{'loss': 0.3891, 'learning_rate': 5.036761413181659e-06, 'epoch': 2.73}

 68%|██████▊   | 2128/3120 [2:57:17<1:19:25,  4.80s/it]
 68%|██████▊   | 2129/3120 [2:57:22<1:18:34,  4.76s/it]
 68%|██████▊   | 2130/3120 [2:57:27<1:20:34,  4.88s/it]
                                                       
{'loss': 0.3639, 'learning_rate': 5.018369513637567e-06, 'epoch': 2.73}

 68%|██████▊   | 2130/3120 [2:57:27<1:20:34,  4.88s/it]
 68%|██████▊   | 2131/3120 [2:57:31<1:16:48,  4.66s/it]
 68%|██████▊   | 2132/3120 [2:57:36<1:17:54,  4.73s/it]
                                                       
{'loss': 0.3498, 'learning_rate': 5.000000000000003e-06, 'epoch': 2.73}

 68%|██████▊   | 2132/3120 [2:57:36<1:17:54,  4.73s/it]
 68%|██████▊   | 2133/3120 [2:57:41<1:21:12,  4.94s/it]
 68%|██████▊   | 2134/3120 [2:57:47<1:23:42,  5.09s/it]
                                                       
{'loss': 0.3427, 'learning_rate': 4.981652954815871e-06, 'epoch': 2.74}

 68%|██████▊   | 2134/3120 [2:57:47<1:23:42,  5.09s/it]
 68%|██████▊   | 2135/3120 [2:57:51<1:21:10,  4.94s/it]
 68%|██████▊   | 2136/3120 [2:57:56<1:20:08,  4.89s/it]
                                                       
{'loss': 0.3843, 'learning_rate': 4.963328460531127e-06, 'epoch': 2.74}

 68%|██████▊   | 2136/3120 [2:57:56<1:20:08,  4.89s/it]
 68%|██████▊   | 2137/3120 [2:58:01<1:18:38,  4.80s/it]
 69%|██████▊   | 2138/3120 [2:58:05<1:16:59,  4.70s/it]
                                                       
{'loss': 0.3662, 'learning_rate': 4.9450265994903666e-06, 'epoch': 2.74}

 69%|██████▊   | 2138/3120 [2:58:05<1:16:59,  4.70s/it]
 69%|██████▊   | 2139/3120 [2:58:10<1:17:40,  4.75s/it]
 69%|██████▊   | 2140/3120 [2:58:15<1:16:52,  4.71s/it]
                                                       
{'loss': 0.3609, 'learning_rate': 4.926747453936509e-06, 'epoch': 2.74}

 69%|██████▊   | 2140/3120 [2:58:15<1:16:52,  4.71s/it]
 69%|██████▊   | 2141/3120 [2:58:20<1:18:18,  4.80s/it]
 69%|██████▊   | 2142/3120 [2:58:25<1:20:25,  4.93s/it]
                                                       
{'loss': 0.3881, 'learning_rate': 4.908491106010368e-06, 'epoch': 2.75}

 69%|██████▊   | 2142/3120 [2:58:25<1:20:25,  4.93s/it]
 69%|██████▊   | 2143/3120 [2:58:30<1:19:26,  4.88s/it]
 69%|██████▊   | 2144/3120 [2:58:34<1:18:58,  4.85s/it]
                                                       
{'loss': 0.3757, 'learning_rate': 4.890257637750332e-06, 'epoch': 2.75}

 69%|██████▊   | 2144/3120 [2:58:34<1:18:58,  4.85s/it]
 69%|██████▉   | 2145/3120 [2:58:40<1:21:56,  5.04s/it]
 69%|██████▉   | 2146/3120 [2:58:45<1:23:45,  5.16s/it]
                                                       
{'loss': 0.3864, 'learning_rate': 4.872047131091961e-06, 'epoch': 2.75}

 69%|██████▉   | 2146/3120 [2:58:45<1:23:45,  5.16s/it]
 69%|██████▉   | 2147/3120 [2:58:50<1:22:03,  5.06s/it]
 69%|██████▉   | 2148/3120 [2:58:55<1:22:22,  5.08s/it]
                                                       
{'loss': 0.3691, 'learning_rate': 4.853859667867641e-06, 'epoch': 2.75}

 69%|██████▉   | 2148/3120 [2:58:55<1:22:22,  5.08s/it]
 69%|██████▉   | 2149/3120 [2:59:00<1:22:47,  5.12s/it]
 69%|██████▉   | 2150/3120 [2:59:05<1:22:20,  5.09s/it]
                                                       
{'loss': 0.3943, 'learning_rate': 4.8356953298062115e-06, 'epoch': 2.76}

 69%|██████▉   | 2150/3120 [2:59:05<1:22:20,  5.09s/it]
 69%|██████▉   | 2151/3120 [2:59:10<1:21:53,  5.07s/it]
 69%|██████▉   | 2152/3120 [2:59:15<1:21:33,  5.06s/it]
                                                       
{'loss': 0.3739, 'learning_rate': 4.817554198532582e-06, 'epoch': 2.76}

 69%|██████▉   | 2152/3120 [2:59:15<1:21:33,  5.06s/it]
 69%|██████▉   | 2153/3120 [2:59:20<1:20:19,  4.98s/it]
 69%|██████▉   | 2154/3120 [2:59:26<1:21:20,  5.05s/it]
                                                       
{'loss': 0.3871, 'learning_rate': 4.799436355567391e-06, 'epoch': 2.76}

 69%|██████▉   | 2154/3120 [2:59:26<1:21:20,  5.05s/it]
 69%|██████▉   | 2155/3120 [2:59:30<1:19:31,  4.94s/it]
 69%|██████▉   | 2156/3120 [2:59:35<1:18:51,  4.91s/it]
                                                       
{'loss': 0.3774, 'learning_rate': 4.781341882326615e-06, 'epoch': 2.76}

 69%|██████▉   | 2156/3120 [2:59:35<1:18:51,  4.91s/it]
 69%|██████▉   | 2157/3120 [2:59:40<1:20:58,  5.05s/it]
 69%|██████▉   | 2158/3120 [2:59:45<1:19:29,  4.96s/it]
                                                       
{'loss': 0.3881, 'learning_rate': 4.763270860121222e-06, 'epoch': 2.77}

 69%|██████▉   | 2158/3120 [2:59:45<1:19:29,  4.96s/it]
 69%|██████▉   | 2159/3120 [2:59:50<1:19:04,  4.94s/it]
 69%|██████▉   | 2160/3120 [2:59:55<1:19:34,  4.97s/it]
                                                       
{'loss': 0.3652, 'learning_rate': 4.745223370156797e-06, 'epoch': 2.77}

 69%|██████▉   | 2160/3120 [2:59:55<1:19:34,  4.97s/it]
 69%|██████▉   | 2161/3120 [3:00:00<1:18:46,  4.93s/it]
 69%|██████▉   | 2162/3120 [3:00:05<1:18:21,  4.91s/it]
                                                       
{'loss': 0.3647, 'learning_rate': 4.727199493533181e-06, 'epoch': 2.77}

 69%|██████▉   | 2162/3120 [3:00:05<1:18:21,  4.91s/it]
 69%|██████▉   | 2163/3120 [3:00:10<1:18:55,  4.95s/it]
 69%|██████▉   | 2164/3120 [3:00:15<1:20:31,  5.05s/it]
                                                       
{'loss': 0.3818, 'learning_rate': 4.709199311244098e-06, 'epoch': 2.77}

 69%|██████▉   | 2164/3120 [3:00:15<1:20:31,  5.05s/it]
 69%|██████▉   | 2165/3120 [3:00:20<1:17:33,  4.87s/it]
 69%|██████▉   | 2166/3120 [3:00:24<1:17:39,  4.88s/it]
                                                       
{'loss': 0.3594, 'learning_rate': 4.691222904176791e-06, 'epoch': 2.78}

 69%|██████▉   | 2166/3120 [3:00:24<1:17:39,  4.88s/it]
 69%|██████▉   | 2167/3120 [3:00:30<1:18:17,  4.93s/it]
 69%|██████▉   | 2168/3120 [3:00:35<1:19:48,  5.03s/it]
                                                       
{'loss': 0.3448, 'learning_rate': 4.673270353111687e-06, 'epoch': 2.78}

 69%|██████▉   | 2168/3120 [3:00:35<1:19:48,  5.03s/it]
 70%|██████▉   | 2169/3120 [3:00:40<1:20:45,  5.10s/it]
 70%|██████▉   | 2170/3120 [3:00:45<1:21:50,  5.17s/it]
                                                       
{'loss': 0.353, 'learning_rate': 4.655341738721989e-06, 'epoch': 2.78}

 70%|██████▉   | 2170/3120 [3:00:45<1:21:50,  5.17s/it]
 70%|██████▉   | 2171/3120 [3:00:50<1:21:15,  5.14s/it]
 70%|██████▉   | 2172/3120 [3:00:55<1:20:18,  5.08s/it]
                                                       
{'loss': 0.37, 'learning_rate': 4.63743714157335e-06, 'epoch': 2.78}

 70%|██████▉   | 2172/3120 [3:00:55<1:20:18,  5.08s/it]
 70%|██████▉   | 2173/3120 [3:01:00<1:19:47,  5.05s/it]
 70%|██████▉   | 2174/3120 [3:01:06<1:20:00,  5.07s/it]
                                                       
{'loss': 0.3812, 'learning_rate': 4.619556642123484e-06, 'epoch': 2.79}

 70%|██████▉   | 2174/3120 [3:01:06<1:20:00,  5.07s/it]
 70%|██████▉   | 2175/3120 [3:01:10<1:18:44,  5.00s/it]
 70%|██████▉   | 2176/3120 [3:01:15<1:19:22,  5.05s/it]
                                                       
{'loss': 0.3642, 'learning_rate': 4.6017003207218294e-06, 'epoch': 2.79}

 70%|██████▉   | 2176/3120 [3:01:15<1:19:22,  5.05s/it]
 70%|██████▉   | 2177/3120 [3:01:20<1:18:08,  4.97s/it]
 70%|██████▉   | 2178/3120 [3:01:25<1:18:59,  5.03s/it]
                                                       
{'loss': 0.3725, 'learning_rate': 4.583868257609171e-06, 'epoch': 2.79}

 70%|██████▉   | 2178/3120 [3:01:25<1:18:59,  5.03s/it]
 70%|██████▉   | 2179/3120 [3:01:31<1:20:48,  5.15s/it]
 70%|██████▉   | 2180/3120 [3:01:36<1:21:55,  5.23s/it]
                                                       
{'loss': 0.3473, 'learning_rate': 4.566060532917288e-06, 'epoch': 2.79}

 70%|██████▉   | 2180/3120 [3:01:36<1:21:55,  5.23s/it]
 70%|██████▉   | 2181/3120 [3:01:41<1:20:14,  5.13s/it]
 70%|██████▉   | 2182/3120 [3:01:46<1:20:11,  5.13s/it]
                                                       
{'loss': 0.3708, 'learning_rate': 4.548277226668585e-06, 'epoch': 2.8}

 70%|██████▉   | 2182/3120 [3:01:46<1:20:11,  5.13s/it]
 70%|██████▉   | 2183/3120 [3:01:52<1:21:06,  5.19s/it]
 70%|███████   | 2184/3120 [3:01:57<1:19:26,  5.09s/it]
                                                       
{'loss': 0.3881, 'learning_rate': 4.530518418775734e-06, 'epoch': 2.8}

 70%|███████   | 2184/3120 [3:01:57<1:19:26,  5.09s/it]
 70%|███████   | 2185/3120 [3:02:02<1:20:50,  5.19s/it]
 70%|███████   | 2186/3120 [3:02:07<1:20:23,  5.16s/it]
                                                       
{'loss': 0.379, 'learning_rate': 4.512784189041328e-06, 'epoch': 2.8}

 70%|███████   | 2186/3120 [3:02:07<1:20:23,  5.16s/it]
 70%|███████   | 2187/3120 [3:02:12<1:18:25,  5.04s/it]
 70%|███████   | 2188/3120 [3:02:17<1:17:11,  4.97s/it]
                                                       
{'loss': 0.4005, 'learning_rate': 4.4950746171575135e-06, 'epoch': 2.81}

 70%|███████   | 2188/3120 [3:02:17<1:17:11,  4.97s/it]
 70%|███████   | 2189/3120 [3:02:21<1:16:25,  4.93s/it]
 70%|███████   | 2190/3120 [3:02:26<1:16:37,  4.94s/it]
                                                       
{'loss': 0.3562, 'learning_rate': 4.477389782705628e-06, 'epoch': 2.81}

 70%|███████   | 2190/3120 [3:02:26<1:16:37,  4.94s/it]
 70%|███████   | 2191/3120 [3:02:32<1:17:38,  5.01s/it]
 70%|███████   | 2192/3120 [3:02:37<1:19:13,  5.12s/it]
                                                       
{'loss': 0.364, 'learning_rate': 4.459729765155842e-06, 'epoch': 2.81}

 70%|███████   | 2192/3120 [3:02:37<1:19:13,  5.12s/it]
 70%|███████   | 2193/3120 [3:02:42<1:17:47,  5.04s/it]
 70%|███████   | 2194/3120 [3:02:46<1:15:21,  4.88s/it]
                                                       
{'loss': 0.3626, 'learning_rate': 4.442094643866817e-06, 'epoch': 2.81}

 70%|███████   | 2194/3120 [3:02:46<1:15:21,  4.88s/it]
 70%|███████   | 2195/3120 [3:02:51<1:16:10,  4.94s/it]
 70%|███████   | 2196/3120 [3:02:56<1:15:19,  4.89s/it]
                                                       
{'loss': 0.3813, 'learning_rate': 4.424484498085335e-06, 'epoch': 2.82}

 70%|███████   | 2196/3120 [3:02:56<1:15:19,  4.89s/it]
 70%|███████   | 2197/3120 [3:03:01<1:15:36,  4.91s/it]
 70%|███████   | 2198/3120 [3:03:06<1:14:31,  4.85s/it]
                                                       
{'loss': 0.3717, 'learning_rate': 4.406899406945938e-06, 'epoch': 2.82}

 70%|███████   | 2198/3120 [3:03:06<1:14:31,  4.85s/it]
 70%|███████   | 2199/3120 [3:03:11<1:13:45,  4.80s/it]
 71%|███████   | 2200/3120 [3:03:15<1:14:00,  4.83s/it]
                                                       
{'loss': 0.3593, 'learning_rate': 4.389339449470592e-06, 'epoch': 2.82}

 71%|███████   | 2200/3120 [3:03:15<1:14:00,  4.83s/it]
 71%|███████   | 2201/3120 [3:03:20<1:14:01,  4.83s/it]
 71%|███████   | 2202/3120 [3:03:25<1:14:20,  4.86s/it]
                                                       
{'loss': 0.3905, 'learning_rate': 4.371804704568309e-06, 'epoch': 2.82}

 71%|███████   | 2202/3120 [3:03:25<1:14:20,  4.86s/it]
 71%|███████   | 2203/3120 [3:03:30<1:16:19,  4.99s/it]
 71%|███████   | 2204/3120 [3:03:35<1:14:43,  4.89s/it]
                                                       
{'loss': 0.3682, 'learning_rate': 4.354295251034811e-06, 'epoch': 2.83}

 71%|███████   | 2204/3120 [3:03:35<1:14:43,  4.89s/it]
 71%|███████   | 2205/3120 [3:03:40<1:15:04,  4.92s/it]
 71%|███████   | 2206/3120 [3:03:45<1:16:54,  5.05s/it]
                                                       
{'loss': 0.3833, 'learning_rate': 4.336811167552167e-06, 'epoch': 2.83}

 71%|███████   | 2206/3120 [3:03:45<1:16:54,  5.05s/it]
 71%|███████   | 2207/3120 [3:03:50<1:15:08,  4.94s/it]
 71%|███████   | 2208/3120 [3:03:56<1:17:49,  5.12s/it]
                                                       
{'loss': 0.3714, 'learning_rate': 4.319352532688444e-06, 'epoch': 2.83}

 71%|███████   | 2208/3120 [3:03:56<1:17:49,  5.12s/it]
 71%|███████   | 2209/3120 [3:04:00<1:15:35,  4.98s/it]
 71%|███████   | 2210/3120 [3:04:05<1:15:56,  5.01s/it]
                                                       
{'loss': 0.3757, 'learning_rate': 4.301919424897339e-06, 'epoch': 2.83}

 71%|███████   | 2210/3120 [3:04:05<1:15:56,  5.01s/it]
 71%|███████   | 2211/3120 [3:04:10<1:14:12,  4.90s/it]
 71%|███████   | 2212/3120 [3:04:15<1:15:34,  4.99s/it]
                                                       
{'loss': 0.3832, 'learning_rate': 4.284511922517853e-06, 'epoch': 2.84}

 71%|███████   | 2212/3120 [3:04:15<1:15:34,  4.99s/it]
 71%|███████   | 2213/3120 [3:04:20<1:16:28,  5.06s/it]
 71%|███████   | 2214/3120 [3:04:25<1:15:50,  5.02s/it]
                                                       
{'loss': 0.355, 'learning_rate': 4.267130103773911e-06, 'epoch': 2.84}

 71%|███████   | 2214/3120 [3:04:25<1:15:50,  5.02s/it]
 71%|███████   | 2215/3120 [3:04:30<1:13:58,  4.90s/it]
 71%|███████   | 2216/3120 [3:04:35<1:12:36,  4.82s/it]
                                                       
{'loss': 0.3941, 'learning_rate': 4.249774046774034e-06, 'epoch': 2.84}

 71%|███████   | 2216/3120 [3:04:35<1:12:36,  4.82s/it]
 71%|███████   | 2217/3120 [3:04:40<1:14:00,  4.92s/it]
 71%|███████   | 2218/3120 [3:04:45<1:12:57,  4.85s/it]
                                                       
{'loss': 0.3579, 'learning_rate': 4.232443829510977e-06, 'epoch': 2.84}

 71%|███████   | 2218/3120 [3:04:45<1:12:57,  4.85s/it]
 71%|███████   | 2219/3120 [3:04:49<1:11:43,  4.78s/it]
 71%|███████   | 2220/3120 [3:04:54<1:12:50,  4.86s/it]
                                                       
{'loss': 0.3482, 'learning_rate': 4.2151395298613675e-06, 'epoch': 2.85}

 71%|███████   | 2220/3120 [3:04:54<1:12:50,  4.86s/it]
 71%|███████   | 2221/3120 [3:04:59<1:11:55,  4.80s/it]
 71%|███████   | 2222/3120 [3:05:04<1:12:27,  4.84s/it]
                                                       
{'loss': 0.3803, 'learning_rate': 4.19786122558538e-06, 'epoch': 2.85}

 71%|███████   | 2222/3120 [3:05:04<1:12:27,  4.84s/it]
 71%|███████▏  | 2223/3120 [3:05:09<1:13:11,  4.90s/it]
 71%|███████▏  | 2224/3120 [3:05:13<1:10:42,  4.74s/it]
                                                       
{'loss': 0.3312, 'learning_rate': 4.180608994326371e-06, 'epoch': 2.85}

 71%|███████▏  | 2224/3120 [3:05:13<1:10:42,  4.74s/it]
 71%|███████▏  | 2225/3120 [3:05:18<1:13:17,  4.91s/it]
 71%|███████▏  | 2226/3120 [3:05:23<1:11:55,  4.83s/it]
                                                       
{'loss': 0.3837, 'learning_rate': 4.163382913610533e-06, 'epoch': 2.85}

 71%|███████▏  | 2226/3120 [3:05:23<1:11:55,  4.83s/it]
 71%|███████▏  | 2227/3120 [3:05:28<1:13:46,  4.96s/it]
 71%|███████▏  | 2228/3120 [3:05:33<1:13:00,  4.91s/it]
                                                       
{'loss': 0.3622, 'learning_rate': 4.1461830608465385e-06, 'epoch': 2.86}

 71%|███████▏  | 2228/3120 [3:05:33<1:13:00,  4.91s/it]
 71%|███████▏  | 2229/3120 [3:05:38<1:12:12,  4.86s/it]
 71%|███████▏  | 2230/3120 [3:05:43<1:12:03,  4.86s/it]
                                                       
{'loss': 0.3751, 'learning_rate': 4.129009513325211e-06, 'epoch': 2.86}

 71%|███████▏  | 2230/3120 [3:05:43<1:12:03,  4.86s/it]
 72%|███████▏  | 2231/3120 [3:05:47<1:10:45,  4.78s/it]
 72%|███████▏  | 2232/3120 [3:05:52<1:11:44,  4.85s/it]
                                                       
{'loss': 0.3627, 'learning_rate': 4.111862348219158e-06, 'epoch': 2.86}

 72%|███████▏  | 2232/3120 [3:05:52<1:11:44,  4.85s/it]
 72%|███████▏  | 2233/3120 [3:05:57<1:10:14,  4.75s/it]
 72%|███████▏  | 2234/3120 [3:06:02<1:11:41,  4.86s/it]
                                                       
{'loss': 0.3627, 'learning_rate': 4.0947416425824334e-06, 'epoch': 2.86}

 72%|███████▏  | 2234/3120 [3:06:02<1:11:41,  4.86s/it]
 72%|███████▏  | 2235/3120 [3:06:07<1:12:15,  4.90s/it]
 72%|███████▏  | 2236/3120 [3:06:12<1:13:15,  4.97s/it]
                                                       
{'loss': 0.3708, 'learning_rate': 4.077647473350201e-06, 'epoch': 2.87}

 72%|███████▏  | 2236/3120 [3:06:12<1:13:15,  4.97s/it]
 72%|███████▏  | 2237/3120 [3:06:17<1:14:29,  5.06s/it]
 72%|███████▏  | 2238/3120 [3:06:22<1:14:19,  5.06s/it]
                                                       
{'loss': 0.3605, 'learning_rate': 4.060579917338362e-06, 'epoch': 2.87}

 72%|███████▏  | 2238/3120 [3:06:22<1:14:19,  5.06s/it]
 72%|███████▏  | 2239/3120 [3:06:27<1:13:08,  4.98s/it]
 72%|███████▏  | 2240/3120 [3:06:32<1:13:08,  4.99s/it]
                                                       
{'loss': 0.3913, 'learning_rate': 4.04353905124324e-06, 'epoch': 2.87}

 72%|███████▏  | 2240/3120 [3:06:32<1:13:08,  4.99s/it]
 72%|███████▏  | 2241/3120 [3:06:37<1:12:38,  4.96s/it]
 72%|███████▏  | 2242/3120 [3:06:42<1:12:53,  4.98s/it]
                                                       
{'loss': 0.3695, 'learning_rate': 4.026524951641204e-06, 'epoch': 2.87}

 72%|███████▏  | 2242/3120 [3:06:42<1:12:53,  4.98s/it]
 72%|███████▏  | 2243/3120 [3:06:47<1:12:48,  4.98s/it]
 72%|███████▏  | 2244/3120 [3:06:52<1:13:18,  5.02s/it]
                                                       
{'loss': 0.3571, 'learning_rate': 4.009537694988372e-06, 'epoch': 2.88}

 72%|███████▏  | 2244/3120 [3:06:52<1:13:18,  5.02s/it]
 72%|███████▏  | 2245/3120 [3:06:57<1:13:28,  5.04s/it]
 72%|███████▏  | 2246/3120 [3:07:02<1:12:05,  4.95s/it]
                                                       
{'loss': 0.3624, 'learning_rate': 3.99257735762021e-06, 'epoch': 2.88}

 72%|███████▏  | 2246/3120 [3:07:02<1:12:05,  4.95s/it]
 72%|███████▏  | 2247/3120 [3:07:07<1:12:22,  4.97s/it]
 72%|███████▏  | 2248/3120 [3:07:12<1:11:58,  4.95s/it]
                                                       
{'loss': 0.3622, 'learning_rate': 3.975644015751234e-06, 'epoch': 2.88}

 72%|███████▏  | 2248/3120 [3:07:12<1:11:58,  4.95s/it]
 72%|███████▏  | 2249/3120 [3:07:17<1:11:54,  4.95s/it]
 72%|███████▏  | 2250/3120 [3:07:22<1:12:51,  5.02s/it]
                                                       
{'loss': 0.3679, 'learning_rate': 3.958737745474638e-06, 'epoch': 2.88}

 72%|███████▏  | 2250/3120 [3:07:22<1:12:51,  5.02s/it]
 72%|███████▏  | 2251/3120 [3:07:27<1:13:10,  5.05s/it]
 72%|███████▏  | 2252/3120 [3:07:32<1:12:15,  5.00s/it]
                                                       
{'loss': 0.3724, 'learning_rate': 3.941858622761975e-06, 'epoch': 2.89}

 72%|███████▏  | 2252/3120 [3:07:32<1:12:15,  5.00s/it]
 72%|███████▏  | 2253/3120 [3:07:37<1:12:03,  4.99s/it]
 72%|███████▏  | 2254/3120 [3:07:42<1:12:44,  5.04s/it]
                                                       
{'loss': 0.3856, 'learning_rate': 3.925006723462806e-06, 'epoch': 2.89}

 72%|███████▏  | 2254/3120 [3:07:42<1:12:44,  5.04s/it]
 72%|███████▏  | 2255/3120 [3:07:48<1:13:35,  5.10s/it]
 72%|███████▏  | 2256/3120 [3:07:52<1:10:49,  4.92s/it]
                                                       
{'loss': 0.3827, 'learning_rate': 3.908182123304344e-06, 'epoch': 2.89}

 72%|███████▏  | 2256/3120 [3:07:52<1:10:49,  4.92s/it]
 72%|███████▏  | 2257/3120 [3:07:57<1:11:57,  5.00s/it]
 72%|███████▏  | 2258/3120 [3:08:02<1:11:02,  4.95s/it]
                                                       
{'loss': 0.3512, 'learning_rate': 3.891384897891148e-06, 'epoch': 2.89}

 72%|███████▏  | 2258/3120 [3:08:02<1:11:02,  4.95s/it]
 72%|███████▏  | 2259/3120 [3:08:07<1:12:12,  5.03s/it]
 72%|███████▏  | 2260/3120 [3:08:13<1:13:03,  5.10s/it]
                                                       
{'loss': 0.3472, 'learning_rate': 3.8746151227047455e-06, 'epoch': 2.9}

 72%|███████▏  | 2260/3120 [3:08:13<1:13:03,  5.10s/it]
 72%|███████▏  | 2261/3120 [3:08:17<1:12:21,  5.05s/it]
 72%|███████▎  | 2262/3120 [3:08:22<1:10:32,  4.93s/it]
                                                       
{'loss': 0.3541, 'learning_rate': 3.857872873103322e-06, 'epoch': 2.9}

 72%|███████▎  | 2262/3120 [3:08:22<1:10:32,  4.93s/it]
 73%|███████▎  | 2263/3120 [3:08:27<1:08:57,  4.83s/it]
 73%|███████▎  | 2264/3120 [3:08:32<1:09:19,  4.86s/it]
                                                       
{'loss': 0.352, 'learning_rate': 3.84115822432137e-06, 'epoch': 2.9}

 73%|███████▎  | 2264/3120 [3:08:32<1:09:19,  4.86s/it]
 73%|███████▎  | 2265/3120 [3:08:36<1:08:33,  4.81s/it]
 73%|███████▎  | 2266/3120 [3:08:41<1:09:19,  4.87s/it]
                                                       
{'loss': 0.3821, 'learning_rate': 3.824471251469352e-06, 'epoch': 2.91}

 73%|███████▎  | 2266/3120 [3:08:41<1:09:19,  4.87s/it]
 73%|███████▎  | 2267/3120 [3:08:46<1:09:11,  4.87s/it]
 73%|███████▎  | 2268/3120 [3:08:51<1:07:36,  4.76s/it]
                                                       
{'loss': 0.358, 'learning_rate': 3.807812029533362e-06, 'epoch': 2.91}

 73%|███████▎  | 2268/3120 [3:08:51<1:07:36,  4.76s/it]
 73%|███████▎  | 2269/3120 [3:08:55<1:06:38,  4.70s/it]
 73%|███████▎  | 2270/3120 [3:09:00<1:07:08,  4.74s/it]
                                                       
{'loss': 0.3785, 'learning_rate': 3.7911806333747847e-06, 'epoch': 2.91}

 73%|███████▎  | 2270/3120 [3:09:00<1:07:08,  4.74s/it]
 73%|███████▎  | 2271/3120 [3:09:05<1:05:33,  4.63s/it]
 73%|███████▎  | 2272/3120 [3:09:09<1:05:26,  4.63s/it]
                                                       
{'loss': 0.3556, 'learning_rate': 3.7745771377299758e-06, 'epoch': 2.91}

 73%|███████▎  | 2272/3120 [3:09:09<1:05:26,  4.63s/it]
 73%|███████▎  | 2273/3120 [3:09:14<1:07:42,  4.80s/it]
 73%|███████▎  | 2274/3120 [3:09:19<1:07:13,  4.77s/it]
                                                       
{'loss': 0.3449, 'learning_rate': 3.7580016172099067e-06, 'epoch': 2.92}

 73%|███████▎  | 2274/3120 [3:09:19<1:07:13,  4.77s/it]
 73%|███████▎  | 2275/3120 [3:09:24<1:07:50,  4.82s/it]
 73%|███████▎  | 2276/3120 [3:09:29<1:08:20,  4.86s/it]
                                                       
{'loss': 0.3615, 'learning_rate': 3.7414541462998446e-06, 'epoch': 2.92}

 73%|███████▎  | 2276/3120 [3:09:29<1:08:20,  4.86s/it]
 73%|███████▎  | 2277/3120 [3:09:34<1:08:53,  4.90s/it]
 73%|███████▎  | 2278/3120 [3:09:39<1:09:08,  4.93s/it]
                                                       
{'loss': 0.3758, 'learning_rate': 3.724934799358998e-06, 'epoch': 2.92}

 73%|███████▎  | 2278/3120 [3:09:39<1:09:08,  4.93s/it]
 73%|███████▎  | 2279/3120 [3:09:44<1:09:57,  4.99s/it]
 73%|███████▎  | 2280/3120 [3:09:49<1:08:49,  4.92s/it]
                                                       
{'loss': 0.3621, 'learning_rate': 3.708443650620206e-06, 'epoch': 2.92}

 73%|███████▎  | 2280/3120 [3:09:49<1:08:49,  4.92s/it]
 73%|███████▎  | 2281/3120 [3:09:54<1:09:43,  4.99s/it]
 73%|███████▎  | 2282/3120 [3:09:59<1:09:12,  4.96s/it]
                                                       
{'loss': 0.3675, 'learning_rate': 3.691980774189591e-06, 'epoch': 2.93}

 73%|███████▎  | 2282/3120 [3:09:59<1:09:12,  4.96s/it]
 73%|███████▎  | 2283/3120 [3:10:04<1:09:39,  4.99s/it]
 73%|███████▎  | 2284/3120 [3:10:09<1:09:24,  4.98s/it]
                                                       
{'loss': 0.3816, 'learning_rate': 3.6755462440462288e-06, 'epoch': 2.93}

 73%|███████▎  | 2284/3120 [3:10:09<1:09:24,  4.98s/it]
 73%|███████▎  | 2285/3120 [3:10:14<1:09:31,  5.00s/it]
 73%|███████▎  | 2286/3120 [3:10:19<1:08:54,  4.96s/it]
                                                       
{'loss': 0.3575, 'learning_rate': 3.659140134041812e-06, 'epoch': 2.93}

 73%|███████▎  | 2286/3120 [3:10:19<1:08:54,  4.96s/it]
 73%|███████▎  | 2287/3120 [3:10:24<1:09:56,  5.04s/it]
 73%|███████▎  | 2288/3120 [3:10:28<1:07:27,  4.87s/it]
                                                       
{'loss': 0.3613, 'learning_rate': 3.6427625179003223e-06, 'epoch': 2.93}

 73%|███████▎  | 2288/3120 [3:10:28<1:07:27,  4.87s/it]
 73%|███████▎  | 2289/3120 [3:10:33<1:05:56,  4.76s/it]
 73%|███████▎  | 2290/3120 [3:10:38<1:07:00,  4.84s/it]
                                                       
{'loss': 0.3693, 'learning_rate': 3.626413469217702e-06, 'epoch': 2.94}

 73%|███████▎  | 2290/3120 [3:10:38<1:07:00,  4.84s/it]
 73%|███████▎  | 2291/3120 [3:10:42<1:05:20,  4.73s/it]
 73%|███████▎  | 2292/3120 [3:10:48<1:07:36,  4.90s/it]
                                                       
{'loss': 0.3807, 'learning_rate': 3.6100930614615204e-06, 'epoch': 2.94}

 73%|███████▎  | 2292/3120 [3:10:48<1:07:36,  4.90s/it]
 73%|███████▎  | 2293/3120 [3:10:53<1:08:01,  4.93s/it]
 74%|███████▎  | 2294/3120 [3:10:58<1:08:04,  4.95s/it]
                                                       
{'loss': 0.3767, 'learning_rate': 3.5938013679706453e-06, 'epoch': 2.94}

 74%|███████▎  | 2294/3120 [3:10:58<1:08:04,  4.95s/it]
 74%|███████▎  | 2295/3120 [3:11:03<1:09:06,  5.03s/it]
 74%|███████▎  | 2296/3120 [3:11:08<1:09:21,  5.05s/it]
                                                       
{'loss': 0.3436, 'learning_rate': 3.5775384619549e-06, 'epoch': 2.94}

 74%|███████▎  | 2296/3120 [3:11:08<1:09:21,  5.05s/it]
 74%|███████▎  | 2297/3120 [3:11:13<1:08:00,  4.96s/it]
 74%|███████▎  | 2298/3120 [3:11:18<1:07:45,  4.95s/it]
                                                       
{'loss': 0.3562, 'learning_rate': 3.5613044164947617e-06, 'epoch': 2.95}

 74%|███████▎  | 2298/3120 [3:11:18<1:07:45,  4.95s/it]
 74%|███████▎  | 2299/3120 [3:11:23<1:09:21,  5.07s/it]
 74%|███████▎  | 2300/3120 [3:11:28<1:08:52,  5.04s/it]
                                                       
{'loss': 0.3736, 'learning_rate': 3.5450993045409997e-06, 'epoch': 2.95}

 74%|███████▎  | 2300/3120 [3:11:28<1:08:52,  5.04s/it]
 74%|███████▍  | 2301/3120 [3:11:33<1:08:03,  4.99s/it]
 74%|███████▍  | 2302/3120 [3:11:38<1:09:11,  5.08s/it]
                                                       
{'loss': 0.3543, 'learning_rate': 3.5289231989143868e-06, 'epoch': 2.95}

 74%|███████▍  | 2302/3120 [3:11:38<1:09:11,  5.08s/it]
 74%|███████▍  | 2303/3120 [3:11:43<1:09:54,  5.13s/it]
 74%|███████▍  | 2304/3120 [3:11:48<1:08:25,  5.03s/it]
                                                       
{'loss': 0.3391, 'learning_rate': 3.5127761723053313e-06, 'epoch': 2.95}

 74%|███████▍  | 2304/3120 [3:11:48<1:08:25,  5.03s/it]
 74%|███████▍  | 2305/3120 [3:11:53<1:08:07,  5.02s/it]
 74%|███████▍  | 2306/3120 [3:11:58<1:09:00,  5.09s/it]
                                                       
{'loss': 0.3482, 'learning_rate': 3.496658297273574e-06, 'epoch': 2.96}

 74%|███████▍  | 2306/3120 [3:11:58<1:09:00,  5.09s/it]
 74%|███████▍  | 2307/3120 [3:12:03<1:08:21,  5.04s/it]
 74%|███████▍  | 2308/3120 [3:12:08<1:07:12,  4.97s/it]
                                                       
{'loss': 0.4039, 'learning_rate': 3.4805696462478634e-06, 'epoch': 2.96}

 74%|███████▍  | 2308/3120 [3:12:08<1:07:12,  4.97s/it]
 74%|███████▍  | 2309/3120 [3:12:13<1:07:17,  4.98s/it]
 74%|███████▍  | 2310/3120 [3:12:19<1:09:21,  5.14s/it]
                                                       
{'loss': 0.3547, 'learning_rate': 3.46451029152562e-06, 'epoch': 2.96}

 74%|███████▍  | 2310/3120 [3:12:19<1:09:21,  5.14s/it]
 74%|███████▍  | 2311/3120 [3:12:23<1:06:44,  4.95s/it]
 74%|███████▍  | 2312/3120 [3:12:28<1:06:41,  4.95s/it]
                                                       
{'loss': 0.3706, 'learning_rate': 3.448480305272619e-06, 'epoch': 2.96}

 74%|███████▍  | 2312/3120 [3:12:28<1:06:41,  4.95s/it]
 74%|███████▍  | 2313/3120 [3:12:33<1:06:07,  4.92s/it]
 74%|███████▍  | 2314/3120 [3:12:38<1:06:48,  4.97s/it]
                                                       
{'loss': 0.3797, 'learning_rate': 3.4324797595226567e-06, 'epoch': 2.97}

 74%|███████▍  | 2314/3120 [3:12:38<1:06:48,  4.97s/it]
 74%|███████▍  | 2315/3120 [3:12:43<1:05:24,  4.88s/it]
 74%|███████▍  | 2316/3120 [3:12:48<1:05:39,  4.90s/it]
                                                       
{'loss': 0.374, 'learning_rate': 3.41650872617724e-06, 'epoch': 2.97}

 74%|███████▍  | 2316/3120 [3:12:48<1:05:39,  4.90s/it]
 74%|███████▍  | 2317/3120 [3:12:52<1:03:53,  4.77s/it]
 74%|███████▍  | 2318/3120 [3:12:58<1:05:55,  4.93s/it]
                                                       
{'loss': 0.3481, 'learning_rate': 3.400567277005247e-06, 'epoch': 2.97}

 74%|███████▍  | 2318/3120 [3:12:58<1:05:55,  4.93s/it]
 74%|███████▍  | 2319/3120 [3:13:02<1:04:34,  4.84s/it]
 74%|███████▍  | 2320/3120 [3:13:07<1:04:33,  4.84s/it]
                                                       
{'loss': 0.3659, 'learning_rate': 3.384655483642624e-06, 'epoch': 2.97}

 74%|███████▍  | 2320/3120 [3:13:07<1:04:33,  4.84s/it]
 74%|███████▍  | 2321/3120 [3:13:12<1:05:04,  4.89s/it]
 74%|███████▍  | 2322/3120 [3:13:17<1:05:09,  4.90s/it]
                                                       
{'loss': 0.3454, 'learning_rate': 3.3687734175920505e-06, 'epoch': 2.98}

 74%|███████▍  | 2322/3120 [3:13:17<1:05:09,  4.90s/it]
 74%|███████▍  | 2323/3120 [3:13:22<1:04:49,  4.88s/it]
 74%|███████▍  | 2324/3120 [3:13:27<1:05:11,  4.91s/it]
                                                       
{'loss': 0.3827, 'learning_rate': 3.352921150222612e-06, 'epoch': 2.98}

 74%|███████▍  | 2324/3120 [3:13:27<1:05:11,  4.91s/it]
 75%|███████▍  | 2325/3120 [3:13:31<1:04:06,  4.84s/it]
 75%|███████▍  | 2326/3120 [3:13:36<1:03:49,  4.82s/it]
                                                       
{'loss': 0.3635, 'learning_rate': 3.337098752769503e-06, 'epoch': 2.98}

 75%|███████▍  | 2326/3120 [3:13:36<1:03:49,  4.82s/it]
 75%|███████▍  | 2327/3120 [3:13:41<1:04:12,  4.86s/it]
 75%|███████▍  | 2328/3120 [3:13:46<1:05:20,  4.95s/it]
                                                       
{'loss': 0.3556, 'learning_rate': 3.321306296333673e-06, 'epoch': 2.98}

 75%|███████▍  | 2328/3120 [3:13:46<1:05:20,  4.95s/it]
 75%|███████▍  | 2329/3120 [3:13:51<1:05:38,  4.98s/it]
 75%|███████▍  | 2330/3120 [3:13:56<1:05:41,  4.99s/it]
                                                       
{'loss': 0.355, 'learning_rate': 3.3055438518815486e-06, 'epoch': 2.99}

 75%|███████▍  | 2330/3120 [3:13:56<1:05:41,  4.99s/it]
 75%|███████▍  | 2331/3120 [3:14:01<1:05:43,  5.00s/it]
 75%|███████▍  | 2332/3120 [3:14:06<1:04:27,  4.91s/it]
                                                       
{'loss': 0.358, 'learning_rate': 3.2898114902446708e-06, 'epoch': 2.99}

 75%|███████▍  | 2332/3120 [3:14:06<1:04:27,  4.91s/it]
 75%|███████▍  | 2333/3120 [3:14:11<1:04:24,  4.91s/it]
 75%|███████▍  | 2334/3120 [3:14:16<1:04:37,  4.93s/it]
                                                       
{'loss': 0.3402, 'learning_rate': 3.274109282119413e-06, 'epoch': 2.99}

 75%|███████▍  | 2334/3120 [3:14:16<1:04:37,  4.93s/it]
 75%|███████▍  | 2335/3120 [3:14:21<1:04:23,  4.92s/it]
 75%|███████▍  | 2336/3120 [3:14:26<1:04:12,  4.91s/it]
                                                       
{'loss': 0.3406, 'learning_rate': 3.2584372980666344e-06, 'epoch': 2.99}

 75%|███████▍  | 2336/3120 [3:14:26<1:04:12,  4.91s/it]
 75%|███████▍  | 2337/3120 [3:14:31<1:03:34,  4.87s/it]
 75%|███████▍  | 2338/3120 [3:14:36<1:04:09,  4.92s/it]
                                                       
{'loss': 0.3545, 'learning_rate': 3.2427956085113875e-06, 'epoch': 3.0}

 75%|███████▍  | 2338/3120 [3:14:36<1:04:09,  4.92s/it]
 75%|███████▍  | 2339/3120 [3:14:40<1:03:41,  4.89s/it]
 75%|███████▌  | 2340/3120 [3:14:45<1:02:44,  4.83s/it]
                                                       
{'loss': 0.3491, 'learning_rate': 3.2271842837425917e-06, 'epoch': 3.0}

 75%|███████▌  | 2340/3120 [3:14:45<1:02:44,  4.83s/it]/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
  warnings.warn(
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  labels = [torch.tensor(x) for x in labels]

 75%|███████▌  | 2341/3120 [3:15:22<3:09:08, 14.57s/it]
 75%|███████▌  | 2342/3120 [3:15:27<2:31:46, 11.70s/it]
                                                       
{'loss': 0.1364, 'learning_rate': 3.2116033939127024e-06, 'epoch': 3.0}

 75%|███████▌  | 2342/3120 [3:15:27<2:31:46, 11.70s/it]
 75%|███████▌  | 2343/3120 [3:15:33<2:06:07,  9.74s/it]
 75%|███████▌  | 2344/3120 [3:15:37<1:46:55,  8.27s/it]
                                                       
{'loss': 0.1197, 'learning_rate': 3.1960530090374277e-06, 'epoch': 3.01}

 75%|███████▌  | 2344/3120 [3:15:37<1:46:55,  8.27s/it]
 75%|███████▌  | 2345/3120 [3:15:43<1:36:59,  7.51s/it]
 75%|███████▌  | 2346/3120 [3:15:48<1:25:58,  6.66s/it]
                                                       
{'loss': 0.1254, 'learning_rate': 3.180533198995379e-06, 'epoch': 3.01}

 75%|███████▌  | 2346/3120 [3:15:48<1:25:58,  6.66s/it]
 75%|███████▌  | 2347/3120 [3:15:53<1:19:37,  6.18s/it]
 75%|███████▌  | 2348/3120 [3:15:58<1:14:20,  5.78s/it]
                                                       
{'loss': 0.1163, 'learning_rate': 3.165044033527789e-06, 'epoch': 3.01}

 75%|███████▌  | 2348/3120 [3:15:58<1:14:20,  5.78s/it]
 75%|███████▌  | 2349/3120 [3:16:03<1:12:25,  5.64s/it]
 75%|███████▌  | 2350/3120 [3:16:08<1:10:32,  5.50s/it]
                                                       
{'loss': 0.1316, 'learning_rate': 3.1495855822381716e-06, 'epoch': 3.01}

 75%|███████▌  | 2350/3120 [3:16:08<1:10:32,  5.50s/it]
 75%|███████▌  | 2351/3120 [3:16:13<1:07:35,  5.27s/it]
 75%|███████▌  | 2352/3120 [3:16:18<1:05:20,  5.10s/it]
                                                       
{'loss': 0.119, 'learning_rate': 3.134157914592032e-06, 'epoch': 3.02}

 75%|███████▌  | 2352/3120 [3:16:18<1:05:20,  5.10s/it]
 75%|███████▌  | 2353/3120 [3:16:23<1:05:10,  5.10s/it]
 75%|███████▌  | 2354/3120 [3:16:28<1:04:26,  5.05s/it]
                                                       
{'loss': 0.1231, 'learning_rate': 3.1187610999165317e-06, 'epoch': 3.02}

 75%|███████▌  | 2354/3120 [3:16:28<1:04:26,  5.05s/it]
 75%|███████▌  | 2355/3120 [3:16:32<1:03:30,  4.98s/it]
 76%|███████▌  | 2356/3120 [3:16:37<1:02:30,  4.91s/it]
                                                       
{'loss': 0.1362, 'learning_rate': 3.1033952074001882e-06, 'epoch': 3.02}

 76%|███████▌  | 2356/3120 [3:16:37<1:02:30,  4.91s/it]
 76%|███████▌  | 2357/3120 [3:16:42<1:03:04,  4.96s/it]
 76%|███████▌  | 2358/3120 [3:16:47<1:03:54,  5.03s/it]
                                                       
{'loss': 0.1191, 'learning_rate': 3.088060306092582e-06, 'epoch': 3.02}

 76%|███████▌  | 2358/3120 [3:16:47<1:03:54,  5.03s/it]
 76%|███████▌  | 2359/3120 [3:16:52<1:01:52,  4.88s/it]
 76%|███████▌  | 2360/3120 [3:16:57<1:01:40,  4.87s/it]
                                                       
{'loss': 0.126, 'learning_rate': 3.0727564649040066e-06, 'epoch': 3.03}

 76%|███████▌  | 2360/3120 [3:16:57<1:01:40,  4.87s/it]
 76%|███████▌  | 2361/3120 [3:17:02<1:02:39,  4.95s/it]
 76%|███████▌  | 2362/3120 [3:17:07<1:01:40,  4.88s/it]
                                                       
{'loss': 0.1242, 'learning_rate': 3.057483752605196e-06, 'epoch': 3.03}

 76%|███████▌  | 2362/3120 [3:17:07<1:01:40,  4.88s/it]
 76%|███████▌  | 2363/3120 [3:17:12<1:02:09,  4.93s/it]
 76%|███████▌  | 2364/3120 [3:17:17<1:04:47,  5.14s/it]
                                                       
{'loss': 0.1365, 'learning_rate': 3.042242237826991e-06, 'epoch': 3.03}

 76%|███████▌  | 2364/3120 [3:17:17<1:04:47,  5.14s/it]
 76%|███████▌  | 2365/3120 [3:17:22<1:04:16,  5.11s/it]
 76%|███████▌  | 2366/3120 [3:17:28<1:04:04,  5.10s/it]
                                                       
{'loss': 0.1274, 'learning_rate': 3.0270319890600465e-06, 'epoch': 3.03}

 76%|███████▌  | 2366/3120 [3:17:28<1:04:04,  5.10s/it]
 76%|███████▌  | 2367/3120 [3:17:33<1:05:53,  5.25s/it]
 76%|███████▌  | 2368/3120 [3:17:38<1:04:58,  5.18s/it]
                                                       
{'loss': 0.1334, 'learning_rate': 3.011853074654515e-06, 'epoch': 3.04}

 76%|███████▌  | 2368/3120 [3:17:38<1:04:58,  5.18s/it]
 76%|███████▌  | 2369/3120 [3:17:43<1:03:55,  5.11s/it]
 76%|███████▌  | 2370/3120 [3:17:48<1:03:31,  5.08s/it]
                                                       
{'loss': 0.1344, 'learning_rate': 2.996705562819747e-06, 'epoch': 3.04}

 76%|███████▌  | 2370/3120 [3:17:48<1:03:31,  5.08s/it]
 76%|███████▌  | 2371/3120 [3:17:53<1:02:05,  4.97s/it]
 76%|███████▌  | 2372/3120 [3:17:58<1:02:43,  5.03s/it]
                                                       
{'loss': 0.1246, 'learning_rate': 2.981589521623973e-06, 'epoch': 3.04}

 76%|███████▌  | 2372/3120 [3:17:58<1:02:43,  5.03s/it]
 76%|███████▌  | 2373/3120 [3:18:03<1:02:40,  5.03s/it]
 76%|███████▌  | 2374/3120 [3:18:08<1:01:02,  4.91s/it]
                                                       
{'loss': 0.1302, 'learning_rate': 2.9665050189940013e-06, 'epoch': 3.04}

 76%|███████▌  | 2374/3120 [3:18:08<1:01:02,  4.91s/it]
 76%|███████▌  | 2375/3120 [3:18:13<1:01:01,  4.92s/it]
 76%|███████▌  | 2376/3120 [3:18:17<59:43,  4.82s/it]  
                                                     
{'loss': 0.1188, 'learning_rate': 2.951452122714926e-06, 'epoch': 3.05}

 76%|███████▌  | 2376/3120 [3:18:17<59:43,  4.82s/it]
 76%|███████▌  | 2377/3120 [3:18:22<1:00:20,  4.87s/it]
 76%|███████▌  | 2378/3120 [3:18:27<1:00:52,  4.92s/it]
                                                       
{'loss': 0.132, 'learning_rate': 2.936430900429805e-06, 'epoch': 3.05}

 76%|███████▌  | 2378/3120 [3:18:27<1:00:52,  4.92s/it]
 76%|███████▋  | 2379/3120 [3:18:32<1:00:18,  4.88s/it]
 76%|███████▋  | 2380/3120 [3:18:37<1:01:15,  4.97s/it]
                                                       
{'loss': 0.1304, 'learning_rate': 2.9214414196393702e-06, 'epoch': 3.05}

 76%|███████▋  | 2380/3120 [3:18:37<1:01:15,  4.97s/it]
 76%|███████▋  | 2381/3120 [3:18:42<1:00:21,  4.90s/it]
 76%|███████▋  | 2382/3120 [3:18:47<1:02:18,  5.07s/it]
                                                       
{'loss': 0.1438, 'learning_rate': 2.906483747701705e-06, 'epoch': 3.05}

 76%|███████▋  | 2382/3120 [3:18:47<1:02:18,  5.07s/it]
 76%|███████▋  | 2383/3120 [3:18:52<1:00:54,  4.96s/it]
 76%|███████▋  | 2384/3120 [3:18:57<1:01:59,  5.05s/it]
                                                       
{'loss': 0.1328, 'learning_rate': 2.8915579518319626e-06, 'epoch': 3.06}

 76%|███████▋  | 2384/3120 [3:18:57<1:01:59,  5.05s/it]
 76%|███████▋  | 2385/3120 [3:19:02<59:56,  4.89s/it]  
 76%|███████▋  | 2386/3120 [3:19:07<1:00:25,  4.94s/it]
                                                       
{'loss': 0.1251, 'learning_rate': 2.8766640991020533e-06, 'epoch': 3.06}

 76%|███████▋  | 2386/3120 [3:19:07<1:00:25,  4.94s/it]
 77%|███████▋  | 2387/3120 [3:19:12<59:13,  4.85s/it]  
 77%|███████▋  | 2388/3120 [3:19:16<58:36,  4.80s/it]
                                                     
{'loss': 0.1324, 'learning_rate': 2.861802256440348e-06, 'epoch': 3.06}

 77%|███████▋  | 2388/3120 [3:19:16<58:36,  4.80s/it]
 77%|███████▋  | 2389/3120 [3:19:21<59:52,  4.91s/it]
 77%|███████▋  | 2390/3120 [3:19:27<1:00:35,  4.98s/it]
                                                       
{'loss': 0.1249, 'learning_rate': 2.8469724906313677e-06, 'epoch': 3.06}

 77%|███████▋  | 2390/3120 [3:19:27<1:00:35,  4.98s/it]
 77%|███████▋  | 2391/3120 [3:19:32<1:01:00,  5.02s/it]
 77%|███████▋  | 2392/3120 [3:19:37<1:01:20,  5.06s/it]
                                                       
{'loss': 0.123, 'learning_rate': 2.8321748683154893e-06, 'epoch': 3.07}

 77%|███████▋  | 2392/3120 [3:19:37<1:01:20,  5.06s/it]
 77%|███████▋  | 2393/3120 [3:19:42<1:00:45,  5.01s/it]
 77%|███████▋  | 2394/3120 [3:19:47<1:00:12,  4.98s/it]
                                                       
{'loss': 0.1236, 'learning_rate': 2.8174094559886535e-06, 'epoch': 3.07}

 77%|███████▋  | 2394/3120 [3:19:47<1:00:12,  4.98s/it]
 77%|███████▋  | 2395/3120 [3:19:51<59:21,  4.91s/it]  
 77%|███████▋  | 2396/3120 [3:19:56<57:45,  4.79s/it]
                                                     
{'loss': 0.1277, 'learning_rate': 2.8026763200020557e-06, 'epoch': 3.07}

 77%|███████▋  | 2396/3120 [3:19:56<57:45,  4.79s/it]
 77%|███████▋  | 2397/3120 [3:20:01<58:39,  4.87s/it]
 77%|███████▋  | 2398/3120 [3:20:05<57:29,  4.78s/it]
                                                     
{'loss': 0.1212, 'learning_rate': 2.7879755265618558e-06, 'epoch': 3.07}

 77%|███████▋  | 2398/3120 [3:20:05<57:29,  4.78s/it]
 77%|███████▋  | 2399/3120 [3:20:10<57:04,  4.75s/it]
 77%|███████▋  | 2400/3120 [3:20:15<56:39,  4.72s/it]
                                                     
{'loss': 0.1219, 'learning_rate': 2.773307141728867e-06, 'epoch': 3.08}

 77%|███████▋  | 2400/3120 [3:20:15<56:39,  4.72s/it]
 77%|███████▋  | 2401/3120 [3:20:20<57:22,  4.79s/it]
 77%|███████▋  | 2402/3120 [3:20:25<58:38,  4.90s/it]
                                                     
{'loss': 0.1199, 'learning_rate': 2.7586712314182775e-06, 'epoch': 3.08}

 77%|███████▋  | 2402/3120 [3:20:25<58:38,  4.90s/it]
 77%|███████▋  | 2403/3120 [3:20:30<57:50,  4.84s/it]
 77%|███████▋  | 2404/3120 [3:20:34<57:46,  4.84s/it]
                                                     
{'loss': 0.134, 'learning_rate': 2.744067861399333e-06, 'epoch': 3.08}

 77%|███████▋  | 2404/3120 [3:20:34<57:46,  4.84s/it]
 77%|███████▋  | 2405/3120 [3:20:40<58:35,  4.92s/it]
 77%|███████▋  | 2406/3120 [3:20:44<56:31,  4.75s/it]
                                                     
{'loss': 0.1293, 'learning_rate': 2.729497097295075e-06, 'epoch': 3.08}

 77%|███████▋  | 2406/3120 [3:20:44<56:31,  4.75s/it]
 77%|███████▋  | 2407/3120 [3:20:49<57:47,  4.86s/it]
 77%|███████▋  | 2408/3120 [3:20:54<58:28,  4.93s/it]
                                                     
{'loss': 0.1308, 'learning_rate': 2.714959004582003e-06, 'epoch': 3.09}

 77%|███████▋  | 2408/3120 [3:20:54<58:28,  4.93s/it]
 77%|███████▋  | 2409/3120 [3:20:59<58:54,  4.97s/it]
 77%|███████▋  | 2410/3120 [3:21:04<57:58,  4.90s/it]
                                                     
{'loss': 0.1208, 'learning_rate': 2.700453648589805e-06, 'epoch': 3.09}

 77%|███████▋  | 2410/3120 [3:21:04<57:58,  4.90s/it]
 77%|███████▋  | 2411/3120 [3:21:09<57:26,  4.86s/it]
 77%|███████▋  | 2412/3120 [3:21:13<56:36,  4.80s/it]
                                                     
{'loss': 0.1265, 'learning_rate': 2.6859810945010687e-06, 'epoch': 3.09}

 77%|███████▋  | 2412/3120 [3:21:13<56:36,  4.80s/it]
 77%|███████▋  | 2413/3120 [3:21:18<57:27,  4.88s/it]
 77%|███████▋  | 2414/3120 [3:21:23<56:39,  4.81s/it]
                                                     
{'loss': 0.1356, 'learning_rate': 2.671541407350975e-06, 'epoch': 3.09}

 77%|███████▋  | 2414/3120 [3:21:23<56:39,  4.81s/it]
 77%|███████▋  | 2415/3120 [3:21:28<55:32,  4.73s/it]
 77%|███████▋  | 2416/3120 [3:21:32<55:26,  4.72s/it]
                                                     
{'loss': 0.1307, 'learning_rate': 2.6571346520270147e-06, 'epoch': 3.1}

 77%|███████▋  | 2416/3120 [3:21:32<55:26,  4.72s/it]
 77%|███████▋  | 2417/3120 [3:21:37<55:10,  4.71s/it]
 78%|███████▊  | 2418/3120 [3:21:42<54:43,  4.68s/it]
                                                     
{'loss': 0.1225, 'learning_rate': 2.642760893268684e-06, 'epoch': 3.1}

 78%|███████▊  | 2418/3120 [3:21:42<54:43,  4.68s/it]
 78%|███████▊  | 2419/3120 [3:21:47<55:55,  4.79s/it]
 78%|███████▊  | 2420/3120 [3:21:52<57:52,  4.96s/it]
                                                     
{'loss': 0.128, 'learning_rate': 2.628420195667214e-06, 'epoch': 3.1}

 78%|███████▊  | 2420/3120 [3:21:52<57:52,  4.96s/it]
 78%|███████▊  | 2421/3120 [3:21:57<58:34,  5.03s/it]
 78%|███████▊  | 2422/3120 [3:22:02<58:06,  4.99s/it]
                                                     
{'loss': 0.1261, 'learning_rate': 2.614112623665259e-06, 'epoch': 3.11}

 78%|███████▊  | 2422/3120 [3:22:02<58:06,  4.99s/it]
 78%|███████▊  | 2423/3120 [3:22:07<58:29,  5.04s/it]
 78%|███████▊  | 2424/3120 [3:22:12<58:31,  5.05s/it]
                                                     
{'loss': 0.1141, 'learning_rate': 2.5998382415566258e-06, 'epoch': 3.11}

 78%|███████▊  | 2424/3120 [3:22:12<58:31,  5.05s/it]
 78%|███████▊  | 2425/3120 [3:22:18<59:21,  5.12s/it]
 78%|███████▊  | 2426/3120 [3:22:23<58:39,  5.07s/it]
                                                     
{'loss': 0.1226, 'learning_rate': 2.5855971134859736e-06, 'epoch': 3.11}

 78%|███████▊  | 2426/3120 [3:22:23<58:39,  5.07s/it]
 78%|███████▊  | 2427/3120 [3:22:28<58:25,  5.06s/it]
 78%|███████▊  | 2428/3120 [3:22:32<57:37,  5.00s/it]
                                                     
{'loss': 0.122, 'learning_rate': 2.5713893034485216e-06, 'epoch': 3.11}

 78%|███████▊  | 2428/3120 [3:22:32<57:37,  5.00s/it]
 78%|███████▊  | 2429/3120 [3:22:37<57:09,  4.96s/it]
 78%|███████▊  | 2430/3120 [3:22:42<56:02,  4.87s/it]
                                                     
{'loss': 0.1346, 'learning_rate': 2.55721487528978e-06, 'epoch': 3.12}

 78%|███████▊  | 2430/3120 [3:22:42<56:02,  4.87s/it]
 78%|███████▊  | 2431/3120 [3:22:47<56:49,  4.95s/it]
 78%|███████▊  | 2432/3120 [3:22:52<56:02,  4.89s/it]
                                                     
{'loss': 0.1393, 'learning_rate': 2.5430738927052346e-06, 'epoch': 3.12}

 78%|███████▊  | 2432/3120 [3:22:52<56:02,  4.89s/it]
 78%|███████▊  | 2433/3120 [3:22:57<55:21,  4.83s/it]
 78%|███████▊  | 2434/3120 [3:23:02<55:44,  4.88s/it]
                                                     
{'loss': 0.1241, 'learning_rate': 2.5289664192400996e-06, 'epoch': 3.12}

 78%|███████▊  | 2434/3120 [3:23:02<55:44,  4.88s/it]
 78%|███████▊  | 2435/3120 [3:23:06<55:41,  4.88s/it]
 78%|███████▊  | 2436/3120 [3:23:11<55:18,  4.85s/it]
                                                     
{'loss': 0.1317, 'learning_rate': 2.514892518288988e-06, 'epoch': 3.12}

 78%|███████▊  | 2436/3120 [3:23:11<55:18,  4.85s/it]
 78%|███████▊  | 2437/3120 [3:23:16<56:20,  4.95s/it]
 78%|███████▊  | 2438/3120 [3:23:21<56:12,  4.94s/it]
                                                     
{'loss': 0.1216, 'learning_rate': 2.5008522530956647e-06, 'epoch': 3.13}

 78%|███████▊  | 2438/3120 [3:23:21<56:12,  4.94s/it]
 78%|███████▊  | 2439/3120 [3:23:26<56:39,  4.99s/it]
 78%|███████▊  | 2440/3120 [3:23:31<56:28,  4.98s/it]
                                                     
{'loss': 0.1126, 'learning_rate': 2.4868456867527315e-06, 'epoch': 3.13}

 78%|███████▊  | 2440/3120 [3:23:31<56:28,  4.98s/it]
 78%|███████▊  | 2441/3120 [3:23:36<56:13,  4.97s/it]
 78%|███████▊  | 2442/3120 [3:23:41<56:34,  5.01s/it]
                                                     
{'loss': 0.1228, 'learning_rate': 2.4728728822013683e-06, 'epoch': 3.13}

 78%|███████▊  | 2442/3120 [3:23:41<56:34,  5.01s/it]
 78%|███████▊  | 2443/3120 [3:23:46<56:41,  5.02s/it]
 78%|███████▊  | 2444/3120 [3:23:51<56:26,  5.01s/it]
                                                     
{'loss': 0.1331, 'learning_rate': 2.4589339022310386e-06, 'epoch': 3.13}

 78%|███████▊  | 2444/3120 [3:23:51<56:26,  5.01s/it]
 78%|███████▊  | 2445/3120 [3:23:57<56:38,  5.03s/it]
 78%|███████▊  | 2446/3120 [3:24:02<57:11,  5.09s/it]
                                                     
{'loss': 0.1179, 'learning_rate': 2.445028809479203e-06, 'epoch': 3.14}

 78%|███████▊  | 2446/3120 [3:24:02<57:11,  5.09s/it]
 78%|███████▊  | 2447/3120 [3:24:07<56:11,  5.01s/it]
 78%|███████▊  | 2448/3120 [3:24:12<56:23,  5.03s/it]
                                                     
{'loss': 0.1303, 'learning_rate': 2.431157666431052e-06, 'epoch': 3.14}

 78%|███████▊  | 2448/3120 [3:24:12<56:23,  5.03s/it]
 78%|███████▊  | 2449/3120 [3:24:17<56:33,  5.06s/it]
 79%|███████▊  | 2450/3120 [3:24:22<57:02,  5.11s/it]
                                                     
{'loss': 0.1261, 'learning_rate': 2.4173205354192065e-06, 'epoch': 3.14}

 79%|███████▊  | 2450/3120 [3:24:22<57:02,  5.11s/it]
 79%|███████▊  | 2451/3120 [3:24:27<56:31,  5.07s/it]
 79%|███████▊  | 2452/3120 [3:24:32<54:56,  4.94s/it]
                                                     
{'loss': 0.1229, 'learning_rate': 2.403517478623456e-06, 'epoch': 3.14}

 79%|███████▊  | 2452/3120 [3:24:32<54:56,  4.94s/it]
 79%|███████▊  | 2453/3120 [3:24:37<55:58,  5.04s/it]
 79%|███████▊  | 2454/3120 [3:24:42<55:49,  5.03s/it]
                                                     
{'loss': 0.1267, 'learning_rate': 2.3897485580704684e-06, 'epoch': 3.15}

 79%|███████▊  | 2454/3120 [3:24:42<55:49,  5.03s/it]
 79%|███████▊  | 2455/3120 [3:24:47<54:56,  4.96s/it]
 79%|███████▊  | 2456/3120 [3:24:52<55:14,  4.99s/it]
                                                     
{'loss': 0.1336, 'learning_rate': 2.3760138356335172e-06, 'epoch': 3.15}

 79%|███████▊  | 2456/3120 [3:24:52<55:14,  4.99s/it]
 79%|███████▉  | 2457/3120 [3:24:56<53:34,  4.85s/it]
 79%|███████▉  | 2458/3120 [3:25:02<54:46,  4.96s/it]
                                                     
{'loss': 0.1331, 'learning_rate': 2.3623133730321923e-06, 'epoch': 3.15}

 79%|███████▉  | 2458/3120 [3:25:02<54:46,  4.96s/it]
 79%|███████▉  | 2459/3120 [3:25:07<54:41,  4.96s/it]
 79%|███████▉  | 2460/3120 [3:25:11<54:26,  4.95s/it]
                                                     
{'loss': 0.1206, 'learning_rate': 2.348647231832131e-06, 'epoch': 3.15}

 79%|███████▉  | 2460/3120 [3:25:11<54:26,  4.95s/it]
 79%|███████▉  | 2461/3120 [3:25:16<54:22,  4.95s/it]
 79%|███████▉  | 2462/3120 [3:25:21<53:29,  4.88s/it]
                                                     
{'loss': 0.1218, 'learning_rate': 2.335015473444754e-06, 'epoch': 3.16}

 79%|███████▉  | 2462/3120 [3:25:21<53:29,  4.88s/it]
 79%|███████▉  | 2463/3120 [3:25:26<53:29,  4.88s/it]
 79%|███████▉  | 2464/3120 [3:25:31<52:56,  4.84s/it]
                                                     
{'loss': 0.1079, 'learning_rate': 2.3214181591269603e-06, 'epoch': 3.16}

 79%|███████▉  | 2464/3120 [3:25:31<52:56,  4.84s/it]
 79%|███████▉  | 2465/3120 [3:25:36<54:18,  4.97s/it]
 79%|███████▉  | 2466/3120 [3:25:41<54:18,  4.98s/it]
                                                     
{'loss': 0.1292, 'learning_rate': 2.30785534998088e-06, 'epoch': 3.16}

 79%|███████▉  | 2466/3120 [3:25:41<54:18,  4.98s/it]
 79%|███████▉  | 2467/3120 [3:25:46<54:05,  4.97s/it]
 79%|███████▉  | 2468/3120 [3:25:51<53:54,  4.96s/it]
                                                     
{'loss': 0.1182, 'learning_rate': 2.2943271069535754e-06, 'epoch': 3.16}

 79%|███████▉  | 2468/3120 [3:25:51<53:54,  4.96s/it]
 79%|███████▉  | 2469/3120 [3:25:56<54:46,  5.05s/it]
 79%|███████▉  | 2470/3120 [3:26:01<54:36,  5.04s/it]
                                                     
{'loss': 0.1204, 'learning_rate': 2.2808334908367914e-06, 'epoch': 3.17}

 79%|███████▉  | 2470/3120 [3:26:01<54:36,  5.04s/it]
 79%|███████▉  | 2471/3120 [3:26:06<53:34,  4.95s/it]
 79%|███████▉  | 2472/3120 [3:26:11<54:11,  5.02s/it]
                                                     
{'loss': 0.1376, 'learning_rate': 2.267374562266662e-06, 'epoch': 3.17}

 79%|███████▉  | 2472/3120 [3:26:11<54:11,  5.02s/it]
 79%|███████▉  | 2473/3120 [3:26:16<55:05,  5.11s/it]
 79%|███████▉  | 2474/3120 [3:26:21<53:47,  5.00s/it]
                                                     
{'loss': 0.1348, 'learning_rate': 2.2539503817234553e-06, 'epoch': 3.17}

 79%|███████▉  | 2474/3120 [3:26:21<53:47,  5.00s/it]
 79%|███████▉  | 2475/3120 [3:26:26<54:04,  5.03s/it]
 79%|███████▉  | 2476/3120 [3:26:32<55:03,  5.13s/it]
                                                     
{'loss': 0.1212, 'learning_rate': 2.240561009531281e-06, 'epoch': 3.17}

 79%|███████▉  | 2476/3120 [3:26:32<55:03,  5.13s/it]
 79%|███████▉  | 2477/3120 [3:26:37<55:06,  5.14s/it]
 79%|███████▉  | 2478/3120 [3:26:42<54:54,  5.13s/it]
                                                     
{'loss': 0.1257, 'learning_rate': 2.227206505857834e-06, 'epoch': 3.18}

 79%|███████▉  | 2478/3120 [3:26:42<54:54,  5.13s/it]
 79%|███████▉  | 2479/3120 [3:26:47<54:52,  5.14s/it]
 79%|███████▉  | 2480/3120 [3:26:52<53:37,  5.03s/it]
                                                     
{'loss': 0.1218, 'learning_rate': 2.2138869307141266e-06, 'epoch': 3.18}

 79%|███████▉  | 2480/3120 [3:26:52<53:37,  5.03s/it]
 80%|███████▉  | 2481/3120 [3:26:57<54:19,  5.10s/it]
 80%|███████▉  | 2482/3120 [3:27:02<54:34,  5.13s/it]
                                                     
{'loss': 0.1254, 'learning_rate': 2.200602343954209e-06, 'epoch': 3.18}

 80%|███████▉  | 2482/3120 [3:27:02<54:34,  5.13s/it]
 80%|███████▉  | 2483/3120 [3:27:07<54:10,  5.10s/it]
 80%|███████▉  | 2484/3120 [3:27:12<52:43,  4.97s/it]
                                                     
{'loss': 0.1212, 'learning_rate': 2.1873528052749094e-06, 'epoch': 3.18}

 80%|███████▉  | 2484/3120 [3:27:12<52:43,  4.97s/it]
 80%|███████▉  | 2485/3120 [3:27:17<52:41,  4.98s/it]
 80%|███████▉  | 2486/3120 [3:27:22<51:31,  4.88s/it]
                                                     
{'loss': 0.1241, 'learning_rate': 2.1741383742155476e-06, 'epoch': 3.19}

 80%|███████▉  | 2486/3120 [3:27:22<51:31,  4.88s/it]
 80%|███████▉  | 2487/3120 [3:27:27<52:41,  5.00s/it]
 80%|███████▉  | 2488/3120 [3:27:32<53:31,  5.08s/it]
                                                     
{'loss': 0.1176, 'learning_rate': 2.1609591101576945e-06, 'epoch': 3.19}

 80%|███████▉  | 2488/3120 [3:27:32<53:31,  5.08s/it]
 80%|███████▉  | 2489/3120 [3:27:37<53:32,  5.09s/it]
 80%|███████▉  | 2490/3120 [3:27:43<54:43,  5.21s/it]
                                                     
{'loss': 0.1265, 'learning_rate': 2.147815072324886e-06, 'epoch': 3.19}

 80%|███████▉  | 2490/3120 [3:27:43<54:43,  5.21s/it]
 80%|███████▉  | 2491/3120 [3:27:48<53:08,  5.07s/it]
 80%|███████▉  | 2492/3120 [3:27:53<53:54,  5.15s/it]
                                                     
{'loss': 0.1251, 'learning_rate': 2.1347063197823648e-06, 'epoch': 3.19}

 80%|███████▉  | 2492/3120 [3:27:53<53:54,  5.15s/it]
 80%|███████▉  | 2493/3120 [3:27:59<55:31,  5.31s/it]
 80%|███████▉  | 2494/3120 [3:28:04<54:45,  5.25s/it]
                                                     
{'loss': 0.1243, 'learning_rate': 2.1216329114368083e-06, 'epoch': 3.2}

 80%|███████▉  | 2494/3120 [3:28:04<54:45,  5.25s/it]
 80%|███████▉  | 2495/3120 [3:28:08<52:24,  5.03s/it]
 80%|████████  | 2496/3120 [3:28:13<53:05,  5.10s/it]
                                                     
{'loss': 0.136, 'learning_rate': 2.1085949060360654e-06, 'epoch': 3.2}

 80%|████████  | 2496/3120 [3:28:13<53:05,  5.10s/it]
 80%|████████  | 2497/3120 [3:28:18<51:06,  4.92s/it]
 80%|████████  | 2498/3120 [3:28:23<51:56,  5.01s/it]
                                                     
{'loss': 0.1278, 'learning_rate': 2.0955923621689034e-06, 'epoch': 3.2}

 80%|████████  | 2498/3120 [3:28:23<51:56,  5.01s/it]
 80%|████████  | 2499/3120 [3:28:28<51:31,  4.98s/it]
 80%|████████  | 2500/3120 [3:28:33<51:32,  4.99s/it]
                                                     
{'loss': 0.1275, 'learning_rate': 2.0826253382647334e-06, 'epoch': 3.21}

 80%|████████  | 2500/3120 [3:28:33<51:32,  4.99s/it]
 80%|████████  | 2501/3120 [3:28:38<50:26,  4.89s/it]
 80%|████████  | 2502/3120 [3:28:43<51:04,  4.96s/it]
                                                     
{'loss': 0.136, 'learning_rate': 2.0696938925933505e-06, 'epoch': 3.21}

 80%|████████  | 2502/3120 [3:28:43<51:04,  4.96s/it]
 80%|████████  | 2503/3120 [3:28:48<51:57,  5.05s/it]
 80%|████████  | 2504/3120 [3:28:53<50:22,  4.91s/it]
                                                     
{'loss': 0.1118, 'learning_rate': 2.056798083264667e-06, 'epoch': 3.21}

 80%|████████  | 2504/3120 [3:28:53<50:22,  4.91s/it]
 80%|████████  | 2505/3120 [3:28:58<51:19,  5.01s/it]
 80%|████████  | 2506/3120 [3:29:04<53:05,  5.19s/it]
                                                     
{'loss': 0.1247, 'learning_rate': 2.0439379682284686e-06, 'epoch': 3.21}

 80%|████████  | 2506/3120 [3:29:04<53:05,  5.19s/it]
 80%|████████  | 2507/3120 [3:29:08<51:02,  5.00s/it]
 80%|████████  | 2508/3120 [3:29:13<49:59,  4.90s/it]
                                                     
{'loss': 0.1344, 'learning_rate': 2.0311136052741274e-06, 'epoch': 3.22}

 80%|████████  | 2508/3120 [3:29:13<49:59,  4.90s/it]
 80%|████████  | 2509/3120 [3:29:17<48:48,  4.79s/it]
 80%|████████  | 2510/3120 [3:29:22<49:48,  4.90s/it]
                                                     
{'loss': 0.123, 'learning_rate': 2.0183250520303653e-06, 'epoch': 3.22}

 80%|████████  | 2510/3120 [3:29:22<49:48,  4.90s/it]
 80%|████████  | 2511/3120 [3:29:27<48:53,  4.82s/it]
 81%|████████  | 2512/3120 [3:29:33<51:38,  5.10s/it]
                                                     
{'loss': 0.113, 'learning_rate': 2.0055723659649907e-06, 'epoch': 3.22}

 81%|████████  | 2512/3120 [3:29:33<51:38,  5.10s/it]
 81%|████████  | 2513/3120 [3:29:38<51:29,  5.09s/it]
 81%|████████  | 2514/3120 [3:29:43<50:20,  4.98s/it]
                                                     
{'loss': 0.1266, 'learning_rate': 1.9928556043846215e-06, 'epoch': 3.22}

 81%|████████  | 2514/3120 [3:29:43<50:20,  4.98s/it]
 81%|████████  | 2515/3120 [3:29:48<50:33,  5.01s/it]
 81%|████████  | 2516/3120 [3:29:53<50:46,  5.04s/it]
                                                     
{'loss': 0.1277, 'learning_rate': 1.9801748244344587e-06, 'epoch': 3.23}

 81%|████████  | 2516/3120 [3:29:53<50:46,  5.04s/it]
 81%|████████  | 2517/3120 [3:29:58<50:50,  5.06s/it]
 81%|████████  | 2518/3120 [3:30:03<51:18,  5.11s/it]
                                                     
{'loss': 0.1283, 'learning_rate': 1.967530083097996e-06, 'epoch': 3.23}

 81%|████████  | 2518/3120 [3:30:03<51:18,  5.11s/it]
 81%|████████  | 2519/3120 [3:30:08<49:58,  4.99s/it]
 81%|████████  | 2520/3120 [3:30:13<49:46,  4.98s/it]
                                                     
{'loss': 0.1212, 'learning_rate': 1.9549214371968008e-06, 'epoch': 3.23}

 81%|████████  | 2520/3120 [3:30:13<49:46,  4.98s/it]
 81%|████████  | 2521/3120 [3:30:17<48:11,  4.83s/it]
 81%|████████  | 2522/3120 [3:30:22<48:47,  4.90s/it]
                                                     
{'loss': 0.1186, 'learning_rate': 1.9423489433902186e-06, 'epoch': 3.23}

 81%|████████  | 2522/3120 [3:30:22<48:47,  4.90s/it]
 81%|████████  | 2523/3120 [3:30:27<49:21,  4.96s/it]
 81%|████████  | 2524/3120 [3:30:33<50:59,  5.13s/it]
                                                     
{'loss': 0.118, 'learning_rate': 1.9298126581751542e-06, 'epoch': 3.24}

 81%|████████  | 2524/3120 [3:30:33<50:59,  5.13s/it]
 81%|████████  | 2525/3120 [3:30:38<50:20,  5.08s/it]
 81%|████████  | 2526/3120 [3:30:43<49:50,  5.03s/it]
                                                     
{'loss': 0.1188, 'learning_rate': 1.917312637885791e-06, 'epoch': 3.24}

 81%|████████  | 2526/3120 [3:30:43<49:50,  5.03s/it]
 81%|████████  | 2527/3120 [3:30:49<51:39,  5.23s/it]
 81%|████████  | 2528/3120 [3:30:53<50:10,  5.09s/it]
                                                     
{'loss': 0.1348, 'learning_rate': 1.9048489386933545e-06, 'epoch': 3.24}

 81%|████████  | 2528/3120 [3:30:53<50:10,  5.09s/it]
 81%|████████  | 2529/3120 [3:30:58<49:46,  5.05s/it]
 81%|████████  | 2530/3120 [3:31:03<48:13,  4.90s/it]
                                                     
{'loss': 0.1427, 'learning_rate': 1.8924216166058574e-06, 'epoch': 3.24}

 81%|████████  | 2530/3120 [3:31:03<48:13,  4.90s/it]
 81%|████████  | 2531/3120 [3:31:08<48:04,  4.90s/it]
 81%|████████  | 2532/3120 [3:31:12<47:15,  4.82s/it]
                                                     
{'loss': 0.1329, 'learning_rate': 1.8800307274678364e-06, 'epoch': 3.25}

 81%|████████  | 2532/3120 [3:31:12<47:15,  4.82s/it]
 81%|████████  | 2533/3120 [3:31:18<48:37,  4.97s/it]
 81%|████████  | 2534/3120 [3:31:22<47:55,  4.91s/it]
                                                     
{'loss': 0.125, 'learning_rate': 1.8676763269601195e-06, 'epoch': 3.25}

 81%|████████  | 2534/3120 [3:31:22<47:55,  4.91s/it]
 81%|████████▏ | 2535/3120 [3:31:27<46:40,  4.79s/it]
 81%|████████▏ | 2536/3120 [3:31:32<46:40,  4.79s/it]
                                                     
{'loss': 0.1309, 'learning_rate': 1.8553584705995564e-06, 'epoch': 3.25}

 81%|████████▏ | 2536/3120 [3:31:32<46:40,  4.79s/it]
 81%|████████▏ | 2537/3120 [3:31:37<47:53,  4.93s/it]
 81%|████████▏ | 2538/3120 [3:31:42<48:51,  5.04s/it]
                                                     
{'loss': 0.129, 'learning_rate': 1.8430772137387853e-06, 'epoch': 3.25}

 81%|████████▏ | 2538/3120 [3:31:42<48:51,  5.04s/it]
 81%|████████▏ | 2539/3120 [3:31:47<48:02,  4.96s/it]
 81%|████████▏ | 2540/3120 [3:31:52<48:49,  5.05s/it]
                                                     
{'loss': 0.1235, 'learning_rate': 1.8308326115659757e-06, 'epoch': 3.26}

 81%|████████▏ | 2540/3120 [3:31:52<48:49,  5.05s/it]
 81%|████████▏ | 2541/3120 [3:31:58<49:33,  5.14s/it]
 81%|████████▏ | 2542/3120 [3:32:02<48:15,  5.01s/it]
                                                     
{'loss': 0.1207, 'learning_rate': 1.8186247191045857e-06, 'epoch': 3.26}

 81%|████████▏ | 2542/3120 [3:32:02<48:15,  5.01s/it]
 82%|████████▏ | 2543/3120 [3:32:07<48:23,  5.03s/it]
 82%|████████▏ | 2544/3120 [3:32:12<47:51,  4.99s/it]
                                                     
{'loss': 0.1282, 'learning_rate': 1.8064535912131032e-06, 'epoch': 3.26}

 82%|████████▏ | 2544/3120 [3:32:12<47:51,  4.99s/it]
 82%|████████▏ | 2545/3120 [3:32:17<46:22,  4.84s/it]
 82%|████████▏ | 2546/3120 [3:32:22<45:54,  4.80s/it]
                                                     
{'loss': 0.1158, 'learning_rate': 1.7943192825848133e-06, 'epoch': 3.26}

 82%|████████▏ | 2546/3120 [3:32:22<45:54,  4.80s/it]
 82%|████████▏ | 2547/3120 [3:32:27<46:14,  4.84s/it]
 82%|████████▏ | 2548/3120 [3:32:32<47:07,  4.94s/it]
                                                     
{'loss': 0.1288, 'learning_rate': 1.7822218477475496e-06, 'epoch': 3.27}

 82%|████████▏ | 2548/3120 [3:32:32<47:07,  4.94s/it]
 82%|████████▏ | 2549/3120 [3:32:37<46:58,  4.94s/it]
 82%|████████▏ | 2550/3120 [3:32:42<46:55,  4.94s/it]
                                                     
{'loss': 0.1245, 'learning_rate': 1.7701613410634367e-06, 'epoch': 3.27}

 82%|████████▏ | 2550/3120 [3:32:42<46:55,  4.94s/it]
 82%|████████▏ | 2551/3120 [3:32:47<47:24,  5.00s/it]
 82%|████████▏ | 2552/3120 [3:32:51<46:26,  4.91s/it]
                                                     
{'loss': 0.1259, 'learning_rate': 1.7581378167286655e-06, 'epoch': 3.27}

 82%|████████▏ | 2552/3120 [3:32:51<46:26,  4.91s/it]
 82%|████████▏ | 2553/3120 [3:32:56<46:36,  4.93s/it]
 82%|████████▏ | 2554/3120 [3:33:01<46:20,  4.91s/it]
                                                     
{'loss': 0.1144, 'learning_rate': 1.7461513287732313e-06, 'epoch': 3.27}

 82%|████████▏ | 2554/3120 [3:33:01<46:20,  4.91s/it]
 82%|████████▏ | 2555/3120 [3:33:06<45:11,  4.80s/it]
 82%|████████▏ | 2556/3120 [3:33:11<46:03,  4.90s/it]
                                                     
{'loss': 0.1242, 'learning_rate': 1.7342019310607062e-06, 'epoch': 3.28}

 82%|████████▏ | 2556/3120 [3:33:11<46:03,  4.90s/it]
 82%|████████▏ | 2557/3120 [3:33:16<47:29,  5.06s/it]
 82%|████████▏ | 2558/3120 [3:33:21<47:14,  5.04s/it]
                                                     
{'loss': 0.1128, 'learning_rate': 1.7222896772879871e-06, 'epoch': 3.28}

 82%|████████▏ | 2558/3120 [3:33:21<47:14,  5.04s/it]
 82%|████████▏ | 2559/3120 [3:33:26<47:10,  5.05s/it]
 82%|████████▏ | 2560/3120 [3:33:32<47:54,  5.13s/it]
                                                     
{'loss': 0.1197, 'learning_rate': 1.7104146209850591e-06, 'epoch': 3.28}

 82%|████████▏ | 2560/3120 [3:33:32<47:54,  5.13s/it]
 82%|████████▏ | 2561/3120 [3:33:36<46:35,  5.00s/it]
 82%|████████▏ | 2562/3120 [3:33:41<45:11,  4.86s/it]
                                                     
{'loss': 0.1195, 'learning_rate': 1.6985768155147498e-06, 'epoch': 3.28}

 82%|████████▏ | 2562/3120 [3:33:41<45:11,  4.86s/it]
 82%|████████▏ | 2563/3120 [3:33:46<45:55,  4.95s/it]
 82%|████████▏ | 2564/3120 [3:33:51<45:32,  4.91s/it]
                                                     
{'loss': 0.1182, 'learning_rate': 1.6867763140724969e-06, 'epoch': 3.29}

 82%|████████▏ | 2564/3120 [3:33:51<45:32,  4.91s/it]
 82%|████████▏ | 2565/3120 [3:33:56<45:11,  4.89s/it]
 82%|████████▏ | 2566/3120 [3:34:01<44:55,  4.86s/it]
                                                     
{'loss': 0.1248, 'learning_rate': 1.6750131696861016e-06, 'epoch': 3.29}

 82%|████████▏ | 2566/3120 [3:34:01<44:55,  4.86s/it]
 82%|████████▏ | 2567/3120 [3:34:05<44:34,  4.84s/it]
 82%|████████▏ | 2568/3120 [3:34:11<45:58,  5.00s/it]
                                                     
{'loss': 0.1273, 'learning_rate': 1.6632874352154982e-06, 'epoch': 3.29}

 82%|████████▏ | 2568/3120 [3:34:11<45:58,  5.00s/it]
 82%|████████▏ | 2569/3120 [3:34:16<46:00,  5.01s/it]
 82%|████████▏ | 2570/3120 [3:34:21<46:35,  5.08s/it]
                                                     
{'loss': 0.1213, 'learning_rate': 1.6515991633525118e-06, 'epoch': 3.29}

 82%|████████▏ | 2570/3120 [3:34:21<46:35,  5.08s/it]
 82%|████████▏ | 2571/3120 [3:34:26<45:53,  5.02s/it]
 82%|████████▏ | 2572/3120 [3:34:31<45:26,  4.98s/it]
                                                     
{'loss': 0.1178, 'learning_rate': 1.6399484066206183e-06, 'epoch': 3.3}

 82%|████████▏ | 2572/3120 [3:34:31<45:26,  4.98s/it]
 82%|████████▏ | 2573/3120 [3:34:35<43:41,  4.79s/it]
 82%|████████▎ | 2574/3120 [3:34:40<44:41,  4.91s/it]
                                                     
{'loss': 0.1275, 'learning_rate': 1.6283352173747148e-06, 'epoch': 3.3}

 82%|████████▎ | 2574/3120 [3:34:40<44:41,  4.91s/it]
 83%|████████▎ | 2575/3120 [3:34:45<43:46,  4.82s/it]
 83%|████████▎ | 2576/3120 [3:34:50<43:54,  4.84s/it]
                                                     
{'loss': 0.136, 'learning_rate': 1.6167596478008817e-06, 'epoch': 3.3}

 83%|████████▎ | 2576/3120 [3:34:50<43:54,  4.84s/it]
 83%|████████▎ | 2577/3120 [3:34:55<44:46,  4.95s/it]
 83%|████████▎ | 2578/3120 [3:35:00<43:51,  4.86s/it]
                                                     
{'loss': 0.1204, 'learning_rate': 1.6052217499161516e-06, 'epoch': 3.31}

 83%|████████▎ | 2578/3120 [3:35:00<43:51,  4.86s/it]
 83%|████████▎ | 2579/3120 [3:35:05<44:07,  4.89s/it]
 83%|████████▎ | 2580/3120 [3:35:10<44:37,  4.96s/it]
                                                     
{'loss': 0.1268, 'learning_rate': 1.5937215755682667e-06, 'epoch': 3.31}

 83%|████████▎ | 2580/3120 [3:35:10<44:37,  4.96s/it]
 83%|████████▎ | 2581/3120 [3:35:15<44:18,  4.93s/it]
 83%|████████▎ | 2582/3120 [3:35:19<43:11,  4.82s/it]
                                                     
{'loss': 0.1272, 'learning_rate': 1.5822591764354512e-06, 'epoch': 3.31}

 83%|████████▎ | 2582/3120 [3:35:19<43:11,  4.82s/it]
 83%|████████▎ | 2583/3120 [3:35:24<43:12,  4.83s/it]
 83%|████████▎ | 2584/3120 [3:35:29<44:00,  4.93s/it]
                                                     
{'loss': 0.1187, 'learning_rate': 1.5708346040261812e-06, 'epoch': 3.31}

 83%|████████▎ | 2584/3120 [3:35:29<44:00,  4.93s/it]
 83%|████████▎ | 2585/3120 [3:35:34<44:03,  4.94s/it]
 83%|████████▎ | 2586/3120 [3:35:39<44:23,  4.99s/it]
                                                     
{'loss': 0.1303, 'learning_rate': 1.559447909678954e-06, 'epoch': 3.32}

 83%|████████▎ | 2586/3120 [3:35:39<44:23,  4.99s/it]
 83%|████████▎ | 2587/3120 [3:35:44<44:04,  4.96s/it]
 83%|████████▎ | 2588/3120 [3:35:49<43:22,  4.89s/it]
                                                     
{'loss': 0.1261, 'learning_rate': 1.5480991445620541e-06, 'epoch': 3.32}

 83%|████████▎ | 2588/3120 [3:35:49<43:22,  4.89s/it]
 83%|████████▎ | 2589/3120 [3:35:54<43:19,  4.90s/it]
 83%|████████▎ | 2590/3120 [3:35:59<43:11,  4.89s/it]
                                                     
{'loss': 0.1215, 'learning_rate': 1.5367883596733157e-06, 'epoch': 3.32}

 83%|████████▎ | 2590/3120 [3:35:59<43:11,  4.89s/it]
 83%|████████▎ | 2591/3120 [3:36:04<44:19,  5.03s/it]
 83%|████████▎ | 2592/3120 [3:36:09<43:19,  4.92s/it]
                                                     
{'loss': 0.1355, 'learning_rate': 1.5255156058399124e-06, 'epoch': 3.32}

 83%|████████▎ | 2592/3120 [3:36:09<43:19,  4.92s/it]
 83%|████████▎ | 2593/3120 [3:36:14<44:20,  5.05s/it]
 83%|████████▎ | 2594/3120 [3:36:19<44:45,  5.11s/it]
                                                     
{'loss': 0.13, 'learning_rate': 1.5142809337181063e-06, 'epoch': 3.33}

 83%|████████▎ | 2594/3120 [3:36:19<44:45,  5.11s/it]
 83%|████████▎ | 2595/3120 [3:36:24<44:38,  5.10s/it]
 83%|████████▎ | 2596/3120 [3:36:29<43:49,  5.02s/it]
                                                     
{'loss': 0.1094, 'learning_rate': 1.5030843937930485e-06, 'epoch': 3.33}

 83%|████████▎ | 2596/3120 [3:36:29<43:49,  5.02s/it]
 83%|████████▎ | 2597/3120 [3:36:35<44:48,  5.14s/it]
 83%|████████▎ | 2598/3120 [3:36:40<44:35,  5.13s/it]
                                                     
{'loss': 0.1251, 'learning_rate': 1.4919260363785215e-06, 'epoch': 3.33}

 83%|████████▎ | 2598/3120 [3:36:40<44:35,  5.13s/it]
 83%|████████▎ | 2599/3120 [3:36:45<45:23,  5.23s/it]
 83%|████████▎ | 2600/3120 [3:36:50<44:02,  5.08s/it]
                                                     
{'loss': 0.1266, 'learning_rate': 1.4808059116167306e-06, 'epoch': 3.33}

 83%|████████▎ | 2600/3120 [3:36:50<44:02,  5.08s/it]
 83%|████████▎ | 2601/3120 [3:36:55<44:26,  5.14s/it]
 83%|████████▎ | 2602/3120 [3:37:00<44:20,  5.14s/it]
                                                     
{'loss': 0.1176, 'learning_rate': 1.4697240694780769e-06, 'epoch': 3.34}

 83%|████████▎ | 2602/3120 [3:37:00<44:20,  5.14s/it]
 83%|████████▎ | 2603/3120 [3:37:05<43:36,  5.06s/it]
 83%|████████▎ | 2604/3120 [3:37:10<44:00,  5.12s/it]
                                                     
{'loss': 0.1426, 'learning_rate': 1.4586805597609333e-06, 'epoch': 3.34}

 83%|████████▎ | 2604/3120 [3:37:10<44:00,  5.12s/it]
 83%|████████▎ | 2605/3120 [3:37:16<44:05,  5.14s/it]
 84%|████████▎ | 2606/3120 [3:37:20<42:55,  5.01s/it]
                                                     
{'loss': 0.1218, 'learning_rate': 1.4476754320914187e-06, 'epoch': 3.34}

 84%|████████▎ | 2606/3120 [3:37:20<42:55,  5.01s/it]
 84%|████████▎ | 2607/3120 [3:37:25<41:49,  4.89s/it]
 84%|████████▎ | 2608/3120 [3:37:30<41:28,  4.86s/it]
                                                     
{'loss': 0.1146, 'learning_rate': 1.4367087359231668e-06, 'epoch': 3.34}

 84%|████████▎ | 2608/3120 [3:37:30<41:28,  4.86s/it]
 84%|████████▎ | 2609/3120 [3:37:35<41:32,  4.88s/it]
 84%|████████▎ | 2610/3120 [3:37:40<42:02,  4.95s/it]
                                                     
{'loss': 0.1221, 'learning_rate': 1.4257805205371233e-06, 'epoch': 3.35}

 84%|████████▎ | 2610/3120 [3:37:40<42:02,  4.95s/it]
 84%|████████▎ | 2611/3120 [3:37:45<41:34,  4.90s/it]
 84%|████████▎ | 2612/3120 [3:37:50<41:49,  4.94s/it]
                                                     
{'loss': 0.1175, 'learning_rate': 1.4148908350413048e-06, 'epoch': 3.35}

 84%|████████▎ | 2612/3120 [3:37:50<41:49,  4.94s/it]
 84%|████████▍ | 2613/3120 [3:37:55<42:10,  4.99s/it]
 84%|████████▍ | 2614/3120 [3:38:00<42:22,  5.02s/it]
                                                     
{'loss': 0.1182, 'learning_rate': 1.4040397283705898e-06, 'epoch': 3.35}

 84%|████████▍ | 2614/3120 [3:38:00<42:22,  5.02s/it]
 84%|████████▍ | 2615/3120 [3:38:04<41:12,  4.90s/it]
 84%|████████▍ | 2616/3120 [3:38:10<41:56,  4.99s/it]
                                                     
{'loss': 0.1177, 'learning_rate': 1.3932272492864984e-06, 'epoch': 3.35}

 84%|████████▍ | 2616/3120 [3:38:10<41:56,  4.99s/it]
 84%|████████▍ | 2617/3120 [3:38:15<42:47,  5.10s/it]
 84%|████████▍ | 2618/3120 [3:38:20<43:38,  5.22s/it]
                                                     
{'loss': 0.1311, 'learning_rate': 1.3824534463769633e-06, 'epoch': 3.36}

 84%|████████▍ | 2618/3120 [3:38:20<43:38,  5.22s/it]
 84%|████████▍ | 2619/3120 [3:38:25<42:34,  5.10s/it]
 84%|████████▍ | 2620/3120 [3:38:30<42:24,  5.09s/it]
                                                     
{'loss': 0.1197, 'learning_rate': 1.3717183680561253e-06, 'epoch': 3.36}

 84%|████████▍ | 2620/3120 [3:38:30<42:24,  5.09s/it]
 84%|████████▍ | 2621/3120 [3:38:35<42:20,  5.09s/it]
 84%|████████▍ | 2622/3120 [3:38:41<42:45,  5.15s/it]
                                                     
{'loss': 0.1288, 'learning_rate': 1.3610220625641002e-06, 'epoch': 3.36}

 84%|████████▍ | 2622/3120 [3:38:41<42:45,  5.15s/it]
 84%|████████▍ | 2623/3120 [3:38:45<41:19,  4.99s/it]
 84%|████████▍ | 2624/3120 [3:38:50<40:08,  4.86s/it]
                                                     
{'loss': 0.1307, 'learning_rate': 1.3503645779667852e-06, 'epoch': 3.36}

 84%|████████▍ | 2624/3120 [3:38:50<40:08,  4.86s/it]
 84%|████████▍ | 2625/3120 [3:38:55<40:01,  4.85s/it]
 84%|████████▍ | 2626/3120 [3:39:00<40:04,  4.87s/it]
                                                     
{'loss': 0.1291, 'learning_rate': 1.339745962155613e-06, 'epoch': 3.37}

 84%|████████▍ | 2626/3120 [3:39:00<40:04,  4.87s/it]
 84%|████████▍ | 2627/3120 [3:39:04<38:34,  4.70s/it]
 84%|████████▍ | 2628/3120 [3:39:09<39:56,  4.87s/it]
                                                     
{'loss': 0.13, 'learning_rate': 1.3291662628473634e-06, 'epoch': 3.37}

 84%|████████▍ | 2628/3120 [3:39:09<39:56,  4.87s/it]
 84%|████████▍ | 2629/3120 [3:39:14<40:45,  4.98s/it]
 84%|████████▍ | 2630/3120 [3:39:20<40:55,  5.01s/it]
                                                     
{'loss': 0.1334, 'learning_rate': 1.3186255275839266e-06, 'epoch': 3.37}

 84%|████████▍ | 2630/3120 [3:39:20<40:55,  5.01s/it]
 84%|████████▍ | 2631/3120 [3:39:24<40:33,  4.98s/it]
 84%|████████▍ | 2632/3120 [3:39:29<40:17,  4.95s/it]
                                                     
{'loss': 0.1305, 'learning_rate': 1.308123803732111e-06, 'epoch': 3.37}

 84%|████████▍ | 2632/3120 [3:39:29<40:17,  4.95s/it]
 84%|████████▍ | 2633/3120 [3:39:35<40:48,  5.03s/it]
 84%|████████▍ | 2634/3120 [3:39:40<41:30,  5.12s/it]
                                                     
{'loss': 0.1264, 'learning_rate': 1.2976611384834148e-06, 'epoch': 3.38}

 84%|████████▍ | 2634/3120 [3:39:40<41:30,  5.12s/it]
 84%|████████▍ | 2635/3120 [3:39:45<41:46,  5.17s/it]
 84%|████████▍ | 2636/3120 [3:39:50<40:36,  5.03s/it]
                                                     
{'loss': 0.1207, 'learning_rate': 1.2872375788538171e-06, 'epoch': 3.38}

 84%|████████▍ | 2636/3120 [3:39:50<40:36,  5.03s/it]
 85%|████████▍ | 2637/3120 [3:39:55<40:14,  5.00s/it]
 85%|████████▍ | 2638/3120 [3:40:00<40:55,  5.09s/it]
                                                     
{'loss': 0.1173, 'learning_rate': 1.2768531716835754e-06, 'epoch': 3.38}

 85%|████████▍ | 2638/3120 [3:40:00<40:55,  5.09s/it]
 85%|████████▍ | 2639/3120 [3:40:05<41:17,  5.15s/it]
 85%|████████▍ | 2640/3120 [3:40:10<39:41,  4.96s/it]
                                                     
{'loss': 0.1223, 'learning_rate': 1.266507963636997e-06, 'epoch': 3.38}

 85%|████████▍ | 2640/3120 [3:40:10<39:41,  4.96s/it]
 85%|████████▍ | 2641/3120 [3:40:15<39:38,  4.97s/it]
 85%|████████▍ | 2642/3120 [3:40:20<39:04,  4.90s/it]
                                                     
{'loss': 0.1222, 'learning_rate': 1.2562020012022512e-06, 'epoch': 3.39}

 85%|████████▍ | 2642/3120 [3:40:20<39:04,  4.90s/it]
 85%|████████▍ | 2643/3120 [3:40:25<39:07,  4.92s/it]
 85%|████████▍ | 2644/3120 [3:40:29<38:10,  4.81s/it]
                                                     
{'loss': 0.1214, 'learning_rate': 1.2459353306911438e-06, 'epoch': 3.39}

 85%|████████▍ | 2644/3120 [3:40:29<38:10,  4.81s/it]
 85%|████████▍ | 2645/3120 [3:40:34<38:19,  4.84s/it]
 85%|████████▍ | 2646/3120 [3:40:39<38:13,  4.84s/it]
                                                     
{'loss': 0.1133, 'learning_rate': 1.2357079982389197e-06, 'epoch': 3.39}

 85%|████████▍ | 2646/3120 [3:40:39<38:13,  4.84s/it]
 85%|████████▍ | 2647/3120 [3:40:44<38:03,  4.83s/it]
 85%|████████▍ | 2648/3120 [3:40:49<38:30,  4.90s/it]
                                                     
{'loss': 0.1208, 'learning_rate': 1.2255200498040432e-06, 'epoch': 3.39}

 85%|████████▍ | 2648/3120 [3:40:49<38:30,  4.90s/it]
 85%|████████▍ | 2649/3120 [3:40:54<38:23,  4.89s/it]
 85%|████████▍ | 2650/3120 [3:40:59<38:46,  4.95s/it]
                                                     
{'loss': 0.1127, 'learning_rate': 1.2153715311680048e-06, 'epoch': 3.4}

 85%|████████▍ | 2650/3120 [3:40:59<38:46,  4.95s/it]
 85%|████████▍ | 2651/3120 [3:41:04<39:21,  5.04s/it]
 85%|████████▌ | 2652/3120 [3:41:09<38:40,  4.96s/it]
                                                     
{'loss': 0.1262, 'learning_rate': 1.2052624879351105e-06, 'epoch': 3.4}

 85%|████████▌ | 2652/3120 [3:41:09<38:40,  4.96s/it]
 85%|████████▌ | 2653/3120 [3:41:14<39:08,  5.03s/it]
 85%|████████▌ | 2654/3120 [3:41:19<39:14,  5.05s/it]
                                                     
{'loss': 0.1257, 'learning_rate': 1.1951929655322692e-06, 'epoch': 3.4}

 85%|████████▌ | 2654/3120 [3:41:19<39:14,  5.05s/it]
 85%|████████▌ | 2655/3120 [3:41:24<38:33,  4.97s/it]
 85%|████████▌ | 2656/3120 [3:41:29<38:55,  5.03s/it]
                                                     
{'loss': 0.1334, 'learning_rate': 1.1851630092088051e-06, 'epoch': 3.41}

 85%|████████▌ | 2656/3120 [3:41:29<38:55,  5.03s/it]
 85%|████████▌ | 2657/3120 [3:41:34<38:02,  4.93s/it]
 85%|████████▌ | 2658/3120 [3:41:39<37:48,  4.91s/it]
                                                     
{'loss': 0.1286, 'learning_rate': 1.175172664036235e-06, 'epoch': 3.41}

 85%|████████▌ | 2658/3120 [3:41:39<37:48,  4.91s/it]
 85%|████████▌ | 2659/3120 [3:41:44<38:39,  5.03s/it]
 85%|████████▌ | 2660/3120 [3:41:49<38:36,  5.04s/it]
                                                     
{'loss': 0.1316, 'learning_rate': 1.1652219749080817e-06, 'epoch': 3.41}

 85%|████████▌ | 2660/3120 [3:41:49<38:36,  5.04s/it]
 85%|████████▌ | 2661/3120 [3:41:54<37:37,  4.92s/it]
 85%|████████▌ | 2662/3120 [3:41:59<38:04,  4.99s/it]
                                                     
{'loss': 0.1305, 'learning_rate': 1.155310986539665e-06, 'epoch': 3.41}

 85%|████████▌ | 2662/3120 [3:41:59<38:04,  4.99s/it]
 85%|████████▌ | 2663/3120 [3:42:04<38:01,  4.99s/it]
 85%|████████▌ | 2664/3120 [3:42:09<38:11,  5.03s/it]
                                                     
{'loss': 0.1179, 'learning_rate': 1.1454397434679022e-06, 'epoch': 3.42}

 85%|████████▌ | 2664/3120 [3:42:09<38:11,  5.03s/it]
 85%|████████▌ | 2665/3120 [3:42:13<37:11,  4.90s/it]
 85%|████████▌ | 2666/3120 [3:42:18<36:45,  4.86s/it]
                                                     
{'loss': 0.1289, 'learning_rate': 1.1356082900511023e-06, 'epoch': 3.42}

 85%|████████▌ | 2666/3120 [3:42:18<36:45,  4.86s/it]
 85%|████████▌ | 2667/3120 [3:42:23<36:49,  4.88s/it]
 86%|████████▌ | 2668/3120 [3:42:28<37:04,  4.92s/it]
                                                     
{'loss': 0.1197, 'learning_rate': 1.12581667046878e-06, 'epoch': 3.42}

 86%|████████▌ | 2668/3120 [3:42:28<37:04,  4.92s/it]
 86%|████████▌ | 2669/3120 [3:42:34<38:24,  5.11s/it]
 86%|████████▌ | 2670/3120 [3:42:39<38:47,  5.17s/it]
                                                     
{'loss': 0.1154, 'learning_rate': 1.116064928721442e-06, 'epoch': 3.42}

 86%|████████▌ | 2670/3120 [3:42:39<38:47,  5.17s/it]
 86%|████████▌ | 2671/3120 [3:42:44<38:18,  5.12s/it]
 86%|████████▌ | 2672/3120 [3:42:49<37:56,  5.08s/it]
                                                     
{'loss': 0.1292, 'learning_rate': 1.1063531086304003e-06, 'epoch': 3.43}

 86%|████████▌ | 2672/3120 [3:42:49<37:56,  5.08s/it]
 86%|████████▌ | 2673/3120 [3:42:54<37:41,  5.06s/it]
 86%|████████▌ | 2674/3120 [3:42:59<36:59,  4.98s/it]
                                                     
{'loss': 0.1225, 'learning_rate': 1.0966812538375738e-06, 'epoch': 3.43}

 86%|████████▌ | 2674/3120 [3:42:59<36:59,  4.98s/it]
 86%|████████▌ | 2675/3120 [3:43:04<37:21,  5.04s/it]
 86%|████████▌ | 2676/3120 [3:43:09<38:05,  5.15s/it]
                                                     
{'loss': 0.1136, 'learning_rate': 1.0870494078052796e-06, 'epoch': 3.43}

 86%|████████▌ | 2676/3120 [3:43:09<38:05,  5.15s/it]
 86%|████████▌ | 2677/3120 [3:43:14<37:04,  5.02s/it]
 86%|████████▌ | 2678/3120 [3:43:19<37:41,  5.12s/it]
                                                     
{'loss': 0.1207, 'learning_rate': 1.0774576138160596e-06, 'epoch': 3.43}

 86%|████████▌ | 2678/3120 [3:43:19<37:41,  5.12s/it]
 86%|████████▌ | 2679/3120 [3:43:24<36:42,  4.99s/it]
 86%|████████▌ | 2680/3120 [3:43:29<36:45,  5.01s/it]
                                                     
{'loss': 0.1334, 'learning_rate': 1.067905914972468e-06, 'epoch': 3.44}

 86%|████████▌ | 2680/3120 [3:43:29<36:45,  5.01s/it]
 86%|████████▌ | 2681/3120 [3:43:34<36:24,  4.98s/it]
 86%|████████▌ | 2682/3120 [3:43:39<37:06,  5.08s/it]
                                                     
{'loss': 0.136, 'learning_rate': 1.0583943541968856e-06, 'epoch': 3.44}

 86%|████████▌ | 2682/3120 [3:43:39<37:06,  5.08s/it]
 86%|████████▌ | 2683/3120 [3:43:45<38:21,  5.27s/it]
 86%|████████▌ | 2684/3120 [3:43:50<37:52,  5.21s/it]
                                                     
{'loss': 0.1279, 'learning_rate': 1.0489229742313223e-06, 'epoch': 3.44}

 86%|████████▌ | 2684/3120 [3:43:50<37:52,  5.21s/it]
 86%|████████▌ | 2685/3120 [3:43:55<37:11,  5.13s/it]
 86%|████████▌ | 2686/3120 [3:44:00<37:09,  5.14s/it]
                                                     
{'loss': 0.1199, 'learning_rate': 1.0394918176372327e-06, 'epoch': 3.44}

 86%|████████▌ | 2686/3120 [3:44:00<37:09,  5.14s/it]
 86%|████████▌ | 2687/3120 [3:44:05<36:55,  5.12s/it]
 86%|████████▌ | 2688/3120 [3:44:10<36:13,  5.03s/it]
                                                     
{'loss': 0.1309, 'learning_rate': 1.0301009267953145e-06, 'epoch': 3.45}

 86%|████████▌ | 2688/3120 [3:44:10<36:13,  5.03s/it]
 86%|████████▌ | 2689/3120 [3:44:15<35:11,  4.90s/it]
 86%|████████▌ | 2690/3120 [3:44:20<35:46,  4.99s/it]
                                                     
{'loss': 0.1151, 'learning_rate': 1.0207503439053279e-06, 'epoch': 3.45}

 86%|████████▌ | 2690/3120 [3:44:20<35:46,  4.99s/it]
 86%|████████▋ | 2691/3120 [3:44:26<37:07,  5.19s/it]
 86%|████████▋ | 2692/3120 [3:44:30<36:13,  5.08s/it]
                                                     
{'loss': 0.1052, 'learning_rate': 1.0114401109859019e-06, 'epoch': 3.45}

 86%|████████▋ | 2692/3120 [3:44:30<36:13,  5.08s/it]
 86%|████████▋ | 2693/3120 [3:44:35<35:28,  4.99s/it]
 86%|████████▋ | 2694/3120 [3:44:41<36:01,  5.07s/it]
                                                     
{'loss': 0.1137, 'learning_rate': 1.0021702698743408e-06, 'epoch': 3.45}

 86%|████████▋ | 2694/3120 [3:44:41<36:01,  5.07s/it]
 86%|████████▋ | 2695/3120 [3:44:46<36:25,  5.14s/it]
 86%|████████▋ | 2696/3120 [3:44:51<35:56,  5.09s/it]
                                                     
{'loss': 0.1258, 'learning_rate': 9.929408622264448e-07, 'epoch': 3.46}

 86%|████████▋ | 2696/3120 [3:44:51<35:56,  5.09s/it]
 86%|████████▋ | 2697/3120 [3:44:56<35:58,  5.10s/it]
 86%|████████▋ | 2698/3120 [3:45:01<35:44,  5.08s/it]
                                                     
{'loss': 0.1157, 'learning_rate': 9.837519295163155e-07, 'epoch': 3.46}

 86%|████████▋ | 2698/3120 [3:45:01<35:44,  5.08s/it]
 87%|████████▋ | 2699/3120 [3:45:06<35:37,  5.08s/it]
 87%|████████▋ | 2700/3120 [3:45:11<34:42,  4.96s/it]
                                                     
{'loss': 0.1131, 'learning_rate': 9.746035130361741e-07, 'epoch': 3.46}

 87%|████████▋ | 2700/3120 [3:45:11<34:42,  4.96s/it]
 87%|████████▋ | 2701/3120 [3:45:16<34:32,  4.95s/it]
 87%|████████▋ | 2702/3120 [3:45:20<34:11,  4.91s/it]
                                                     
{'loss': 0.1176, 'learning_rate': 9.65495653896179e-07, 'epoch': 3.46}

 87%|████████▋ | 2702/3120 [3:45:20<34:11,  4.91s/it]
 87%|████████▋ | 2703/3120 [3:45:25<34:00,  4.89s/it]
 87%|████████▋ | 2704/3120 [3:45:30<34:03,  4.91s/it]
                                                     
{'loss': 0.1165, 'learning_rate': 9.564283930242258e-07, 'epoch': 3.47}

 87%|████████▋ | 2704/3120 [3:45:30<34:03,  4.91s/it]
 87%|████████▋ | 2705/3120 [3:45:35<33:23,  4.83s/it]
 87%|████████▋ | 2706/3120 [3:45:40<33:29,  4.85s/it]
                                                     
{'loss': 0.1168, 'learning_rate': 9.474017711657835e-07, 'epoch': 3.47}

 87%|████████▋ | 2706/3120 [3:45:40<33:29,  4.85s/it]
 87%|████████▋ | 2707/3120 [3:45:45<33:09,  4.82s/it]
 87%|████████▋ | 2708/3120 [3:45:50<33:39,  4.90s/it]
                                                     
{'loss': 0.1243, 'learning_rate': 9.384158288836987e-07, 'epoch': 3.47}

 87%|████████▋ | 2708/3120 [3:45:50<33:39,  4.90s/it]
 87%|████████▋ | 2709/3120 [3:45:54<33:21,  4.87s/it]
 87%|████████▋ | 2710/3120 [3:45:59<33:33,  4.91s/it]
                                                     
{'loss': 0.1134, 'learning_rate': 9.294706065580172e-07, 'epoch': 3.47}

 87%|████████▋ | 2710/3120 [3:45:59<33:33,  4.91s/it]
 87%|████████▋ | 2711/3120 [3:46:05<34:18,  5.03s/it]
 87%|████████▋ | 2712/3120 [3:46:10<34:18,  5.04s/it]
                                                     
{'loss': 0.1404, 'learning_rate': 9.205661443857994e-07, 'epoch': 3.48}

 87%|████████▋ | 2712/3120 [3:46:10<34:18,  5.04s/it]
 87%|████████▋ | 2713/3120 [3:46:15<35:07,  5.18s/it]
 87%|████████▋ | 2714/3120 [3:46:21<35:09,  5.20s/it]
                                                     
{'loss': 0.1233, 'learning_rate': 9.117024823809462e-07, 'epoch': 3.48}

 87%|████████▋ | 2714/3120 [3:46:21<35:09,  5.20s/it]
 87%|████████▋ | 2715/3120 [3:46:26<35:03,  5.19s/it]
 87%|████████▋ | 2716/3120 [3:46:31<34:51,  5.18s/it]
                                                     
{'loss': 0.1295, 'learning_rate': 9.028796603740097e-07, 'epoch': 3.48}

 87%|████████▋ | 2716/3120 [3:46:31<34:51,  5.18s/it]
 87%|████████▋ | 2717/3120 [3:46:36<34:10,  5.09s/it]
 87%|████████▋ | 2718/3120 [3:46:41<33:57,  5.07s/it]
                                                     
{'loss': 0.1089, 'learning_rate': 8.940977180120247e-07, 'epoch': 3.48}

 87%|████████▋ | 2718/3120 [3:46:41<33:57,  5.07s/it]
 87%|████████▋ | 2719/3120 [3:46:46<33:45,  5.05s/it]
 87%|████████▋ | 2720/3120 [3:46:51<33:30,  5.03s/it]
                                                     
{'loss': 0.1252, 'learning_rate': 8.853566947583259e-07, 'epoch': 3.49}

 87%|████████▋ | 2720/3120 [3:46:51<33:30,  5.03s/it]
 87%|████████▋ | 2721/3120 [3:46:55<32:44,  4.92s/it]
 87%|████████▋ | 2722/3120 [3:47:00<32:18,  4.87s/it]
                                                     
{'loss': 0.1215, 'learning_rate': 8.766566298923651e-07, 'epoch': 3.49}

 87%|████████▋ | 2722/3120 [3:47:00<32:18,  4.87s/it]
 87%|████████▋ | 2723/3120 [3:47:05<31:47,  4.80s/it]
 87%|████████▋ | 2724/3120 [3:47:10<31:56,  4.84s/it]
                                                     
{'loss': 0.1299, 'learning_rate': 8.67997562509546e-07, 'epoch': 3.49}

 87%|████████▋ | 2724/3120 [3:47:10<31:56,  4.84s/it]
 87%|████████▋ | 2725/3120 [3:47:15<32:35,  4.95s/it]
 87%|████████▋ | 2726/3120 [3:47:20<32:35,  4.96s/it]
                                                     
{'loss': 0.1278, 'learning_rate': 8.593795315210341e-07, 'epoch': 3.49}

 87%|████████▋ | 2726/3120 [3:47:20<32:35,  4.96s/it]
 87%|████████▋ | 2727/3120 [3:47:25<32:54,  5.02s/it]
 87%|████████▋ | 2728/3120 [3:47:30<31:57,  4.89s/it]
                                                     
{'loss': 0.1151, 'learning_rate': 8.508025756535987e-07, 'epoch': 3.5}

 87%|████████▋ | 2728/3120 [3:47:30<31:57,  4.89s/it]
 87%|████████▋ | 2729/3120 [3:47:35<32:40,  5.02s/it]
 88%|████████▊ | 2730/3120 [3:47:40<32:20,  4.98s/it]
                                                     
{'loss': 0.1218, 'learning_rate': 8.42266733449425e-07, 'epoch': 3.5}

 88%|████████▊ | 2730/3120 [3:47:40<32:20,  4.98s/it]
 88%|████████▊ | 2731/3120 [3:47:45<31:45,  4.90s/it]
 88%|████████▊ | 2732/3120 [3:47:49<31:40,  4.90s/it]
                                                     
{'loss': 0.1245, 'learning_rate': 8.337720432659513e-07, 'epoch': 3.5}

 88%|████████▊ | 2732/3120 [3:47:49<31:40,  4.90s/it]
 88%|████████▊ | 2733/3120 [3:47:54<31:43,  4.92s/it]
 88%|████████▊ | 2734/3120 [3:47:59<31:38,  4.92s/it]
                                                     
{'loss': 0.1219, 'learning_rate': 8.253185432756816e-07, 'epoch': 3.51}

 88%|████████▊ | 2734/3120 [3:47:59<31:38,  4.92s/it]
 88%|████████▊ | 2735/3120 [3:48:04<31:54,  4.97s/it]
 88%|████████▊ | 2736/3120 [3:48:09<31:43,  4.96s/it]
                                                     
{'loss': 0.1172, 'learning_rate': 8.169062714660347e-07, 'epoch': 3.51}

 88%|████████▊ | 2736/3120 [3:48:09<31:43,  4.96s/it]
 88%|████████▊ | 2737/3120 [3:48:14<30:46,  4.82s/it]
 88%|████████▊ | 2738/3120 [3:48:19<30:50,  4.85s/it]
                                                     
{'loss': 0.1283, 'learning_rate': 8.085352656391576e-07, 'epoch': 3.51}

 88%|████████▊ | 2738/3120 [3:48:19<30:50,  4.85s/it]
 88%|████████▊ | 2739/3120 [3:48:24<30:56,  4.87s/it]
 88%|████████▊ | 2740/3120 [3:48:28<30:16,  4.78s/it]
                                                     
{'loss': 0.1163, 'learning_rate': 8.002055634117578e-07, 'epoch': 3.51}

 88%|████████▊ | 2740/3120 [3:48:28<30:16,  4.78s/it]
 88%|████████▊ | 2741/3120 [3:48:33<30:45,  4.87s/it]
 88%|████████▊ | 2742/3120 [3:48:39<31:22,  4.98s/it]
                                                     
{'loss': 0.1344, 'learning_rate': 7.919172022149458e-07, 'epoch': 3.52}

 88%|████████▊ | 2742/3120 [3:48:39<31:22,  4.98s/it]
 88%|████████▊ | 2743/3120 [3:48:43<30:40,  4.88s/it]
 88%|████████▊ | 2744/3120 [3:48:48<30:52,  4.93s/it]
                                                     
{'loss': 0.1333, 'learning_rate': 7.836702192940493e-07, 'epoch': 3.52}

 88%|████████▊ | 2744/3120 [3:48:48<30:52,  4.93s/it]
 88%|████████▊ | 2745/3120 [3:48:53<31:11,  4.99s/it]
 88%|████████▊ | 2746/3120 [3:48:59<31:31,  5.06s/it]
                                                     
{'loss': 0.1251, 'learning_rate': 7.754646517084585e-07, 'epoch': 3.52}

 88%|████████▊ | 2746/3120 [3:48:59<31:31,  5.06s/it]
 88%|████████▊ | 2747/3120 [3:49:03<30:57,  4.98s/it]
 88%|████████▊ | 2748/3120 [3:49:08<30:03,  4.85s/it]
                                                     
{'loss': 0.1139, 'learning_rate': 7.673005363314578e-07, 'epoch': 3.52}

 88%|████████▊ | 2748/3120 [3:49:08<30:03,  4.85s/it]
 88%|████████▊ | 2749/3120 [3:49:13<29:53,  4.83s/it]
 88%|████████▊ | 2750/3120 [3:49:18<31:17,  5.07s/it]
                                                     
{'loss': 0.1332, 'learning_rate': 7.591779098500584e-07, 'epoch': 3.53}

 88%|████████▊ | 2750/3120 [3:49:18<31:17,  5.07s/it]
 88%|████████▊ | 2751/3120 [3:49:24<31:52,  5.18s/it]
 88%|████████▊ | 2752/3120 [3:49:29<31:44,  5.17s/it]
                                                     
{'loss': 0.1195, 'learning_rate': 7.510968087648262e-07, 'epoch': 3.53}

 88%|████████▊ | 2752/3120 [3:49:29<31:44,  5.17s/it]
 88%|████████▊ | 2753/3120 [3:49:34<31:16,  5.11s/it]
 88%|████████▊ | 2754/3120 [3:49:39<30:49,  5.05s/it]
                                                     
{'loss': 0.1266, 'learning_rate': 7.430572693897342e-07, 'epoch': 3.53}

 88%|████████▊ | 2754/3120 [3:49:39<30:49,  5.05s/it]
 88%|████████▊ | 2755/3120 [3:49:44<30:14,  4.97s/it]
 88%|████████▊ | 2756/3120 [3:49:49<31:04,  5.12s/it]
                                                     
{'loss': 0.1197, 'learning_rate': 7.350593278519824e-07, 'epoch': 3.53}

 88%|████████▊ | 2756/3120 [3:49:49<31:04,  5.12s/it]
 88%|████████▊ | 2757/3120 [3:49:54<31:08,  5.15s/it]
 88%|████████▊ | 2758/3120 [3:49:59<30:20,  5.03s/it]
                                                     
{'loss': 0.1265, 'learning_rate': 7.271030200918483e-07, 'epoch': 3.54}

 88%|████████▊ | 2758/3120 [3:49:59<30:20,  5.03s/it]
 88%|████████▊ | 2759/3120 [3:50:04<30:32,  5.08s/it]
 88%|████████▊ | 2760/3120 [3:50:09<30:31,  5.09s/it]
                                                     
{'loss': 0.1323, 'learning_rate': 7.19188381862519e-07, 'epoch': 3.54}

 88%|████████▊ | 2760/3120 [3:50:09<30:31,  5.09s/it]
 88%|████████▊ | 2761/3120 [3:50:14<30:12,  5.05s/it]
 89%|████████▊ | 2762/3120 [3:50:20<30:34,  5.12s/it]
                                                     
{'loss': 0.1295, 'learning_rate': 7.113154487299267e-07, 'epoch': 3.54}

 89%|████████▊ | 2762/3120 [3:50:20<30:34,  5.12s/it]
 89%|████████▊ | 2763/3120 [3:50:24<29:31,  4.96s/it]
 89%|████████▊ | 2764/3120 [3:50:29<29:32,  4.98s/it]
                                                     
{'loss': 0.1257, 'learning_rate': 7.034842560726008e-07, 'epoch': 3.54}

 89%|████████▊ | 2764/3120 [3:50:29<29:32,  4.98s/it]
 89%|████████▊ | 2765/3120 [3:50:34<29:38,  5.01s/it]
 89%|████████▊ | 2766/3120 [3:50:39<29:43,  5.04s/it]
                                                     
{'loss': 0.1181, 'learning_rate': 6.956948390814977e-07, 'epoch': 3.55}

 89%|████████▊ | 2766/3120 [3:50:39<29:43,  5.04s/it]
 89%|████████▊ | 2767/3120 [3:50:44<29:11,  4.96s/it]
 89%|████████▊ | 2768/3120 [3:50:49<28:24,  4.84s/it]
                                                     
{'loss': 0.1172, 'learning_rate': 6.879472327598502e-07, 'epoch': 3.55}

 89%|████████▊ | 2768/3120 [3:50:49<28:24,  4.84s/it]
 89%|████████▉ | 2769/3120 [3:50:54<28:17,  4.84s/it]
 89%|████████▉ | 2770/3120 [3:50:58<28:07,  4.82s/it]
                                                     
{'loss': 0.1159, 'learning_rate': 6.802414719230021e-07, 'epoch': 3.55}

 89%|████████▉ | 2770/3120 [3:50:58<28:07,  4.82s/it]
 89%|████████▉ | 2771/3120 [3:51:03<28:07,  4.84s/it]
 89%|████████▉ | 2772/3120 [3:51:08<28:12,  4.86s/it]
                                                     
{'loss': 0.1198, 'learning_rate': 6.725775911982602e-07, 'epoch': 3.55}

 89%|████████▉ | 2772/3120 [3:51:08<28:12,  4.86s/it]
 89%|████████▉ | 2773/3120 [3:51:13<28:10,  4.87s/it]
 89%|████████▉ | 2774/3120 [3:51:18<28:36,  4.96s/it]
                                                     
{'loss': 0.1183, 'learning_rate': 6.649556250247325e-07, 'epoch': 3.56}

 89%|████████▉ | 2774/3120 [3:51:18<28:36,  4.96s/it]
 89%|████████▉ | 2775/3120 [3:51:24<29:23,  5.11s/it]
 89%|████████▉ | 2776/3120 [3:51:28<28:24,  4.96s/it]
                                                     
{'loss': 0.123, 'learning_rate': 6.573756076531779e-07, 'epoch': 3.56}

 89%|████████▉ | 2776/3120 [3:51:28<28:24,  4.96s/it]
 89%|████████▉ | 2777/3120 [3:51:33<27:40,  4.84s/it]
 89%|████████▉ | 2778/3120 [3:51:38<27:54,  4.90s/it]
                                                     
{'loss': 0.1302, 'learning_rate': 6.498375731458529e-07, 'epoch': 3.56}

 89%|████████▉ | 2778/3120 [3:51:38<27:54,  4.90s/it]
 89%|████████▉ | 2779/3120 [3:51:43<28:14,  4.97s/it]
 89%|████████▉ | 2780/3120 [3:51:48<27:46,  4.90s/it]
                                                     
{'loss': 0.1213, 'learning_rate': 6.423415553763479e-07, 'epoch': 3.56}

 89%|████████▉ | 2780/3120 [3:51:48<27:46,  4.90s/it]
 89%|████████▉ | 2781/3120 [3:51:53<28:17,  5.01s/it]
 89%|████████▉ | 2782/3120 [3:51:58<27:38,  4.91s/it]
                                                     
{'loss': 0.1143, 'learning_rate': 6.348875880294536e-07, 'epoch': 3.57}

 89%|████████▉ | 2782/3120 [3:51:58<27:38,  4.91s/it]
 89%|████████▉ | 2783/3120 [3:52:03<27:49,  4.95s/it]
 89%|████████▉ | 2784/3120 [3:52:08<27:42,  4.95s/it]
                                                     
{'loss': 0.127, 'learning_rate': 6.274757046009871e-07, 'epoch': 3.57}

 89%|████████▉ | 2784/3120 [3:52:08<27:42,  4.95s/it]
 89%|████████▉ | 2785/3120 [3:52:13<27:52,  4.99s/it]
 89%|████████▉ | 2786/3120 [3:52:18<28:17,  5.08s/it]
                                                     
{'loss': 0.1211, 'learning_rate': 6.201059383976671e-07, 'epoch': 3.57}

 89%|████████▉ | 2786/3120 [3:52:18<28:17,  5.08s/it]
 89%|████████▉ | 2787/3120 [3:52:23<28:25,  5.12s/it]
 89%|████████▉ | 2788/3120 [3:52:28<28:07,  5.08s/it]
                                                     
{'loss': 0.123, 'learning_rate': 6.127783225369377e-07, 'epoch': 3.57}

 89%|████████▉ | 2788/3120 [3:52:28<28:07,  5.08s/it]
 89%|████████▉ | 2789/3120 [3:52:33<27:23,  4.97s/it]
 89%|████████▉ | 2790/3120 [3:52:38<27:06,  4.93s/it]
                                                     
{'loss': 0.1254, 'learning_rate': 6.054928899468427e-07, 'epoch': 3.58}

 89%|████████▉ | 2790/3120 [3:52:38<27:06,  4.93s/it]
 89%|████████▉ | 2791/3120 [3:52:43<27:03,  4.94s/it]
 89%|████████▉ | 2792/3120 [3:52:48<27:16,  4.99s/it]
                                                     
{'loss': 0.1124, 'learning_rate': 5.982496733658582e-07, 'epoch': 3.58}

 89%|████████▉ | 2792/3120 [3:52:48<27:16,  4.99s/it]
 90%|████████▉ | 2793/3120 [3:52:53<27:04,  4.97s/it]
 90%|████████▉ | 2794/3120 [3:52:58<26:52,  4.95s/it]
                                                     
{'loss': 0.1232, 'learning_rate': 5.910487053427616e-07, 'epoch': 3.58}

 90%|████████▉ | 2794/3120 [3:52:58<26:52,  4.95s/it]
 90%|████████▉ | 2795/3120 [3:53:03<26:54,  4.97s/it]
 90%|████████▉ | 2796/3120 [3:53:08<27:01,  5.00s/it]
                                                     
{'loss': 0.123, 'learning_rate': 5.83890018236476e-07, 'epoch': 3.58}

 90%|████████▉ | 2796/3120 [3:53:08<27:01,  5.00s/it]
 90%|████████▉ | 2797/3120 [3:53:13<27:00,  5.02s/it]
 90%|████████▉ | 2798/3120 [3:53:18<26:28,  4.93s/it]
                                                     
{'loss': 0.1266, 'learning_rate': 5.767736442159256e-07, 'epoch': 3.59}

 90%|████████▉ | 2798/3120 [3:53:18<26:28,  4.93s/it]
 90%|████████▉ | 2799/3120 [3:53:23<26:55,  5.03s/it]
 90%|████████▉ | 2800/3120 [3:53:28<26:08,  4.90s/it]
                                                     
{'loss': 0.1382, 'learning_rate': 5.696996152598966e-07, 'epoch': 3.59}

 90%|████████▉ | 2800/3120 [3:53:28<26:08,  4.90s/it]
 90%|████████▉ | 2801/3120 [3:53:32<25:49,  4.86s/it]
 90%|████████▉ | 2802/3120 [3:53:37<25:35,  4.83s/it]
                                                     
{'loss': 0.1271, 'learning_rate': 5.626679631568832e-07, 'epoch': 3.59}

 90%|████████▉ | 2802/3120 [3:53:37<25:35,  4.83s/it]
 90%|████████▉ | 2803/3120 [3:53:42<25:27,  4.82s/it]
 90%|████████▉ | 2804/3120 [3:53:46<25:07,  4.77s/it]
                                                     
{'loss': 0.1204, 'learning_rate': 5.556787195049573e-07, 'epoch': 3.59}

 90%|████████▉ | 2804/3120 [3:53:46<25:07,  4.77s/it]
 90%|████████▉ | 2805/3120 [3:53:51<25:07,  4.79s/it]
 90%|████████▉ | 2806/3120 [3:53:56<25:21,  4.84s/it]
                                                     
{'loss': 0.1105, 'learning_rate': 5.487319157116178e-07, 'epoch': 3.6}

 90%|████████▉ | 2806/3120 [3:53:56<25:21,  4.84s/it]
 90%|████████▉ | 2807/3120 [3:54:02<26:12,  5.02s/it]
 90%|█████████ | 2808/3120 [3:54:07<25:50,  4.97s/it]
                                                     
{'loss': 0.1189, 'learning_rate': 5.418275829936537e-07, 'epoch': 3.6}

 90%|█████████ | 2808/3120 [3:54:07<25:50,  4.97s/it]
 90%|█████████ | 2809/3120 [3:54:12<25:53,  5.00s/it]
 90%|█████████ | 2810/3120 [3:54:16<25:36,  4.96s/it]
                                                     
{'loss': 0.1171, 'learning_rate': 5.349657523769991e-07, 'epoch': 3.6}

 90%|█████████ | 2810/3120 [3:54:17<25:36,  4.96s/it]
 90%|█████████ | 2811/3120 [3:54:21<25:21,  4.92s/it]
 90%|█████████ | 2812/3120 [3:54:26<25:22,  4.94s/it]
                                                     
{'loss': 0.128, 'learning_rate': 5.281464546965953e-07, 'epoch': 3.61}

 90%|█████████ | 2812/3120 [3:54:26<25:22,  4.94s/it]
 90%|█████████ | 2813/3120 [3:54:31<25:02,  4.90s/it]
 90%|█████████ | 2814/3120 [3:54:36<24:54,  4.88s/it]
                                                     
{'loss': 0.1279, 'learning_rate': 5.213697205962631e-07, 'epoch': 3.61}

 90%|█████████ | 2814/3120 [3:54:36<24:54,  4.88s/it]
 90%|█████████ | 2815/3120 [3:54:41<25:37,  5.04s/it]
 90%|█████████ | 2816/3120 [3:54:46<25:04,  4.95s/it]
                                                     
{'loss': 0.1265, 'learning_rate': 5.146355805285452e-07, 'epoch': 3.61}

 90%|█████████ | 2816/3120 [3:54:46<25:04,  4.95s/it]
 90%|█████████ | 2817/3120 [3:54:51<24:59,  4.95s/it]
 90%|█████████ | 2818/3120 [3:54:56<24:55,  4.95s/it]
                                                     
{'loss': 0.1182, 'learning_rate': 5.079440647545852e-07, 'epoch': 3.61}

 90%|█████████ | 2818/3120 [3:54:56<24:55,  4.95s/it]
 90%|█████████ | 2819/3120 [3:55:01<24:22,  4.86s/it]
 90%|█████████ | 2820/3120 [3:55:06<24:44,  4.95s/it]
                                                     
{'loss': 0.1316, 'learning_rate': 5.012952033439844e-07, 'epoch': 3.62}

 90%|█████████ | 2820/3120 [3:55:06<24:44,  4.95s/it]
 90%|█████████ | 2821/3120 [3:55:11<24:39,  4.95s/it]
 90%|█████████ | 2822/3120 [3:55:15<24:10,  4.87s/it]
                                                     
{'loss': 0.1137, 'learning_rate': 4.946890261746706e-07, 'epoch': 3.62}

 90%|█████████ | 2822/3120 [3:55:15<24:10,  4.87s/it]
 90%|█████████ | 2823/3120 [3:55:20<23:59,  4.85s/it]
 91%|█████████ | 2824/3120 [3:55:25<23:51,  4.84s/it]
                                                     
{'loss': 0.1154, 'learning_rate': 4.881255629327608e-07, 'epoch': 3.62}

 91%|█████████ | 2824/3120 [3:55:25<23:51,  4.84s/it]
 91%|█████████ | 2825/3120 [3:55:30<23:54,  4.86s/it]
 91%|█████████ | 2826/3120 [3:55:35<24:02,  4.91s/it]
                                                     
{'loss': 0.1171, 'learning_rate': 4.816048431124265e-07, 'epoch': 3.62}

 91%|█████████ | 2826/3120 [3:55:35<24:02,  4.91s/it]
 91%|█████████ | 2827/3120 [3:55:40<24:16,  4.97s/it]
 91%|█████████ | 2828/3120 [3:55:45<24:13,  4.98s/it]
                                                     
{'loss': 0.1264, 'learning_rate': 4.7512689601576843e-07, 'epoch': 3.63}

 91%|█████████ | 2828/3120 [3:55:45<24:13,  4.98s/it]
 91%|█████████ | 2829/3120 [3:55:50<23:58,  4.94s/it]
 91%|█████████ | 2830/3120 [3:55:55<24:07,  4.99s/it]
                                                     
{'loss': 0.1184, 'learning_rate': 4.686917507526756e-07, 'epoch': 3.63}

 91%|█████████ | 2830/3120 [3:55:55<24:07,  4.99s/it]
 91%|█████████ | 2831/3120 [3:56:00<23:40,  4.92s/it]
 91%|█████████ | 2832/3120 [3:56:05<23:18,  4.86s/it]
                                                     
{'loss': 0.1295, 'learning_rate': 4.6229943624069963e-07, 'epoch': 3.63}

 91%|█████████ | 2832/3120 [3:56:05<23:18,  4.86s/it]
 91%|█████████ | 2833/3120 [3:56:09<23:02,  4.82s/it]
 91%|█████████ | 2834/3120 [3:56:14<23:15,  4.88s/it]
                                                     
{'loss': 0.1232, 'learning_rate': 4.5594998120492505e-07, 'epoch': 3.63}

 91%|█████████ | 2834/3120 [3:56:14<23:15,  4.88s/it]
 91%|█████████ | 2835/3120 [3:56:20<23:45,  5.00s/it]
 91%|█████████ | 2836/3120 [3:56:24<23:05,  4.88s/it]
                                                     
{'loss': 0.1172, 'learning_rate': 4.4964341417784165e-07, 'epoch': 3.64}

 91%|█████████ | 2836/3120 [3:56:24<23:05,  4.88s/it]
 91%|█████████ | 2837/3120 [3:56:29<23:22,  4.96s/it]
 91%|█████████ | 2838/3120 [3:56:34<22:49,  4.85s/it]
                                                     
{'loss': 0.1111, 'learning_rate': 4.4337976349920763e-07, 'epoch': 3.64}

 91%|█████████ | 2838/3120 [3:56:34<22:49,  4.85s/it]
 91%|█████████ | 2839/3120 [3:56:39<22:32,  4.81s/it]
 91%|█████████ | 2840/3120 [3:56:44<22:37,  4.85s/it]
                                                     
{'loss': 0.1286, 'learning_rate': 4.3715905731593233e-07, 'epoch': 3.64}

 91%|█████████ | 2840/3120 [3:56:44<22:37,  4.85s/it]
 91%|█████████ | 2841/3120 [3:56:49<22:49,  4.91s/it]
 91%|█████████ | 2842/3120 [3:56:54<22:57,  4.95s/it]
                                                     
{'loss': 0.1141, 'learning_rate': 4.309813235819471e-07, 'epoch': 3.64}

 91%|█████████ | 2842/3120 [3:56:54<22:57,  4.95s/it]
 91%|█████████ | 2843/3120 [3:56:58<22:08,  4.80s/it]
 91%|█████████ | 2844/3120 [3:57:03<22:10,  4.82s/it]
                                                     
{'loss': 0.1203, 'learning_rate': 4.248465900580734e-07, 'epoch': 3.65}

 91%|█████████ | 2844/3120 [3:57:03<22:10,  4.82s/it]
 91%|█████████ | 2845/3120 [3:57:08<21:57,  4.79s/it]
 91%|█████████ | 2846/3120 [3:57:13<22:02,  4.83s/it]
                                                     
{'loss': 0.117, 'learning_rate': 4.1875488431190934e-07, 'epoch': 3.65}

 91%|█████████ | 2846/3120 [3:57:13<22:02,  4.83s/it]
 91%|█████████▏| 2847/3120 [3:57:18<22:32,  4.95s/it]
 91%|█████████▏| 2848/3120 [3:57:22<21:59,  4.85s/it]
                                                     
{'loss': 0.1156, 'learning_rate': 4.127062337176935e-07, 'epoch': 3.65}

 91%|█████████▏| 2848/3120 [3:57:22<21:59,  4.85s/it]
 91%|█████████▏| 2849/3120 [3:57:28<22:25,  4.97s/it]
 91%|█████████▏| 2850/3120 [3:57:33<22:50,  5.07s/it]
                                                     
{'loss': 0.1264, 'learning_rate': 4.0670066545619224e-07, 'epoch': 3.65}

 91%|█████████▏| 2850/3120 [3:57:33<22:50,  5.07s/it]
 91%|█████████▏| 2851/3120 [3:57:38<21:58,  4.90s/it]
 91%|█████████▏| 2852/3120 [3:57:42<21:43,  4.86s/it]
                                                     
{'loss': 0.1208, 'learning_rate': 4.0073820651457043e-07, 'epoch': 3.66}

 91%|█████████▏| 2852/3120 [3:57:42<21:43,  4.86s/it]
 91%|█████████▏| 2853/3120 [3:57:47<21:56,  4.93s/it]
 91%|█████████▏| 2854/3120 [3:57:52<22:03,  4.97s/it]
                                                     
{'loss': 0.1171, 'learning_rate': 3.9481888368627764e-07, 'epoch': 3.66}

 91%|█████████▏| 2854/3120 [3:57:52<22:03,  4.97s/it]
 92%|█████████▏| 2855/3120 [3:57:57<21:35,  4.89s/it]
 92%|█████████▏| 2856/3120 [3:58:02<21:15,  4.83s/it]
                                                     
{'loss': 0.118, 'learning_rate': 3.889427235709153e-07, 'epoch': 3.66}

 92%|█████████▏| 2856/3120 [3:58:02<21:15,  4.83s/it]
 92%|█████████▏| 2857/3120 [3:58:07<21:08,  4.82s/it]
 92%|█████████▏| 2858/3120 [3:58:12<22:06,  5.06s/it]
                                                     
{'loss': 0.1444, 'learning_rate': 3.8310975257413343e-07, 'epoch': 3.66}

 92%|█████████▏| 2858/3120 [3:58:12<22:06,  5.06s/it]
 92%|█████████▏| 2859/3120 [3:58:17<21:53,  5.03s/it]
 92%|█████████▏| 2860/3120 [3:58:22<21:19,  4.92s/it]
                                                     
{'loss': 0.1221, 'learning_rate': 3.773199969074959e-07, 'epoch': 3.67}

 92%|█████████▏| 2860/3120 [3:58:22<21:19,  4.92s/it]
 92%|█████████▏| 2861/3120 [3:58:27<21:29,  4.98s/it]
 92%|█████████▏| 2862/3120 [3:58:32<21:42,  5.05s/it]
                                                     
{'loss': 0.1296, 'learning_rate': 3.715734825883766e-07, 'epoch': 3.67}

 92%|█████████▏| 2862/3120 [3:58:32<21:42,  5.05s/it]
 92%|█████████▏| 2863/3120 [3:58:37<20:58,  4.90s/it]
 92%|█████████▏| 2864/3120 [3:58:42<21:05,  4.94s/it]
                                                     
{'loss': 0.109, 'learning_rate': 3.658702354398325e-07, 'epoch': 3.67}

 92%|█████████▏| 2864/3120 [3:58:42<21:05,  4.94s/it]
 92%|█████████▏| 2865/3120 [3:58:47<21:38,  5.09s/it]
 92%|█████████▏| 2866/3120 [3:58:52<21:26,  5.07s/it]
                                                     
{'loss': 0.1116, 'learning_rate': 3.6021028109049063e-07, 'epoch': 3.67}

 92%|█████████▏| 2866/3120 [3:58:52<21:26,  5.07s/it]
 92%|█████████▏| 2867/3120 [3:58:57<21:17,  5.05s/it]
 92%|█████████▏| 2868/3120 [3:59:03<21:45,  5.18s/it]
                                                     
{'loss': 0.1114, 'learning_rate': 3.5459364497443696e-07, 'epoch': 3.68}

 92%|█████████▏| 2868/3120 [3:59:03<21:45,  5.18s/it]
 92%|█████████▏| 2869/3120 [3:59:08<21:42,  5.19s/it]
 92%|█████████▏| 2870/3120 [3:59:13<21:20,  5.12s/it]
                                                     
{'loss': 0.1193, 'learning_rate': 3.4902035233109644e-07, 'epoch': 3.68}

 92%|█████████▏| 2870/3120 [3:59:13<21:20,  5.12s/it]
 92%|█████████▏| 2871/3120 [3:59:18<20:51,  5.03s/it]
 92%|█████████▏| 2872/3120 [3:59:23<20:40,  5.00s/it]
                                                     
{'loss': 0.1159, 'learning_rate': 3.4349042820512325e-07, 'epoch': 3.68}

 92%|█████████▏| 2872/3120 [3:59:23<20:40,  5.00s/it]
 92%|█████████▏| 2873/3120 [3:59:28<20:39,  5.02s/it]
 92%|█████████▏| 2874/3120 [3:59:32<20:14,  4.94s/it]
                                                     
{'loss': 0.1081, 'learning_rate': 3.38003897446284e-07, 'epoch': 3.68}

 92%|█████████▏| 2874/3120 [3:59:32<20:14,  4.94s/it]
 92%|█████████▏| 2875/3120 [3:59:37<20:01,  4.90s/it]
 92%|█████████▏| 2876/3120 [3:59:42<20:16,  4.99s/it]
                                                     
{'loss': 0.1281, 'learning_rate': 3.325607847093537e-07, 'epoch': 3.69}

 92%|█████████▏| 2876/3120 [3:59:43<20:16,  4.99s/it]
 92%|█████████▏| 2877/3120 [3:59:47<19:58,  4.93s/it]
 92%|█████████▏| 2878/3120 [3:59:52<19:53,  4.93s/it]
                                                     
{'loss': 0.1353, 'learning_rate': 3.2716111445399324e-07, 'epoch': 3.69}

 92%|█████████▏| 2878/3120 [3:59:52<19:53,  4.93s/it]
 92%|█████████▏| 2879/3120 [3:59:57<19:50,  4.94s/it]
 92%|█████████▏| 2880/3120 [4:00:01<18:59,  4.75s/it]
                                                     
{'loss': 0.121, 'learning_rate': 3.2180491094465414e-07, 'epoch': 3.69}

 92%|█████████▏| 2880/3120 [4:00:02<18:59,  4.75s/it]
 92%|█████████▏| 2881/3120 [4:00:06<19:09,  4.81s/it]
 92%|█████████▏| 2882/3120 [4:00:11<19:12,  4.84s/it]
                                                     
{'loss': 0.126, 'learning_rate': 3.164921982504565e-07, 'epoch': 3.69}

 92%|█████████▏| 2882/3120 [4:00:11<19:12,  4.84s/it]
 92%|█████████▏| 2883/3120 [4:00:16<19:01,  4.82s/it]
 92%|█████████▏| 2884/3120 [4:00:21<19:11,  4.88s/it]
                                                     
{'loss': 0.126, 'learning_rate': 3.112230002450889e-07, 'epoch': 3.7}

 92%|█████████▏| 2884/3120 [4:00:21<19:11,  4.88s/it]
 92%|█████████▏| 2885/3120 [4:00:27<19:47,  5.05s/it]
 92%|█████████▎| 2886/3120 [4:00:31<19:29,  5.00s/it]
                                                     
{'loss': 0.1314, 'learning_rate': 3.059973406066963e-07, 'epoch': 3.7}

 92%|█████████▎| 2886/3120 [4:00:31<19:29,  5.00s/it]
 93%|█████████▎| 2887/3120 [4:00:36<19:17,  4.97s/it]
 93%|█████████▎| 2888/3120 [4:00:41<18:57,  4.90s/it]
                                                     
{'loss': 0.1231, 'learning_rate': 3.0081524281777687e-07, 'epoch': 3.7}

 93%|█████████▎| 2888/3120 [4:00:41<18:57,  4.90s/it]
 93%|█████████▎| 2889/3120 [4:00:46<18:59,  4.93s/it]
 93%|█████████▎| 2890/3120 [4:00:51<19:15,  5.02s/it]
                                                     
{'loss': 0.1258, 'learning_rate': 2.9567673016507646e-07, 'epoch': 3.71}

 93%|█████████▎| 2890/3120 [4:00:51<19:15,  5.02s/it]
 93%|█████████▎| 2891/3120 [4:00:57<19:42,  5.17s/it]
 93%|█████████▎| 2892/3120 [4:01:02<19:30,  5.13s/it]
                                                     
{'loss': 0.1364, 'learning_rate': 2.905818257394799e-07, 'epoch': 3.71}

 93%|█████████▎| 2892/3120 [4:01:02<19:30,  5.13s/it]
 93%|█████████▎| 2893/3120 [4:01:07<19:06,  5.05s/it]
 93%|█████████▎| 2894/3120 [4:01:12<19:06,  5.07s/it]
                                                     
{'loss': 0.1183, 'learning_rate': 2.855305524359131e-07, 'epoch': 3.71}

 93%|█████████▎| 2894/3120 [4:01:12<19:06,  5.07s/it]
 93%|█████████▎| 2895/3120 [4:01:18<19:54,  5.31s/it]
 93%|█████████▎| 2896/3120 [4:01:23<20:02,  5.37s/it]
                                                     
{'loss': 0.1205, 'learning_rate': 2.805229329532344e-07, 'epoch': 3.71}

 93%|█████████▎| 2896/3120 [4:01:23<20:02,  5.37s/it]
 93%|█████████▎| 2897/3120 [4:01:28<18:50,  5.07s/it]
 93%|█████████▎| 2898/3120 [4:01:33<18:34,  5.02s/it]
                                                     
{'loss': 0.1203, 'learning_rate': 2.7555898979413796e-07, 'epoch': 3.72}

 93%|█████████▎| 2898/3120 [4:01:33<18:34,  5.02s/it]
 93%|█████████▎| 2899/3120 [4:01:37<18:04,  4.91s/it]
 93%|█████████▎| 2900/3120 [4:01:42<18:03,  4.92s/it]
                                                     
{'loss': 0.1065, 'learning_rate': 2.706387452650494e-07, 'epoch': 3.72}

 93%|█████████▎| 2900/3120 [4:01:42<18:03,  4.92s/it]
 93%|█████████▎| 2901/3120 [4:01:47<18:04,  4.95s/it]
 93%|█████████▎| 2902/3120 [4:01:52<17:58,  4.95s/it]
                                                     
{'loss': 0.1149, 'learning_rate': 2.657622214760247e-07, 'epoch': 3.72}

 93%|█████████▎| 2902/3120 [4:01:52<17:58,  4.95s/it]
 93%|█████████▎| 2903/3120 [4:01:57<18:09,  5.02s/it]
 93%|█████████▎| 2904/3120 [4:02:02<17:42,  4.92s/it]
                                                     
{'loss': 0.1283, 'learning_rate': 2.609294403406537e-07, 'epoch': 3.72}

 93%|█████████▎| 2904/3120 [4:02:02<17:42,  4.92s/it]
 93%|█████████▎| 2905/3120 [4:02:07<17:50,  4.98s/it]
 93%|█████████▎| 2906/3120 [4:02:13<18:20,  5.14s/it]
                                                     
{'loss': 0.1139, 'learning_rate': 2.56140423575959e-07, 'epoch': 3.73}

 93%|█████████▎| 2906/3120 [4:02:13<18:20,  5.14s/it]
 93%|█████████▎| 2907/3120 [4:02:17<17:56,  5.05s/it]
 93%|█████████▎| 2908/3120 [4:02:22<17:20,  4.91s/it]
                                                     
{'loss': 0.1173, 'learning_rate': 2.513951927023017e-07, 'epoch': 3.73}

 93%|█████████▎| 2908/3120 [4:02:22<17:20,  4.91s/it]
 93%|█████████▎| 2909/3120 [4:02:27<16:59,  4.83s/it]
 93%|█████████▎| 2910/3120 [4:02:32<17:06,  4.89s/it]
                                                     
{'loss': 0.1178, 'learning_rate': 2.4669376904328244e-07, 'epoch': 3.73}

 93%|█████████▎| 2910/3120 [4:02:32<17:06,  4.89s/it]
 93%|█████████▎| 2911/3120 [4:02:36<16:54,  4.86s/it]
 93%|█████████▎| 2912/3120 [4:02:41<16:44,  4.83s/it]
                                                     
{'loss': 0.1232, 'learning_rate': 2.420361737256438e-07, 'epoch': 3.73}

 93%|█████████▎| 2912/3120 [4:02:41<16:44,  4.83s/it]
 93%|█████████▎| 2913/3120 [4:02:46<16:53,  4.89s/it]
 93%|█████████▎| 2914/3120 [4:02:52<17:19,  5.05s/it]
                                                     
{'loss': 0.1231, 'learning_rate': 2.3742242767917921e-07, 'epoch': 3.74}

 93%|█████████▎| 2914/3120 [4:02:52<17:19,  5.05s/it]
 93%|█████████▎| 2915/3120 [4:02:57<17:14,  5.05s/it]
 93%|█████████▎| 2916/3120 [4:03:01<16:44,  4.92s/it]
                                                     
{'loss': 0.1208, 'learning_rate': 2.3285255163663535e-07, 'epoch': 3.74}

 93%|█████████▎| 2916/3120 [4:03:01<16:44,  4.92s/it]
 93%|█████████▎| 2917/3120 [4:03:06<16:24,  4.85s/it]
 94%|█████████▎| 2918/3120 [4:03:12<17:30,  5.20s/it]
                                                     
{'loss': 0.1088, 'learning_rate': 2.2832656613362092e-07, 'epoch': 3.74}

 94%|█████████▎| 2918/3120 [4:03:12<17:30,  5.20s/it]
 94%|█████████▎| 2919/3120 [4:03:17<17:06,  5.11s/it]
 94%|█████████▎| 2920/3120 [4:03:22<16:40,  5.00s/it]
                                                     
{'loss': 0.1195, 'learning_rate': 2.2384449150851695e-07, 'epoch': 3.74}

 94%|█████████▎| 2920/3120 [4:03:22<16:40,  5.00s/it]
 94%|█████████▎| 2921/3120 [4:03:26<16:18,  4.92s/it]
 94%|█████████▎| 2922/3120 [4:03:31<16:17,  4.94s/it]
                                                     
{'loss': 0.1203, 'learning_rate': 2.1940634790238003e-07, 'epoch': 3.75}

 94%|█████████▎| 2922/3120 [4:03:31<16:17,  4.94s/it]
 94%|█████████▎| 2923/3120 [4:03:36<16:02,  4.88s/it]
 94%|█████████▎| 2924/3120 [4:03:42<16:51,  5.16s/it]
                                                     
{'loss': 0.1131, 'learning_rate': 2.1501215525885245e-07, 'epoch': 3.75}

 94%|█████████▎| 2924/3120 [4:03:42<16:51,  5.16s/it]
 94%|█████████▍| 2925/3120 [4:03:47<16:20,  5.03s/it]
 94%|█████████▍| 2926/3120 [4:03:52<16:16,  5.03s/it]
                                                     
{'loss': 0.1247, 'learning_rate': 2.1066193332407891e-07, 'epoch': 3.75}

 94%|█████████▍| 2926/3120 [4:03:52<16:16,  5.03s/it]
 94%|█████████▍| 2927/3120 [4:03:57<16:11,  5.03s/it]
 94%|█████████▍| 2928/3120 [4:04:02<16:18,  5.10s/it]
                                                     
{'loss': 0.1202, 'learning_rate': 2.063557016466111e-07, 'epoch': 3.75}

 94%|█████████▍| 2928/3120 [4:04:02<16:18,  5.10s/it]
 94%|█████████▍| 2929/3120 [4:04:07<16:10,  5.08s/it]
 94%|█████████▍| 2930/3120 [4:04:12<15:55,  5.03s/it]
                                                     
{'loss': 0.1264, 'learning_rate': 2.0209347957732328e-07, 'epoch': 3.76}

 94%|█████████▍| 2930/3120 [4:04:12<15:55,  5.03s/it]
 94%|█████████▍| 2931/3120 [4:04:17<15:27,  4.91s/it]
 94%|█████████▍| 2932/3120 [4:04:21<15:00,  4.79s/it]
                                                     
{'loss': 0.1156, 'learning_rate': 1.978752862693212e-07, 'epoch': 3.76}

 94%|█████████▍| 2932/3120 [4:04:21<15:00,  4.79s/it]
 94%|█████████▍| 2933/3120 [4:04:26<15:16,  4.90s/it]
 94%|█████████▍| 2934/3120 [4:04:31<15:07,  4.88s/it]
                                                     
{'loss': 0.1261, 'learning_rate': 1.9370114067785995e-07, 'epoch': 3.76}

 94%|█████████▍| 2934/3120 [4:04:31<15:07,  4.88s/it]
 94%|█████████▍| 2935/3120 [4:04:36<15:14,  4.94s/it]
 94%|█████████▍| 2936/3120 [4:04:41<15:17,  4.99s/it]
                                                     
{'loss': 0.1253, 'learning_rate': 1.8957106156026084e-07, 'epoch': 3.76}

 94%|█████████▍| 2936/3120 [4:04:41<15:17,  4.99s/it]
 94%|█████████▍| 2937/3120 [4:04:47<15:32,  5.09s/it]
 94%|█████████▍| 2938/3120 [4:04:51<15:09,  5.00s/it]
                                                     
{'loss': 0.1213, 'learning_rate': 1.854850674758213e-07, 'epoch': 3.77}

 94%|█████████▍| 2938/3120 [4:04:51<15:09,  5.00s/it]
 94%|█████████▍| 2939/3120 [4:04:57<15:39,  5.19s/it]
 94%|█████████▍| 2940/3120 [4:05:02<15:12,  5.07s/it]
                                                     
{'loss': 0.1227, 'learning_rate': 1.8144317678573497e-07, 'epoch': 3.77}

 94%|█████████▍| 2940/3120 [4:05:02<15:12,  5.07s/it]
 94%|█████████▍| 2941/3120 [4:05:06<14:36,  4.89s/it]
 94%|█████████▍| 2942/3120 [4:05:11<14:43,  4.96s/it]
                                                     
{'loss': 0.1287, 'learning_rate': 1.7744540765300743e-07, 'epoch': 3.77}

 94%|█████████▍| 2942/3120 [4:05:11<14:43,  4.96s/it]
 94%|█████████▍| 2943/3120 [4:05:17<14:43,  4.99s/it]
 94%|█████████▍| 2944/3120 [4:05:21<14:17,  4.87s/it]
                                                     
{'loss': 0.1139, 'learning_rate': 1.7349177804237837e-07, 'epoch': 3.77}

 94%|█████████▍| 2944/3120 [4:05:21<14:17,  4.87s/it]
 94%|█████████▍| 2945/3120 [4:05:26<14:30,  4.97s/it]
 94%|█████████▍| 2946/3120 [4:05:31<14:07,  4.87s/it]
                                                     
{'loss': 0.126, 'learning_rate': 1.6958230572023504e-07, 'epoch': 3.78}

 94%|█████████▍| 2946/3120 [4:05:31<14:07,  4.87s/it]
 94%|█████████▍| 2947/3120 [4:05:36<14:06,  4.89s/it]
 94%|█████████▍| 2948/3120 [4:05:41<14:07,  4.93s/it]
                                                     
{'loss': 0.1169, 'learning_rate': 1.6571700825453674e-07, 'epoch': 3.78}

 94%|█████████▍| 2948/3120 [4:05:41<14:07,  4.93s/it]
 95%|█████████▍| 2949/3120 [4:05:46<13:56,  4.89s/it]
 95%|█████████▍| 2950/3120 [4:05:51<13:48,  4.87s/it]
                                                     
{'loss': 0.1208, 'learning_rate': 1.618959030147349e-07, 'epoch': 3.78}

 95%|█████████▍| 2950/3120 [4:05:51<13:48,  4.87s/it]
 95%|█████████▍| 2951/3120 [4:05:55<13:47,  4.89s/it]
 95%|█████████▍| 2952/3120 [4:06:00<13:35,  4.85s/it]
                                                     
{'loss': 0.1237, 'learning_rate': 1.5811900717169537e-07, 'epoch': 3.78}

 95%|█████████▍| 2952/3120 [4:06:00<13:35,  4.85s/it]
 95%|█████████▍| 2953/3120 [4:06:05<13:37,  4.89s/it]
 95%|█████████▍| 2954/3120 [4:06:10<13:19,  4.81s/it]
                                                     
{'loss': 0.1337, 'learning_rate': 1.5438633769761845e-07, 'epoch': 3.79}

 95%|█████████▍| 2954/3120 [4:06:10<13:19,  4.81s/it]
 95%|█████████▍| 2955/3120 [4:06:15<13:06,  4.77s/it]
 95%|█████████▍| 2956/3120 [4:06:20<13:18,  4.87s/it]
                                                     
{'loss': 0.1102, 'learning_rate': 1.506979113659679e-07, 'epoch': 3.79}

 95%|█████████▍| 2956/3120 [4:06:20<13:18,  4.87s/it]
 95%|█████████▍| 2957/3120 [4:06:25<13:32,  4.99s/it]
 95%|█████████▍| 2958/3120 [4:06:30<13:35,  5.03s/it]
                                                     
{'loss': 0.1215, 'learning_rate': 1.4705374475138978e-07, 'epoch': 3.79}

 95%|█████████▍| 2958/3120 [4:06:30<13:35,  5.03s/it]
 95%|█████████▍| 2959/3120 [4:06:35<13:25,  5.01s/it]
 95%|█████████▍| 2960/3120 [4:06:40<13:14,  4.96s/it]
                                                     
{'loss': 0.1297, 'learning_rate': 1.4345385422964043e-07, 'epoch': 3.79}

 95%|█████████▍| 2960/3120 [4:06:40<13:14,  4.96s/it]
 95%|█████████▍| 2961/3120 [4:06:45<13:11,  4.98s/it]
 95%|█████████▍| 2962/3120 [4:06:50<13:29,  5.12s/it]
                                                     
{'loss': 0.1351, 'learning_rate': 1.398982559775175e-07, 'epoch': 3.8}

 95%|█████████▍| 2962/3120 [4:06:50<13:29,  5.12s/it]
 95%|█████████▍| 2963/3120 [4:06:56<13:32,  5.17s/it]
 95%|█████████▌| 2964/3120 [4:07:01<13:20,  5.13s/it]
                                                     
{'loss': 0.1203, 'learning_rate': 1.3638696597277678e-07, 'epoch': 3.8}

 95%|█████████▌| 2964/3120 [4:07:01<13:20,  5.13s/it]
 95%|█████████▌| 2965/3120 [4:07:05<12:55,  5.00s/it]
 95%|█████████▌| 2966/3120 [4:07:10<12:42,  4.95s/it]
                                                     
{'loss': 0.117, 'learning_rate': 1.3291999999407e-07, 'epoch': 3.8}

 95%|█████████▌| 2966/3120 [4:07:10<12:42,  4.95s/it]
 95%|█████████▌| 2967/3120 [4:07:15<12:44,  4.99s/it]
 95%|█████████▌| 2968/3120 [4:07:20<12:47,  5.05s/it]
                                                     
{'loss': 0.1234, 'learning_rate': 1.2949737362087156e-07, 'epoch': 3.81}

 95%|█████████▌| 2968/3120 [4:07:20<12:47,  5.05s/it]
 95%|█████████▌| 2969/3120 [4:07:25<12:13,  4.85s/it]
 95%|█████████▌| 2970/3120 [4:07:30<12:17,  4.92s/it]
                                                     
{'loss': 0.1173, 'learning_rate': 1.2611910223340408e-07, 'epoch': 3.81}

 95%|█████████▌| 2970/3120 [4:07:30<12:17,  4.92s/it]
 95%|█████████▌| 2971/3120 [4:07:35<12:06,  4.88s/it]
 95%|█████████▌| 2972/3120 [4:07:40<12:00,  4.87s/it]
                                                     
{'loss': 0.1239, 'learning_rate': 1.227852010125752e-07, 'epoch': 3.81}

 95%|█████████▌| 2972/3120 [4:07:40<12:00,  4.87s/it]
 95%|█████████▌| 2973/3120 [4:07:45<12:02,  4.91s/it]
 95%|█████████▌| 2974/3120 [4:07:50<12:13,  5.03s/it]
                                                     
{'loss': 0.1202, 'learning_rate': 1.1949568493990428e-07, 'epoch': 3.81}

 95%|█████████▌| 2974/3120 [4:07:50<12:13,  5.03s/it]
 95%|█████████▌| 2975/3120 [4:07:54<11:52,  4.91s/it]
 95%|█████████▌| 2976/3120 [4:07:59<11:38,  4.85s/it]
                                                     
{'loss': 0.1185, 'learning_rate': 1.1625056879746133e-07, 'epoch': 3.82}

 95%|█████████▌| 2976/3120 [4:07:59<11:38,  4.85s/it]
 95%|█████████▌| 2977/3120 [4:08:04<11:33,  4.85s/it]
 95%|█████████▌| 2978/3120 [4:08:09<11:33,  4.88s/it]
                                                     
{'loss': 0.1214, 'learning_rate': 1.1304986716779265e-07, 'epoch': 3.82}

 95%|█████████▌| 2978/3120 [4:08:09<11:33,  4.88s/it]
 95%|█████████▌| 2979/3120 [4:08:14<11:32,  4.91s/it]
 96%|█████████▌| 2980/3120 [4:08:19<11:14,  4.82s/it]
                                                     
{'loss': 0.1281, 'learning_rate': 1.0989359443386305e-07, 'epoch': 3.82}

 96%|█████████▌| 2980/3120 [4:08:19<11:14,  4.82s/it]
 96%|█████████▌| 2981/3120 [4:08:24<11:14,  4.86s/it]
 96%|█████████▌| 2982/3120 [4:08:28<11:11,  4.86s/it]
                                                     
{'loss': 0.1163, 'learning_rate': 1.0678176477898372e-07, 'epoch': 3.82}

 96%|█████████▌| 2982/3120 [4:08:28<11:11,  4.86s/it]
 96%|█████████▌| 2983/3120 [4:08:33<11:05,  4.86s/it]
 96%|█████████▌| 2984/3120 [4:08:38<10:55,  4.82s/it]
                                                     
{'loss': 0.1177, 'learning_rate': 1.0371439218675671e-07, 'epoch': 3.83}

 96%|█████████▌| 2984/3120 [4:08:38<10:55,  4.82s/it]
 96%|█████████▌| 2985/3120 [4:08:43<10:56,  4.86s/it]
 96%|█████████▌| 2986/3120 [4:08:48<10:51,  4.86s/it]
                                                     
{'loss': 0.121, 'learning_rate': 1.0069149044100502e-07, 'epoch': 3.83}

 96%|█████████▌| 2986/3120 [4:08:48<10:51,  4.86s/it]
 96%|█████████▌| 2987/3120 [4:08:53<11:11,  5.05s/it]
 96%|█████████▌| 2988/3120 [4:08:59<11:14,  5.11s/it]
                                                     
{'loss': 0.1335, 'learning_rate': 9.771307312571254e-08, 'epoch': 3.83}

 96%|█████████▌| 2988/3120 [4:08:59<11:14,  5.11s/it]
 96%|█████████▌| 2989/3120 [4:09:04<11:21,  5.20s/it]
 96%|█████████▌| 2990/3120 [4:09:09<11:09,  5.15s/it]
                                                     
{'loss': 0.1128, 'learning_rate': 9.47791536249676e-08, 'epoch': 3.83}

 96%|█████████▌| 2990/3120 [4:09:09<11:09,  5.15s/it]
 96%|█████████▌| 2991/3120 [4:09:14<11:08,  5.19s/it]
 96%|█████████▌| 2992/3120 [4:09:20<11:07,  5.22s/it]
                                                     
{'loss': 0.1182, 'learning_rate': 9.188974512289617e-08, 'epoch': 3.84}

 96%|█████████▌| 2992/3120 [4:09:20<11:07,  5.22s/it]
 96%|█████████▌| 2993/3120 [4:09:24<10:29,  4.96s/it]
 96%|█████████▌| 2994/3120 [4:09:29<10:21,  4.94s/it]
                                                     
{'loss': 0.1181, 'learning_rate': 8.90448606036054e-08, 'epoch': 3.84}

 96%|█████████▌| 2994/3120 [4:09:29<10:21,  4.94s/it]
 96%|█████████▌| 2995/3120 [4:09:34<10:24,  4.99s/it]
 96%|█████████▌| 2996/3120 [4:09:39<10:16,  4.97s/it]
                                                     
{'loss': 0.1223, 'learning_rate': 8.624451285112689e-08, 'epoch': 3.84}

 96%|█████████▌| 2996/3120 [4:09:39<10:16,  4.97s/it]
 96%|█████████▌| 2997/3120 [4:09:43<09:56,  4.85s/it]
 96%|█████████▌| 2998/3120 [4:09:48<09:52,  4.86s/it]
                                                     
{'loss': 0.1284, 'learning_rate': 8.348871444935902e-08, 'epoch': 3.84}

 96%|█████████▌| 2998/3120 [4:09:48<09:52,  4.86s/it]
 96%|█████████▌| 2999/3120 [4:09:53<09:53,  4.91s/it]
 96%|█████████▌| 3000/3120 [4:09:58<09:46,  4.89s/it]
                                                     
{'loss': 0.1234, 'learning_rate': 8.077747778200474e-08, 'epoch': 3.85}

 96%|█████████▌| 3000/3120 [4:09:58<09:46,  4.89s/it]
 96%|█████████▌| 3001/3120 [4:10:03<09:43,  4.90s/it]
 96%|█████████▌| 3002/3120 [4:10:08<09:35,  4.88s/it]
                                                     
{'loss': 0.1184, 'learning_rate': 7.811081503252382e-08, 'epoch': 3.85}

 96%|█████████▌| 3002/3120 [4:10:08<09:35,  4.88s/it]
 96%|█████████▋| 3003/3120 [4:10:13<09:31,  4.89s/it]
 96%|█████████▋| 3004/3120 [4:10:18<09:24,  4.87s/it]
                                                     
{'loss': 0.1162, 'learning_rate': 7.54887381840752e-08, 'epoch': 3.85}

 96%|█████████▋| 3004/3120 [4:10:18<09:24,  4.87s/it]
 96%|█████████▋| 3005/3120 [4:10:23<09:26,  4.93s/it]
 96%|█████████▋| 3006/3120 [4:10:28<09:20,  4.91s/it]
                                                     
{'loss': 0.1188, 'learning_rate': 7.291125901946027e-08, 'epoch': 3.85}

 96%|█████████▋| 3006/3120 [4:10:28<09:20,  4.91s/it]
 96%|█████████▋| 3007/3120 [4:10:32<09:11,  4.88s/it]
 96%|█████████▋| 3008/3120 [4:10:37<09:11,  4.92s/it]
                                                     
{'loss': 0.1242, 'learning_rate': 7.037838912107298e-08, 'epoch': 3.86}

 96%|█████████▋| 3008/3120 [4:10:37<09:11,  4.92s/it]
 96%|█████████▋| 3009/3120 [4:10:42<09:02,  4.89s/it]
 96%|█████████▋| 3010/3120 [4:10:47<09:06,  4.97s/it]
                                                     
{'loss': 0.1298, 'learning_rate': 6.789013987084647e-08, 'epoch': 3.86}

 96%|█████████▋| 3010/3120 [4:10:47<09:06,  4.97s/it]
 97%|█████████▋| 3011/3120 [4:10:52<09:02,  4.98s/it]
 97%|█████████▋| 3012/3120 [4:10:57<08:46,  4.88s/it]
                                                     
{'loss': 0.1172, 'learning_rate': 6.544652245020433e-08, 'epoch': 3.86}

 97%|█████████▋| 3012/3120 [4:10:57<08:46,  4.88s/it]
 97%|█████████▋| 3013/3120 [4:11:02<08:37,  4.84s/it]
 97%|█████████▋| 3014/3120 [4:11:07<08:34,  4.86s/it]
                                                     
{'loss': 0.1113, 'learning_rate': 6.304754784000721e-08, 'epoch': 3.86}

 97%|█████████▋| 3014/3120 [4:11:07<08:34,  4.86s/it]
 97%|█████████▋| 3015/3120 [4:11:11<08:28,  4.84s/it]
 97%|█████████▋| 3016/3120 [4:11:16<08:21,  4.82s/it]
                                                     
{'loss': 0.1147, 'learning_rate': 6.069322682050516e-08, 'epoch': 3.87}

 97%|█████████▋| 3016/3120 [4:11:16<08:21,  4.82s/it]
 97%|█████████▋| 3017/3120 [4:11:21<08:13,  4.79s/it]
 97%|█████████▋| 3018/3120 [4:11:26<08:09,  4.80s/it]
                                                     
{'loss': 0.1156, 'learning_rate': 5.838356997128869e-08, 'epoch': 3.87}

 97%|█████████▋| 3018/3120 [4:11:26<08:09,  4.80s/it]
 97%|█████████▋| 3019/3120 [4:11:31<08:29,  5.05s/it]
 97%|█████████▋| 3020/3120 [4:11:36<08:24,  5.05s/it]
                                                     
{'loss': 0.124, 'learning_rate': 5.611858767124001e-08, 'epoch': 3.87}

 97%|█████████▋| 3020/3120 [4:11:36<08:24,  5.05s/it]
 97%|█████████▋| 3021/3120 [4:11:42<08:32,  5.17s/it]
 97%|█████████▋| 3022/3120 [4:11:47<08:18,  5.09s/it]
                                                     
{'loss': 0.1228, 'learning_rate': 5.389829009849079e-08, 'epoch': 3.87}

 97%|█████████▋| 3022/3120 [4:11:47<08:18,  5.09s/it]
 97%|█████████▋| 3023/3120 [4:11:52<08:07,  5.02s/it]
 97%|█████████▋| 3024/3120 [4:11:56<07:53,  4.93s/it]
                                                     
{'loss': 0.1131, 'learning_rate': 5.1722687230369995e-08, 'epoch': 3.88}

 97%|█████████▋| 3024/3120 [4:11:56<07:53,  4.93s/it]
 97%|█████████▋| 3025/3120 [4:12:01<07:43,  4.88s/it]
 97%|█████████▋| 3026/3120 [4:12:06<07:36,  4.85s/it]
                                                     
{'loss': 0.1222, 'learning_rate': 4.959178884336502e-08, 'epoch': 3.88}

 97%|█████████▋| 3026/3120 [4:12:06<07:36,  4.85s/it]
 97%|█████████▋| 3027/3120 [4:12:11<07:42,  4.98s/it]
 97%|█████████▋| 3028/3120 [4:12:16<07:31,  4.91s/it]
                                                     
{'loss': 0.1182, 'learning_rate': 4.7505604513072845e-08, 'epoch': 3.88}

 97%|█████████▋| 3028/3120 [4:12:16<07:31,  4.91s/it]
 97%|█████████▋| 3029/3120 [4:12:21<07:36,  5.01s/it]
 97%|█████████▋| 3030/3120 [4:12:26<07:35,  5.06s/it]
                                                     
{'loss': 0.1168, 'learning_rate': 4.5464143614162294e-08, 'epoch': 3.88}

 97%|█████████▋| 3030/3120 [4:12:26<07:35,  5.06s/it]
 97%|█████████▋| 3031/3120 [4:12:31<07:16,  4.90s/it]
 97%|█████████▋| 3032/3120 [4:12:36<07:15,  4.95s/it]
                                                     
{'loss': 0.1135, 'learning_rate': 4.346741532032628e-08, 'epoch': 3.89}

 97%|█████████▋| 3032/3120 [4:12:36<07:15,  4.95s/it]
 97%|█████████▋| 3033/3120 [4:12:41<07:00,  4.83s/it]
 97%|█████████▋| 3034/3120 [4:12:45<06:51,  4.78s/it]
                                                     
{'loss': 0.1099, 'learning_rate': 4.151542860424407e-08, 'epoch': 3.89}

 97%|█████████▋| 3034/3120 [4:12:45<06:51,  4.78s/it]
 97%|█████████▋| 3035/3120 [4:12:50<06:50,  4.83s/it]
 97%|█████████▋| 3036/3120 [4:12:55<06:40,  4.76s/it]
                                                     
{'loss': 0.1169, 'learning_rate': 3.96081922375402e-08, 'epoch': 3.89}

 97%|█████████▋| 3036/3120 [4:12:55<06:40,  4.76s/it]
 97%|█████████▋| 3037/3120 [4:12:59<06:33,  4.74s/it]
 97%|█████████▋| 3038/3120 [4:13:04<06:20,  4.65s/it]
                                                     
{'loss': 0.1141, 'learning_rate': 3.7745714790745623e-08, 'epoch': 3.89}

 97%|█████████▋| 3038/3120 [4:13:04<06:20,  4.65s/it]
 97%|█████████▋| 3039/3120 [4:13:09<06:28,  4.79s/it]
 97%|█████████▋| 3040/3120 [4:13:13<06:16,  4.70s/it]
                                                     
{'loss': 0.1189, 'learning_rate': 3.592800463325663e-08, 'epoch': 3.9}

 97%|█████████▋| 3040/3120 [4:13:13<06:16,  4.70s/it]
 97%|█████████▋| 3041/3120 [4:13:18<06:12,  4.72s/it]
 98%|█████████▊| 3042/3120 [4:13:23<06:16,  4.83s/it]
                                                     
{'loss': 0.1184, 'learning_rate': 3.4155069933301535e-08, 'epoch': 3.9}

 98%|█████████▊| 3042/3120 [4:13:23<06:16,  4.83s/it]
 98%|█████████▊| 3043/3120 [4:13:28<06:10,  4.82s/it]
 98%|█████████▊| 3044/3120 [4:13:33<06:01,  4.76s/it]
                                                     
{'loss': 0.1265, 'learning_rate': 3.242691865790071e-08, 'epoch': 3.9}

 98%|█████████▊| 3044/3120 [4:13:33<06:01,  4.76s/it]
 98%|█████████▊| 3045/3120 [4:13:38<06:00,  4.80s/it]
 98%|█████████▊| 3046/3120 [4:13:42<05:55,  4.80s/it]
                                                     
{'loss': 0.1156, 'learning_rate': 3.074355857282996e-08, 'epoch': 3.91}

 98%|█████████▊| 3046/3120 [4:13:42<05:55,  4.80s/it]
 98%|█████████▊| 3047/3120 [4:13:47<05:46,  4.74s/it]
 98%|█████████▊| 3048/3120 [4:13:52<05:54,  4.92s/it]
                                                     
{'loss': 0.1207, 'learning_rate': 2.9104997242590528e-08, 'epoch': 3.91}

 98%|█████████▊| 3048/3120 [4:13:52<05:54,  4.92s/it]
 98%|█████████▊| 3049/3120 [4:13:57<05:43,  4.84s/it]
 98%|█████████▊| 3050/3120 [4:14:03<05:53,  5.05s/it]
                                                     
{'loss': 0.1205, 'learning_rate': 2.7511242030368035e-08, 'epoch': 3.91}

 98%|█████████▊| 3050/3120 [4:14:03<05:53,  5.05s/it]
 98%|█████████▊| 3051/3120 [4:14:07<05:40,  4.93s/it]
 98%|█████████▊| 3052/3120 [4:14:12<05:36,  4.95s/it]
                                                     
{'loss': 0.1271, 'learning_rate': 2.5962300098008042e-08, 'epoch': 3.91}

 98%|█████████▊| 3052/3120 [4:14:12<05:36,  4.95s/it]
 98%|█████████▊| 3053/3120 [4:14:17<05:33,  4.98s/it]
 98%|█████████▊| 3054/3120 [4:14:22<05:30,  5.01s/it]
                                                     
{'loss': 0.1281, 'learning_rate': 2.4458178405974974e-08, 'epoch': 3.92}

 98%|█████████▊| 3054/3120 [4:14:22<05:30,  5.01s/it]
 98%|█████████▊| 3055/3120 [4:14:27<05:24,  4.99s/it]
 98%|█████████▊| 3056/3120 [4:14:32<05:19,  4.99s/it]
                                                     
{'loss': 0.124, 'learning_rate': 2.2998883713326592e-08, 'epoch': 3.92}

 98%|█████████▊| 3056/3120 [4:14:32<05:19,  4.99s/it]
 98%|█████████▊| 3057/3120 [4:14:37<05:14,  5.00s/it]
 98%|█████████▊| 3058/3120 [4:14:42<05:04,  4.92s/it]
                                                     
{'loss': 0.1141, 'learning_rate': 2.1584422577682894e-08, 'epoch': 3.92}

 98%|█████████▊| 3058/3120 [4:14:42<05:04,  4.92s/it]
 98%|█████████▊| 3059/3120 [4:14:47<05:02,  4.96s/it]
 98%|█████████▊| 3060/3120 [4:14:52<04:57,  4.96s/it]
                                                     
{'loss': 0.1134, 'learning_rate': 2.0214801355192826e-08, 'epoch': 3.92}

 98%|█████████▊| 3060/3120 [4:14:52<04:57,  4.96s/it]
 98%|█████████▊| 3061/3120 [4:14:57<04:55,  5.00s/it]
 98%|█████████▊| 3062/3120 [4:15:02<04:53,  5.06s/it]
                                                     
{'loss': 0.1182, 'learning_rate': 1.889002620051206e-08, 'epoch': 3.93}

 98%|█████████▊| 3062/3120 [4:15:02<04:53,  5.06s/it]
 98%|█████████▊| 3063/3120 [4:15:07<04:48,  5.06s/it]
 98%|█████████▊| 3064/3120 [4:15:12<04:37,  4.96s/it]
                                                     
{'loss': 0.1066, 'learning_rate': 1.761010306676969e-08, 'epoch': 3.93}

 98%|█████████▊| 3064/3120 [4:15:12<04:37,  4.96s/it]
 98%|█████████▊| 3065/3120 [4:15:17<04:35,  5.00s/it]
 98%|█████████▊| 3066/3120 [4:15:22<04:27,  4.95s/it]
                                                     
{'loss': 0.1203, 'learning_rate': 1.6375037705543827e-08, 'epoch': 3.93}

 98%|█████████▊| 3066/3120 [4:15:22<04:27,  4.95s/it]
 98%|█████████▊| 3067/3120 [4:15:27<04:25,  5.01s/it]
 98%|█████████▊| 3068/3120 [4:15:32<04:21,  5.03s/it]
                                                     
{'loss': 0.1186, 'learning_rate': 1.518483566683826e-08, 'epoch': 3.93}

 98%|█████████▊| 3068/3120 [4:15:32<04:21,  5.03s/it]
 98%|█████████▊| 3069/3120 [4:15:37<04:13,  4.96s/it]
 98%|█████████▊| 3070/3120 [4:15:42<04:10,  5.01s/it]
                                                     
{'loss': 0.1259, 'learning_rate': 1.4039502299050266e-08, 'epoch': 3.94}

 98%|█████████▊| 3070/3120 [4:15:42<04:10,  5.01s/it]
 98%|█████████▊| 3071/3120 [4:15:47<04:05,  5.02s/it]
 98%|█████████▊| 3072/3120 [4:15:52<04:03,  5.06s/it]
                                                     
{'loss': 0.1161, 'learning_rate': 1.2939042748955078e-08, 'epoch': 3.94}

 98%|█████████▊| 3072/3120 [4:15:52<04:03,  5.06s/it]
 98%|█████████▊| 3073/3120 [4:15:57<03:54,  5.00s/it]
 99%|█████████▊| 3074/3120 [4:16:02<03:50,  5.01s/it]
                                                     
{'loss': 0.1165, 'learning_rate': 1.1883461961678112e-08, 'epoch': 3.94}

 99%|█████████▊| 3074/3120 [4:16:02<03:50,  5.01s/it]
 99%|█████████▊| 3075/3120 [4:16:07<03:40,  4.90s/it]
 99%|█████████▊| 3076/3120 [4:16:11<03:28,  4.75s/it]
                                                     
{'loss': 0.116, 'learning_rate': 1.0872764680671666e-08, 'epoch': 3.94}

 99%|█████████▊| 3076/3120 [4:16:11<03:28,  4.75s/it]
 99%|█████████▊| 3077/3120 [4:16:16<03:21,  4.69s/it]
 99%|█████████▊| 3078/3120 [4:16:21<03:23,  4.84s/it]
                                                     
{'loss': 0.1178, 'learning_rate': 9.906955447697153e-09, 'epoch': 3.95}

 99%|█████████▊| 3078/3120 [4:16:21<03:23,  4.84s/it]
 99%|█████████▊| 3079/3120 [4:16:26<03:19,  4.88s/it]
 99%|█████████▊| 3080/3120 [4:16:31<03:13,  4.83s/it]
                                                     
{'loss': 0.1203, 'learning_rate': 8.986038602802894e-09, 'epoch': 3.95}

 99%|█████████▊| 3080/3120 [4:16:31<03:13,  4.83s/it]
 99%|█████████▉| 3081/3120 [4:16:36<03:13,  4.97s/it]
 99%|█████████▉| 3082/3120 [4:16:41<03:10,  5.01s/it]
                                                     
{'loss': 0.1177, 'learning_rate': 8.110018284304132e-09, 'epoch': 3.95}

 99%|█████████▉| 3082/3120 [4:16:41<03:10,  5.01s/it]
 99%|█████████▉| 3083/3120 [4:16:46<03:04,  4.98s/it]
 99%|█████████▉| 3084/3120 [4:16:51<02:54,  4.84s/it]
                                                     
{'loss': 0.117, 'learning_rate': 7.278898428764169e-09, 'epoch': 3.95}

 99%|█████████▉| 3084/3120 [4:16:51<02:54,  4.84s/it]
 99%|█████████▉| 3085/3120 [4:16:56<02:51,  4.91s/it]
 99%|█████████▉| 3086/3120 [4:17:00<02:46,  4.89s/it]
                                                     
{'loss': 0.1251, 'learning_rate': 6.492682770979919e-09, 'epoch': 3.96}

 99%|█████████▉| 3086/3120 [4:17:00<02:46,  4.89s/it]
 99%|█████████▉| 3087/3120 [4:17:05<02:41,  4.90s/it]
 99%|█████████▉| 3088/3120 [4:17:10<02:34,  4.84s/it]
                                                     
{'loss': 0.1221, 'learning_rate': 5.751374843961932e-09, 'epoch': 3.96}

 99%|█████████▉| 3088/3120 [4:17:10<02:34,  4.84s/it]
 99%|█████████▉| 3089/3120 [4:17:15<02:29,  4.83s/it]
 99%|█████████▉| 3090/3120 [4:17:20<02:29,  4.98s/it]
                                                     
{'loss': 0.1227, 'learning_rate': 5.054977978916631e-09, 'epoch': 3.96}

 99%|█████████▉| 3090/3120 [4:17:20<02:29,  4.98s/it]
 99%|█████████▉| 3091/3120 [4:17:25<02:23,  4.95s/it]
 99%|█████████▉| 3092/3120 [4:17:30<02:21,  5.07s/it]
                                                     
{'loss': 0.1306, 'learning_rate': 4.403495305237426e-09, 'epoch': 3.96}

 99%|█████████▉| 3092/3120 [4:17:30<02:21,  5.07s/it]
 99%|█████████▉| 3093/3120 [4:17:35<02:15,  5.04s/it]
 99%|█████████▉| 3094/3120 [4:17:40<02:08,  4.95s/it]
                                                     
{'loss': 0.1129, 'learning_rate': 3.7969297504858445e-09, 'epoch': 3.97}

 99%|█████████▉| 3094/3120 [4:17:40<02:08,  4.95s/it]
 99%|█████████▉| 3095/3120 [4:17:45<02:04,  4.98s/it]
 99%|█████████▉| 3096/3120 [4:17:50<01:59,  4.97s/it]
                                                     
{'loss': 0.1208, 'learning_rate': 3.2352840403804264e-09, 'epoch': 3.97}

 99%|█████████▉| 3096/3120 [4:17:50<01:59,  4.97s/it]
 99%|█████████▉| 3097/3120 [4:17:55<01:54,  4.97s/it]
 99%|█████████▉| 3098/3120 [4:18:00<01:48,  4.93s/it]
                                                     
{'loss': 0.1283, 'learning_rate': 2.7185606987811808e-09, 'epoch': 3.97}

 99%|█████████▉| 3098/3120 [4:18:00<01:48,  4.93s/it]
 99%|█████████▉| 3099/3120 [4:18:05<01:43,  4.94s/it]
 99%|█████████▉| 3100/3120 [4:18:10<01:38,  4.94s/it]
                                                     
{'loss': 0.1171, 'learning_rate': 2.246762047685147e-09, 'epoch': 3.97}

 99%|█████████▉| 3100/3120 [4:18:10<01:38,  4.94s/it]
 99%|█████████▉| 3101/3120 [4:18:15<01:34,  5.00s/it]
 99%|█████████▉| 3102/3120 [4:18:20<01:28,  4.93s/it]
                                                     
{'loss': 0.1164, 'learning_rate': 1.8198902072097402e-09, 'epoch': 3.98}

 99%|█████████▉| 3102/3120 [4:18:20<01:28,  4.93s/it]
 99%|█████████▉| 3103/3120 [4:18:25<01:22,  4.87s/it]
 99%|█████████▉| 3104/3120 [4:18:29<01:17,  4.84s/it]
                                                     
{'loss': 0.1199, 'learning_rate': 1.437947095582759e-09, 'epoch': 3.98}

 99%|█████████▉| 3104/3120 [4:18:29<01:17,  4.84s/it]
100%|█████████▉| 3105/3120 [4:18:35<01:14,  4.96s/it]
100%|█████████▉| 3106/3120 [4:18:39<01:08,  4.90s/it]
                                                     
{'loss': 0.1165, 'learning_rate': 1.1009344291401657e-09, 'epoch': 3.98}

100%|█████████▉| 3106/3120 [4:18:39<01:08,  4.90s/it]
100%|█████████▉| 3107/3120 [4:18:44<01:03,  4.91s/it]
100%|█████████▉| 3108/3120 [4:18:49<00:58,  4.89s/it]
                                                     
{'loss': 0.1148, 'learning_rate': 8.088537223116533e-10, 'epoch': 3.98}

100%|█████████▉| 3108/3120 [4:18:49<00:58,  4.89s/it]
100%|█████████▉| 3109/3120 [4:18:54<00:53,  4.89s/it]
100%|█████████▉| 3110/3120 [4:18:59<00:49,  4.97s/it]
                                                     
{'loss': 0.1127, 'learning_rate': 5.617062876195345e-10, 'epoch': 3.99}

100%|█████████▉| 3110/3120 [4:18:59<00:49,  4.97s/it]
100%|█████████▉| 3111/3120 [4:19:04<00:43,  4.83s/it]
100%|█████████▉| 3112/3120 [4:19:09<00:39,  4.88s/it]
                                                     
{'loss': 0.1129, 'learning_rate': 3.594932356654202e-10, 'epoch': 3.99}

100%|█████████▉| 3112/3120 [4:19:09<00:39,  4.88s/it]
100%|█████████▉| 3113/3120 [4:19:13<00:33,  4.76s/it]
100%|█████████▉| 3114/3120 [4:19:18<00:29,  4.84s/it]
                                                     
{'loss': 0.1164, 'learning_rate': 2.02215475132439e-10, 'epoch': 3.99}

100%|█████████▉| 3114/3120 [4:19:18<00:29,  4.84s/it]
100%|█████████▉| 3115/3120 [4:19:23<00:24,  4.99s/it]
100%|█████████▉| 3116/3120 [4:19:28<00:19,  4.93s/it]
                                                     
{'loss': 0.1217, 'learning_rate': 8.987371278079693e-11, 'epoch': 3.99}

100%|█████████▉| 3116/3120 [4:19:28<00:19,  4.93s/it]
100%|█████████▉| 3117/3120 [4:19:33<00:14,  4.96s/it]
100%|█████████▉| 3118/3120 [4:19:38<00:09,  4.94s/it]
                                                     
{'loss': 0.1212, 'learning_rate': 2.246845343667481e-11, 'epoch': 4.0}

100%|█████████▉| 3118/3120 [4:19:38<00:09,  4.94s/it]
100%|█████████▉| 3119/3120 [4:19:43<00:04,  4.88s/it]
100%|██████████| 3120/3120 [4:19:47<00:00,  4.72s/it]
                                                     
{'loss': 0.1224, 'learning_rate': 0.0, 'epoch': 4.0}

100%|██████████| 3120/3120 [4:19:47<00:00,  4.72s/it]/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
  warnings.warn(

                                                     
{'train_runtime': 15617.9007, 'train_samples_per_second': 12.785, 'train_steps_per_second': 0.2, 'train_loss': 0.610505388624584, 'epoch': 4.0}

100%|██████████| 3120/3120 [4:20:17<00:00,  4.72s/it]
100%|██████████| 3120/3120 [4:20:17<00:00,  5.01s/it]
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
[2024-04-21 04:02:35,619] [INFO] [launch.py:348:main] Process 274 exits successfully.
[2024-04-21 04:02:36,620] [INFO] [launch.py:348:main] Process 271 exits successfully.
[2024-04-21 04:02:36,620] [INFO] [launch.py:348:main] Process 268 exits successfully.
[2024-04-21 04:02:36,620] [INFO] [launch.py:348:main] Process 273 exits successfully.
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
[2024-04-21 04:02:37,621] [INFO] [launch.py:348:main] Process 270 exits successfully.
[2024-04-21 04:02:37,622] [INFO] [launch.py:348:main] Process 272 exits successfully.
[2024-04-21 04:02:37,622] [INFO] [launch.py:348:main] Process 269 exits successfully.
[2024-04-21 04:02:39,624] [INFO] [launch.py:348:main] Process 267 exits successfully.