681 KiB
681 KiB
[2024-04-20 23:35:42,048] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[2024-04-20 23:35:43,623] [WARNING] [runner.py:202:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only.
[2024-04-20 23:35:43,624] [INFO] [runner.py:568:main] cmd = /opt/conda/bin/python3.8 -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMCwgMSwgMiwgMywgNCwgNSwgNiwgN119 --master_addr=127.0.0.1 --master_port=29500 --enable_each_rank_log=None train.py --model_name_or_path /workspace/qwen2_4b/ --data_path refgpt_train.json --output_dir qwen2_4b_4epoch_refgpt --num_train_epochs 4 --model_max_length 2048 --per_device_train_batch_size 1 --per_device_eval_batch_size 2 --gradient_accumulation_steps 8 --evaluation_strategy no --save_strategy steps --save_steps 780 --save_total_limit 10 --learning_rate 2e-5 --warmup_ratio 0.05 --logging_steps 2 --lr_scheduler_type cosine --report_to tensorboard --bf16 True --deepspeed ds_config.json --use_lora False
[2024-04-20 23:35:44,554] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_TC=136
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.9.9-1+cuda11.3
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_DEV_PACKAGE_VERSION=2.9.9-1
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_MIN_NCHANNELS=4
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_NET_PLUGIN=none
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_VERSION=2.12.10
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_SOCKET_IFNAME=eth
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_DEBUG=INFO
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_HCA=mlx5
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_GID_INDEX=3
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_QPS_PER_CONNECTION=8
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_TIMEOUT=22
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_SL=5
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_PACKAGE=libnccl2=2.9.9-1+cuda11.3
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_PACKAGE_NAME=libnccl2
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_PACKAGE_VERSION=2.9.9-1
[2024-04-20 23:35:46,123] [INFO] [launch.py:145:main] WORLD INFO DICT: {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]}
[2024-04-20 23:35:46,123] [INFO] [launch.py:151:main] nnodes=1, num_local_procs=8, node_rank=0
[2024-04-20 23:35:46,123] [INFO] [launch.py:162:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]})
[2024-04-20 23:35:46,123] [INFO] [launch.py:163:main] dist_world_size=8
[2024-04-20 23:35:46,123] [INFO] [launch.py:165:main] Setting CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
[2024-04-20 23:35:46,128] [INFO] [launch.py:253:main] process 267 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=0', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,131] [INFO] [launch.py:253:main] process 268 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=1', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,134] [INFO] [launch.py:253:main] process 269 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=2', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,137] [INFO] [launch.py:253:main] process 270 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=3', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,140] [INFO] [launch.py:253:main] process 271 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=4', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,143] [INFO] [launch.py:253:main] process 272 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=5', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,147] [INFO] [launch.py:253:main] process 273 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=6', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,151] [INFO] [launch.py:253:main] process 274 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=7', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:49,984] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:49,990] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:49,999] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,015] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,036] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,050] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,054] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,067] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[2024-04-20 23:35:52,257] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-04-20 23:35:52,261] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-04-20 23:35:52,261] [INFO] [comm.py:668:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
[2024-04-20 23:35:52,266] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 6
[2024-04-20 23:35:52,300] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-04-20 23:35:52,301] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 4
[INFO] [2024-04-20 23:35:52] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 7
[2024-04-20 23:35:52,309] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 2
[2024-04-20 23:35:52,325] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 5
[2024-04-20 23:35:52,340] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 1
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 3
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 0
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 3: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 5: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 6: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 1: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 4: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 2: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 7: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.24s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.43s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.39s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.35s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.13s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.22s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.20s/it]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.20s/it]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
[INFO] [2024-04-20 23:36:35] train.py : Formatting inputs...
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Plugin name set by env to libnccl-net-none.so
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Using network IB
NCCL version 2.12.10+cuda11.3
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO Plugin name set by env to libnccl-net-none.so
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO Plugin name set by env to libnccl-net-none.so
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO Plugin name set by env to libnccl-net-none.so
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO Plugin name set by env to libnccl-net-none.so
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO Plugin name set by env to libnccl-net-none.so
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO Plugin name set by env to libnccl-net-none.so
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO Plugin name set by env to libnccl-net-none.so
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Setting affinity for GPU 0 to 0fffff
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Setting affinity for GPU 2 to 0fffff
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Setting affinity for GPU 3 to 0fffff
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Setting affinity for GPU 1 to 0fffff
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 00/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 01/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 02/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 03/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 04/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 05/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 06/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 07/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 08/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 09/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 10/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 11/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 12/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 13/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 14/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 15/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 16/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 17/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 18/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 19/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 20/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 21/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 22/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 23/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 00 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 00 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 00 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 00 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 00 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 00 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 01 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 01 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 01 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 01 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 01 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 01 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 02 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 02 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 02 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 02 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 02 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 00 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 02 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 03 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 03 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 03 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 03 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 01 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 03 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 00 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 04 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 04 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 04 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 04 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 02 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 04 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 01 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 05 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 05 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 05 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 05 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 03 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 05 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 06 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 02 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 06 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 06 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 03 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 04 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 06 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 06 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 07 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 07 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 07 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 04 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 05 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 03 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 07 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 07 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 08 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 08 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 05 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 08 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 04 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 06 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 08 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 08 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 09 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 09 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 09 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 06 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 05 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 07 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 09 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 10 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 09 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 10 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 10 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 07 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 08 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 06 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 10 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 11 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 10 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 11 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 11 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 08 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 09 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 07 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 11 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 11 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 12 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 12 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 09 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 12 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 10 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 08 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 12 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 13 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 12 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 13 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 13 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 09 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 10 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 11 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 14 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 14 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 13 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 14 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 10 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 11 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 15 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 14 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 15 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 11 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 12 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 16 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 15 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 16 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 12 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 13 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 17 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 16 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 17 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 14 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 13 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 18 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 17 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 18 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 15 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 14 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 19 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 18 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 19 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 16 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 20 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 13 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 15 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 15 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 19 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 17 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 20 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 21 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 12 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 14 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 16 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 18 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 21 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 20 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 22 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 17 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 19 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 21 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 22 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 23 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 18 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 20 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 23 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 19 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 21 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 20 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 22 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 21 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 23 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 22 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 23 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 22 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 23 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 13 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 16 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 15 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 14 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 17 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 15 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 18 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 16 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 19 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 16 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 17 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 20 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 17 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 18 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 21 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 18 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 19 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 22 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 19 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 20 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 23 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 20 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 21 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 21 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 22 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 22 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 23 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 23 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 00 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 01 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 02 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 03 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 04 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 05 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 06 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 07 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 08 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 09 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 10 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 11 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 12 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 13 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 00 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 14 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 01 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 15 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 02 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 16 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 03 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 17 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 04 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 18 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 05 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 19 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 06 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 20 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 07 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 21 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 08 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 22 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 09 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 23 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 10 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 11 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 12 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 13 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 00 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 00 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 00 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 00 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 14 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 01 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 00 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 01 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 01 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 01 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 01 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 02 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 15 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 02 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 02 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 02 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 02 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 03 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 16 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 03 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 03 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 03 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 03 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 04 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 17 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 04 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 04 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 04 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 04 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 18 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 05 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 05 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 05 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 05 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 19 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 06 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 06 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 06 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 06 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 20 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 07 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 07 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 07 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 07 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 21 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 08 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 08 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 05 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 08 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 22 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 08 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 09 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 09 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 06 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 09 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 23 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 10 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 09 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 10 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 07 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 10 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 10 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 11 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 11 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 08 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 11 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 12 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 12 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 11 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 09 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 12 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 13 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 13 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 13 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 12 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 14 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 14 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 14 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 13 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 15 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 15 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 15 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 14 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 16 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 16 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 16 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 15 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 17 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 17 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 17 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 16 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 18 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 18 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 18 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 17 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 19 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 19 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 18 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 19 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 10 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 20 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 20 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 19 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 21 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 20 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 21 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 20 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 22 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 21 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 22 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 21 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 23 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 22 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 23 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 22 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 23 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 23 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 11 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 12 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 13 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 14 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 15 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 16 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 17 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 18 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 19 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 20 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 21 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 22 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 23 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO comm 0x7f3ffc008fb0 rank 5 nranks 8 cudaDev 5 busId 60 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO comm 0x7f3114008fb0 rank 3 nranks 8 cudaDev 3 busId 40 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO comm 0x7f3794008fb0 rank 1 nranks 8 cudaDev 1 busId 20 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO comm 0x7f4f34008fb0 rank 7 nranks 8 cudaDev 7 busId 80 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO comm 0x7f53a0008fb0 rank 2 nranks 8 cudaDev 2 busId 30 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO comm 0x7fed4c008fb0 rank 0 nranks 8 cudaDev 0 busId 10 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO comm 0x7fa874008fb0 rank 6 nranks 8 cudaDev 6 busId 70 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Launch mode Parallel
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO comm 0x7f4758008fb0 rank 4 nranks 8 cudaDev 4 busId 50 - Init COMPLETE
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Data Num: 49920
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Sample 34992 of the training set: {'input_ids': tensor([104600, 12, 44636, 102430, 100280, 102064, 11095, 15843, 71,
1174, 15459, 58463, 99449, 100280, 102064, 104995, 16, 18,
15, 86402, 102064, 107339, 72881, 70769, 3837, 17714, 58463,
99449, 33108, 109033, 104995, 23, 11, 15, 15, 15,
102266, 31838, 37029, 1773, 106676, 72881, 9370, 101468, 105570,
42140, 34204, 92894, 58463, 99449, 100280, 102064, 101468, 9370,
59743, 33108, 1773, 92894, 101194, 101468, 109824, 29524, 106676,
106758, 104995, 16, 15, 15, 102266, 105899, 101705, 72881,
9909, 39120, 644, 48272, 44636, 102430, 72881, 3837, 23,
15, 102266, 105899, 44956, 99727, 72881, 9909, 42, 84,
352, 48272, 112964, 106835, 33108, 105906, 112545, 23, 15,
102266, 105899, 104600, 72881, 1773, 18493, 104152, 104600, 12,
44636, 102430, 100280, 102064, 15946, 3837, 101043, 104600, 72881,
5373, 44636, 102430, 72881, 33108, 106676, 72881, 18493, 16,
24, 101186, 103982, 18830, 108201, 100169, 1773, 26940, 99470,
104600, 72881, 25067, 99461, 33956, 33477, 61443, 34204, 22,
101186, 3837, 20412, 101888, 58463, 99449, 113228, 105871, 82832,
3837, 103957, 112964, 17340, 103993, 109755, 9909, 73743, 99915,
11095, 25079, 33108, 1425, 292, 4378, 5942, 74276, 104600,
12, 44636, 102430, 100280, 102064, 9370, 101460, 105761, 104363,
20412, 101514, 99740, 78685, 3837, 99740, 78685, 116966, 100405,
104610, 99278, 102357, 70074, 9370, 119301, 119301, 111947, 111947,
9370, 102124, 90395, 100136, 104250, 70074, 47872, 102478, 1773,
33975, 25, 220, 108386, 37945, 56007, 107809, 104169, 99526,
99526, 104600, 12, 44636, 102430, 100280, 102064, 101037, 26850,
71703, 25, 220, 108386, 3837, 103942, 73670, 1773, 104600,
12, 44636, 102430, 100280, 102064, 101909, 67071, 104995, 16,
18, 15, 86402, 102064, 107339, 72881, 70769, 3837, 99558,
114021, 58463, 99449, 33108, 109033, 100361, 3837, 104678, 104995,
23, 15, 15, 15, 102266, 18493, 37029, 100001, 102064,
1773, 90919, 3837, 106676, 72881, 37029, 104346, 102538, 3837,
99427, 71304, 92894, 58463, 99449, 100280, 102064, 9370, 59743,
33108, 1773, 110075, 37029, 104346, 106204, 109824, 100630, 101705,
72881, 9909, 39120, 644, 3837, 94237, 16, 15, 15,
102266, 37029, 3837, 99558, 18493, 106676, 106758, 64359, 44636,
102430, 72881, 5373, 44956, 99727, 72881, 9909, 42, 84,
352, 3837, 94237, 23, 15, 102266, 37029, 7552, 101034,
104600, 72881, 9909, 18493, 112964, 106835, 33108, 105906, 112545,
18830, 23, 15, 102266, 37029, 74276, 18493, 100001, 104600,
12, 44636, 102430, 100280, 102064, 15946, 3837, 104600, 72881,
5373, 44636, 102430, 72881, 33108, 106676, 72881, 18830, 108201,
100169, 3837, 73670, 109234, 26939, 16, 24, 101186, 103982,
1773, 100654, 100146, 91111, 99470, 104600, 72881, 25067, 114292,
101888, 58463, 99449, 113228, 105871, 82832, 3837, 105985, 22,
101186, 104435, 65577, 3837, 103957, 99250, 112964, 17340, 109755,
1773, 116880, 3837, 104600, 12, 44636, 102430, 100280, 102064,
9370, 101460, 105761, 115093, 103926, 101514, 99740, 78685, 5373,
116966, 100405, 100136, 16530, 99278, 102357, 70074, 3837, 91572,
104250, 70074, 47872, 102478, 3407, 33975, 25, 49434, 239,
104958, 104600, 12, 44636, 102430, 100280, 102064, 118619, 99740,
78685, 37945, 56007, 100001, 99740, 78685, 104139, 100772, 11319,
106998, 64471, 33108, 109023, 99604, 26850, 71703, 25, 54851,
9370, 3837, 104600, 12, 44636, 102430, 100280, 102064, 104877,
109023, 99883, 3837, 100629, 33126, 104653, 99740, 78685, 1773,
100001, 99740, 78685, 9370, 116966, 115390, 3837, 102119, 16530,
111240, 102357, 70074, 3837, 105884, 116966, 13343, 102274, 99792,
103148, 3837, 106085, 109023, 101047, 111817, 78685, 3837, 29524,
2073, 89, 33590, 2073, 23815, 854, 49567, 116966, 13343,
107689, 118216, 102073, 102357, 70074, 1773, 104043, 3837, 104600,
12, 44636, 102430, 100280, 102064, 18493, 116966, 13343, 102119,
110921, 106050, 70074, 47872, 102478, 3837, 105064, 33108, 109023,
103943, 100687, 102665, 3837, 99519, 109023, 101158, 70074, 47872,
102064, 3837, 101970, 70074, 47872, 73670, 51463, 101970, 99689,
64559, 1773, 106279, 3837, 104600, 12, 44636, 102430, 100280,
102064, 9370, 99740, 78685, 100772, 33108, 109023, 101208, 3837,
99558, 20742, 99601, 100733, 33071, 5373, 116966, 9370, 100405,
33071, 101034, 104250, 70074, 47872, 102478, 17447, 1773, 151643]), 'labels': tensor([ -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
33975, 25, 220, 108386, 37945, 56007, 107809, 104169, 99526,
99526, 104600, 12, 44636, 102430, 100280, 102064, 101037, 26850,
71703, 25, 220, 108386, 3837, 103942, 73670, 1773, 104600,
12, 44636, 102430, 100280, 102064, 101909, 67071, 104995, 16,
18, 15, 86402, 102064, 107339, 72881, 70769, 3837, 99558,
114021, 58463, 99449, 33108, 109033, 100361, 3837, 104678, 104995,
23, 15, 15, 15, 102266, 18493, 37029, 100001, 102064,
1773, 90919, 3837, 106676, 72881, 37029, 104346, 102538, 3837,
99427, 71304, 92894, 58463, 99449, 100280, 102064, 9370, 59743,
33108, 1773, 110075, 37029, 104346, 106204, 109824, 100630, 101705,
72881, 9909, 39120, 644, 3837, 94237, 16, 15, 15,
102266, 37029, 3837, 99558, 18493, 106676, 106758, 64359, 44636,
102430, 72881, 5373, 44956, 99727, 72881, 9909, 42, 84,
352, 3837, 94237, 23, 15, 102266, 37029, 7552, 101034,
104600, 72881, 9909, 18493, 112964, 106835, 33108, 105906, 112545,
18830, 23, 15, 102266, 37029, 74276, 18493, 100001, 104600,
12, 44636, 102430, 100280, 102064, 15946, 3837, 104600, 72881,
5373, 44636, 102430, 72881, 33108, 106676, 72881, 18830, 108201,
100169, 3837, 73670, 109234, 26939, 16, 24, 101186, 103982,
1773, 100654, 100146, 91111, 99470, 104600, 72881, 25067, 114292,
101888, 58463, 99449, 113228, 105871, 82832, 3837, 105985, 22,
101186, 104435, 65577, 3837, 103957, 99250, 112964, 17340, 109755,
1773, 116880, 3837, 104600, 12, 44636, 102430, 100280, 102064,
9370, 101460, 105761, 115093, 103926, 101514, 99740, 78685, 5373,
116966, 100405, 100136, 16530, 99278, 102357, 70074, 3837, 91572,
104250, 70074, 47872, 102478, 3407, 33975, 25, 49434, 239,
104958, 104600, 12, 44636, 102430, 100280, 102064, 118619, 99740,
78685, 37945, 56007, 100001, 99740, 78685, 104139, 100772, 11319,
106998, 64471, 33108, 109023, 99604, 26850, 71703, 25, 54851,
9370, 3837, 104600, 12, 44636, 102430, 100280, 102064, 104877,
109023, 99883, 3837, 100629, 33126, 104653, 99740, 78685, 1773,
100001, 99740, 78685, 9370, 116966, 115390, 3837, 102119, 16530,
111240, 102357, 70074, 3837, 105884, 116966, 13343, 102274, 99792,
103148, 3837, 106085, 109023, 101047, 111817, 78685, 3837, 29524,
2073, 89, 33590, 2073, 23815, 854, 49567, 116966, 13343,
107689, 118216, 102073, 102357, 70074, 1773, 104043, 3837, 104600,
12, 44636, 102430, 100280, 102064, 18493, 116966, 13343, 102119,
110921, 106050, 70074, 47872, 102478, 3837, 105064, 33108, 109023,
103943, 100687, 102665, 3837, 99519, 109023, 101158, 70074, 47872,
102064, 3837, 101970, 70074, 47872, 73670, 51463, 101970, 99689,
64559, 1773, 106279, 3837, 104600, 12, 44636, 102430, 100280,
102064, 9370, 99740, 78685, 100772, 33108, 109023, 101208, 3837,
99558, 20742, 99601, 100733, 33071, 5373, 116966, 9370, 100405,
33071, 101034, 104250, 70074, 47872, 102478, 17447, 1773, 151643])}, torch.Size([603]).
孟-高棉诸语言Mon-Khmer languages南亚诸语言大约130种语言组成的语族，为南亚和东南亚大约8,000万人所使用。越南语的使用者远远多于其他南亚诸语言使用者的总和。其他有很多使用者的语言如越南北部大约100万人使用的芒语（Muong），高棉语，80万人使用的库阿语（Kuay），缅甸南部和泰国部分地区80万人使用的孟语。在所有的孟-高棉诸语言中，只有孟语、高棉语和越南语在19世纪以前有书写传统。《古孟语》已经确证写于7世纪，是关于南亚起源的手稿，后来缅甸人进行了改编（参阅Mon kingdom和Indic writing systems）。孟-高棉诸语言的典型语音特征是大量的母音，母音发音简单而不是带呼吸声的叽叽嘎嘎的表达，并且缺乏声调差异。Human: 你好，请问你能给我讲讲孟-高棉诸语言吗？
Assistant: 你好，当然可以。孟-高棉诸语言是一个由大约130种语言组成的语族，主要分布在南亚和东南亚地区，共有大约8000万人在使用这些语言。其中，越南语使用人数最多，远超其他南亚诸语言的总和。还有一些使用人数较多的语言包括芒语（Muong，约100万人使用，主要在越南北部）、高棉语、库阿语（Kuay，约80万人使用）以及孟语（在缅甸南部和泰国部分地区有80万人使用）。在这些孟-高棉诸语言中，孟语、高棉语和越南语有书写传统，可以追溯到19世纪以前。特别的是，《古孟语》是一部关于南亚起源的手稿，早在7世纪就有记录，后来被缅甸人改编。总的来说，孟-高棉诸语言的典型语音特点是拥有大量的母音、发音简单且不带呼吸声，同时缺乏声调差异。
Human: 我了解到孟-高棉诸语言有很多种母音，请问这些母音有什么特点？比如说是否和汉语不同？
Assistant: 是的，孟-高棉诸语言对比汉语来说，具有更丰富的母音。这些母音的发音比较简单，通常不伴随呼吸声，也就是说发音时声音比较单纯，不像汉语中的浊音，如“z”，“zh”等发音时伴随着喉咙里的呼吸声。此外，孟-高棉诸语言在发音时通常不会有明显的声调差异，这一点和汉语也有明显区别，因为汉语是一种声调语言，不同的声调可以表示不同的词义。总之，孟-高棉诸语言的母音特点和汉语相比，主要表现在丰富性、发音的简单性以及缺乏声调差异上。<|endoftext|>
[INFO] [2024-04-20 23:38:55] train.py : Sample 46308 of the training set: {'input_ids': tensor([100349, 102028, 85361, 112272, 16, 24, 17, 23, 21216,
17, 15, 15, 15, 7948, 53, 329, 318, 320,
47, 3433, 72, 1020, 1579, 859, 701, 28507, 104328,
100378, 104727, 1773, 16, 24, 19, 15, 104227, 107886,
3837, 42411, 99992, 30709, 101430, 101193, 102508, 99418, 104534,
3837, 103922, 99364, 113259, 100378, 103132, 1773, 102888, 3837,
42411, 104247, 26940, 120036, 99686, 100027, 25067, 9909, 28085,
3891, 665, 11, 220, 16, 24, 20, 18, 7552,
104195, 9370, 106575, 1773, 42411, 115999, 62926, 100656, 108598,
34187, 101251, 105651, 62945, 38035, 100378, 26940, 106020, 100211,
101989, 25067, 9909, 3036, 4264, 4290, 24511, 11, 220,
16, 24, 20, 21, 48272, 75882, 34718, 108438, 100648,
102375, 102461, 107309, 109452, 120411, 110803, 106411, 106681, 1773,
103957, 3837, 42411, 14777, 87256, 100152, 99487, 67279, 99959,
16530, 25710, 9370, 99619, 8863, 1773, 42411, 11622, 99497,
116170, 102627, 9909, 2082, 78728, 794, 3352, 7725, 7552,
100172, 26940, 104264, 100145, 25067, 7, 97215, 782, 83657,
19379, 11, 220, 16, 24, 20, 24, 69515, 11622,
100861, 102261, 93488, 100172, 26940, 108481, 82847, 113724, 25067,
9909, 3428, 54102, 35782, 11, 220, 16, 24, 21,
23, 48272, 62926, 11622, 104156, 100841, 105115, 100721, 105658,
103639, 100172, 26940, 100397, 99244, 108571, 99232, 25067, 9909,
53, 558, 323, 47775, 361, 11, 220, 16, 24,
21, 17, 48272, 24562, 105352, 100132, 100648, 102375, 3837,
100161, 101961, 104428, 100648, 103430, 1773, 33975, 25, 220,
14880, 106525, 100349, 102028, 13935, 101190, 100222, 101316, 105518,
26850, 71703, 25, 10236, 121, 245, 102028, 13935, 101190,
100222, 101316, 9909, 78425, 92112, 318, 3837, 35987, 13072,
53, 329, 318, 393, 3433, 72, 1020, 1579, 859,
7552, 109182, 104328, 100378, 104727, 3837, 100648, 111978, 114984,
17, 15, 101186, 19, 15, 104227, 1773, 102965, 3837,
42411, 107709, 102508, 99418, 17447, 104247, 104534, 3837, 101889,
106438, 100378, 100650, 1773, 100648, 104727, 103132, 104847, 100694,
19108, 3837, 104033, 16, 24, 20, 21, 104300, 100378,
26940, 106020, 100211, 101989, 25067, 9909, 3036, 4264, 4290,
24511, 48272, 99652, 108438, 42411, 102375, 102461, 107309, 13935,
109452, 120411, 99787, 110803, 106411, 106681, 1773, 104043, 3837,
106473, 115999, 106650, 92894, 102280, 104869, 90395, 57218, 108171,
100527, 114224, 71817, 38182, 99721, 3407, 33975, 25, 220,
42411, 104727, 34187, 102224, 100527, 104869, 26850, 71703, 25,
10236, 121, 245, 102028, 13935, 101190, 100222, 101316, 104727,
105593, 80942, 103215, 9370, 100378, 1773, 77557, 91111, 106020,
100211, 101989, 25067, 9909, 16, 24, 20, 21, 7948,
3837, 107119, 102461, 107309, 13935, 109452, 120411, 64359, 26940,
104264, 100145, 25067, 9909, 16, 24, 20, 24, 7948,
3837, 107119, 2082, 78728, 794, 3352, 7725, 64359, 26940,
100397, 99244, 108571, 99232, 25067, 9909, 16, 24, 21,
17, 7948, 3837, 107119, 104156, 107318, 105115, 13935, 99462,
99458, 103639, 7552, 101034, 26940, 108481, 82847, 113724, 25067,
9909, 16, 24, 21, 23, 7948, 3837, 107119, 98237,
13935, 23384, 93488, 74276, 100001, 100378, 99902, 106844, 108124,
104727, 101262, 3837, 97706, 17714, 42411, 57218, 108171, 102280,
114224, 107298, 104257, 100133, 3407, 33975, 25, 220, 42411,
99557, 103928, 101968, 102152, 107258, 105815, 100145, 11319, 4710,
71703, 25, 10236, 121, 245, 102028, 13935, 101190, 100222,
101316, 57218, 108171, 101968, 102152, 107258, 105815, 100145, 1773,
45181, 113460, 27369, 100192, 3837, 42411, 57218, 102461, 107309,
13935, 109452, 120411, 9909, 6828, 12597, 665, 63204, 354,
7552, 104388, 90395, 18493, 104727, 9370, 100378, 26940, 106020,
100211, 101989, 25067, 15946, 57218, 99517, 99721, 1773, 91572,
3837, 106473, 99798, 57218, 2082, 78728, 794, 3352, 7725,
104388, 62926, 104727, 34187, 99517, 107119, 9370, 100378, 26940,
104264, 100145, 25067, 1773, 104043, 3837, 106473, 57218, 98237,
13935, 23384, 93488, 99721, 104421, 34187, 100378, 26940, 108481,
82847, 113724, 87243, 110133, 102280, 104328, 114224, 104156, 107318,
105115, 13935, 99462, 99458, 103639, 9909, 34, 27009, 422,
1952, 56023, 7552, 107258, 103430, 100145, 3837, 111733, 26940,
100397, 99244, 108571, 99232, 25067, 9370, 104727, 3407, 33975,
25, 220, 100648, 104040, 33108, 100772, 102021, 26850, 71703,
25, 51461, 117, 16038, 113460, 27369, 3837, 100349, 102028,
13935, 101190, 100222, 101316, 9370, 104040, 33108, 100772, 101321,
42411, 106614, 109362, 101968, 57566, 33108, 102550, 3837, 104033,
101968, 9370, 110803, 105997, 1773, 100648, 100378, 100663, 101207,
23031, 101968, 100782, 63836, 62926, 101107, 104258, 9370, 101280,
33108, 110803, 1773, 77557, 91111, 106020, 100211, 101989, 25067,
108438, 102461, 107309, 13935, 109452, 120411, 110803, 106411, 106681,
24968, 92894, 100663, 29524, 26940, 104264, 100145, 25067, 9909,
107119, 2082, 78728, 794, 3352, 7725, 64359, 26940, 108481,
82847, 113724, 25067, 9909, 107119, 98237, 13935, 23384, 93488,
7552, 49567, 74763, 100629, 105790, 105117, 1773, 91572, 3837,
42411, 104026, 100137, 67279, 99959, 104339, 3837, 57218, 108171,
100527, 114224, 99721, 96050, 107222, 100378, 102707, 104847, 19108,
1773, 151643]), 'labels': tensor([ -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, 33975, 25, 220,
14880, 106525, 100349, 102028, 13935, 101190, 100222, 101316, 105518,
26850, 71703, 25, 10236, 121, 245, 102028, 13935, 101190,
100222, 101316, 9909, 78425, 92112, 318, 3837, 35987, 13072,
53, 329, 318, 393, 3433, 72, 1020, 1579, 859,
7552, 109182, 104328, 100378, 104727, 3837, 100648, 111978, 114984,
17, 15, 101186, 19, 15, 104227, 1773, 102965, 3837,
42411, 107709, 102508, 99418, 17447, 104247, 104534, 3837, 101889,
106438, 100378, 100650, 1773, 100648, 104727, 103132, 104847, 100694,
19108, 3837, 104033, 16, 24, 20, 21, 104300, 100378,
26940, 106020, 100211, 101989, 25067, 9909, 3036, 4264, 4290,
24511, 48272, 99652, 108438, 42411, 102375, 102461, 107309, 13935,
109452, 120411, 99787, 110803, 106411, 106681, 1773, 104043, 3837,
106473, 115999, 106650, 92894, 102280, 104869, 90395, 57218, 108171,
100527, 114224, 71817, 38182, 99721, 3407, 33975, 25, 220,
42411, 104727, 34187, 102224, 100527, 104869, 26850, 71703, 25,
10236, 121, 245, 102028, 13935, 101190, 100222, 101316, 104727,
105593, 80942, 103215, 9370, 100378, 1773, 77557, 91111, 106020,
100211, 101989, 25067, 9909, 16, 24, 20, 21, 7948,
3837, 107119, 102461, 107309, 13935, 109452, 120411, 64359, 26940,
104264, 100145, 25067, 9909, 16, 24, 20, 24, 7948,
3837, 107119, 2082, 78728, 794, 3352, 7725, 64359, 26940,
100397, 99244, 108571, 99232, 25067, 9909, 16, 24, 21,
17, 7948, 3837, 107119, 104156, 107318, 105115, 13935, 99462,
99458, 103639, 7552, 101034, 26940, 108481, 82847, 113724, 25067,
9909, 16, 24, 21, 23, 7948, 3837, 107119, 98237,
13935, 23384, 93488, 74276, 100001, 100378, 99902, 106844, 108124,
104727, 101262, 3837, 97706, 17714, 42411, 57218, 108171, 102280,
114224, 107298, 104257, 100133, 3407, 33975, 25, 220, 42411,
99557, 103928, 101968, 102152, 107258, 105815, 100145, 11319, 4710,
71703, 25, 10236, 121, 245, 102028, 13935, 101190, 100222,
101316, 57218, 108171, 101968, 102152, 107258, 105815, 100145, 1773,
45181, 113460, 27369, 100192, 3837, 42411, 57218, 102461, 107309,
13935, 109452, 120411, 9909, 6828, 12597, 665, 63204, 354,
7552, 104388, 90395, 18493, 104727, 9370, 100378, 26940, 106020,
100211, 101989, 25067, 15946, 57218, 99517, 99721, 1773, 91572,
3837, 106473, 99798, 57218, 2082, 78728, 794, 3352, 7725,
104388, 62926, 104727, 34187, 99517, 107119, 9370, 100378, 26940,
104264, 100145, 25067, 1773, 104043, 3837, 106473, 57218, 98237,
13935, 23384, 93488, 99721, 104421, 34187, 100378, 26940, 108481,
82847, 113724, 87243, 110133, 102280, 104328, 114224, 104156, 107318,
105115, 13935, 99462, 99458, 103639, 9909, 34, 27009, 422,
1952, 56023, 7552, 107258, 103430, 100145, 3837, 111733, 26940,
100397, 99244, 108571, 99232, 25067, 9370, 104727, 3407, 33975,
25, 220, 100648, 104040, 33108, 100772, 102021, 26850, 71703,
25, 51461, 117, 16038, 113460, 27369, 3837, 100349, 102028,
13935, 101190, 100222, 101316, 9370, 104040, 33108, 100772, 101321,
42411, 106614, 109362, 101968, 57566, 33108, 102550, 3837, 104033,
101968, 9370, 110803, 105997, 1773, 100648, 100378, 100663, 101207,
23031, 101968, 100782, 63836, 62926, 101107, 104258, 9370, 101280,
33108, 110803, 1773, 77557, 91111, 106020, 100211, 101989, 25067,
108438, 102461, 107309, 13935, 109452, 120411, 110803, 106411, 106681,
24968, 92894, 100663, 29524, 26940, 104264, 100145, 25067, 9909,
107119, 2082, 78728, 794, 3352, 7725, 64359, 26940, 108481,
82847, 113724, 25067, 9909, 107119, 98237, 13935, 23384, 93488,
7552, 49567, 74763, 100629, 105790, 105117, 1773, 91572, 3837,
42411, 104026, 100137, 67279, 99959, 104339, 3837, 57218, 108171,
100527, 114224, 99721, 96050, 107222, 100378, 102707, 104847, 19108,
1773, 151643])}, torch.Size([740]).
罗杰华汀1928～2000年Vadim (Plemiannikov), Roger法国电影导演。1940年代中期，他有一小段时间从事舞台剧演员，之后便转入电影生涯。最早，他担任《茱丽叶》（Juliette, 1953）一片的助理。他执导并共同编写了高度成功的异色电影《上帝创造女人》（And God Created Woman, 1956），该片奠定了他的妻子碧姬芭铎性感象征的地位。后来，他一再利用这个包赢不输的套数。他用史绰柏（Annette Stroyberg）拍《危险关系》(Dangerous Liaisons, 1959)，用珍芳达拍《太空英雌》（Barbarella, 1968），并用凯萨琳丹妮芙拍《乱世姐妹花》（Vice and Virtue, 1962），前两位都是他的妻子，最后一位则是他的情人。Human: 请告诉我罗杰·瓦迪姆是谁？
Assistant: 罗杰·瓦迪姆（Roger Vadim，全名Vadim Plemiannikov）是一位法国电影导演，他的职业生涯始于20世纪40年代。最初，他曾在舞台剧上担任演员，然后转向电影领域。他的导演生涯取得了许多成功，尤其是1956年的电影《上帝创造女人》（And God Created Woman），它奠定了他妻子碧姬·芭铎成为性感象征的地位。此外，他还执导了许多其他著名影片，并与多位知名女星进行过合作。
Human: 他导演了哪些知名影片？
Assistant: 罗杰·瓦迪姆导演了一些广受欢迎的电影。例如，《上帝创造女人》（1956年，主演碧姬·芭铎）、《危险关系》（1959年，主演Annette Stroyberg）、《乱世姐妹花》（1962年，主演凯瑟琳·德纳芙）以及《太空英雌》（1968年，主演简·方达）。这些电影不仅确立了他的导演地位，还为他与多位著名女星的合作提供了平台。
Human: 他跟那些女性明星有过私人关系？
Assistant: 罗杰·瓦迪姆与多位女性明星有过私人关系。从所提供的信息来看，他与碧姬·芭铎（Brigitte Bardot）结婚，并在导演的电影《上帝创造女人》中与她合作。同时，他还曾与Annette Stroyberg结婚并导演了她主演的电影《危险关系》。此外，他还与简·方达合作拍摄了电影《太空英雌》，并与著名法国女星凯瑟琳·德纳芙（Catherine Deneuve）有过情人关系，同时也是《乱世姐妹花》的导演。
Human: 他的风格和特点是什么？
Assistant: 根据所提供的信息，罗杰·瓦迪姆的风格和特点在于他善于发掘女性美和魅力，尤其是女性的性感一面。他的电影作品往往以女性为主角并表现她们的美丽和性感。例如，《上帝创造女人》奠定了碧姬·芭铎性感象征的地位；其他作品如《危险关系》（主演Annette Stroyberg）、《太空英雌》（主演简·方达）等也具有相似的特点。同时，他运用这种包赢的方法，与多位知名女星合作，在当时的电影市场上取得了成功。<|endoftext|>
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
[WARNING] [2024-04-20 23:38:55] logging.py : Detected kernel version 4.19.91, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
[INFO] [2024-04-20 23:39:01] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 0
[INFO] [2024-04-20 23:39:11] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:39:21] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:39:31] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:39:41] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:39:51] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:01] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:11] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:21] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:31] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:41] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:51] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:01] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:11] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
[INFO] [2024-04-20 23:41:21] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:31] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:41] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:43] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 5
[INFO] [2024-04-20 23:41:43] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 1
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 4
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 6
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 7
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 3
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 2
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 2: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 0: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 5: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 4: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 1: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 3: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 6: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 7: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Setting affinity for GPU 3 to 0fffff
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Setting affinity for GPU 1 to 0fffff
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Setting affinity for GPU 0 to 0fffff
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Setting affinity for GPU 2 to 0fffff
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 00/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 01/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 02/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 03/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 04/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 05/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 06/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 07/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 08/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 09/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 10/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 11/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 12/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 13/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 14/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 15/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 16/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 17/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 18/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 19/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 20/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 21/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 22/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 23/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 00 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 00 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 00 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 00 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 00 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 01 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 00 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 01 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 01 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 01 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 01 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 02 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 01 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 02 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 02 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 02 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 03 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 02 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 02 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 03 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 03 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 03 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 03 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 04 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 03 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 04 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 04 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 05 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 04 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 04 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 04 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 05 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 05 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 06 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 05 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 05 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 06 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 06 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 07 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 06 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 00 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 06 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 05 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 07 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 07 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 08 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 00 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 07 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 01 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 06 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 08 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 08 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 09 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 01 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 08 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 02 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 07 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 09 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 09 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 07 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 02 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 10 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 09 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 03 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 08 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 10 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 10 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 08 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 03 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 10 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 04 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 09 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 11 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 11 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 11 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 09 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 04 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 11 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 05 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 12 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 12 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 12 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 10 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 10 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 05 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 12 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 06 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 13 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 11 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 11 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 06 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 13 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 13 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 07 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 13 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 14 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 12 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 12 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 07 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 14 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 08 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 14 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 15 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 13 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 08 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 15 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 09 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 15 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 14 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 09 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 16 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 10 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 15 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 16 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 10 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 13 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 14 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 16 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 17 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 11 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 16 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 17 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 11 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 14 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 15 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 17 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 18 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 12 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 17 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 18 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 15 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 12 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 18 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 19 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 13 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 18 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 19 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 16 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 13 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 19 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 20 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 14 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 19 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 20 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 17 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 14 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 20 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 21 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 15 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 16 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 20 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 21 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 18 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 15 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 21 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 22 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 16 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 21 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 22 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 16 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 22 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 23 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 22 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 17 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 23 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 17 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 23 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 18 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 19 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 20 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 21 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 22 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 17 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 23 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 18 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 19 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 20 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 21 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 22 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 23 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 23 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 18 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 19 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 19 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 20 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 20 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 21 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 21 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 22 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 22 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 23 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 23 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 00 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 01 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 02 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 03 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 04 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 05 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 06 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 00 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 00 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 00 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 00 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 07 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 01 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 01 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 01 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 01 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 08 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 02 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 09 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 02 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 03 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 10 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 03 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 11 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 04 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 04 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 12 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 05 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 05 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 02 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 00 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 13 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 06 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 06 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 03 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 14 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 01 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 07 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 07 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 04 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 15 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 02 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 08 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 08 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 05 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 16 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 03 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 09 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 09 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 02 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 06 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 17 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 04 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 10 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 10 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 03 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 07 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 18 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 05 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 11 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 11 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 04 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 19 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 08 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 06 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 12 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 12 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 05 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 09 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 20 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 07 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 13 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 13 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 06 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 10 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 08 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 14 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 14 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 07 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 11 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 09 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 15 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 15 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 08 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 12 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 10 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 16 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 16 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 09 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 13 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 21 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 11 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 17 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 17 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 10 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 14 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 22 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 18 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 18 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 11 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 15 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 23 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 19 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 00 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 20 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 19 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 12 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 16 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 01 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 21 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 17 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 13 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 20 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 02 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 12 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 22 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 21 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 18 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 13 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 23 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 22 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 14 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 03 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 19 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 14 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 23 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 20 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 15 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 21 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 16 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 15 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 22 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 16 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 23 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 17 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 04 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 18 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 05 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 19 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 17 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 06 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 20 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 18 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 07 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 21 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 19 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 08 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 22 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 20 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 09 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 23 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 21 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 10 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 22 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 11 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 23 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 12 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 13 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 14 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 15 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 16 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 17 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 18 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 19 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 20 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 21 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 22 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 23 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO comm 0x7f51f8008fb0 rank 7 nranks 8 cudaDev 7 busId 80 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO comm 0x7f4a10008fb0 rank 4 nranks 8 cudaDev 4 busId 50 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO comm 0x7f5660008fb0 rank 2 nranks 8 cudaDev 2 busId 30 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO comm 0x7f33d4008fb0 rank 3 nranks 8 cudaDev 3 busId 40 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO comm 0x7f3a58008fb0 rank 1 nranks 8 cudaDev 1 busId 20 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO comm 0x7ff06c008fb0 rank 0 nranks 8 cudaDev 0 busId 10 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Launch mode Parallel
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO comm 0x7fab28008fb0 rank 6 nranks 8 cudaDev 6 busId 70 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO comm 0x7f42b0008fb0 rank 5 nranks 8 cudaDev 5 busId 60 - Init COMPLETE
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
0%| | 0/3120 [00:00<?, ?it/s]train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
{'loss': 1.3137, 'learning_rate': 2.564102564102564e-07, 'epoch': 0.0}
{'loss': 1.2524, 'learning_rate': 5.128205128205128e-07, 'epoch': 0.01}
{'loss': 1.2224, 'learning_rate': 7.692307692307694e-07, 'epoch': 0.01}
{'loss': 1.2209, 'learning_rate': 1.0256410256410257e-06, 'epoch': 0.01}
{'loss': 1.1776, 'learning_rate': 1.282051282051282e-06, 'epoch': 0.01}
{'loss': 1.124, 'learning_rate': 1.5384615384615387e-06, 'epoch': 0.02}
{'loss': 1.1545, 'learning_rate': 1.794871794871795e-06, 'epoch': 0.02}
{'loss': 1.1266, 'learning_rate': 2.0512820512820513e-06, 'epoch': 0.02}
{'loss': 1.1284, 'learning_rate': 2.307692307692308e-06, 'epoch': 0.02}
{'loss': 1.0904, 'learning_rate': 2.564102564102564e-06, 'epoch': 0.03}
{'loss': 1.0894, 'learning_rate': 2.8205128205128207e-06, 'epoch': 0.03}
{'loss': 1.0946, 'learning_rate': 3.0769230769230774e-06, 'epoch': 0.03}
{'loss': 1.1264, 'learning_rate': 3.3333333333333333e-06, 'epoch': 0.03}
{'loss': 1.0495, 'learning_rate': 3.58974358974359e-06, 'epoch': 0.04}
{'loss': 1.0859, 'learning_rate': 3.846153846153847e-06, 'epoch': 0.04}
{'loss': 1.0676, 'learning_rate': 4.102564102564103e-06, 'epoch': 0.04}
{'loss': 1.0675, 'learning_rate': 4.358974358974359e-06, 'epoch': 0.04}
{'loss': 1.0887, 'learning_rate': 4.615384615384616e-06, 'epoch': 0.05}
{'loss': 1.0623, 'learning_rate': 4.871794871794872e-06, 'epoch': 0.05}
{'loss': 1.0979, 'learning_rate': 5.128205128205128e-06, 'epoch': 0.05}
{'loss': 1.0907, 'learning_rate': 5.384615384615385e-06, 'epoch': 0.05}
{'loss': 1.1248, 'learning_rate': 5.641025641025641e-06, 'epoch': 0.06}
{'loss': 1.0717, 'learning_rate': 5.897435897435898e-06, 'epoch': 0.06}
{'loss': 1.0472, 'learning_rate': 6.153846153846155e-06, 'epoch': 0.06}
{'loss': 1.0468, 'learning_rate': 6.410256410256412e-06, 'epoch': 0.06}
{'loss': 1.1036, 'learning_rate': 6.666666666666667e-06, 'epoch': 0.07}
{'loss': 1.0443, 'learning_rate': 6.923076923076923e-06, 'epoch': 0.07}
{'loss': 1.1326, 'learning_rate': 7.17948717948718e-06, 'epoch': 0.07}
{'loss': 1.0333, 'learning_rate': 7.435897435897437e-06, 'epoch': 0.07}
{'loss': 1.0917, 'learning_rate': 7.692307692307694e-06, 'epoch': 0.08}
{'loss': 1.1045, 'learning_rate': 7.948717948717949e-06, 'epoch': 0.08}
{'loss': 1.0835, 'learning_rate': 8.205128205128205e-06, 'epoch': 0.08}
{'loss': 1.0796, 'learning_rate': 8.461538461538462e-06, 'epoch': 0.08}
{'loss': 1.1246, 'learning_rate': 8.717948717948719e-06, 'epoch': 0.09}
{'loss': 1.0481, 'learning_rate': 8.974358974358976e-06, 'epoch': 0.09}
{'loss': 1.0502, 'learning_rate': 9.230769230769232e-06, 'epoch': 0.09}
{'loss': 1.1048, 'learning_rate': 9.487179487179487e-06, 'epoch': 0.09}
{'loss': 1.1112, 'learning_rate': 9.743589743589744e-06, 'epoch': 0.1}
{'loss': 1.1319, 'learning_rate': 1e-05, 'epoch': 0.1}
{'loss': 1.095, 'learning_rate': 1.0256410256410256e-05, 'epoch': 0.1}
{'loss': 1.0906, 'learning_rate': 1.0512820512820514e-05, 'epoch': 0.11}
{'loss': 1.1147, 'learning_rate': 1.076923076923077e-05, 'epoch': 0.11}
{'loss': 1.0714, 'learning_rate': 1.1025641025641028e-05, 'epoch': 0.11}
{'loss': 1.1125, 'learning_rate': 1.1282051282051283e-05, 'epoch': 0.11}
{'loss': 1.0838, 'learning_rate': 1.1538461538461538e-05, 'epoch': 0.12}
{'loss': 1.1119, 'learning_rate': 1.1794871794871796e-05, 'epoch': 0.12}
{'loss': 1.1097, 'learning_rate': 1.2051282051282051e-05, 'epoch': 0.12}
{'loss': 1.1344, 'learning_rate': 1.230769230769231e-05, 'epoch': 0.12}
{'loss': 1.1108, 'learning_rate': 1.2564102564102565e-05, 'epoch': 0.13}
{'loss': 1.1308, 'learning_rate': 1.2820512820512823e-05, 'epoch': 0.13}
{'loss': 1.1198, 'learning_rate': 1.3076923076923078e-05, 'epoch': 0.13}
{'loss': 1.1589, 'learning_rate': 1.3333333333333333e-05, 'epoch': 0.13}
{'loss': 1.1473, 'learning_rate': 1.3589743589743592e-05, 'epoch': 0.14}
{'loss': 1.0719, 'learning_rate': 1.3846153846153847e-05, 'epoch': 0.14}
{'loss': 1.1418, 'learning_rate': 1.4102564102564105e-05, 'epoch': 0.14}
{'loss': 1.1056, 'learning_rate': 1.435897435897436e-05, 'epoch': 0.14}
{'loss': 1.1027, 'learning_rate': 1.4615384615384615e-05, 'epoch': 0.15}
{'loss': 1.1641, 'learning_rate': 1.4871794871794874e-05, 'epoch': 0.15}
{'loss': 1.094, 'learning_rate': 1.5128205128205129e-05, 'epoch': 0.15}
{'loss': 1.1528, 'learning_rate': 1.5384615384615387e-05, 'epoch': 0.15}
{'loss': 1.1395, 'learning_rate': 1.5641025641025644e-05, 'epoch': 0.16}
{'loss': 1.1329, 'learning_rate': 1.5897435897435897e-05, 'epoch': 0.16}
{'loss': 1.0875, 'learning_rate': 1.6153846153846154e-05, 'epoch': 0.16}
{'loss': 1.1137, 'learning_rate': 1.641025641025641e-05, 'epoch': 0.16}
{'loss': 1.1722, 'learning_rate': 1.6666666666666667e-05, 'epoch': 0.17}
{'loss': 1.0949, 'learning_rate': 1.6923076923076924e-05, 'epoch': 0.17}
{'loss': 1.1418, 'learning_rate': 1.717948717948718e-05, 'epoch': 0.17}
{'loss': 1.158, 'learning_rate': 1.7435897435897438e-05, 'epoch': 0.17}
{'loss': 1.1303, 'learning_rate': 1.7692307692307694e-05, 'epoch': 0.18}
{'loss': 1.189, 'learning_rate': 1.794871794871795e-05, 'epoch': 0.18}
{'loss': 1.1031, 'learning_rate': 1.8205128205128208e-05, 'epoch': 0.18}
{'loss': 1.1348, 'learning_rate': 1.8461538461538465e-05, 'epoch': 0.18}
{'loss': 1.1124, 'learning_rate': 1.8717948717948718e-05, 'epoch': 0.19}
{'loss': 1.1338, 'learning_rate': 1.8974358974358975e-05, 'epoch': 0.19}
{'loss': 1.1585, 'learning_rate': 1.923076923076923e-05, 'epoch': 0.19}
{'loss': 1.1598, 'learning_rate': 1.9487179487179488e-05, 'epoch': 0.19}
{'loss': 1.186, 'learning_rate': 1.9743589743589745e-05, 'epoch': 0.2}
{'loss': 1.1578, 'learning_rate': 2e-05, 'epoch': 0.2}
{'loss': 1.2017, 'learning_rate': 1.9999977531546567e-05, 'epoch': 0.2}
{'loss': 1.1756, 'learning_rate': 1.999991012628722e-05, 'epoch': 0.21}
{'loss': 1.1565, 'learning_rate': 1.9999797784524866e-05, 'epoch': 0.21}
{'loss': 1.1721, 'learning_rate': 1.999964050676434e-05, 'epoch': 0.21}
{'loss': 1.1388, 'learning_rate': 1.9999438293712382e-05, 'epoch': 0.21}
{'loss': 1.1761, 'learning_rate': 1.999919114627769e-05, 'epoch': 0.22}
{'loss': 1.1619, 'learning_rate': 1.999889906557086e-05, 'epoch': 0.22}
{'loss': 1.1452, 'learning_rate': 1.999856205290442e-05, 'epoch': 0.22}
{'loss': 1.1364, 'learning_rate': 1.9998180109792793e-05, 'epoch': 0.22}
{'loss': 1.1926, 'learning_rate': 1.999775323795232e-05, 'epoch': 0.23}
{'loss': 1.1646, 'learning_rate': 1.9997281439301218e-05, 'epoch': 0.23}
{'loss': 1.1484, 'learning_rate': 1.999676471595962e-05, 'epoch': 0.23}
{'loss': 1.1563, 'learning_rate': 1.9996203070249516e-05, 'epoch': 0.23}
{'loss': 1.1448, 'learning_rate': 1.9995596504694764e-05, 'epoch': 0.24}
{'loss': 1.1539, 'learning_rate': 1.9994945022021085e-05, 'epoch': 0.24}
{'loss': 1.1047, 'learning_rate': 1.999424862515604e-05, 'epoch': 0.24}
{'loss': 1.145, 'learning_rate': 1.999350731722902e-05, 'epoch': 0.24}
{'loss': 1.2109, 'learning_rate': 1.9992721101571238e-05, 'epoch': 0.25}
{'loss': 1.1766, 'learning_rate': 1.9991889981715696e-05, 'epoch': 0.25}
{'loss': 1.1013, 'learning_rate': 1.99910139613972e-05, 'epoch': 0.25}
{'loss': 1.1406, 'learning_rate': 1.9990093044552304e-05, 'epoch': 0.25}
{'loss': 1.1561, 'learning_rate': 1.998912723531933e-05, 'epoch': 0.26}
{'loss': 1.1512, 'learning_rate': 1.9988116538038326e-05, 'epoch': 0.26}
{'loss': 1.1941, 'learning_rate': 1.9987060957251047e-05, 'epoch': 0.26}
{'loss': 1.1429, 'learning_rate': 1.9985960497700953e-05, 'epoch': 0.26}
{'loss': 1.1689, 'learning_rate': 1.9984815164333163e-05, 'epoch': 0.27}
{'loss': 1.201, 'learning_rate': 1.998362496229446e-05, 'epoch': 0.27}
{'loss': 1.1526, 'learning_rate': 1.998238989693323e-05, 'epoch': 0.27}
{'loss': 1.1902, 'learning_rate': 1.998110997379949e-05, 'epoch': 0.27}
{'loss': 1.1751, 'learning_rate': 1.997978519864481e-05, 'epoch': 0.28}
{'loss': 1.1176, 'learning_rate': 1.997841557742232e-05, 'epoch': 0.28}
{'loss': 1.1814, 'learning_rate': 1.9977001116286675e-05, 'epoch': 0.28}
{'loss': 1.1603, 'learning_rate': 1.9975541821594028e-05, 'epoch': 0.28}
{'loss': 1.1679, 'learning_rate': 1.9974037699901993e-05, 'epoch': 0.29}
{'loss': 1.1865, 'learning_rate': 1.9972488757969635e-05, 'epoch': 0.29}
{'loss': 1.1097, 'learning_rate': 1.9970895002757413e-05, 'epoch': 0.29}
{'loss': 1.1651, 'learning_rate': 1.9969256441427173e-05, 'epoch': 0.29}
{'loss': 1.1874, 'learning_rate': 1.9967573081342103e-05, 'epoch': 0.3}
{'loss': 1.1893, 'learning_rate': 1.99658449300667e-05, 'epoch': 0.3}
{'loss': 1.1387, 'learning_rate': 1.9964071995366744e-05, 'epoch': 0.3}
{'loss': 1.155, 'learning_rate': 1.9962254285209256e-05, 'epoch': 0.31}
{'loss': 1.1563, 'learning_rate': 1.9960391807762462e-05, 'epoch': 0.31}
{'loss': 1.1502, 'learning_rate': 1.9958484571395756e-05, 'epoch': 0.31}
{'loss': 1.15, 'learning_rate': 1.9956532584679676e-05, 'epoch': 0.31}
{'loss': 1.2085, 'learning_rate': 1.995453585638584e-05, 'epoch': 0.32}
{'loss': 1.0918, 'learning_rate': 1.995249439548693e-05, 'epoch': 0.32}
{'loss': 1.1681, 'learning_rate': 1.9950408211156637e-05, 'epoch': 0.32}
{'loss': 1.1875, 'learning_rate': 1.994827731276963e-05, 'epoch': 0.32}
{'loss': 1.1399, 'learning_rate': 1.9946101709901513e-05, 'epoch': 0.33}
{'loss': 1.1883, 'learning_rate': 1.994388141232876e-05, 'epoch': 0.33}
{'loss': 1.1678, 'learning_rate': 1.9941616430028713e-05, 'epoch': 0.33}
{'loss': 1.2118, 'learning_rate': 1.9939306773179498e-05, 'epoch': 0.33}
{'loss': 1.1649, 'learning_rate': 1.9936952452159995e-05, 'epoch': 0.34}
{'loss': 1.1545, 'learning_rate': 1.9934553477549795e-05, 'epoch': 0.34}
{'loss': 1.1819, 'learning_rate': 1.9932109860129154e-05, 'epoch': 0.34}
{'loss': 1.1457, 'learning_rate': 1.992962161087893e-05, 'epoch': 0.34}
{'loss': 1.1614, 'learning_rate': 1.992708874098054e-05, 'epoch': 0.35}
{'loss': 1.1435, 'learning_rate': 1.9924511261815928e-05, 'epoch': 0.35}
{'loss': 1.1353, 'learning_rate': 1.992188918496748e-05, 'epoch': 0.35}
{'loss': 1.1642, 'learning_rate': 1.9919222522217998e-05, 'epoch': 0.35}
{'loss': 1.1312, 'learning_rate': 1.991651128555064e-05, 'epoch': 0.36}
{'loss': 1.1559, 'learning_rate': 1.9913755487148874e-05, 'epoch': 0.36}
{'loss': 1.1477, 'learning_rate': 1.9910955139396395e-05, 'epoch': 0.36}
{'loss': 1.1672, 'learning_rate': 1.9908110254877107e-05, 'epoch': 0.36}
{'loss': 1.1704, 'learning_rate': 1.990522084637503e-05, 'epoch': 0.37}
{'loss': 1.1242, 'learning_rate': 1.990228692687429e-05, 'epoch': 0.37}
{'loss': 1.164, 'learning_rate': 1.9899308509558998e-05, 'epoch': 0.37}
{'loss': 1.1559, 'learning_rate': 1.9896285607813245e-05, 'epoch': 0.37}
{'loss': 1.1351, 'learning_rate': 1.9893218235221016e-05, 'epoch': 0.38}
{'loss': 1.2004, 'learning_rate': 1.989010640556614e-05, 'epoch': 0.38}
{'loss': 1.2014, 'learning_rate': 1.988695013283221e-05, 'epoch': 0.38}
{'loss': 1.136, 'learning_rate': 1.988374943120254e-05, 'epoch': 0.38}
{'loss': 1.1745, 'learning_rate': 1.9880504315060096e-05, 'epoch': 0.39}
{'loss': 1.1619, 'learning_rate': 1.9877214798987428e-05, 'epoch': 0.39}
{'loss': 1.1502, 'learning_rate': 1.9873880897766597e-05, 'epoch': 0.39}
{'loss': 1.1542, 'learning_rate': 1.9870502626379127e-05, 'epoch': 0.39}
{'loss': 1.104, 'learning_rate': 1.9867080000005933e-05, 'epoch': 0.4}
{'loss': 1.1589, 'learning_rate': 1.9863613034027224e-05, 'epoch': 0.4}
{'loss': 1.1237, 'learning_rate': 1.9860101744022485e-05, 'epoch': 0.4}
{'loss': 1.1635, 'learning_rate': 1.985654614577036e-05, 'epoch': 0.41}
{'loss': 1.1501, 'learning_rate': 1.985294625524861e-05, 'epoch': 0.41}
{'loss': 1.1633, 'learning_rate': 1.9849302088634034e-05, 'epoch': 0.41}
{'loss': 1.1703, 'learning_rate': 1.9845613662302384e-05, 'epoch': 0.41}
{'loss': 1.1496, 'learning_rate': 1.9841880992828306e-05, 'epoch': 0.42}
{'loss': 1.22, 'learning_rate': 1.9838104096985266e-05, 'epoch': 0.42}
{'loss': 1.1372, 'learning_rate': 1.9834282991745465e-05, 'epoch': 0.42}
{'loss': 1.1988, 'learning_rate': 1.9830417694279766e-05, 'epoch': 0.42}
{'loss': 1.1268, 'learning_rate': 1.9826508221957624e-05, 'epoch': 0.43}
{'loss': 1.1662, 'learning_rate': 1.9822554592346996e-05, 'epoch': 0.43}
{'loss': 1.1371, 'learning_rate': 1.981855682321427e-05, 'epoch': 0.43}
{'loss': 1.1876, 'learning_rate': 1.981451493252418e-05, 'epoch': 0.43}
{'loss': 1.1379, 'learning_rate': 1.981042893843974e-05, 'epoch': 0.44}
{'loss': 1.1569, 'learning_rate': 1.9806298859322143e-05, 'epoch': 0.44}
{'loss': 1.0988, 'learning_rate': 1.980212471373068e-05, 'epoch': 0.44}
{'loss': 1.137, 'learning_rate': 1.979790652042268e-05, 'epoch': 0.44}
{'loss': 1.1034, 'learning_rate': 1.979364429835339e-05, 'epoch': 0.45}
{'loss': 1.1428, 'learning_rate': 1.9789338066675924e-05, 'epoch': 0.45}
{'loss': 1.1757, 'learning_rate': 1.978498784474115e-05, 'epoch': 0.45}
{'loss': 1.186, 'learning_rate': 1.978059365209762e-05, 'epoch': 0.45}
{'loss': 1.1064, 'learning_rate': 1.9776155508491482e-05, 'epoch': 0.46}
{'loss': 1.2007, 'learning_rate': 1.977167343386638e-05, 'epoch': 0.46}
{'loss': 1.2118, 'learning_rate': 1.9767147448363366e-05, 'epoch': 0.46}
{'loss': 1.134, 'learning_rate': 1.9762577572320826e-05, 'epoch': 0.46}
{'loss': 1.1829, 'learning_rate': 1.9757963826274357e-05, 'epoch': 0.47}
{'loss': 1.1299, 'learning_rate': 1.975330623095672e-05, 'epoch': 0.47}
{'loss': 1.198, 'learning_rate': 1.97486048072977e-05, 'epoch': 0.47}
{'loss': 1.1158, 'learning_rate': 1.9743859576424044e-05, 'epoch': 0.47}
{'loss': 1.167, 'learning_rate': 1.9739070559659347e-05, 'epoch': 0.48}
{'loss': 1.1606, 'learning_rate': 1.9734237778523977e-05, 'epoch': 0.48}
{'loss': 1.1466, 'learning_rate': 1.972936125473495e-05, 'epoch': 0.48}
{'loss': 1.1417, 'learning_rate': 1.9724441010205865e-05, 'epoch': 0.48}
{'loss': 1.192, 'learning_rate': 1.9719477067046768e-05, 'epoch': 0.49}
{'loss': 1.1691, 'learning_rate': 1.971446944756409e-05, 'epoch': 0.49}
{'loss': 1.1153, 'learning_rate': 1.9709418174260523e-05, 'epoch': 0.49}
{'loss': 1.1259, 'learning_rate': 1.9704323269834925e-05, 'epoch': 0.49}
{'loss': 1.1723, 'learning_rate': 1.9699184757182225e-05, 'epoch': 0.5}
{'loss': 1.1526, 'learning_rate': 1.9694002659393306e-05, 'epoch': 0.5}
{'loss': 1.1425, 'learning_rate': 1.9688776999754913e-05, 'epoch': 0.5}
{'loss': 1.1615, 'learning_rate': 1.9683507801749547e-05, 'epoch': 0.51}
{'loss': 1.144, 'learning_rate': 1.9678195089055347e-05, 'epoch': 0.51}
{'loss': 1.0944, 'learning_rate': 1.9672838885546007e-05, 'epoch': 0.51}
{'loss': 1.112, 'learning_rate': 1.966743921529065e-05, 'epoch': 0.51}
{'loss': 1.1535, 'learning_rate': 1.9661996102553716e-05, 'epoch': 0.52}
{'loss': 1.1526, 'learning_rate': 1.965650957179488e-05, 'epoch': 0.52}
{'loss': 1.157, 'learning_rate': 1.9650979647668908e-05, 'epoch': 0.52}
{'loss': 1.1447, 'learning_rate': 1.9645406355025565e-05, 'epoch': 0.52}
{'loss': 1.1192, 'learning_rate': 1.963978971890951e-05, 'epoch': 0.53}
{'loss': 1.19, 'learning_rate': 1.963412976456017e-05, 'epoch': 0.53}
{'loss': 1.1789, 'learning_rate': 1.9628426517411625e-05, 'epoch': 0.53}
{'loss': 1.204, 'learning_rate': 1.9622680003092503e-05, 'epoch': 0.53}
{'loss': 1.1377, 'learning_rate': 1.9616890247425867e-05, 'epoch': 0.54}
{'loss': 1.1645, 'learning_rate': 1.9611057276429085e-05, 'epoch': 0.54}
{'loss': 1.1234, 'learning_rate': 1.9605181116313725e-05, 'epoch': 0.54}
{'loss': 1.124, 'learning_rate': 1.9599261793485432e-05, 'epoch': 0.54}
{'loss': 1.1944, 'learning_rate': 1.959329933454381e-05, 'epoch': 0.55}
{'loss': 1.1968, 'learning_rate': 1.958729376628231e-05, 'epoch': 0.55}
{'loss': 1.1025, 'learning_rate': 1.9581245115688093e-05, 'epoch': 0.55}
{'loss': 1.1932, 'learning_rate': 1.957515340994193e-05, 'epoch': 0.55}
{'loss': 1.1497, 'learning_rate': 1.9569018676418053e-05, 'epoch': 0.56}
{'loss': 1.1787, 'learning_rate': 1.956284094268407e-05, 'epoch': 0.56}
{'loss': 1.1771, 'learning_rate': 1.9556620236500794e-05, 'epoch': 0.56}
{'loss': 1.1826, 'learning_rate': 1.955035658582216e-05, 'epoch': 0.56}
{'loss': 1.1726, 'learning_rate': 1.9544050018795076e-05, 'epoch': 0.57}
{'loss': 1.1397, 'learning_rate': 1.9537700563759303e-05, 'epoch': 0.57}
{'loss': 1.1658, 'learning_rate': 1.9531308249247328e-05, 'epoch': 0.57}
{'loss': 1.1805, 'learning_rate': 1.9524873103984234e-05, 'epoch': 0.57}
{'loss': 1.1383, 'learning_rate': 1.9518395156887574e-05, 'epoch': 0.58}
{'loss': 1.1558, 'learning_rate': 1.9511874437067243e-05, 'epoch': 0.58}
{'loss': 1.1584, 'learning_rate': 1.950531097382533e-05, 'epoch': 0.58}
{'loss': 1.2029, 'learning_rate': 1.949870479665602e-05, 'epoch': 0.58}
{'loss': 1.1418, 'learning_rate': 1.9492055935245418e-05, 'epoch': 0.59}
{'loss': 1.176, 'learning_rate': 1.9485364419471454e-05, 'epoch': 0.59}
{'loss': 1.1742, 'learning_rate': 1.9478630279403737e-05, 'epoch': 0.59}
{'loss': 1.0948, 'learning_rate': 1.9471853545303407e-05, 'epoch': 0.59}
{'loss': 1.1567, 'learning_rate': 1.9465034247623005e-05, 'epoch': 0.6}
{'loss': 1.1644, 'learning_rate': 1.9458172417006347e-05, 'epoch': 0.6}
{'loss': 1.1226, 'learning_rate': 1.9451268084288385e-05, 'epoch': 0.6}
{'loss': 1.0994, 'learning_rate': 1.9444321280495045e-05, 'epoch': 0.61}
{'loss': 1.1429, 'learning_rate': 1.943733203684312e-05, 'epoch': 0.61}
{'loss': 1.1411, 'learning_rate': 1.9430300384740108e-05, 'epoch': 0.61}
{'loss': 1.1425, 'learning_rate': 1.942322635578408e-05, 'epoch': 0.61}
{'loss': 1.1447, 'learning_rate': 1.9416109981763526e-05, 'epoch': 0.62}
{'loss': 1.1677, 'learning_rate': 1.940895129465724e-05, 'epoch': 0.62}
{'loss': 1.1862, 'learning_rate': 1.9401750326634144e-05, 'epoch': 0.62}
{'loss': 1.1923, 'learning_rate': 1.939450711005316e-05, 'epoch': 0.62}
{'loss': 1.202, 'learning_rate': 1.9387221677463064e-05, 'epoch': 0.63}
{'loss': 1.1419, 'learning_rate': 1.9379894061602335e-05, 'epoch': 0.63}
{'loss': 1.111, 'learning_rate': 1.9372524295399014e-05, 'epoch': 0.63}
{'loss': 1.1734, 'learning_rate': 1.936511241197055e-05, 'epoch': 0.63}
{'loss': 1.1437, 'learning_rate': 1.9357658444623655e-05, 'epoch': 0.64}
{'loss': 1.1303, 'learning_rate': 1.9350162426854152e-05, 'epoch': 0.64}
{'loss': 1.1779, 'learning_rate': 1.9342624392346826e-05, 'epoch': 0.64}
{'loss': 1.1967, 'learning_rate': 1.933504437497527e-05, 'epoch': 0.64}
{'loss': 1.1155, 'learning_rate': 1.9327422408801744e-05, 'epoch': 0.65}
{'loss': 1.1579, 'learning_rate': 1.9319758528076998e-05, 'epoch': 0.65}
{'loss': 1.144, 'learning_rate': 1.9312052767240153e-05, 'epoch': 0.65}
{'loss': 1.1733, 'learning_rate': 1.93043051609185e-05, 'epoch': 0.65}
{'loss': 1.1563, 'learning_rate': 1.92965157439274e-05, 'epoch': 0.66}
{'loss': 1.1467, 'learning_rate': 1.9288684551270072e-05, 'epoch': 0.66}
{'loss': 1.212, 'learning_rate': 1.9280811618137486e-05, 'epoch': 0.66}
{'loss': 1.1307, 'learning_rate': 1.9272896979908153e-05, 'epoch': 0.66}
{'loss': 1.1368, 'learning_rate': 1.9264940672148018e-05, 'epoch': 0.67}
{'loss': 1.1799, 'learning_rate': 1.9256942730610268e-05, 'epoch': 0.67}
{'loss': 1.1466, 'learning_rate': 1.9248903191235177e-05, 'epoch': 0.67}
{'loss': 1.2009, 'learning_rate': 1.9240822090149944e-05, 'epoch': 0.67}
{'loss': 1.1678, 'learning_rate': 1.9232699463668543e-05, 'epoch': 0.68}
{'loss': 1.1532, 'learning_rate': 1.9224535348291543e-05, 'epoch': 0.68}
{'loss': 1.1784, 'learning_rate': 1.9216329780705955e-05, 'epoch': 0.68}
{'loss': 1.1642, 'learning_rate': 1.9208082797785057e-05, 'epoch': 0.68}
{'loss': 1.1825, 'learning_rate': 1.9199794436588244e-05, 'epoch': 0.69}
{'loss': 1.1526, 'learning_rate': 1.9191464734360844e-05, 'epoch': 0.69}
{'loss': 1.1468, 'learning_rate': 1.9183093728533966e-05, 'epoch': 0.69}
{'loss': 1.1444, 'learning_rate': 1.917468145672432e-05, 'epoch': 0.69}
{'loss': 1.1357, 'learning_rate': 1.916622795673405e-05, 'epoch': 0.7}
{'loss': 1.1569, 'learning_rate': 1.9157733266550577e-05, 'epoch': 0.7}
{'loss': 1.1081, 'learning_rate': 1.9149197424346405e-05, 'epoch': 0.7}
{'loss': 1.1765, 'learning_rate': 1.9140620468478967e-05, 'epoch': 0.71}
{'loss': 1.2001, 'learning_rate': 1.913200243749046e-05, 'epoch': 0.71}
{'loss': 1.1429, 'learning_rate': 1.9123343370107636e-05, 'epoch': 0.71}
{'loss': 1.1898, 'learning_rate': 1.9114643305241678e-05, 'epoch': 0.71}
{'loss': 1.1473, 'learning_rate': 1.910590228198798e-05, 'epoch': 0.72}
{'loss': 1.1505, 'learning_rate': 1.9097120339625994e-05, 'epoch': 0.72}
{'loss': 1.1208, 'learning_rate': 1.9088297517619056e-05, 'epoch': 0.72}
{'loss': 1.1064, 'learning_rate': 1.9079433855614203e-05, 'epoch': 0.72}
{'loss': 1.1549, 'learning_rate': 1.9070529393441985e-05, 'epoch': 0.73}
{'loss': 1.1308, 'learning_rate': 1.9061584171116302e-05, 'epoch': 0.73}
{'loss': 1.1827, 'learning_rate': 1.9052598228834217e-05, 'epoch': 0.73}
{'loss': 1.1394, 'learning_rate': 1.9043571606975776e-05, 'epoch': 0.73}
{'loss': 1.1002, 'learning_rate': 1.9034504346103825e-05, 'epoch': 0.74}
{'loss': 1.1308, 'learning_rate': 1.9025396486963827e-05, 'epoch': 0.74}
{'loss': 1.1007, 'learning_rate': 1.9016248070483686e-05, 'epoch': 0.74}
{'loss': 1.1288, 'learning_rate': 1.900705913777356e-05, 'epoch': 0.74}
{'loss': 1.1672, 'learning_rate': 1.8997829730125662e-05, 'epoch': 0.75}
{'loss': 1.099, 'learning_rate': 1.89885598890141e-05, 'epoch': 0.75}
{'loss': 1.1658, 'learning_rate': 1.8979249656094672e-05, 'epoch': 0.75}
{'loss': 1.1251, 'learning_rate': 1.8969899073204687e-05, 'epoch': 0.75}
{'loss': 1.1284, 'learning_rate': 1.896050818236277e-05, 'epoch': 0.76}
{'loss': 1.1105, 'learning_rate': 1.895107702576868e-05, 'epoch': 0.76}
{'loss': 1.1285, 'learning_rate': 1.8941605645803115e-05, 'epoch': 0.76}
{'loss': 1.1648, 'learning_rate': 1.8932094085027534e-05, 'epoch': 0.76}
{'loss': 1.1517, 'learning_rate': 1.8922542386183942e-05, 'epoch': 0.77}
{'loss': 1.1652, 'learning_rate': 1.891295059219472e-05, 'epoch': 0.77}
{'loss': 1.1424, 'learning_rate': 1.8903318746162428e-05, 'epoch': 0.77}
{'loss': 1.1157, 'learning_rate': 1.88936468913696e-05, 'epoch': 0.77}
{'loss': 1.1597, 'learning_rate': 1.888393507127856e-05, 'epoch': 0.78}
{'loss': 1.1169, 'learning_rate': 1.8874183329531222e-05, 'epoch': 0.78}
{'loss': 1.1528, 'learning_rate': 1.88643917099489e-05, 'epoch': 0.78}
{'loss': 1.1508, 'learning_rate': 1.8854560256532098e-05, 'epoch': 0.78}
{'loss': 1.138, 'learning_rate': 1.8844689013460335e-05, 'epoch': 0.79}
{'loss': 1.1486, 'learning_rate': 1.883477802509192e-05, 'epoch': 0.79}
{'loss': 1.1427, 'learning_rate': 1.8824827335963767e-05, 'epoch': 0.79}
{'loss': 1.1143, 'learning_rate': 1.88148369907912e-05, 'epoch': 0.79}
{'loss': 1.1366, 'learning_rate': 1.8804807034467735e-05, 'epoch': 0.8}
{'loss': 1.1975, 'learning_rate': 1.879473751206489e-05, 'epoch': 0.8}
{'loss': 1.1549, 'learning_rate': 1.8784628468831997e-05, 'epoch': 0.8}
{'loss': 1.1532, 'learning_rate': 1.877447995019596e-05, 'epoch': 0.81}
{'loss': 1.1389, 'learning_rate': 1.876429200176108e-05, 'epoch': 0.81}
{'loss': 1.1319, 'learning_rate': 1.875406466930886e-05, 'epoch': 0.81}
{'loss': 1.1022, 'learning_rate': 1.874379799879775e-05, 'epoch': 0.81}
{'loss': 1.1313, 'learning_rate': 1.8733492036363007e-05, 'epoch': 0.82}
{'loss': 1.1646, 'learning_rate': 1.872314682831643e-05, 'epoch': 0.82}
{'loss': 1.208, 'learning_rate': 1.8712762421146185e-05, 'epoch': 0.82}
{'loss': 1.1441, 'learning_rate': 1.8702338861516587e-05, 'epoch': 0.82}
{'loss': 1.1564, 'learning_rate': 1.8691876196267892e-05, 'epoch': 0.83}
{'loss': 1.1334, 'learning_rate': 1.8681374472416075e-05, 'epoch': 0.83}
{'loss': 1.0704, 'learning_rate': 1.867083373715264e-05, 'epoch': 0.83}
{'loss': 1.1261, 'learning_rate': 1.866025403784439e-05, 'epoch': 0.83}
{'loss': 1.1484, 'learning_rate': 1.8649635422033218e-05, 'epoch': 0.84}
{'loss': 1.1134, 'learning_rate': 1.86389779374359e-05, 'epoch': 0.84}
{'loss': 1.1219, 'learning_rate': 1.862828163194388e-05, 'epoch': 0.84}
{'loss': 1.1314, 'learning_rate': 1.861754655362304e-05, 'epoch': 0.84}
{'loss': 1.1601, 'learning_rate': 1.8606772750713503e-05, 'epoch': 0.85}
{'loss': 1.168, 'learning_rate': 1.8595960271629412e-05, 'epoch': 0.85}
{'loss': 1.1485, 'learning_rate': 1.8585109164958698e-05, 'epoch': 0.85}
{'loss': 1.1637, 'learning_rate': 1.857421947946288e-05, 'epoch': 0.85}
{'loss': 1.1241, 'learning_rate': 1.8563291264076834e-05, 'epoch': 0.86}
{'loss': 1.1053, 'learning_rate': 1.8552324567908585e-05, 'epoch': 0.86}
{'loss': 1.1382, 'learning_rate': 1.8541319440239066e-05, 'epoch': 0.86}
{'loss': 1.1319, 'learning_rate': 1.8530275930521925e-05, 'epoch': 0.86}
{'loss': 1.0963, 'learning_rate': 1.851919408838327e-05, 'epoch': 0.87}
{'loss': 1.127, 'learning_rate': 1.8508073963621482e-05, 'epoch': 0.87}
{'loss': 1.1608, 'learning_rate': 1.8496915606206952e-05, 'epoch': 0.87}
{'loss': 1.158, 'learning_rate': 1.8485719066281894e-05, 'epoch': 0.87}
{'loss': 1.1696, 'learning_rate': 1.847448439416009e-05, 'epoch': 0.88}
{'loss': 1.1744, 'learning_rate': 1.8463211640326686e-05, 'epoch': 0.88}
{'loss': 1.1363, 'learning_rate': 1.845190085543795e-05, 'epoch': 0.88}
{'loss': 1.1001, 'learning_rate': 1.8440552090321047e-05, 'epoch': 0.88}
{'loss': 1.1647, 'learning_rate': 1.842916539597382e-05, 'epoch': 0.89}
{'loss': 1.119, 'learning_rate': 1.8417740823564552e-05, 'epoch': 0.89}
{'loss': 1.1718, 'learning_rate': 1.8406278424431737e-05, 'epoch': 0.89}
{'loss': 1.1101, 'learning_rate': 1.839477825008385e-05, 'epoch': 0.89}
{'loss': 1.1245, 'learning_rate': 1.8383240352199118e-05, 'epoch': 0.9}
{'loss': 1.1056, 'learning_rate': 1.8371664782625287e-05, 'epoch': 0.9}
{'loss': 1.1578, 'learning_rate': 1.8360051593379383e-05, 'epoch': 0.9}
{'loss': 1.1112, 'learning_rate': 1.8348400836647492e-05, 'epoch': 0.91}
{'loss': 1.1375, 'learning_rate': 1.8336712564784506e-05, 'epoch': 0.91}
{'loss': 1.1008, 'learning_rate': 1.83249868303139e-05, 'epoch': 0.91}
{'loss': 1.1385, 'learning_rate': 1.8313223685927507e-05, 'epoch': 0.91}
{'loss': 1.0728, 'learning_rate': 1.8301423184485253e-05, 'epoch': 0.92}
{'loss': 1.1382, 'learning_rate': 1.8289585379014942e-05, 'epoch': 0.92}
{'loss': 1.1376, 'learning_rate': 1.827771032271201e-05, 'epoch': 0.92}
{'loss': 1.2051, 'learning_rate': 1.8265798068939295e-05, 'epoch': 0.92}
{'loss': 1.1694, 'learning_rate': 1.825384867122677e-05, 'epoch': 0.93}
{'loss': 1.1092, 'learning_rate': 1.8241862183271338e-05, 'epoch': 0.93}
{'loss': 1.1343, 'learning_rate': 1.8229838658936566e-05, 'epoch': 0.93}
{'loss': 1.1233, 'learning_rate': 1.821777815225245e-05, 'epoch': 0.93}
{'loss': 1.0899, 'learning_rate': 1.8205680717415188e-05, 'epoch': 0.94}
{'loss': 1.1064, 'learning_rate': 1.81935464087869e-05, 'epoch': 0.94}
{'loss': 1.1538, 'learning_rate': 1.8181375280895417e-05, 'epoch': 0.94}
{'loss': 1.1558, 'learning_rate': 1.8169167388434024e-05, 'epoch': 0.94}
{'loss': 1.1142, 'learning_rate': 1.815692278626122e-05, 'epoch': 0.95}
{'loss': 1.1397, 'learning_rate': 1.8144641529400445e-05, 'epoch': 0.95}
{'loss': 1.1285, 'learning_rate': 1.8132323673039883e-05, 'epoch': 0.95}
{'loss': 1.1637, 'learning_rate': 1.8119969272532164e-05, 'epoch': 0.95}
{'loss': 1.1269, 'learning_rate': 1.8107578383394145e-05, 'epoch': 0.96}
{'loss': 1.1177, 'learning_rate': 1.8095151061306647e-05, 'epoch': 0.96}
{'loss': 1.1103, 'learning_rate': 1.808268736211421e-05, 'epoch': 0.96}
{'loss': 1.1156, 'learning_rate': 1.8070187341824848e-05, 'epoch': 0.96}
{'loss': 1.0974, 'learning_rate': 1.8057651056609784e-05, 'epoch': 0.97}
{'loss': 1.1858, 'learning_rate': 1.8045078562803203e-05, 'epoch': 0.97}
{'loss': 1.1328, 'learning_rate': 1.8032469916902004e-05, 'epoch': 0.97}
{'loss': 1.1207, 'learning_rate': 1.8019825175565544e-05, 'epoch': 0.97}
{'loss': 1.1782, 'learning_rate': 1.800714439561538e-05, 'epoch': 0.98}
{'loss': 1.1429, 'learning_rate': 1.7994427634035016e-05, 'epoch': 0.98}
{'loss': 1.1238, 'learning_rate': 1.7981674947969635e-05, 'epoch': 0.98}
{'loss': 1.1004, 'learning_rate': 1.7968886394725876e-05, 'epoch': 0.98}
{'loss': 1.096, 'learning_rate': 1.7956062031771533e-05, 'epoch': 0.99}
{'loss': 1.085, 'learning_rate': 1.7943201916735337e-05, 'epoch': 0.99}
{'loss': 1.1077, 'learning_rate': 1.793030610740665e-05, 'epoch': 0.99}
{'loss': 1.104, 'learning_rate': 1.791737466173527e-05, 'epoch': 0.99}
{'loss': 1.1207, 'learning_rate': 1.7904407637831097e-05, 'epoch': 1.0}
{'loss': 1.1076, 'learning_rate': 1.789140509396394e-05, 'epoch': 1.0}
25%|██▌ | 780/3120 [1:04:30<3:18:41, 5.09s/it]/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
{'loss': 0.8392, 'learning_rate': 1.7878367088563195e-05, 'epoch': 1.0}
{'loss': 0.7721, 'learning_rate': 1.7865293680217636e-05, 'epoch': 1.01}
{'loss': 0.7814, 'learning_rate': 1.7852184927675113e-05, 'epoch': 1.01}
{'loss': 0.7843, 'learning_rate': 1.7839040889842307e-05, 'epoch': 1.01}
{'loss': 0.7827, 'learning_rate': 1.7825861625784457e-05, 'epoch': 1.01}
{'loss': 0.8196, 'learning_rate': 1.7812647194725093e-05, 'epoch': 1.02}
{'loss': 0.8131, 'learning_rate': 1.779939765604579e-05, 'epoch': 1.02}
{'loss': 0.7676, 'learning_rate': 1.7786113069285877e-05, 'epoch': 1.02}
{'loss': 0.8016, 'learning_rate': 1.777279349414217e-05, 'epoch': 1.02}
{'loss': 0.7868, 'learning_rate': 1.7759438990468726e-05, 'epoch': 1.03}
{'loss': 0.8294, 'learning_rate': 1.7746049618276545e-05, 'epoch': 1.03}
{'loss': 0.7904, 'learning_rate': 1.7732625437733338e-05, 'epoch': 1.03}
{'loss': 0.7852, 'learning_rate': 1.771916650916321e-05, 'epoch': 1.03}
{'loss': 0.772, 'learning_rate': 1.7705672893046425e-05, 'epoch': 1.04}
{'loss': 0.772, 'learning_rate': 1.7692144650019125e-05, 'epoch': 1.04}
{'loss': 0.7656, 'learning_rate': 1.767858184087304e-05, 'epoch': 1.04}
{'loss': 0.7976, 'learning_rate': 1.7664984526555247e-05, 'epoch': 1.04}
{'loss': 0.8023, 'learning_rate': 1.765135276816787e-05, 'epoch': 1.05}
{'loss': 0.7723, 'learning_rate': 1.7637686626967815e-05, 'epoch': 1.05}
{'loss': 0.8008, 'learning_rate': 1.7623986164366487e-05, 'epoch': 1.05}
{'loss': 0.7881, 'learning_rate': 1.7610251441929532e-05, 'epoch': 1.05}
{'loss': 0.7723, 'learning_rate': 1.7596482521376546e-05, 'epoch': 1.06}
{'loss': 0.7637, 'learning_rate': 1.7582679464580796e-05, 'epoch': 1.06}
{'loss': 0.8181, 'learning_rate': 1.7568842333568952e-05, 'epoch': 1.06}
{'loss': 0.8291, 'learning_rate': 1.75549711905208e-05, 'epoch': 1.06}
{'loss': 0.7597, 'learning_rate': 1.7541066097768965e-05, 'epoch': 1.07}
{'loss': 0.8401, 'learning_rate': 1.7527127117798635e-05, 'epoch': 1.07}
{'loss': 0.8012, 'learning_rate': 1.7513154313247273e-05, 'epoch': 1.07}
{'loss': 0.8042, 'learning_rate': 1.7499147746904337e-05, 'epoch': 1.07}
{'loss': 0.7931, 'learning_rate': 1.7485107481711014e-05, 'epoch': 1.08}
{'loss': 0.815, 'learning_rate': 1.7471033580759903e-05, 'epoch': 1.08}
{'loss': 0.8042, 'learning_rate': 1.7456926107294765e-05, 'epoch': 1.08}
{'loss': 0.7814, 'learning_rate': 1.7442785124710227e-05, 'epoch': 1.08}
{'loss': 0.7983, 'learning_rate': 1.742861069655148e-05, 'epoch': 1.09}
{'loss': 0.8045, 'learning_rate': 1.741440288651403e-05, 'epoch': 1.09}
{'loss': 0.7718, 'learning_rate': 1.7400161758443377e-05, 'epoch': 1.09}
{'loss': 0.8006, 'learning_rate': 1.7385887376334743e-05, 'epoch': 1.09}
{'loss': 0.7937, 'learning_rate': 1.737157980433279e-05, 'epoch': 1.1}
{'loss': 0.7888, 'learning_rate': 1.735723910673132e-05, 'epoch': 1.1}
{'loss': 0.8135, 'learning_rate': 1.7342865347972987e-05, 'epoch': 1.1}
{'loss': 0.805, 'learning_rate': 1.7328458592649028e-05, 'epoch': 1.11}
{'loss': 0.7874, 'learning_rate': 1.7314018905498932e-05, 'epoch': 1.11}
{'loss': 0.773, 'learning_rate': 1.7299546351410197e-05, 'epoch': 1.11}
{'loss': 0.8026, 'learning_rate': 1.7285040995418003e-05, 'epoch': 1.11}
{'loss': 0.8232, 'learning_rate': 1.7270502902704925e-05, 'epoch': 1.12}
{'loss': 0.802, 'learning_rate': 1.7255932138600665e-05, 'epoch': 1.12}
{'loss': 0.797, 'learning_rate': 1.7241328768581725e-05, 'epoch': 1.12}
{'loss': 0.7957, 'learning_rate': 1.7226692858271133e-05, 'epoch': 1.12}
{'loss': 0.7776, 'learning_rate': 1.7212024473438145e-05, 'epoch': 1.13}
{'loss': 0.7955, 'learning_rate': 1.7197323679997943e-05, 'epoch': 1.13}
{'loss': 0.781, 'learning_rate': 1.718259054401135e-05, 'epoch': 1.13}
{'loss': 0.79, 'learning_rate': 1.7167825131684516e-05, 'epoch': 1.13}
{'loss': 0.8113, 'learning_rate': 1.7153027509368637e-05, 'epoch': 1.14}
{'loss': 0.8052, 'learning_rate': 1.7138197743559656e-05, 'epoch': 1.14}
{'loss': 0.7755, 'learning_rate': 1.7123335900897946e-05, 'epoch': 1.14}
{'loss': 0.818, 'learning_rate': 1.7108442048168038e-05, 'epoch': 1.14}
{'loss': 0.8166, 'learning_rate': 1.7093516252298296e-05, 'epoch': 1.15}
{'loss': 0.8, 'learning_rate': 1.707855858036063e-05, 'epoch': 1.15}
{'loss': 0.804, 'learning_rate': 1.7063569099570196e-05, 'epoch': 1.15}
{'loss': 0.8087, 'learning_rate': 1.7048547877285078e-05, 'epoch': 1.15}
{'loss': 0.7976, 'learning_rate': 1.7033494981006003e-05, 'epoch': 1.16}
{'loss': 0.7594, 'learning_rate': 1.7018410478376033e-05, 'epoch': 1.16}
{'loss': 0.7686, 'learning_rate': 1.7003294437180254e-05, 'epoch': 1.16}
{'loss': 0.7879, 'learning_rate': 1.6988146925345487e-05, 'epoch': 1.16}
{'loss': 0.8057, 'learning_rate': 1.6972968010939953e-05, 'epoch': 1.17}
{'loss': 0.8118, 'learning_rate': 1.695775776217301e-05, 'epoch': 1.17}
{'loss': 0.7613, 'learning_rate': 1.6942516247394806e-05, 'epoch': 1.17}
{'loss': 0.8186, 'learning_rate': 1.6927243535095995e-05, 'epoch': 1.17}
{'loss': 0.8214, 'learning_rate': 1.6911939693907422e-05, 'epoch': 1.18}
{'loss': 0.7938, 'learning_rate': 1.6896604792599813e-05, 'epoch': 1.18}
{'loss': 0.7804, 'learning_rate': 1.6881238900083472e-05, 'epoch': 1.18}
{'loss': 0.8361, 'learning_rate': 1.686584208540797e-05, 'epoch': 1.18}
{'loss': 0.7873, 'learning_rate': 1.685041441776183e-05, 'epoch': 1.19}
{'loss': 0.8094, 'learning_rate': 1.6834955966472214e-05, 'epoch': 1.19}
{'loss': 0.8054, 'learning_rate': 1.6819466801004622e-05, 'epoch': 1.19}
{'loss': 0.7698, 'learning_rate': 1.6803946990962577e-05, 'epoch': 1.19}
{'loss': 0.7648, 'learning_rate': 1.67883966060873e-05, 'epoch': 1.2}
{'loss': 0.7801, 'learning_rate': 1.6772815716257414e-05, 'epoch': 1.2}
{'loss': 0.8163, 'learning_rate': 1.6757204391488617e-05, 'epoch': 1.2}
{'loss': 0.8108, 'learning_rate': 1.6741562701933366e-05, 'epoch': 1.21}
{'loss': 0.8093, 'learning_rate': 1.672589071788059e-05, 'epoch': 1.21}
{'loss': 0.8412, 'learning_rate': 1.671018850975533e-05, 'epoch': 1.21}
{'loss': 0.7669, 'learning_rate': 1.6694456148118453e-05, 'epoch': 1.21}
{'loss': 0.7766, 'learning_rate': 1.6678693703666327e-05, 'epoch': 1.22}
{'loss': 0.8172, 'learning_rate': 1.66629012472305e-05, 'epoch': 1.22}
{'loss': 0.7943, 'learning_rate': 1.664707884977739e-05, 'epoch': 1.22}
{'loss': 0.7944, 'learning_rate': 1.6631226582407954e-05, 'epoch': 1.22}
{'loss': 0.7759, 'learning_rate': 1.661534451635738e-05, 'epoch': 1.23}
{'loss': 0.7853, 'learning_rate': 1.6599432722994754e-05, 'epoch': 1.23}
{'loss': 0.7879, 'learning_rate': 1.6583491273822763e-05, 'epoch': 1.23}
{'loss': 0.7834, 'learning_rate': 1.6567520240477344e-05, 'epoch': 1.23}
{'loss': 0.7992, 'learning_rate': 1.655151969472738e-05, 'epoch': 1.24}
{'loss': 0.801, 'learning_rate': 1.653548970847438e-05, 'epoch': 1.24}
{'loss': 0.7843, 'learning_rate': 1.6519430353752138e-05, 'epoch': 1.24}
{'loss': 0.7752, 'learning_rate': 1.6503341702726428e-05, 'epoch': 1.24}
{'loss': 0.8022, 'learning_rate': 1.6487223827694673e-05, 'epoch': 1.25}
{'loss': 0.8492, 'learning_rate': 1.6471076801085618e-05, 'epoch': 1.25}
{'loss': 0.7996, 'learning_rate': 1.6454900695459e-05, 'epoch': 1.25}
{'loss': 0.7611, 'learning_rate': 1.643869558350524e-05, 'epoch': 1.25}
{'loss': 0.7562, 'learning_rate': 1.6422461538045104e-05, 'epoch': 1.26}
{'loss': 0.7883, 'learning_rate': 1.6406198632029358e-05, 'epoch': 1.26}
{'loss': 0.7932, 'learning_rate': 1.638990693853848e-05, 'epoch': 1.26}
{'loss': 0.811, 'learning_rate': 1.63735865307823e-05, 'epoch': 1.26}
{'loss': 0.7592, 'learning_rate': 1.6357237482099682e-05, 'epoch': 1.27}
{'loss': 0.7816, 'learning_rate': 1.6340859865958193e-05, 'epoch': 1.27}
{'loss': 0.788, 'learning_rate': 1.6324453755953772e-05, 'epoch': 1.27}
{'loss': 0.8024, 'learning_rate': 1.630801922581041e-05, 'epoch': 1.27}
{'loss': 0.7902, 'learning_rate': 1.6291556349379794e-05, 'epoch': 1.28}
{'loss': 0.791, 'learning_rate': 1.6275065200641004e-05, 'epoch': 1.28}
{'loss': 0.7769, 'learning_rate': 1.6258545853700157e-05, 'epoch': 1.28}
{'loss': 0.8325, 'learning_rate': 1.6241998382790095e-05, 'epoch': 1.28}
{'loss': 0.7926, 'learning_rate': 1.622542286227003e-05, 'epoch': 1.29}
{'loss': 0.8153, 'learning_rate': 1.620881936662522e-05, 'epoch': 1.29}
{'loss': 0.7851, 'learning_rate': 1.6192187970466646e-05, 'epoch': 1.29}
{'loss': 0.7744, 'learning_rate': 1.617552874853065e-05, 'epoch': 1.29}
{'loss': 0.7978, 'learning_rate': 1.615884177567863e-05, 'epoch': 1.3}
{'loss': 0.7764, 'learning_rate': 1.6142127126896682e-05, 'epoch': 1.3}
{'loss': 0.7693, 'learning_rate': 1.6125384877295255e-05, 'epoch': 1.3}
{'loss': 0.8497, 'learning_rate': 1.6108615102108854e-05, 'epoch': 1.31}
{'loss': 0.7874, 'learning_rate': 1.6091817876695655e-05, 'epoch': 1.31}
{'loss': 0.7844, 'learning_rate': 1.6074993276537198e-05, 'epoch': 1.31}
{'loss': 0.8288, 'learning_rate': 1.6058141377238026e-05, 'epoch': 1.31}
{'loss': 0.8386, 'learning_rate': 1.6041262254525362e-05, 'epoch': 1.32}
{'loss': 0.7879, 'learning_rate': 1.602435598424877e-05, 'epoch': 1.32}
{'loss': 0.8181, 'learning_rate': 1.600742264237979e-05, 'epoch': 1.32}
{'loss': 0.7813, 'learning_rate': 1.599046230501163e-05, 'epoch': 1.32}
{'loss': 0.8164, 'learning_rate': 1.5973475048358795e-05, 'epoch': 1.33}
{'loss': 0.8212, 'learning_rate': 1.5956460948756765e-05, 'epoch': 1.33}
{'loss': 0.7928, 'learning_rate': 1.593942008266164e-05, 'epoch': 1.33}
{'loss': 0.7925, 'learning_rate': 1.5922352526649803e-05, 'epoch': 1.33}
{'loss': 0.7599, 'learning_rate': 1.5905258357417567e-05, 'epoch': 1.34}
{'loss': 0.7971, 'learning_rate': 1.5888137651780847e-05, 'epoch': 1.34}
{'loss': 0.8046, 'learning_rate': 1.5870990486674794e-05, 'epoch': 1.34}
{'loss': 0.7811, 'learning_rate': 1.585381693915346e-05, 'epoch': 1.34}
{'loss': 0.7737, 'learning_rate': 1.583661708638947e-05, 'epoch': 1.35}
{'loss': 0.7909, 'learning_rate': 1.581939100567363e-05, 'epoch': 1.35}
{'loss': 0.8154, 'learning_rate': 1.580213877441462e-05, 'epoch': 1.35}
{'loss': 0.831, 'learning_rate': 1.5784860470138633e-05, 'epoch': 1.35}
{'loss': 0.7842, 'learning_rate': 1.5767556170489027e-05, 'epoch': 1.36}
{'loss': 0.7851, 'learning_rate': 1.5750225953225968e-05, 'epoch': 1.36}
{'loss': 0.8102, 'learning_rate': 1.573286989622609e-05, 'epoch': 1.36}
{'loss': 0.842, 'learning_rate': 1.5715488077482152e-05, 'epoch': 1.36}
{'loss': 0.7943, 'learning_rate': 1.5698080575102662e-05, 'epoch': 1.37}
{'loss': 0.7573, 'learning_rate': 1.568064746731156e-05, 'epoch': 1.37}
{'loss': 0.837, 'learning_rate': 1.5663188832447833e-05, 'epoch': 1.37}
{'loss': 0.8127, 'learning_rate': 1.5645704748965193e-05, 'epoch': 1.37}
{'loss': 0.7727, 'learning_rate': 1.5628195295431696e-05, 'epoch': 1.38}
{'loss': 0.7898, 'learning_rate': 1.5610660550529413e-05, 'epoch': 1.38}
{'loss': 0.7841, 'learning_rate': 1.5593100593054063e-05, 'epoch': 1.38}
{'loss': 0.8299, 'learning_rate': 1.557551550191467e-05, 'epoch': 1.38}
{'loss': 0.8236, 'learning_rate': 1.5557905356133182e-05, 'epoch': 1.39}
{'loss': 0.8314, 'learning_rate': 1.554027023484416e-05, 'epoch': 1.39}
{'loss': 0.8049, 'learning_rate': 1.5522610217294377e-05, 'epoch': 1.39}
{'loss': 0.7687, 'learning_rate': 1.550492538284249e-05, 'epoch': 1.39}
{'loss': 0.8071, 'learning_rate': 1.5487215810958675e-05, 'epoch': 1.4}
{'loss': 0.7917, 'learning_rate': 1.5469481581224274e-05, 'epoch': 1.4}
{'loss': 0.7914, 'learning_rate': 1.545172277333142e-05, 'epoch': 1.4}
{'loss': 0.8045, 'learning_rate': 1.5433939467082713e-05, 'epoch': 1.41}
{'loss': 0.7788, 'learning_rate': 1.5416131742390827e-05, 'epoch': 1.41}
{'loss': 0.851, 'learning_rate': 1.5398299679278172e-05, 'epoch': 1.41}
{'loss': 0.7832, 'learning_rate': 1.5380443357876516e-05, 'epoch': 1.41}
{'loss': 0.7591, 'learning_rate': 1.5362562858426655e-05, 'epoch': 1.42}
{'loss': 0.767, 'learning_rate': 1.5344658261278013e-05, 'epoch': 1.42}
{'loss': 0.8645, 'learning_rate': 1.5326729646888314e-05, 'epoch': 1.42}
{'loss': 0.7996, 'learning_rate': 1.530877709582321e-05, 'epoch': 1.42}
{'loss': 0.8467, 'learning_rate': 1.5290800688755906e-05, 'epoch': 1.43}
{'loss': 0.8371, 'learning_rate': 1.527280050646682e-05, 'epoch': 1.43}
{'loss': 0.8251, 'learning_rate': 1.5254776629843204e-05, 'epoch': 1.43}
{'loss': 0.8115, 'learning_rate': 1.523672913987878e-05, 'epoch': 1.43}
{'loss': 0.8108, 'learning_rate': 1.5218658117673389e-05, 'epoch': 1.44}
{'loss': 0.7774, 'learning_rate': 1.5200563644432614e-05, 'epoch': 1.44}
{'loss': 0.7728, 'learning_rate': 1.518244580146742e-05, 'epoch': 1.44}
{'loss': 0.7847, 'learning_rate': 1.5164304670193792e-05, 'epoch': 1.44}
{'loss': 0.8291, 'learning_rate': 1.5146140332132359e-05, 'epoch': 1.45}
{'loss': 0.7675, 'learning_rate': 1.5127952868908043e-05, 'epoch': 1.45}
{'loss': 0.8281, 'learning_rate': 1.5109742362249673e-05, 'epoch': 1.45}
{'loss': 0.8432, 'learning_rate': 1.5091508893989633e-05, 'epoch': 1.45}
{'loss': 0.7984, 'learning_rate': 1.5073252546063493e-05, 'epoch': 1.46}
{'loss': 0.8033, 'learning_rate': 1.5054973400509631e-05, 'epoch': 1.46}
{'loss': 0.8412, 'learning_rate': 1.5036671539468879e-05, 'epoch': 1.46}
{'loss': 0.7544, 'learning_rate': 1.5018347045184132e-05, 'epoch': 1.46}
{'loss': 0.7962, 'learning_rate': 1.5000000000000002e-05, 'epoch': 1.47}
{'loss': 0.8351, 'learning_rate': 1.4981630486362435e-05, 'epoch': 1.47}
{'loss': 0.7913, 'learning_rate': 1.4963238586818346e-05, 'epoch': 1.47}
{'loss': 0.8417, 'learning_rate': 1.4944824384015238e-05, 'epoch': 1.47}
{'loss': 0.766, 'learning_rate': 1.4926387960700843e-05, 'epoch': 1.48}
{'loss': 0.7899, 'learning_rate': 1.4907929399722752e-05, 'epoch': 1.48}
{'loss': 0.7746, 'learning_rate': 1.488944878402802e-05, 'epoch': 1.48}
{'loss': 0.8103, 'learning_rate': 1.4870946196662822e-05, 'epoch': 1.48}
{'loss': 0.8422, 'learning_rate': 1.4852421720772064e-05, 'epoch': 1.49}
{'loss': 0.8136, 'learning_rate': 1.4833875439599004e-05, 'epoch': 1.49}
{'loss': 0.7957, 'learning_rate': 1.4815307436484898e-05, 'epoch': 1.49}
{'loss': 0.8081, 'learning_rate': 1.4796717794868606e-05, 'epoch': 1.49}
{'loss': 0.8154, 'learning_rate': 1.4778106598286235e-05, 'epoch': 1.5}
{'loss': 0.8193, 'learning_rate': 1.4759473930370738e-05, 'epoch': 1.5}
{'loss': 0.7687, 'learning_rate': 1.4740819874851562e-05, 'epoch': 1.5}
{'loss': 0.7366, 'learning_rate': 1.4722144515554263e-05, 'epoch': 1.51}
{'loss': 0.7733, 'learning_rate': 1.4703447936400135e-05, 'epoch': 1.51}
{'loss': 0.8318, 'learning_rate': 1.4684730221405814e-05, 'epoch': 1.51}
{'loss': 0.8014, 'learning_rate': 1.4665991454682924e-05, 'epoch': 1.51}
{'loss': 0.7706, 'learning_rate': 1.4647231720437687e-05, 'epoch': 1.52}
{'loss': 0.8441, 'learning_rate': 1.4628451102970546e-05, 'epoch': 1.52}
{'loss': 0.7787, 'learning_rate': 1.4609649686675786e-05, 'epoch': 1.52}
{'loss': 0.8178, 'learning_rate': 1.4590827556041158e-05, 'epoch': 1.52}
{'loss': 0.823, 'learning_rate': 1.4571984795647494e-05, 'epoch': 1.53}
{'loss': 0.8072, 'learning_rate': 1.4553121490168335e-05, 'epoch': 1.53}
{'loss': 0.8222, 'learning_rate': 1.4534237724369534e-05, 'epoch': 1.53}
{'loss': 0.776, 'learning_rate': 1.4515333583108896e-05, 'epoch': 1.53}
{'loss': 0.7757, 'learning_rate': 1.4496409151335786e-05, 'epoch': 1.54}
{'loss': 0.8204, 'learning_rate': 1.4477464514090745e-05, 'epoch': 1.54}
{'loss': 0.7843, 'learning_rate': 1.4458499756505116e-05, 'epoch': 1.54}
{'loss': 0.8112, 'learning_rate': 1.443951496380065e-05, 'epoch': 1.54}
{'loss': 0.8246, 'learning_rate': 1.4420510221289137e-05, 'epoch': 1.55}
{'loss': 0.7993, 'learning_rate': 1.4401485614372009e-05, 'epoch': 1.55}
{'loss': 0.8087, 'learning_rate': 1.4382441228539962e-05, 'epoch': 1.55}
{'loss': 0.7602, 'learning_rate': 1.4363377149372584e-05, 'epoch': 1.55}
{'loss': 0.7935, 'learning_rate': 1.434429346253794e-05, 'epoch': 1.56}
{'loss': 0.8377, 'learning_rate': 1.4325190253792222e-05, 'epoch': 1.56}
{'loss': 0.8346, 'learning_rate': 1.430606760897934e-05, 'epoch': 1.56}
{'loss': 0.7851, 'learning_rate': 1.4286925614030542e-05, 'epoch': 1.56}
{'loss': 0.787, 'learning_rate': 1.4267764354964038e-05, 'epoch': 1.57}
{'loss': 0.8389, 'learning_rate': 1.4248583917884595e-05, 'epoch': 1.57}
{'loss': 0.7794, 'learning_rate': 1.4229384388983167e-05, 'epoch': 1.57}
{'loss': 0.8116, 'learning_rate': 1.4210165854536495e-05, 'epoch': 1.57}
{'loss': 0.8146, 'learning_rate': 1.4190928400906731e-05, 'epoch': 1.58}
{'loss': 0.8344, 'learning_rate': 1.4171672114541042e-05, 'epoch': 1.58}
{'loss': 0.8259, 'learning_rate': 1.415239708197122e-05, 'epoch': 1.58}
{'loss': 0.7878, 'learning_rate': 1.4133103389813302e-05, 'epoch': 1.58}
{'loss': 0.7825, 'learning_rate': 1.411379112476717e-05, 'epoch': 1.59}
{'loss': 0.7542, 'learning_rate': 1.409446037361617e-05, 'epoch': 1.59}
{'loss': 0.8001, 'learning_rate': 1.4075111223226721e-05, 'epoch': 1.59}
{'loss': 0.8038, 'learning_rate': 1.4055743760547918e-05, 'epoch': 1.59}
{'loss': 0.7916, 'learning_rate': 1.4036358072611147e-05, 'epoch': 1.6}
{'loss': 0.8181, 'learning_rate': 1.4016954246529697e-05, 'epoch': 1.6}
{'loss': 0.8473, 'learning_rate': 1.3997532369498357e-05, 'epoch': 1.6}
{'loss': 0.7634, 'learning_rate': 1.3978092528793032e-05, 'epoch': 1.61}
{'loss': 0.8079, 'learning_rate': 1.3958634811770361e-05, 'epoch': 1.61}
{'loss': 0.8156, 'learning_rate': 1.39391593058673e-05, 'epoch': 1.61}
{'loss': 0.7625, 'learning_rate': 1.3919666098600753e-05, 'epoch': 1.61}
{'loss': 0.803, 'learning_rate': 1.3900155277567157e-05, 'epoch': 1.62}
{'loss': 0.8364, 'learning_rate': 1.3880626930442115e-05, 'epoch': 1.62}
{'loss': 0.7946, 'learning_rate': 1.3861081144979975e-05, 'epoch': 1.62}
{'loss': 0.8737, 'learning_rate': 1.3841518009013446e-05, 'epoch': 1.62}
{'loss': 0.8138, 'learning_rate': 1.382193761045322e-05, 'epoch': 1.63}
{'loss': 0.7597, 'learning_rate': 1.380234003728754e-05, 'epoch': 1.63}
{'loss': 0.8276, 'learning_rate': 1.378272537758185e-05, 'epoch': 1.63}
{'loss': 0.8108, 'learning_rate': 1.3763093719478357e-05, 'epoch': 1.63}
{'loss': 0.8137, 'learning_rate': 1.3743445151195658e-05, 'epoch': 1.64}
{'loss': 0.8055, 'learning_rate': 1.3723779761028349e-05, 'epoch': 1.64}
{'loss': 0.7624, 'learning_rate': 1.37040976373466e-05, 'epoch': 1.64}
{'loss': 0.862, 'learning_rate': 1.36843988685958e-05, 'epoch': 1.64}
{'loss': 0.7821, 'learning_rate': 1.3664683543296114e-05, 'epoch': 1.65}
{'loss': 0.758, 'learning_rate': 1.3644951750042115e-05, 'epoch': 1.65}
{'loss': 0.8178, 'learning_rate': 1.3625203577502384e-05, 'epoch': 1.65}
{'loss': 0.8348, 'learning_rate': 1.3605439114419095e-05, 'epoch': 1.65}
{'loss': 0.8298, 'learning_rate': 1.3585658449607632e-05, 'epoch': 1.66}
{'loss': 0.8194, 'learning_rate': 1.3565861671956189e-05, 'epoch': 1.66}
{'loss': 0.8231, 'learning_rate': 1.3546048870425356e-05, 'epoch': 1.66}
{'loss': 0.8115, 'learning_rate': 1.352622013404774e-05, 'epoch': 1.66}
{'loss': 0.7935, 'learning_rate': 1.3506375551927546e-05, 'epoch': 1.67}
{'loss': 0.7573, 'learning_rate': 1.3486515213240188e-05, 'epoch': 1.67}
{'loss': 0.819, 'learning_rate': 1.3466639207231882e-05, 'epoch': 1.67}
{'loss': 0.8032, 'learning_rate': 1.3446747623219256e-05, 'epoch': 1.67}
{'loss': 0.7717, 'learning_rate': 1.3426840550588933e-05, 'epoch': 1.68}
{'loss': 0.7919, 'learning_rate': 1.3406918078797142e-05, 'epoch': 1.68}
{'loss': 0.7638, 'learning_rate': 1.3386980297369308e-05, 'epoch': 1.68}
{'loss': 0.805, 'learning_rate': 1.3367027295899652e-05, 'epoch': 1.68}
{'loss': 0.8378, 'learning_rate': 1.3347059164050796e-05, 'epoch': 1.69}
{'loss': 0.8601, 'learning_rate': 1.332707599155334e-05, 'epoch': 1.69}
{'loss': 0.827, 'learning_rate': 1.3307077868205487e-05, 'epoch': 1.69}
{'loss': 0.8289, 'learning_rate': 1.3287064883872614e-05, 'epoch': 1.69}
{'loss': 0.8072, 'learning_rate': 1.3267037128486883e-05, 'epoch': 1.7}
{'loss': 0.7881, 'learning_rate': 1.3246994692046837e-05, 'epoch': 1.7}
{'loss': 0.7684, 'learning_rate': 1.3226937664616977e-05, 'epoch': 1.7}
{'loss': 0.7952, 'learning_rate': 1.3206866136327387e-05, 'epoch': 1.71}
{'loss': 0.8064, 'learning_rate': 1.3186780197373306e-05, 'epoch': 1.71}
{'loss': 0.8248, 'learning_rate': 1.3166679938014728e-05, 'epoch': 1.71}
{'loss': 0.7767, 'learning_rate': 1.3146565448576002e-05, 'epoch': 1.71}
{'loss': 0.7899, 'learning_rate': 1.3126436819445423e-05, 'epoch': 1.72}
{'loss': 0.7839, 'learning_rate': 1.3106294141074825e-05, 'epoch': 1.72}
{'loss': 0.7953, 'learning_rate': 1.308613750397917e-05, 'epoch': 1.72}
{'loss': 0.7935, 'learning_rate': 1.3065966998736155e-05, 'epoch': 1.72}
{'loss': 0.8131, 'learning_rate': 1.304578271598579e-05, 'epoch': 1.73}
{'loss': 0.7642, 'learning_rate': 1.302558474643e-05, 'epoch': 1.73}
{'loss': 0.7821, 'learning_rate': 1.300537318083221e-05, 'epoch': 1.73}
{'loss': 0.8327, 'learning_rate': 1.2985148110016947e-05, 'epoch': 1.73}
{'loss': 0.7973, 'learning_rate': 1.2964909624869422e-05, 'epoch': 1.74}
{'loss': 0.7698, 'learning_rate': 1.2944657816335124e-05, 'epoch': 1.74}
{'loss': 0.8402, 'learning_rate': 1.2924392775419421e-05, 'epoch': 1.74}
{'loss': 0.7686, 'learning_rate': 1.2904114593187136e-05, 'epoch': 1.74}
{'loss': 0.8225, 'learning_rate': 1.2883823360762149e-05, 'epoch': 1.75}
{'loss': 0.7842, 'learning_rate': 1.2863519169326984e-05, 'epoch': 1.75}
{'loss': 0.8096, 'learning_rate': 1.284320211012239e-05, 'epoch': 1.75}
{'loss': 0.7605, 'learning_rate': 1.2822872274446958e-05, 'epoch': 1.75}
{'loss': 0.7653, 'learning_rate': 1.280252975365667e-05, 'epoch': 1.76}
{'loss': 0.8186, 'learning_rate': 1.2782174639164528e-05, 'epoch': 1.76}
{'loss': 0.8052, 'learning_rate': 1.276180702244012e-05, 'epoch': 1.76}
{'loss': 0.8106, 'learning_rate': 1.2741426995009214e-05, 'epoch': 1.76}
{'loss': 0.8028, 'learning_rate': 1.2721034648453353e-05, 'epoch': 1.77}
{'loss': 0.7903, 'learning_rate': 1.2700630074409427e-05, 'epoch': 1.77}
{'loss': 0.7951, 'learning_rate': 1.268021336456929e-05, 'epoch': 1.77}
{'loss': 0.7863, 'learning_rate': 1.2659784610679318e-05, 'epoch': 1.77}
{'loss': 0.8114, 'learning_rate': 1.2639343904540008e-05, 'epoch': 1.78}
{'loss': 0.7654, 'learning_rate': 1.2618891338005574e-05, 'epoch': 1.78}
{'loss': 0.8028, 'learning_rate': 1.2598427002983523e-05, 'epoch': 1.78}
{'loss': 0.8131, 'learning_rate': 1.2577950991434249e-05, 'epoch': 1.78}
{'loss': 0.8281, 'learning_rate': 1.2557463395370609e-05, 'epoch': 1.79}
{'loss': 0.779, 'learning_rate': 1.2536964306857526e-05, 'epoch': 1.79}
{'loss': 0.7945, 'learning_rate': 1.2516453818011567e-05, 'epoch': 1.79}
{'loss': 0.7965, 'learning_rate': 1.2495932021000516e-05, 'epoch': 1.79}
{'loss': 0.7763, 'learning_rate': 1.247539900804299e-05, 'epoch': 1.8}
{'loss': 0.8241, 'learning_rate': 1.2454854871407993e-05, 'epoch': 1.8}
{'loss': 0.8058, 'learning_rate': 1.2434299703414523e-05, 'epoch': 1.8}
{'loss': 0.8203, 'learning_rate': 1.2413733596431141e-05, 'epoch': 1.81}
{'loss': 0.7938, 'learning_rate': 1.2393156642875579e-05, 'epoch': 1.81}
{'loss': 0.7953, 'learning_rate': 1.2372568935214298e-05, 'epoch': 1.81}
{'loss': 0.782, 'learning_rate': 1.2351970565962083e-05, 'epoch': 1.81}
{'loss': 0.8193, 'learning_rate': 1.2331361627681645e-05, 'epoch': 1.82}
{'loss': 0.8184, 'learning_rate': 1.2310742212983167e-05, 'epoch': 1.82}
{'loss': 0.8209, 'learning_rate': 1.2290112414523927e-05, 'epoch': 1.82}
{'loss': 0.7982, 'learning_rate': 1.2269472325007858e-05, 'epoch': 1.82}
{'loss': 0.779, 'learning_rate': 1.2248822037185137e-05, 'epoch': 1.83}
{'loss': 0.7905, 'learning_rate': 1.222816164385177e-05, 'epoch': 1.83}
{'loss': 0.743, 'learning_rate': 1.2207491237849174e-05, 'epoch': 1.83}
{'loss': 0.8098, 'learning_rate': 1.218681091206376e-05, 'epoch': 1.83}
{'loss': 0.7813, 'learning_rate': 1.2166120759426515e-05, 'epoch': 1.84}
{'loss': 0.7944, 'learning_rate': 1.2145420872912586e-05, 'epoch': 1.84}
{'loss': 0.8159, 'learning_rate': 1.2124711345540861e-05, 'epoch': 1.84}
{'loss': 0.7834, 'learning_rate': 1.2103992270373548e-05, 'epoch': 1.84}
{'loss': 0.8172, 'learning_rate': 1.2083263740515764e-05, 'epoch': 1.85}
{'loss': 0.7996, 'learning_rate': 1.2062525849115107e-05, 'epoch': 1.85}
{'loss': 0.8369, 'learning_rate': 1.2041778689361254e-05, 'epoch': 1.85}
{'loss': 0.767, 'learning_rate': 1.2021022354485514e-05, 'epoch': 1.85}
{'loss': 0.8078, 'learning_rate': 1.2000256937760446e-05, 'epoch': 1.86}
{'loss': 0.7994, 'learning_rate': 1.1979482532499402e-05, 'epoch': 1.86}
{'loss': 0.7912, 'learning_rate': 1.1958699232056135e-05, 'epoch': 1.86}
{'loss': 0.81, 'learning_rate': 1.1937907129824371e-05, 'epoch': 1.86}
{'loss': 0.8002, 'learning_rate': 1.1917106319237386e-05, 'epoch': 1.87}
{'loss': 0.7966, 'learning_rate': 1.1896296893767588e-05, 'epoch': 1.87}
{'loss': 0.8234, 'learning_rate': 1.1875478946926094e-05, 'epoch': 1.87}
{'loss': 0.7378, 'learning_rate': 1.1854652572262324e-05, 'epoch': 1.87}
{'loss': 0.7878, 'learning_rate': 1.1833817863363563e-05, 'epoch': 1.88}
{'loss': 0.7829, 'learning_rate': 1.1812974913854545e-05, 'epoch': 1.88}
{'loss': 0.7498, 'learning_rate': 1.1792123817397041e-05, 'epoch': 1.88}
{'loss': 0.7399, 'learning_rate': 1.1771264667689428e-05, 'epoch': 1.88}
{'loss': 0.8065, 'learning_rate': 1.1750397558466273e-05, 'epoch': 1.89}
{'loss': 0.7602, 'learning_rate': 1.1729522583497913e-05, 'epoch': 1.89}
{'loss': 0.7641, 'learning_rate': 1.1708639836590024e-05, 'epoch': 1.89}
{'loss': 0.8267, 'learning_rate': 1.1687749411583213e-05, 'epoch': 1.89}
{'loss': 0.7885, 'learning_rate': 1.1666851402352587e-05, 'epoch': 1.9}
{'loss': 0.7623, 'learning_rate': 1.164594590280734e-05, 'epoch': 1.9}
{'loss': 0.7529, 'learning_rate': 1.1625033006890316e-05, 'epoch': 1.9}
{'loss': 0.7962, 'learning_rate': 1.1604112808577603e-05, 'epoch': 1.91}
{'loss': 0.7621, 'learning_rate': 1.15831854018781e-05, 'epoch': 1.91}
{'loss': 0.7941, 'learning_rate': 1.1562250880833103e-05, 'epoch': 1.91}
{'loss': 0.8247, 'learning_rate': 1.154130933951587e-05, 'epoch': 1.91}
{'loss': 0.8127, 'learning_rate': 1.1520360872031208e-05, 'epoch': 1.92}
{'loss': 0.7675, 'learning_rate': 1.1499405572515059e-05, 'epoch': 1.92}
{'loss': 0.7718, 'learning_rate': 1.147844353513405e-05, 'epoch': 1.92}
{'loss': 0.839, 'learning_rate': 1.1457474854085095e-05, 'epoch': 1.92}
{'loss': 0.7915, 'learning_rate': 1.1436499623594963e-05, 'epoch': 1.93}
{'loss': 0.8186, 'learning_rate': 1.1415517937919846e-05, 'epoch': 1.93}
{'loss': 0.8163, 'learning_rate': 1.139452989134496e-05, 'epoch': 1.93}
{'loss': 0.8156, 'learning_rate': 1.1373535578184083e-05, 'epoch': 1.93}
{'loss': 0.8194, 'learning_rate': 1.1352535092779174e-05, 'epoch': 1.94}
{'loss': 0.7787, 'learning_rate': 1.1331528529499909e-05, 'epoch': 1.94}
{'loss': 0.7798, 'learning_rate': 1.1310515982743293e-05, 'epoch': 1.94}
{'loss': 0.7784, 'learning_rate': 1.1289497546933212e-05, 'epoch': 1.94}
{'loss': 0.7827, 'learning_rate': 1.1268473316520007e-05, 'epoch': 1.95}
{'loss': 0.7978, 'learning_rate': 1.124744338598008e-05, 'epoch': 1.95}
{'loss': 0.7453, 'learning_rate': 1.1226407849815422e-05, 'epoch': 1.95}
{'loss': 0.8078, 'learning_rate': 1.1205366802553231e-05, 'epoch': 1.95}
{'loss': 0.7637, 'learning_rate': 1.1184320338745467e-05, 'epoch': 1.96}
{'loss': 0.7807, 'learning_rate': 1.1163268552968422e-05, 'epoch': 1.96}
{'loss': 0.8054, 'learning_rate': 1.1142211539822318e-05, 'epoch': 1.96}
{'loss': 0.826, 'learning_rate': 1.112114939393085e-05, 'epoch': 1.96}
{'loss': 0.8095, 'learning_rate': 1.1100082209940795e-05, 'epoch': 1.97}
{'loss': 0.7496, 'learning_rate': 1.1079010082521557e-05, 'epoch': 1.97}
{'loss': 0.8398, 'learning_rate': 1.1057933106364758e-05, 'epoch': 1.97}
{'loss': 0.7913, 'learning_rate': 1.1036851376183812e-05, 'epoch': 1.97}
{'loss': 0.8017, 'learning_rate': 1.101576498671349e-05, 'epoch': 1.98}
{'loss': 0.7392, 'learning_rate': 1.0994674032709514e-05, 'epoch': 1.98}
{'loss': 0.7795, 'learning_rate': 1.0973578608948094e-05, 'epoch': 1.98}
{'loss': 0.7834, 'learning_rate': 1.095247881022555e-05, 'epoch': 1.98}
{'loss': 0.7992, 'learning_rate': 1.0931374731357843e-05, 'epoch': 1.99}
{'loss': 0.8043, 'learning_rate': 1.091026646718018e-05, 'epoch': 1.99}
{'loss': 0.7868, 'learning_rate': 1.0889154112546569e-05, 'epoch': 1.99}
{'loss': 0.8144, 'learning_rate': 1.0868037762329405e-05, 'epoch': 1.99}
{'loss': 0.7629, 'learning_rate': 1.0846917511419031e-05, 'epoch': 2.0}
{'loss': 0.8088, 'learning_rate': 1.0825793454723325e-05, 'epoch': 2.0}
50%|█████ | 1560/3120 [2:09:35<2:10:47, 5.03s/it]/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
{'loss': 0.4308, 'learning_rate': 1.0804665687167262e-05, 'epoch': 2.0}
{'loss': 0.4056, 'learning_rate': 1.0783534303692493e-05, 'epoch': 2.01}
{'loss': 0.4122, 'learning_rate': 1.0762399399256917e-05, 'epoch': 2.01}
{'loss': 0.4125, 'learning_rate': 1.0741261068834266e-05, 'epoch': 2.01}
{'loss': 0.3983, 'learning_rate': 1.0720119407413647e-05, 'epoch': 2.01}
{'loss': 0.3843, 'learning_rate': 1.0698974509999159e-05, 'epoch': 2.02}
{'loss': 0.4024, 'learning_rate': 1.0677826471609423e-05, 'epoch': 2.02}
{'loss': 0.3865, 'learning_rate': 1.0656675387277183e-05, 'epoch': 2.02}
{'loss': 0.3953, 'learning_rate': 1.0635521352048873e-05, 'epoch': 2.02}
{'loss': 0.3734, 'learning_rate': 1.0614364460984178e-05, 'epoch': 2.03}
{'loss': 0.3556, 'learning_rate': 1.0593204809155628e-05, 'epoch': 2.03}
{'loss': 0.3826, 'learning_rate': 1.057204249164815e-05, 'epoch': 2.03}
{'loss': 0.3836, 'learning_rate': 1.0550877603558656e-05, 'epoch': 2.03}
{'loss': 0.4046, 'learning_rate': 1.0529710239995606e-05, 'epoch': 2.04}
{'loss': 0.3772, 'learning_rate': 1.0508540496078582e-05, 'epoch': 2.04}
{'loss': 0.3808, 'learning_rate': 1.0487368466937866e-05, 'epoch': 2.04}
{'loss': 0.367, 'learning_rate': 1.0466194247714008e-05, 'epoch': 2.04}
{'loss': 0.3947, 'learning_rate': 1.0445017933557404e-05, 'epoch': 2.05}
{'loss': 0.3663, 'learning_rate': 1.0423839619627854e-05, 'epoch': 2.05}
{'loss': 0.4022, 'learning_rate': 1.0402659401094154e-05, 'epoch': 2.05}
{'loss': 0.3824, 'learning_rate': 1.0381477373133652e-05, 'epoch': 2.05}
{'loss': 0.3954, 'learning_rate': 1.036029363093183e-05, 'epoch': 2.06}
{'loss': 0.4046, 'learning_rate': 1.0339108269681874e-05, 'epoch': 2.06}
{'loss': 0.4107, 'learning_rate': 1.0317921384584245e-05, 'epoch': 2.06}
{'loss': 0.3856, 'learning_rate': 1.0296733070846252e-05, 'epoch': 2.06}
{'loss': 0.375, 'learning_rate': 1.0275543423681622e-05, 'epoch': 2.07}
{'loss': 0.4128, 'learning_rate': 1.0254352538310075e-05, 'epoch': 2.07}
{'loss': 0.3887, 'learning_rate': 1.0233160509956893e-05, 'epoch': 2.07}
{'loss': 0.3704, 'learning_rate': 1.0211967433852501e-05, 'epoch': 2.07}
{'loss': 0.382, 'learning_rate': 1.0190773405232024e-05, 'epoch': 2.08}
{'loss': 0.3955, 'learning_rate': 1.0169578519334873e-05, 'epoch': 2.08}
{'loss': 0.3985, 'learning_rate': 1.014838287140431e-05, 'epoch': 2.08}
{'loss': 0.3869, 'learning_rate': 1.012718655668702e-05, 'epoch': 2.08}
{'loss': 0.393, 'learning_rate': 1.010598967043268e-05, 'epoch': 2.09}
{'loss': 0.3714, 'learning_rate': 1.0084792307893551e-05, 'epoch': 2.09}
{'loss': 0.3715, 'learning_rate': 1.0063594564324014e-05, 'epoch': 2.09}
{'loss': 0.3817, 'learning_rate': 1.0042396534980176e-05, 'epoch': 2.09}
{'loss': 0.394, 'learning_rate': 1.0021198315119426e-05, 'epoch': 2.1}
{'loss': 0.3823, 'learning_rate': 1e-05, 'epoch': 2.1}
{'loss': 0.3811, 'learning_rate': 9.97880168488058e-06, 'epoch': 2.1}
{'loss': 0.4121, 'learning_rate': 9.957603465019827e-06, 'epoch': 2.11}
{'loss': 0.3859, 'learning_rate': 9.936405435675991e-06, 'epoch': 2.11}
{'loss': 0.3806, 'learning_rate': 9.91520769210645e-06, 'epoch': 2.11}
{'loss': 0.385, 'learning_rate': 9.894010329567322e-06, 'epoch': 2.11}
{'loss': 0.3755, 'learning_rate': 9.872813443312984e-06, 'epoch': 2.12}
{'loss': 0.4136, 'learning_rate': 9.851617128595694e-06, 'epoch': 2.12}
{'loss': 0.3898, 'learning_rate': 9.830421480665128e-06, 'epoch': 2.12}
{'loss': 0.4021, 'learning_rate': 9.809226594767979e-06, 'epoch': 2.12}
{'loss': 0.3928, 'learning_rate': 9.788032566147506e-06, 'epoch': 2.13}
{'loss': 0.3639, 'learning_rate': 9.766839490043108e-06, 'epoch': 2.13}
{'loss': 0.3843, 'learning_rate': 9.745647461689932e-06, 'epoch': 2.13}
{'loss': 0.3798, 'learning_rate': 9.724456576318383e-06, 'epoch': 2.13}
{'loss': 0.3915, 'learning_rate': 9.703266929153751e-06, 'epoch': 2.14}
{'loss': 0.3685, 'learning_rate': 9.682078615415755e-06, 'epoch': 2.14}
{'loss': 0.4122, 'learning_rate': 9.660891730318129e-06, 'epoch': 2.14}
{'loss': 0.3752, 'learning_rate': 9.63970636906817e-06, 'epoch': 2.14}
{'loss': 0.3927, 'learning_rate': 9.618522626866351e-06, 'epoch': 2.15}
{'loss': 0.4052, 'learning_rate': 9.597340598905851e-06, 'epoch': 2.15}
{'loss': 0.3777, 'learning_rate': 9.57616038037215e-06, 'epoch': 2.15}
{'loss': 0.4091, 'learning_rate': 9.554982066442601e-06, 'epoch': 2.15}
{'loss': 0.3733, 'learning_rate': 9.533805752285992e-06, 'epoch': 2.16}
{'loss': 0.3721, 'learning_rate': 9.512631533062138e-06, 'epoch': 2.16}
{'loss': 0.3936, 'learning_rate': 9.49145950392142e-06, 'epoch': 2.16}
{'loss': 0.3851, 'learning_rate': 9.470289760004398e-06, 'epoch': 2.16}
{'loss': 0.3796, 'learning_rate': 9.449122396441344e-06, 'epoch': 2.17}
{'loss': 0.3848, 'learning_rate': 9.427957508351852e-06, 'epoch': 2.17}
{'loss': 0.3836, 'learning_rate': 9.406795190844376e-06, 'epoch': 2.17}
{'loss': 0.3959, 'learning_rate': 9.385635539015824e-06, 'epoch': 2.17}
{'loss': 0.4006, 'learning_rate': 9.364478647951132e-06, 'epoch': 2.18}
{'loss': 0.3844, 'learning_rate': 9.343324612722819e-06, 'epoch': 2.18}
{'loss': 0.4128, 'learning_rate': 9.32217352839058e-06, 'epoch': 2.18}
{'loss': 0.4061, 'learning_rate': 9.301025490000843e-06, 'epoch': 2.18}
{'loss': 0.3827, 'learning_rate': 9.279880592586355e-06, 'epoch': 2.19}
{'loss': 0.4085, 'learning_rate': 9.25873893116574e-06, 'epoch': 2.19}
{'loss': 0.3793, 'learning_rate': 9.237600600743086e-06, 'epoch': 2.19}
{'loss': 0.3724, 'learning_rate': 9.216465696307513e-06, 'epoch': 2.19}
{'loss': 0.3611, 'learning_rate': 9.195334312832742e-06, 'epoch': 2.2}
{'loss': 0.3698, 'learning_rate': 9.174206545276678e-06, 'epoch': 2.2}
{'loss': 0.3819, 'learning_rate': 9.153082488580969e-06, 'epoch': 2.2}
{'loss': 0.3903, 'learning_rate': 9.131962237670599e-06, 'epoch': 2.21}
{'loss': 0.3894, 'learning_rate': 9.11084588745343e-06, 'epoch': 2.21}
{'loss': 0.3942, 'learning_rate': 9.089733532819825e-06, 'epoch': 2.21}
{'loss': 0.3799, 'learning_rate': 9.068625268642162e-06, 'epoch': 2.21}
{'loss': 0.3966, 'learning_rate': 9.047521189774456e-06, 'epoch': 2.22}
{'loss': 0.3808, 'learning_rate': 9.026421391051908e-06, 'epoch': 2.22}
{'loss': 0.3848, 'learning_rate': 9.005325967290489e-06, 'epoch': 2.22}
{'loss': 0.4027, 'learning_rate': 8.984235013286512e-06, 'epoch': 2.22}
{'loss': 0.3943, 'learning_rate': 8.963148623816191e-06, 'epoch': 2.23}
{'loss': 0.3921, 'learning_rate': 8.942066893635245e-06, 'epoch': 2.23}
{'loss': 0.3841, 'learning_rate': 8.920989917478446e-06, 'epoch': 2.23}
{'loss': 0.3639, 'learning_rate': 8.899917790059208e-06, 'epoch': 2.23}
{'loss': 0.3924, 'learning_rate': 8.878850606069152e-06, 'epoch': 2.24}
{'loss': 0.4187, 'learning_rate': 8.857788460177685e-06, 'epoch': 2.24}
{'loss': 0.3605, 'learning_rate': 8.836731447031581e-06, 'epoch': 2.24}
{'loss': 0.3867, 'learning_rate': 8.815679661254538e-06, 'epoch': 2.24}
{'loss': 0.391, 'learning_rate': 8.79463319744677e-06, 'epoch': 2.25}
{'loss': 0.3901, 'learning_rate': 8.77359215018458e-06, 'epoch': 2.25}
{'loss': 0.3709, 'learning_rate': 8.752556614019924e-06, 'epoch': 2.25}
{'loss': 0.4087, 'learning_rate': 8.731526683479991e-06, 'epoch': 2.25}
{'loss': 0.3871, 'learning_rate': 8.710502453066791e-06, 'epoch': 2.26}
{'loss': 0.4008, 'learning_rate': 8.689484017256712e-06, 'epoch': 2.26}
{'loss': 0.3916, 'learning_rate': 8.668471470500094e-06, 'epoch': 2.26}
{'loss': 0.3945, 'learning_rate': 8.647464907220833e-06, 'epoch': 2.26}
{'loss': 0.3975, 'learning_rate': 8.626464421815919e-06, 'epoch': 2.27}
{'loss': 0.3856, 'learning_rate': 8.605470108655046e-06, 'epoch': 2.27}
{'loss': 0.3875, 'learning_rate': 8.584482062080154e-06, 'epoch': 2.27}
{'loss': 0.3793, 'learning_rate': 8.56350037640504e-06, 'epoch': 2.27}
{'loss': 0.3688, 'learning_rate': 8.542525145914907e-06, 'epoch': 2.28}
{'loss': 0.3756, 'learning_rate': 8.521556464865955e-06, 'epoch': 2.28}
{'loss': 0.3982, 'learning_rate': 8.500594427484946e-06, 'epoch': 2.28}
{'loss': 0.3684, 'learning_rate': 8.479639127968793e-06, 'epoch': 2.28}
{'loss': 0.3618, 'learning_rate': 8.458690660484134e-06, 'epoch': 2.29}
{'loss': 0.3696, 'learning_rate': 8.4377491191669e-06, 'epoch': 2.29}
{'loss': 0.3897, 'learning_rate': 8.416814598121901e-06, 'epoch': 2.29}
{'loss': 0.3597, 'learning_rate': 8.395887191422397e-06, 'epoch': 2.29}
{'loss': 0.3656, 'learning_rate': 8.374966993109689e-06, 'epoch': 2.3}
{'loss': 0.3702, 'learning_rate': 8.35405409719266e-06, 'epoch': 2.3}
{'loss': 0.3971, 'learning_rate': 8.333148597647414e-06, 'epoch': 2.3}
{'loss': 0.3746, 'learning_rate': 8.312250588416792e-06, 'epoch': 2.31}
{'loss': 0.3684, 'learning_rate': 8.291360163409978e-06, 'epoch': 2.31}
{'loss': 0.3889, 'learning_rate': 8.27047741650209e-06, 'epoch': 2.31}
{'loss': 0.4051, 'learning_rate': 8.249602441533727e-06, 'epoch': 2.31}
{'loss': 0.39, 'learning_rate': 8.228735332310575e-06, 'epoch': 2.32}
{'loss': 0.3554, 'learning_rate': 8.207876182602959e-06, 'epoch': 2.32}
{'loss': 0.3847, 'learning_rate': 8.187025086145458e-06, 'epoch': 2.32}
{'loss': 0.3708, 'learning_rate': 8.16618213663644e-06, 'epoch': 2.32}
{'loss': 0.3767, 'learning_rate': 8.14534742773768e-06, 'epoch': 2.33}
{'loss': 0.3775, 'learning_rate': 8.12452105307391e-06, 'epoch': 2.33}
{'loss': 0.4184, 'learning_rate': 8.103703106232416e-06, 'epoch': 2.33}
{'loss': 0.3827, 'learning_rate': 8.082893680762619e-06, 'epoch': 2.33}
{'loss': 0.3907, 'learning_rate': 8.062092870175629e-06, 'epoch': 2.34}
{'loss': 0.3817, 'learning_rate': 8.041300767943867e-06, 'epoch': 2.34}
{'loss': 0.3809, 'learning_rate': 8.0205174675006e-06, 'epoch': 2.34}
{'loss': 0.3952, 'learning_rate': 7.999743062239557e-06, 'epoch': 2.34}
{'loss': 0.3844, 'learning_rate': 7.978977645514488e-06, 'epoch': 2.35}
{'loss': 0.385, 'learning_rate': 7.958221310638749e-06, 'epoch': 2.35}
{'loss': 0.3926, 'learning_rate': 7.937474150884898e-06, 'epoch': 2.35}
{'loss': 0.3856, 'learning_rate': 7.916736259484239e-06, 'epoch': 2.35}
{'loss': 0.3861, 'learning_rate': 7.896007729626457e-06, 'epoch': 2.36}
{'loss': 0.3853, 'learning_rate': 7.875288654459144e-06, 'epoch': 2.36}
{'loss': 0.3897, 'learning_rate': 7.854579127087418e-06, 'epoch': 2.36}
{'loss': 0.3908, 'learning_rate': 7.833879240573487e-06, 'epoch': 2.36}
{'loss': 0.4056, 'learning_rate': 7.813189087936243e-06, 'epoch': 2.37}
{'loss': 0.378, 'learning_rate': 7.792508762150833e-06, 'epoch': 2.37}
{'loss': 0.3971, 'learning_rate': 7.771838356148231e-06, 'epoch': 2.37}
{'loss': 0.3823, 'learning_rate': 7.751177962814867e-06, 'epoch': 2.37}
{'loss': 0.3894, 'learning_rate': 7.730527674992143e-06, 'epoch': 2.38}
{'loss': 0.3829, 'learning_rate': 7.709887585476075e-06, 'epoch': 2.38}
{'loss': 0.3749, 'learning_rate': 7.689257787016834e-06, 'epoch': 2.38}
{'loss': 0.3979, 'learning_rate': 7.668638372318359e-06, 'epoch': 2.38}
{'loss': 0.3682, 'learning_rate': 7.648029434037915e-06, 'epoch': 2.39}
{'loss': 0.3408, 'learning_rate': 7.627431064785705e-06, 'epoch': 2.39}
{'loss': 0.3749, 'learning_rate': 7.606843357124426e-06, 'epoch': 2.39}
{'loss': 0.3857, 'learning_rate': 7.5862664035688604e-06, 'epoch': 2.39}
{'loss': 0.391, 'learning_rate': 7.565700296585483e-06, 'epoch': 2.4}
{'loss': 0.3709, 'learning_rate': 7.545145128592009e-06, 'epoch': 2.4}
{'loss': 0.3874, 'learning_rate': 7.524600991957013e-06, 'epoch': 2.4}
{'loss': 0.3881, 'learning_rate': 7.504067978999484e-06, 'epoch': 2.41}
{'loss': 0.3869, 'learning_rate': 7.483546181988437e-06, 'epoch': 2.41}
{'loss': 0.3638, 'learning_rate': 7.463035693142473e-06, 'epoch': 2.41}
{'loss': 0.3865, 'learning_rate': 7.442536604629396e-06, 'epoch': 2.41}
{'loss': 0.4102, 'learning_rate': 7.422049008565757e-06, 'epoch': 2.42}
{'loss': 0.4113, 'learning_rate': 7.40157299701648e-06, 'epoch': 2.42}
{'loss': 0.4165, 'learning_rate': 7.38110866199443e-06, 'epoch': 2.42}
{'loss': 0.3775, 'learning_rate': 7.360656095459995e-06, 'epoch': 2.42}
{'loss': 0.3756, 'learning_rate': 7.340215389320686e-06, 'epoch': 2.43}
{'loss': 0.3758, 'learning_rate': 7.31978663543071e-06, 'epoch': 2.43}
{'loss': 0.3839, 'learning_rate': 7.299369925590575e-06, 'epoch': 2.43}
{'loss': 0.3921, 'learning_rate': 7.278965351546648e-06, 'epoch': 2.43}
{'loss': 0.4005, 'learning_rate': 7.258573004990789e-06, 'epoch': 2.44}
{'loss': 0.3916, 'learning_rate': 7.2381929775598835e-06, 'epoch': 2.44}
{'loss': 0.3971, 'learning_rate': 7.217825360835475e-06, 'epoch': 2.44}
{'loss': 0.3681, 'learning_rate': 7.197470246343333e-06, 'epoch': 2.44}
{'loss': 0.3765, 'learning_rate': 7.1771277255530456e-06, 'epoch': 2.45}
{'loss': 0.381, 'learning_rate': 7.156797889877613e-06, 'epoch': 2.45}
{'loss': 0.3638, 'learning_rate': 7.136480830673018e-06, 'epoch': 2.45}
{'loss': 0.3783, 'learning_rate': 7.116176639237853e-06, 'epoch': 2.45}
{'loss': 0.3704, 'learning_rate': 7.095885406812866e-06, 'epoch': 2.46}
{'loss': 0.3908, 'learning_rate': 7.075607224580581e-06, 'epoch': 2.46}
{'loss': 0.3999, 'learning_rate': 7.05534218366488e-06, 'epoch': 2.46}
{'loss': 0.3768, 'learning_rate': 7.035090375130581e-06, 'epoch': 2.46}
{'loss': 0.3601, 'learning_rate': 7.014851889983058e-06, 'epoch': 2.47}
{'loss': 0.3643, 'learning_rate': 6.994626819167789e-06, 'epoch': 2.47}
{'loss': 0.3719, 'learning_rate': 6.974415253570003e-06, 'epoch': 2.47}
{'loss': 0.3831, 'learning_rate': 6.9542172840142105e-06, 'epoch': 2.47}
{'loss': 0.402, 'learning_rate': 6.934033001263847e-06, 'epoch': 2.48}
{'loss': 0.3967, 'learning_rate': 6.913862496020831e-06, 'epoch': 2.48}
{'loss': 0.3879, 'learning_rate': 6.893705858925179e-06, 'epoch': 2.48}
{'loss': 0.4073, 'learning_rate': 6.873563180554583e-06, 'epoch': 2.48}
{'loss': 0.3656, 'learning_rate': 6.853434551424001e-06, 'epoch': 2.49}
{'loss': 0.3959, 'learning_rate': 6.833320061985278e-06, 'epoch': 2.49}
{'loss': 0.3569, 'learning_rate': 6.813219802626698e-06, 'epoch': 2.49}
{'loss': 0.3647, 'learning_rate': 6.793133863672616e-06, 'epoch': 2.49}
{'loss': 0.3795, 'learning_rate': 6.773062335383024e-06, 'epoch': 2.5}
{'loss': 0.3523, 'learning_rate': 6.7530053079531664e-06, 'epoch': 2.5}
{'loss': 0.3683, 'learning_rate': 6.73296287151312e-06, 'epoch': 2.5}
{'loss': 0.3993, 'learning_rate': 6.712935116127389e-06, 'epoch': 2.51}
{'loss': 0.3766, 'learning_rate': 6.692922131794517e-06, 'epoch': 2.51}
{'loss': 0.3865, 'learning_rate': 6.672924008446663e-06, 'epoch': 2.51}
{'loss': 0.3695, 'learning_rate': 6.652940835949208e-06, 'epoch': 2.51}
{'loss': 0.3803, 'learning_rate': 6.632972704100349e-06, 'epoch': 2.52}
{'loss': 0.3885, 'learning_rate': 6.6130197026306945e-06, 'epoch': 2.52}
{'loss': 0.38, 'learning_rate': 6.59308192120286e-06, 'epoch': 2.52}
{'loss': 0.3645, 'learning_rate': 6.573159449411071e-06, 'epoch': 2.52}
{'loss': 0.3649, 'learning_rate': 6.553252376780749e-06, 'epoch': 2.53}
{'loss': 0.3767, 'learning_rate': 6.533360792768122e-06, 'epoch': 2.53}
{'loss': 0.3762, 'learning_rate': 6.513484786759818e-06, 'epoch': 2.53}
{'loss': 0.3885, 'learning_rate': 6.4936244480724575e-06, 'epoch': 2.53}
{'loss': 0.3901, 'learning_rate': 6.473779865952263e-06, 'epoch': 2.54}
{'loss': 0.3846, 'learning_rate': 6.453951129574644e-06, 'epoch': 2.54}
{'loss': 0.3772, 'learning_rate': 6.434138328043815e-06, 'epoch': 2.54}
{'loss': 0.3927, 'learning_rate': 6.4143415503923676e-06, 'epoch': 2.54}
{'loss': 0.3657, 'learning_rate': 6.39456088558091e-06, 'epoch': 2.55}
{'loss': 0.377, 'learning_rate': 6.374796422497622e-06, 'epoch': 2.55}
{'loss': 0.3522, 'learning_rate': 6.355048249957887e-06, 'epoch': 2.55}
{'loss': 0.378, 'learning_rate': 6.335316456703891e-06, 'epoch': 2.55}
{'loss': 0.386, 'learning_rate': 6.315601131404201e-06, 'epoch': 2.56}
{'loss': 0.3726, 'learning_rate': 6.295902362653401e-06, 'epoch': 2.56}
{'loss': 0.385, 'learning_rate': 6.276220238971653e-06, 'epoch': 2.56}
{'loss': 0.36, 'learning_rate': 6.256554848804343e-06, 'epoch': 2.56}
{'loss': 0.3847, 'learning_rate': 6.236906280521646e-06, 'epoch': 2.57}
{'loss': 0.3653, 'learning_rate': 6.2172746224181524e-06, 'epoch': 2.57}
{'loss': 0.3703, 'learning_rate': 6.197659962712461e-06, 'epoch': 2.57}
{'loss': 0.3693, 'learning_rate': 6.178062389546784e-06, 'epoch': 2.57}
{'loss': 0.3625, 'learning_rate': 6.158481990986558e-06, 'epoch': 2.58}
{'loss': 0.3673, 'learning_rate': 6.138918855020028e-06, 'epoch': 2.58}
{'loss': 0.3656, 'learning_rate': 6.119373069557889e-06, 'epoch': 2.58}
{'loss': 0.3595, 'learning_rate': 6.099844722432844e-06, 'epoch': 2.58}
{'loss': 0.3788, 'learning_rate': 6.080333901399252e-06, 'epoch': 2.59}
{'loss': 0.3646, 'learning_rate': 6.060840694132701e-06, 'epoch': 2.59}
{'loss': 0.3802, 'learning_rate': 6.041365188229641e-06, 'epoch': 2.59}
{'loss': 0.3416, 'learning_rate': 6.021907471206971e-06, 'epoch': 2.59}
{'loss': 0.3886, 'learning_rate': 6.002467630501646e-06, 'epoch': 2.6}
{'loss': 0.3918, 'learning_rate': 5.983045753470308e-06, 'epoch': 2.6}
{'loss': 0.3844, 'learning_rate': 5.963641927388854e-06, 'epoch': 2.6}
{'loss': 0.3844, 'learning_rate': 5.944256239452085e-06, 'epoch': 2.61}
{'loss': 0.3726, 'learning_rate': 5.924888776773281e-06, 'epoch': 2.61}
{'loss': 0.3803, 'learning_rate': 5.905539626383831e-06, 'epoch': 2.61}
{'loss': 0.3422, 'learning_rate': 5.886208875232833e-06, 'epoch': 2.61}
{'loss': 0.3681, 'learning_rate': 5.866896610186701e-06, 'epoch': 2.62}
{'loss': 0.3803, 'learning_rate': 5.847602918028785e-06, 'epoch': 2.62}
{'loss': 0.366, 'learning_rate': 5.82832788545896e-06, 'epoch': 2.62}
{'loss': 0.3866, 'learning_rate': 5.809071599093272e-06, 'epoch': 2.62}
{'loss': 0.3796, 'learning_rate': 5.789834145463506e-06, 'epoch': 2.63}
{'loss': 0.4013, 'learning_rate': 5.7706156110168384e-06, 'epoch': 2.63}
{'loss': 0.3908, 'learning_rate': 5.7514160821154085e-06, 'epoch': 2.63}
{'loss': 0.3768, 'learning_rate': 5.732235645035964e-06, 'epoch': 2.63}
{'loss': 0.3917, 'learning_rate': 5.713074385969457e-06, 'epoch': 2.64}
{'loss': 0.3823, 'learning_rate': 5.693932391020664e-06, 'epoch': 2.64}
{'loss': 0.3765, 'learning_rate': 5.67480974620778e-06, 'epoch': 2.64}
{'loss': 0.3858, 'learning_rate': 5.65570653746206e-06, 'epoch': 2.64}
{'loss': 0.3673, 'learning_rate': 5.63662285062742e-06, 'epoch': 2.65}
{'loss': 0.3838, 'learning_rate': 5.617558771460039e-06, 'epoch': 2.65}
{'loss': 0.3876, 'learning_rate': 5.598514385627997e-06, 'epoch': 2.65}
{'loss': 0.3825, 'learning_rate': 5.579489778710867e-06, 'epoch': 2.65}
{'loss': 0.3756, 'learning_rate': 5.56048503619935e-06, 'epoch': 2.66}
{'loss': 0.3673, 'learning_rate': 5.541500243494889e-06, 'epoch': 2.66}
{'loss': 0.3816, 'learning_rate': 5.522535485909258e-06, 'epoch': 2.66}
{'loss': 0.3637, 'learning_rate': 5.5035908486642195e-06, 'epoch': 2.66}
{'loss': 0.3577, 'learning_rate': 5.484666416891109e-06, 'epoch': 2.67}
{'loss': 0.3672, 'learning_rate': 5.465762275630471e-06, 'epoch': 2.67}
{'loss': 0.3735, 'learning_rate': 5.446878509831668e-06, 'epoch': 2.67}
{'loss': 0.3645, 'learning_rate': 5.428015204352508e-06, 'epoch': 2.67}
{'loss': 0.3621, 'learning_rate': 5.409172443958844e-06, 'epoch': 2.68}
{'loss': 0.3423, 'learning_rate': 5.390350313324214e-06, 'epoch': 2.68}
{'loss': 0.3821, 'learning_rate': 5.371548897029457e-06, 'epoch': 2.68}
{'loss': 0.353, 'learning_rate': 5.352768279562315e-06, 'epoch': 2.68}
{'loss': 0.378, 'learning_rate': 5.334008545317082e-06, 'epoch': 2.69}
{'loss': 0.352, 'learning_rate': 5.315269778594189e-06, 'epoch': 2.69}
{'loss': 0.3678, 'learning_rate': 5.2965520635998676e-06, 'epoch': 2.69}
{'loss': 0.3559, 'learning_rate': 5.277855484445736e-06, 'epoch': 2.69}
{'loss': 0.3828, 'learning_rate': 5.259180125148442e-06, 'epoch': 2.7}
{'loss': 0.3932, 'learning_rate': 5.240526069629265e-06, 'epoch': 2.7}
{'loss': 0.3874, 'learning_rate': 5.22189340171377e-06, 'epoch': 2.7}
{'loss': 0.3908, 'learning_rate': 5.203282205131395e-06, 'epoch': 2.71}
{'loss': 0.3731, 'learning_rate': 5.184692563515104e-06, 'epoch': 2.71}
{'loss': 0.3504, 'learning_rate': 5.166124560401002e-06, 'epoch': 2.71}
{'loss': 0.3724, 'learning_rate': 5.147578279227943e-06, 'epoch': 2.71}
{'loss': 0.3595, 'learning_rate': 5.129053803337181e-06, 'epoch': 2.72}
{'loss': 0.3831, 'learning_rate': 5.110551215971981e-06, 'epoch': 2.72}
{'loss': 0.3842, 'learning_rate': 5.0920706002772525e-06, 'epoch': 2.72}
{'loss': 0.3923, 'learning_rate': 5.073612039299157e-06, 'epoch': 2.72}
{'loss': 0.3539, 'learning_rate': 5.055175615984766e-06, 'epoch': 2.73}
{'loss': 0.3891, 'learning_rate': 5.036761413181659e-06, 'epoch': 2.73}
{'loss': 0.3639, 'learning_rate': 5.018369513637567e-06, 'epoch': 2.73}
{'loss': 0.3498, 'learning_rate': 5.000000000000003e-06, 'epoch': 2.73}
{'loss': 0.3427, 'learning_rate': 4.981652954815871e-06, 'epoch': 2.74}
{'loss': 0.3843, 'learning_rate': 4.963328460531127e-06, 'epoch': 2.74}
{'loss': 0.3662, 'learning_rate': 4.9450265994903666e-06, 'epoch': 2.74}
{'loss': 0.3609, 'learning_rate': 4.926747453936509e-06, 'epoch': 2.74}
{'loss': 0.3881, 'learning_rate': 4.908491106010368e-06, 'epoch': 2.75}
{'loss': 0.3757, 'learning_rate': 4.890257637750332e-06, 'epoch': 2.75}
{'loss': 0.3864, 'learning_rate': 4.872047131091961e-06, 'epoch': 2.75}
{'loss': 0.3691, 'learning_rate': 4.853859667867641e-06, 'epoch': 2.75}
{'loss': 0.3943, 'learning_rate': 4.8356953298062115e-06, 'epoch': 2.76}
{'loss': 0.3739, 'learning_rate': 4.817554198532582e-06, 'epoch': 2.76}
{'loss': 0.3871, 'learning_rate': 4.799436355567391e-06, 'epoch': 2.76}
{'loss': 0.3774, 'learning_rate': 4.781341882326615e-06, 'epoch': 2.76}
{'loss': 0.3881, 'learning_rate': 4.763270860121222e-06, 'epoch': 2.77}
{'loss': 0.3652, 'learning_rate': 4.745223370156797e-06, 'epoch': 2.77}
{'loss': 0.3647, 'learning_rate': 4.727199493533181e-06, 'epoch': 2.77}
{'loss': 0.3818, 'learning_rate': 4.709199311244098e-06, 'epoch': 2.77}
{'loss': 0.3594, 'learning_rate': 4.691222904176791e-06, 'epoch': 2.78}
{'loss': 0.3448, 'learning_rate': 4.673270353111687e-06, 'epoch': 2.78}
{'loss': 0.353, 'learning_rate': 4.655341738721989e-06, 'epoch': 2.78}
{'loss': 0.37, 'learning_rate': 4.63743714157335e-06, 'epoch': 2.78}
{'loss': 0.3812, 'learning_rate': 4.619556642123484e-06, 'epoch': 2.79}
{'loss': 0.3642, 'learning_rate': 4.6017003207218294e-06, 'epoch': 2.79}
{'loss': 0.3725, 'learning_rate': 4.583868257609171e-06, 'epoch': 2.79}
{'loss': 0.3473, 'learning_rate': 4.566060532917288e-06, 'epoch': 2.79}
{'loss': 0.3708, 'learning_rate': 4.548277226668585e-06, 'epoch': 2.8}
{'loss': 0.3881, 'learning_rate': 4.530518418775734e-06, 'epoch': 2.8}
{'loss': 0.379, 'learning_rate': 4.512784189041328e-06, 'epoch': 2.8}
{'loss': 0.4005, 'learning_rate': 4.4950746171575135e-06, 'epoch': 2.81}
{'loss': 0.3562, 'learning_rate': 4.477389782705628e-06, 'epoch': 2.81}
{'loss': 0.364, 'learning_rate': 4.459729765155842e-06, 'epoch': 2.81}
{'loss': 0.3626, 'learning_rate': 4.442094643866817e-06, 'epoch': 2.81}
{'loss': 0.3813, 'learning_rate': 4.424484498085335e-06, 'epoch': 2.82}
{'loss': 0.3717, 'learning_rate': 4.406899406945938e-06, 'epoch': 2.82}
{'loss': 0.3593, 'learning_rate': 4.389339449470592e-06, 'epoch': 2.82}
{'loss': 0.3905, 'learning_rate': 4.371804704568309e-06, 'epoch': 2.82}
{'loss': 0.3682, 'learning_rate': 4.354295251034811e-06, 'epoch': 2.83}
{'loss': 0.3833, 'learning_rate': 4.336811167552167e-06, 'epoch': 2.83}
{'loss': 0.3714, 'learning_rate': 4.319352532688444e-06, 'epoch': 2.83}
{'loss': 0.3757, 'learning_rate': 4.301919424897339e-06, 'epoch': 2.83}
{'loss': 0.3832, 'learning_rate': 4.284511922517853e-06, 'epoch': 2.84}
{'loss': 0.355, 'learning_rate': 4.267130103773911e-06, 'epoch': 2.84}
{'loss': 0.3941, 'learning_rate': 4.249774046774034e-06, 'epoch': 2.84}
{'loss': 0.3579, 'learning_rate': 4.232443829510977e-06, 'epoch': 2.84}
{'loss': 0.3482, 'learning_rate': 4.2151395298613675e-06, 'epoch': 2.85}
{'loss': 0.3803, 'learning_rate': 4.19786122558538e-06, 'epoch': 2.85}
{'loss': 0.3312, 'learning_rate': 4.180608994326371e-06, 'epoch': 2.85}
{'loss': 0.3837, 'learning_rate': 4.163382913610533e-06, 'epoch': 2.85}
{'loss': 0.3622, 'learning_rate': 4.1461830608465385e-06, 'epoch': 2.86}
{'loss': 0.3751, 'learning_rate': 4.129009513325211e-06, 'epoch': 2.86}
{'loss': 0.3627, 'learning_rate': 4.111862348219158e-06, 'epoch': 2.86}
{'loss': 0.3627, 'learning_rate': 4.0947416425824334e-06, 'epoch': 2.86}
{'loss': 0.3708, 'learning_rate': 4.077647473350201e-06, 'epoch': 2.87}
{'loss': 0.3605, 'learning_rate': 4.060579917338362e-06, 'epoch': 2.87}
{'loss': 0.3913, 'learning_rate': 4.04353905124324e-06, 'epoch': 2.87}
{'loss': 0.3695, 'learning_rate': 4.026524951641204e-06, 'epoch': 2.87}
{'loss': 0.3571, 'learning_rate': 4.009537694988372e-06, 'epoch': 2.88}
{'loss': 0.3624, 'learning_rate': 3.99257735762021e-06, 'epoch': 2.88}
{'loss': 0.3622, 'learning_rate': 3.975644015751234e-06, 'epoch': 2.88}
{'loss': 0.3679, 'learning_rate': 3.958737745474638e-06, 'epoch': 2.88}
{'loss': 0.3724, 'learning_rate': 3.941858622761975e-06, 'epoch': 2.89}
{'loss': 0.3856, 'learning_rate': 3.925006723462806e-06, 'epoch': 2.89}
{'loss': 0.3827, 'learning_rate': 3.908182123304344e-06, 'epoch': 2.89}
{'loss': 0.3512, 'learning_rate': 3.891384897891148e-06, 'epoch': 2.89}
{'loss': 0.3472, 'learning_rate': 3.8746151227047455e-06, 'epoch': 2.9}
{'loss': 0.3541, 'learning_rate': 3.857872873103322e-06, 'epoch': 2.9}
{'loss': 0.352, 'learning_rate': 3.84115822432137e-06, 'epoch': 2.9}
{'loss': 0.3821, 'learning_rate': 3.824471251469352e-06, 'epoch': 2.91}
{'loss': 0.358, 'learning_rate': 3.807812029533362e-06, 'epoch': 2.91}
{'loss': 0.3785, 'learning_rate': 3.7911806333747847e-06, 'epoch': 2.91}
{'loss': 0.3556, 'learning_rate': 3.7745771377299758e-06, 'epoch': 2.91}
{'loss': 0.3449, 'learning_rate': 3.7580016172099067e-06, 'epoch': 2.92}
{'loss': 0.3615, 'learning_rate': 3.7414541462998446e-06, 'epoch': 2.92}
{'loss': 0.3758, 'learning_rate': 3.724934799358998e-06, 'epoch': 2.92}
{'loss': 0.3621, 'learning_rate': 3.708443650620206e-06, 'epoch': 2.92}
{'loss': 0.3675, 'learning_rate': 3.691980774189591e-06, 'epoch': 2.93}
{'loss': 0.3816, 'learning_rate': 3.6755462440462288e-06, 'epoch': 2.93}
{'loss': 0.3575, 'learning_rate': 3.659140134041812e-06, 'epoch': 2.93}
{'loss': 0.3613, 'learning_rate': 3.6427625179003223e-06, 'epoch': 2.93}
{'loss': 0.3693, 'learning_rate': 3.626413469217702e-06, 'epoch': 2.94}
{'loss': 0.3807, 'learning_rate': 3.6100930614615204e-06, 'epoch': 2.94}
{'loss': 0.3767, 'learning_rate': 3.5938013679706453e-06, 'epoch': 2.94}
{'loss': 0.3436, 'learning_rate': 3.5775384619549e-06, 'epoch': 2.94}
{'loss': 0.3562, 'learning_rate': 3.5613044164947617e-06, 'epoch': 2.95}
{'loss': 0.3736, 'learning_rate': 3.5450993045409997e-06, 'epoch': 2.95}
{'loss': 0.3543, 'learning_rate': 3.5289231989143868e-06, 'epoch': 2.95}
{'loss': 0.3391, 'learning_rate': 3.5127761723053313e-06, 'epoch': 2.95}
{'loss': 0.3482, 'learning_rate': 3.496658297273574e-06, 'epoch': 2.96}
{'loss': 0.4039, 'learning_rate': 3.4805696462478634e-06, 'epoch': 2.96}
{'loss': 0.3547, 'learning_rate': 3.46451029152562e-06, 'epoch': 2.96}
{'loss': 0.3706, 'learning_rate': 3.448480305272619e-06, 'epoch': 2.96}
{'loss': 0.3797, 'learning_rate': 3.4324797595226567e-06, 'epoch': 2.97}
{'loss': 0.374, 'learning_rate': 3.41650872617724e-06, 'epoch': 2.97}
{'loss': 0.3481, 'learning_rate': 3.400567277005247e-06, 'epoch': 2.97}
{'loss': 0.3659, 'learning_rate': 3.384655483642624e-06, 'epoch': 2.97}
{'loss': 0.3454, 'learning_rate': 3.3687734175920505e-06, 'epoch': 2.98}
{'loss': 0.3827, 'learning_rate': 3.352921150222612e-06, 'epoch': 2.98}
{'loss': 0.3635, 'learning_rate': 3.337098752769503e-06, 'epoch': 2.98}
{'loss': 0.3556, 'learning_rate': 3.321306296333673e-06, 'epoch': 2.98}
{'loss': 0.355, 'learning_rate': 3.3055438518815486e-06, 'epoch': 2.99}
{'loss': 0.358, 'learning_rate': 3.2898114902446708e-06, 'epoch': 2.99}
{'loss': 0.3402, 'learning_rate': 3.274109282119413e-06, 'epoch': 2.99}
{'loss': 0.3406, 'learning_rate': 3.2584372980666344e-06, 'epoch': 2.99}
{'loss': 0.3545, 'learning_rate': 3.2427956085113875e-06, 'epoch': 3.0}
{'loss': 0.3491, 'learning_rate': 3.2271842837425917e-06, 'epoch': 3.0}
75%|███████▌ | 2340/3120 [3:14:45<1:02:44, 4.83s/it]/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
{'loss': 0.1364, 'learning_rate': 3.2116033939127024e-06, 'epoch': 3.0}
{'loss': 0.1197, 'learning_rate': 3.1960530090374277e-06, 'epoch': 3.01}
{'loss': 0.1254, 'learning_rate': 3.180533198995379e-06, 'epoch': 3.01}
{'loss': 0.1163, 'learning_rate': 3.165044033527789e-06, 'epoch': 3.01}
{'loss': 0.1316, 'learning_rate': 3.1495855822381716e-06, 'epoch': 3.01}
{'loss': 0.119, 'learning_rate': 3.134157914592032e-06, 'epoch': 3.02}
{'loss': 0.1231, 'learning_rate': 3.1187610999165317e-06, 'epoch': 3.02}
{'loss': 0.1362, 'learning_rate': 3.1033952074001882e-06, 'epoch': 3.02}
{'loss': 0.1191, 'learning_rate': 3.088060306092582e-06, 'epoch': 3.02}
{'loss': 0.126, 'learning_rate': 3.0727564649040066e-06, 'epoch': 3.03}
{'loss': 0.1242, 'learning_rate': 3.057483752605196e-06, 'epoch': 3.03}
{'loss': 0.1365, 'learning_rate': 3.042242237826991e-06, 'epoch': 3.03}
{'loss': 0.1274, 'learning_rate': 3.0270319890600465e-06, 'epoch': 3.03}
{'loss': 0.1334, 'learning_rate': 3.011853074654515e-06, 'epoch': 3.04}
{'loss': 0.1344, 'learning_rate': 2.996705562819747e-06, 'epoch': 3.04}
{'loss': 0.1246, 'learning_rate': 2.981589521623973e-06, 'epoch': 3.04}
{'loss': 0.1302, 'learning_rate': 2.9665050189940013e-06, 'epoch': 3.04}
{'loss': 0.1188, 'learning_rate': 2.951452122714926e-06, 'epoch': 3.05}
{'loss': 0.132, 'learning_rate': 2.936430900429805e-06, 'epoch': 3.05}
{'loss': 0.1304, 'learning_rate': 2.9214414196393702e-06, 'epoch': 3.05}
{'loss': 0.1438, 'learning_rate': 2.906483747701705e-06, 'epoch': 3.05}
{'loss': 0.1328, 'learning_rate': 2.8915579518319626e-06, 'epoch': 3.06}
{'loss': 0.1251, 'learning_rate': 2.8766640991020533e-06, 'epoch': 3.06}
{'loss': 0.1324, 'learning_rate': 2.861802256440348e-06, 'epoch': 3.06}
{'loss': 0.1249, 'learning_rate': 2.8469724906313677e-06, 'epoch': 3.06}
{'loss': 0.123, 'learning_rate': 2.8321748683154893e-06, 'epoch': 3.07}
{'loss': 0.1236, 'learning_rate': 2.8174094559886535e-06, 'epoch': 3.07}
{'loss': 0.1277, 'learning_rate': 2.8026763200020557e-06, 'epoch': 3.07}
{'loss': 0.1212, 'learning_rate': 2.7879755265618558e-06, 'epoch': 3.07}
{'loss': 0.1219, 'learning_rate': 2.773307141728867e-06, 'epoch': 3.08}
{'loss': 0.1199, 'learning_rate': 2.7586712314182775e-06, 'epoch': 3.08}
{'loss': 0.134, 'learning_rate': 2.744067861399333e-06, 'epoch': 3.08}
{'loss': 0.1293, 'learning_rate': 2.729497097295075e-06, 'epoch': 3.08}
{'loss': 0.1308, 'learning_rate': 2.714959004582003e-06, 'epoch': 3.09}
{'loss': 0.1208, 'learning_rate': 2.700453648589805e-06, 'epoch': 3.09}
{'loss': 0.1265, 'learning_rate': 2.6859810945010687e-06, 'epoch': 3.09}
{'loss': 0.1356, 'learning_rate': 2.671541407350975e-06, 'epoch': 3.09}
{'loss': 0.1307, 'learning_rate': 2.6571346520270147e-06, 'epoch': 3.1}
{'loss': 0.1225, 'learning_rate': 2.642760893268684e-06, 'epoch': 3.1}
{'loss': 0.128, 'learning_rate': 2.628420195667214e-06, 'epoch': 3.1}
{'loss': 0.1261, 'learning_rate': 2.614112623665259e-06, 'epoch': 3.11}
{'loss': 0.1141, 'learning_rate': 2.5998382415566258e-06, 'epoch': 3.11}
{'loss': 0.1226, 'learning_rate': 2.5855971134859736e-06, 'epoch': 3.11}
{'loss': 0.122, 'learning_rate': 2.5713893034485216e-06, 'epoch': 3.11}
{'loss': 0.1346, 'learning_rate': 2.55721487528978e-06, 'epoch': 3.12}
{'loss': 0.1393, 'learning_rate': 2.5430738927052346e-06, 'epoch': 3.12}
{'loss': 0.1241, 'learning_rate': 2.5289664192400996e-06, 'epoch': 3.12}
{'loss': 0.1317, 'learning_rate': 2.514892518288988e-06, 'epoch': 3.12}
{'loss': 0.1216, 'learning_rate': 2.5008522530956647e-06, 'epoch': 3.13}
{'loss': 0.1126, 'learning_rate': 2.4868456867527315e-06, 'epoch': 3.13}
{'loss': 0.1228, 'learning_rate': 2.4728728822013683e-06, 'epoch': 3.13}
{'loss': 0.1331, 'learning_rate': 2.4589339022310386e-06, 'epoch': 3.13}
{'loss': 0.1179, 'learning_rate': 2.445028809479203e-06, 'epoch': 3.14}
{'loss': 0.1303, 'learning_rate': 2.431157666431052e-06, 'epoch': 3.14}
{'loss': 0.1261, 'learning_rate': 2.4173205354192065e-06, 'epoch': 3.14}
{'loss': 0.1229, 'learning_rate': 2.403517478623456e-06, 'epoch': 3.14}
{'loss': 0.1267, 'learning_rate': 2.3897485580704684e-06, 'epoch': 3.15}
{'loss': 0.1336, 'learning_rate': 2.3760138356335172e-06, 'epoch': 3.15}
{'loss': 0.1331, 'learning_rate': 2.3623133730321923e-06, 'epoch': 3.15}
{'loss': 0.1206, 'learning_rate': 2.348647231832131e-06, 'epoch': 3.15}
{'loss': 0.1218, 'learning_rate': 2.335015473444754e-06, 'epoch': 3.16}
{'loss': 0.1079, 'learning_rate': 2.3214181591269603e-06, 'epoch': 3.16}
{'loss': 0.1292, 'learning_rate': 2.30785534998088e-06, 'epoch': 3.16}
{'loss': 0.1182, 'learning_rate': 2.2943271069535754e-06, 'epoch': 3.16}
{'loss': 0.1204, 'learning_rate': 2.2808334908367914e-06, 'epoch': 3.17}
{'loss': 0.1376, 'learning_rate': 2.267374562266662e-06, 'epoch': 3.17}
{'loss': 0.1348, 'learning_rate': 2.2539503817234553e-06, 'epoch': 3.17}
{'loss': 0.1212, 'learning_rate': 2.240561009531281e-06, 'epoch': 3.17}
{'loss': 0.1257, 'learning_rate': 2.227206505857834e-06, 'epoch': 3.18}
{'loss': 0.1218, 'learning_rate': 2.2138869307141266e-06, 'epoch': 3.18}
{'loss': 0.1254, 'learning_rate': 2.200602343954209e-06, 'epoch': 3.18}
{'loss': 0.1212, 'learning_rate': 2.1873528052749094e-06, 'epoch': 3.18}
{'loss': 0.1241, 'learning_rate': 2.1741383742155476e-06, 'epoch': 3.19}
{'loss': 0.1176, 'learning_rate': 2.1609591101576945e-06, 'epoch': 3.19}
{'loss': 0.1265, 'learning_rate': 2.147815072324886e-06, 'epoch': 3.19}
{'loss': 0.1251, 'learning_rate': 2.1347063197823648e-06, 'epoch': 3.19}
{'loss': 0.1243, 'learning_rate': 2.1216329114368083e-06, 'epoch': 3.2}
{'loss': 0.136, 'learning_rate': 2.1085949060360654e-06, 'epoch': 3.2}
{'loss': 0.1278, 'learning_rate': 2.0955923621689034e-06, 'epoch': 3.2}
{'loss': 0.1275, 'learning_rate': 2.0826253382647334e-06, 'epoch': 3.21}
{'loss': 0.136, 'learning_rate': 2.0696938925933505e-06, 'epoch': 3.21}
{'loss': 0.1118, 'learning_rate': 2.056798083264667e-06, 'epoch': 3.21}
{'loss': 0.1247, 'learning_rate': 2.0439379682284686e-06, 'epoch': 3.21}
{'loss': 0.1344, 'learning_rate': 2.0311136052741274e-06, 'epoch': 3.22}
{'loss': 0.123, 'learning_rate': 2.0183250520303653e-06, 'epoch': 3.22}
{'loss': 0.113, 'learning_rate': 2.0055723659649907e-06, 'epoch': 3.22}
{'loss': 0.1266, 'learning_rate': 1.9928556043846215e-06, 'epoch': 3.22}
{'loss': 0.1277, 'learning_rate': 1.9801748244344587e-06, 'epoch': 3.23}
{'loss': 0.1283, 'learning_rate': 1.967530083097996e-06, 'epoch': 3.23}
{'loss': 0.1212, 'learning_rate': 1.9549214371968008e-06, 'epoch': 3.23}
{'loss': 0.1186, 'learning_rate': 1.9423489433902186e-06, 'epoch': 3.23}
{'loss': 0.118, 'learning_rate': 1.9298126581751542e-06, 'epoch': 3.24}
{'loss': 0.1188, 'learning_rate': 1.917312637885791e-06, 'epoch': 3.24}
{'loss': 0.1348, 'learning_rate': 1.9048489386933545e-06, 'epoch': 3.24}
{'loss': 0.1427, 'learning_rate': 1.8924216166058574e-06, 'epoch': 3.24}
{'loss': 0.1329, 'learning_rate': 1.8800307274678364e-06, 'epoch': 3.25}
{'loss': 0.125, 'learning_rate': 1.8676763269601195e-06, 'epoch': 3.25}
{'loss': 0.1309, 'learning_rate': 1.8553584705995564e-06, 'epoch': 3.25}
{'loss': 0.129, 'learning_rate': 1.8430772137387853e-06, 'epoch': 3.25}
{'loss': 0.1235, 'learning_rate': 1.8308326115659757e-06, 'epoch': 3.26}
{'loss': 0.1207, 'learning_rate': 1.8186247191045857e-06, 'epoch': 3.26}
{'loss': 0.1282, 'learning_rate': 1.8064535912131032e-06, 'epoch': 3.26}
{'loss': 0.1158, 'learning_rate': 1.7943192825848133e-06, 'epoch': 3.26}
{'loss': 0.1288, 'learning_rate': 1.7822218477475496e-06, 'epoch': 3.27}
{'loss': 0.1245, 'learning_rate': 1.7701613410634367e-06, 'epoch': 3.27}
{'loss': 0.1259, 'learning_rate': 1.7581378167286655e-06, 'epoch': 3.27}
{'loss': 0.1144, 'learning_rate': 1.7461513287732313e-06, 'epoch': 3.27}
{'loss': 0.1242, 'learning_rate': 1.7342019310607062e-06, 'epoch': 3.28}
{'loss': 0.1128, 'learning_rate': 1.7222896772879871e-06, 'epoch': 3.28}
{'loss': 0.1197, 'learning_rate': 1.7104146209850591e-06, 'epoch': 3.28}
{'loss': 0.1195, 'learning_rate': 1.6985768155147498e-06, 'epoch': 3.28}
{'loss': 0.1182, 'learning_rate': 1.6867763140724969e-06, 'epoch': 3.29}
{'loss': 0.1248, 'learning_rate': 1.6750131696861016e-06, 'epoch': 3.29}
{'loss': 0.1273, 'learning_rate': 1.6632874352154982e-06, 'epoch': 3.29}
{'loss': 0.1213, 'learning_rate': 1.6515991633525118e-06, 'epoch': 3.29}
{'loss': 0.1178, 'learning_rate': 1.6399484066206183e-06, 'epoch': 3.3}
{'loss': 0.1275, 'learning_rate': 1.6283352173747148e-06, 'epoch': 3.3}
{'loss': 0.136, 'learning_rate': 1.6167596478008817e-06, 'epoch': 3.3}
{'loss': 0.1204, 'learning_rate': 1.6052217499161516e-06, 'epoch': 3.31}
{'loss': 0.1268, 'learning_rate': 1.5937215755682667e-06, 'epoch': 3.31}
{'loss': 0.1272, 'learning_rate': 1.5822591764354512e-06, 'epoch': 3.31}
{'loss': 0.1187, 'learning_rate': 1.5708346040261812e-06, 'epoch': 3.31}
{'loss': 0.1303, 'learning_rate': 1.559447909678954e-06, 'epoch': 3.32}
{'loss': 0.1261, 'learning_rate': 1.5480991445620541e-06, 'epoch': 3.32}
{'loss': 0.1215, 'learning_rate': 1.5367883596733157e-06, 'epoch': 3.32}
{'loss': 0.1355, 'learning_rate': 1.5255156058399124e-06, 'epoch': 3.32}
{'loss': 0.13, 'learning_rate': 1.5142809337181063e-06, 'epoch': 3.33}
{'loss': 0.1094, 'learning_rate': 1.5030843937930485e-06, 'epoch': 3.33}
{'loss': 0.1251, 'learning_rate': 1.4919260363785215e-06, 'epoch': 3.33}
{'loss': 0.1266, 'learning_rate': 1.4808059116167306e-06, 'epoch': 3.33}
{'loss': 0.1176, 'learning_rate': 1.4697240694780769e-06, 'epoch': 3.34}
{'loss': 0.1426, 'learning_rate': 1.4586805597609333e-06, 'epoch': 3.34}
{'loss': 0.1218, 'learning_rate': 1.4476754320914187e-06, 'epoch': 3.34}
{'loss': 0.1146, 'learning_rate': 1.4367087359231668e-06, 'epoch': 3.34}
{'loss': 0.1221, 'learning_rate': 1.4257805205371233e-06, 'epoch': 3.35}
{'loss': 0.1175, 'learning_rate': 1.4148908350413048e-06, 'epoch': 3.35}
{'loss': 0.1182, 'learning_rate': 1.4040397283705898e-06, 'epoch': 3.35}
{'loss': 0.1177, 'learning_rate': 1.3932272492864984e-06, 'epoch': 3.35}
{'loss': 0.1311, 'learning_rate': 1.3824534463769633e-06, 'epoch': 3.36}
{'loss': 0.1197, 'learning_rate': 1.3717183680561253e-06, 'epoch': 3.36}
{'loss': 0.1288, 'learning_rate': 1.3610220625641002e-06, 'epoch': 3.36}
{'loss': 0.1307, 'learning_rate': 1.3503645779667852e-06, 'epoch': 3.36}
{'loss': 0.1291, 'learning_rate': 1.339745962155613e-06, 'epoch': 3.37}
{'loss': 0.13, 'learning_rate': 1.3291662628473634e-06, 'epoch': 3.37}
{'loss': 0.1334, 'learning_rate': 1.3186255275839266e-06, 'epoch': 3.37}
{'loss': 0.1305, 'learning_rate': 1.308123803732111e-06, 'epoch': 3.37}
{'loss': 0.1264, 'learning_rate': 1.2976611384834148e-06, 'epoch': 3.38}
{'loss': 0.1207, 'learning_rate': 1.2872375788538171e-06, 'epoch': 3.38}
{'loss': 0.1173, 'learning_rate': 1.2768531716835754e-06, 'epoch': 3.38}
{'loss': 0.1223, 'learning_rate': 1.266507963636997e-06, 'epoch': 3.38}
{'loss': 0.1222, 'learning_rate': 1.2562020012022512e-06, 'epoch': 3.39}
{'loss': 0.1214, 'learning_rate': 1.2459353306911438e-06, 'epoch': 3.39}
{'loss': 0.1133, 'learning_rate': 1.2357079982389197e-06, 'epoch': 3.39}
{'loss': 0.1208, 'learning_rate': 1.2255200498040432e-06, 'epoch': 3.39}
{'loss': 0.1127, 'learning_rate': 1.2153715311680048e-06, 'epoch': 3.4}
{'loss': 0.1262, 'learning_rate': 1.2052624879351105e-06, 'epoch': 3.4}
{'loss': 0.1257, 'learning_rate': 1.1951929655322692e-06, 'epoch': 3.4}
{'loss': 0.1334, 'learning_rate': 1.1851630092088051e-06, 'epoch': 3.41}
{'loss': 0.1286, 'learning_rate': 1.175172664036235e-06, 'epoch': 3.41}
{'loss': 0.1316, 'learning_rate': 1.1652219749080817e-06, 'epoch': 3.41}
{'loss': 0.1305, 'learning_rate': 1.155310986539665e-06, 'epoch': 3.41}
{'loss': 0.1179, 'learning_rate': 1.1454397434679022e-06, 'epoch': 3.42}
{'loss': 0.1289, 'learning_rate': 1.1356082900511023e-06, 'epoch': 3.42}
{'loss': 0.1197, 'learning_rate': 1.12581667046878e-06, 'epoch': 3.42}
{'loss': 0.1154, 'learning_rate': 1.116064928721442e-06, 'epoch': 3.42}
{'loss': 0.1292, 'learning_rate': 1.1063531086304003e-06, 'epoch': 3.43}
{'loss': 0.1225, 'learning_rate': 1.0966812538375738e-06, 'epoch': 3.43}
{'loss': 0.1136, 'learning_rate': 1.0870494078052796e-06, 'epoch': 3.43}
{'loss': 0.1207, 'learning_rate': 1.0774576138160596e-06, 'epoch': 3.43}
{'loss': 0.1334, 'learning_rate': 1.067905914972468e-06, 'epoch': 3.44}
{'loss': 0.136, 'learning_rate': 1.0583943541968856e-06, 'epoch': 3.44}
{'loss': 0.1279, 'learning_rate': 1.0489229742313223e-06, 'epoch': 3.44}
{'loss': 0.1199, 'learning_rate': 1.0394918176372327e-06, 'epoch': 3.44}
{'loss': 0.1309, 'learning_rate': 1.0301009267953145e-06, 'epoch': 3.45}
{'loss': 0.1151, 'learning_rate': 1.0207503439053279e-06, 'epoch': 3.45}
{'loss': 0.1052, 'learning_rate': 1.0114401109859019e-06, 'epoch': 3.45}
{'loss': 0.1137, 'learning_rate': 1.0021702698743408e-06, 'epoch': 3.45}
{'loss': 0.1258, 'learning_rate': 9.929408622264448e-07, 'epoch': 3.46}
{'loss': 0.1157, 'learning_rate': 9.837519295163155e-07, 'epoch': 3.46}
{'loss': 0.1131, 'learning_rate': 9.746035130361741e-07, 'epoch': 3.46}
{'loss': 0.1176, 'learning_rate': 9.65495653896179e-07, 'epoch': 3.46}
{'loss': 0.1165, 'learning_rate': 9.564283930242258e-07, 'epoch': 3.47}
{'loss': 0.1168, 'learning_rate': 9.474017711657835e-07, 'epoch': 3.47}
{'loss': 0.1243, 'learning_rate': 9.384158288836987e-07, 'epoch': 3.47}
{'loss': 0.1134, 'learning_rate': 9.294706065580172e-07, 'epoch': 3.47}
{'loss': 0.1404, 'learning_rate': 9.205661443857994e-07, 'epoch': 3.48}
{'loss': 0.1233, 'learning_rate': 9.117024823809462e-07, 'epoch': 3.48}
{'loss': 0.1295, 'learning_rate': 9.028796603740097e-07, 'epoch': 3.48}
{'loss': 0.1089, 'learning_rate': 8.940977180120247e-07, 'epoch': 3.48}
{'loss': 0.1252, 'learning_rate': 8.853566947583259e-07, 'epoch': 3.49}
{'loss': 0.1215, 'learning_rate': 8.766566298923651e-07, 'epoch': 3.49}
{'loss': 0.1299, 'learning_rate': 8.67997562509546e-07, 'epoch': 3.49}
{'loss': 0.1278, 'learning_rate': 8.593795315210341e-07, 'epoch': 3.49}
{'loss': 0.1151, 'learning_rate': 8.508025756535987e-07, 'epoch': 3.5}
{'loss': 0.1218, 'learning_rate': 8.42266733449425e-07, 'epoch': 3.5}
{'loss': 0.1245, 'learning_rate': 8.337720432659513e-07, 'epoch': 3.5}
{'loss': 0.1219, 'learning_rate': 8.253185432756816e-07, 'epoch': 3.51}
{'loss': 0.1172, 'learning_rate': 8.169062714660347e-07, 'epoch': 3.51}
{'loss': 0.1283, 'learning_rate': 8.085352656391576e-07, 'epoch': 3.51}
{'loss': 0.1163, 'learning_rate': 8.002055634117578e-07, 'epoch': 3.51}
{'loss': 0.1344, 'learning_rate': 7.919172022149458e-07, 'epoch': 3.52}
{'loss': 0.1333, 'learning_rate': 7.836702192940493e-07, 'epoch': 3.52}
{'loss': 0.1251, 'learning_rate': 7.754646517084585e-07, 'epoch': 3.52}
{'loss': 0.1139, 'learning_rate': 7.673005363314578e-07, 'epoch': 3.52}
{'loss': 0.1332, 'learning_rate': 7.591779098500584e-07, 'epoch': 3.53}
{'loss': 0.1195, 'learning_rate': 7.510968087648262e-07, 'epoch': 3.53}
{'loss': 0.1266, 'learning_rate': 7.430572693897342e-07, 'epoch': 3.53}
{'loss': 0.1197, 'learning_rate': 7.350593278519824e-07, 'epoch': 3.53}
{'loss': 0.1265, 'learning_rate': 7.271030200918483e-07, 'epoch': 3.54}
{'loss': 0.1323, 'learning_rate': 7.19188381862519e-07, 'epoch': 3.54}
{'loss': 0.1295, 'learning_rate': 7.113154487299267e-07, 'epoch': 3.54}
{'loss': 0.1257, 'learning_rate': 7.034842560726008e-07, 'epoch': 3.54}
{'loss': 0.1181, 'learning_rate': 6.956948390814977e-07, 'epoch': 3.55}
{'loss': 0.1172, 'learning_rate': 6.879472327598502e-07, 'epoch': 3.55}
{'loss': 0.1159, 'learning_rate': 6.802414719230021e-07, 'epoch': 3.55}
{'loss': 0.1198, 'learning_rate': 6.725775911982602e-07, 'epoch': 3.55}
{'loss': 0.1183, 'learning_rate': 6.649556250247325e-07, 'epoch': 3.56}
{'loss': 0.123, 'learning_rate': 6.573756076531779e-07, 'epoch': 3.56}
{'loss': 0.1302, 'learning_rate': 6.498375731458529e-07, 'epoch': 3.56}
{'loss': 0.1213, 'learning_rate': 6.423415553763479e-07, 'epoch': 3.56}
{'loss': 0.1143, 'learning_rate': 6.348875880294536e-07, 'epoch': 3.57}
{'loss': 0.127, 'learning_rate': 6.274757046009871e-07, 'epoch': 3.57}
{'loss': 0.1211, 'learning_rate': 6.201059383976671e-07, 'epoch': 3.57}
{'loss': 0.123, 'learning_rate': 6.127783225369377e-07, 'epoch': 3.57}
{'loss': 0.1254, 'learning_rate': 6.054928899468427e-07, 'epoch': 3.58}
{'loss': 0.1124, 'learning_rate': 5.982496733658582e-07, 'epoch': 3.58}
{'loss': 0.1232, 'learning_rate': 5.910487053427616e-07, 'epoch': 3.58}
{'loss': 0.123, 'learning_rate': 5.83890018236476e-07, 'epoch': 3.58}
{'loss': 0.1266, 'learning_rate': 5.767736442159256e-07, 'epoch': 3.59}
{'loss': 0.1382, 'learning_rate': 5.696996152598966e-07, 'epoch': 3.59}
{'loss': 0.1271, 'learning_rate': 5.626679631568832e-07, 'epoch': 3.59}
{'loss': 0.1204, 'learning_rate': 5.556787195049573e-07, 'epoch': 3.59}
{'loss': 0.1105, 'learning_rate': 5.487319157116178e-07, 'epoch': 3.6}
{'loss': 0.1189, 'learning_rate': 5.418275829936537e-07, 'epoch': 3.6}
{'loss': 0.1171, 'learning_rate': 5.349657523769991e-07, 'epoch': 3.6}
{'loss': 0.128, 'learning_rate': 5.281464546965953e-07, 'epoch': 3.61}
{'loss': 0.1279, 'learning_rate': 5.213697205962631e-07, 'epoch': 3.61}
{'loss': 0.1265, 'learning_rate': 5.146355805285452e-07, 'epoch': 3.61}
{'loss': 0.1182, 'learning_rate': 5.079440647545852e-07, 'epoch': 3.61}
{'loss': 0.1316, 'learning_rate': 5.012952033439844e-07, 'epoch': 3.62}
{'loss': 0.1137, 'learning_rate': 4.946890261746706e-07, 'epoch': 3.62}
{'loss': 0.1154, 'learning_rate': 4.881255629327608e-07, 'epoch': 3.62}
{'loss': 0.1171, 'learning_rate': 4.816048431124265e-07, 'epoch': 3.62}
{'loss': 0.1264, 'learning_rate': 4.7512689601576843e-07, 'epoch': 3.63}
{'loss': 0.1184, 'learning_rate': 4.686917507526756e-07, 'epoch': 3.63}
{'loss': 0.1295, 'learning_rate': 4.6229943624069963e-07, 'epoch': 3.63}
{'loss': 0.1232, 'learning_rate': 4.5594998120492505e-07, 'epoch': 3.63}
{'loss': 0.1172, 'learning_rate': 4.4964341417784165e-07, 'epoch': 3.64}
{'loss': 0.1111, 'learning_rate': 4.4337976349920763e-07, 'epoch': 3.64}
{'loss': 0.1286, 'learning_rate': 4.3715905731593233e-07, 'epoch': 3.64}
{'loss': 0.1141, 'learning_rate': 4.309813235819471e-07, 'epoch': 3.64}
{'loss': 0.1203, 'learning_rate': 4.248465900580734e-07, 'epoch': 3.65}
{'loss': 0.117, 'learning_rate': 4.1875488431190934e-07, 'epoch': 3.65}
{'loss': 0.1156, 'learning_rate': 4.127062337176935e-07, 'epoch': 3.65}
{'loss': 0.1264, 'learning_rate': 4.0670066545619224e-07, 'epoch': 3.65}
{'loss': 0.1208, 'learning_rate': 4.0073820651457043e-07, 'epoch': 3.66}
{'loss': 0.1171, 'learning_rate': 3.9481888368627764e-07, 'epoch': 3.66}
{'loss': 0.118, 'learning_rate': 3.889427235709153e-07, 'epoch': 3.66}
{'loss': 0.1444, 'learning_rate': 3.8310975257413343e-07, 'epoch': 3.66}
{'loss': 0.1221, 'learning_rate': 3.773199969074959e-07, 'epoch': 3.67}
{'loss': 0.1296, 'learning_rate': 3.715734825883766e-07, 'epoch': 3.67}
{'loss': 0.109, 'learning_rate': 3.658702354398325e-07, 'epoch': 3.67}
{'loss': 0.1116, 'learning_rate': 3.6021028109049063e-07, 'epoch': 3.67}
{'loss': 0.1114, 'learning_rate': 3.5459364497443696e-07, 'epoch': 3.68}
{'loss': 0.1193, 'learning_rate': 3.4902035233109644e-07, 'epoch': 3.68}
{'loss': 0.1159, 'learning_rate': 3.4349042820512325e-07, 'epoch': 3.68}
{'loss': 0.1081, 'learning_rate': 3.38003897446284e-07, 'epoch': 3.68}
{'loss': 0.1281, 'learning_rate': 3.325607847093537e-07, 'epoch': 3.69}
{'loss': 0.1353, 'learning_rate': 3.2716111445399324e-07, 'epoch': 3.69}
{'loss': 0.121, 'learning_rate': 3.2180491094465414e-07, 'epoch': 3.69}
{'loss': 0.126, 'learning_rate': 3.164921982504565e-07, 'epoch': 3.69}
{'loss': 0.126, 'learning_rate': 3.112230002450889e-07, 'epoch': 3.7}
{'loss': 0.1314, 'learning_rate': 3.059973406066963e-07, 'epoch': 3.7}
{'loss': 0.1231, 'learning_rate': 3.0081524281777687e-07, 'epoch': 3.7}
{'loss': 0.1258, 'learning_rate': 2.9567673016507646e-07, 'epoch': 3.71}
{'loss': 0.1364, 'learning_rate': 2.905818257394799e-07, 'epoch': 3.71}
{'loss': 0.1183, 'learning_rate': 2.855305524359131e-07, 'epoch': 3.71}
{'loss': 0.1205, 'learning_rate': 2.805229329532344e-07, 'epoch': 3.71}
{'loss': 0.1203, 'learning_rate': 2.7555898979413796e-07, 'epoch': 3.72}
{'loss': 0.1065, 'learning_rate': 2.706387452650494e-07, 'epoch': 3.72}
{'loss': 0.1149, 'learning_rate': 2.657622214760247e-07, 'epoch': 3.72}
{'loss': 0.1283, 'learning_rate': 2.609294403406537e-07, 'epoch': 3.72}
{'loss': 0.1139, 'learning_rate': 2.56140423575959e-07, 'epoch': 3.73}
{'loss': 0.1173, 'learning_rate': 2.513951927023017e-07, 'epoch': 3.73}
{'loss': 0.1178, 'learning_rate': 2.4669376904328244e-07, 'epoch': 3.73}
{'loss': 0.1232, 'learning_rate': 2.420361737256438e-07, 'epoch': 3.73}
{'loss': 0.1231, 'learning_rate': 2.3742242767917921e-07, 'epoch': 3.74}
{'loss': 0.1208, 'learning_rate': 2.3285255163663535e-07, 'epoch': 3.74}
{'loss': 0.1088, 'learning_rate': 2.2832656613362092e-07, 'epoch': 3.74}
{'loss': 0.1195, 'learning_rate': 2.2384449150851695e-07, 'epoch': 3.74}
{'loss': 0.1203, 'learning_rate': 2.1940634790238003e-07, 'epoch': 3.75}
{'loss': 0.1131, 'learning_rate': 2.1501215525885245e-07, 'epoch': 3.75}
{'loss': 0.1247, 'learning_rate': 2.1066193332407891e-07, 'epoch': 3.75}
{'loss': 0.1202, 'learning_rate': 2.063557016466111e-07, 'epoch': 3.75}
{'loss': 0.1264, 'learning_rate': 2.0209347957732328e-07, 'epoch': 3.76}
{'loss': 0.1156, 'learning_rate': 1.978752862693212e-07, 'epoch': 3.76}
{'loss': 0.1261, 'learning_rate': 1.9370114067785995e-07, 'epoch': 3.76}
{'loss': 0.1253, 'learning_rate': 1.8957106156026084e-07, 'epoch': 3.76}
{'loss': 0.1213, 'learning_rate': 1.854850674758213e-07, 'epoch': 3.77}
{'loss': 0.1227, 'learning_rate': 1.8144317678573497e-07, 'epoch': 3.77}
{'loss': 0.1287, 'learning_rate': 1.7744540765300743e-07, 'epoch': 3.77}
{'loss': 0.1139, 'learning_rate': 1.7349177804237837e-07, 'epoch': 3.77}
{'loss': 0.126, 'learning_rate': 1.6958230572023504e-07, 'epoch': 3.78}
{'loss': 0.1169, 'learning_rate': 1.6571700825453674e-07, 'epoch': 3.78}
{'loss': 0.1208, 'learning_rate': 1.618959030147349e-07, 'epoch': 3.78}
{'loss': 0.1237, 'learning_rate': 1.5811900717169537e-07, 'epoch': 3.78}
{'loss': 0.1337, 'learning_rate': 1.5438633769761845e-07, 'epoch': 3.79}
{'loss': 0.1102, 'learning_rate': 1.506979113659679e-07, 'epoch': 3.79}
{'loss': 0.1215, 'learning_rate': 1.4705374475138978e-07, 'epoch': 3.79}
{'loss': 0.1297, 'learning_rate': 1.4345385422964043e-07, 'epoch': 3.79}
{'loss': 0.1351, 'learning_rate': 1.398982559775175e-07, 'epoch': 3.8}
{'loss': 0.1203, 'learning_rate': 1.3638696597277678e-07, 'epoch': 3.8}
{'loss': 0.117, 'learning_rate': 1.3291999999407e-07, 'epoch': 3.8}
{'loss': 0.1234, 'learning_rate': 1.2949737362087156e-07, 'epoch': 3.81}
{'loss': 0.1173, 'learning_rate': 1.2611910223340408e-07, 'epoch': 3.81}
{'loss': 0.1239, 'learning_rate': 1.227852010125752e-07, 'epoch': 3.81}
{'loss': 0.1202, 'learning_rate': 1.1949568493990428e-07, 'epoch': 3.81}
{'loss': 0.1185, 'learning_rate': 1.1625056879746133e-07, 'epoch': 3.82}
{'loss': 0.1214, 'learning_rate': 1.1304986716779265e-07, 'epoch': 3.82}
{'loss': 0.1281, 'learning_rate': 1.0989359443386305e-07, 'epoch': 3.82}
{'loss': 0.1163, 'learning_rate': 1.0678176477898372e-07, 'epoch': 3.82}
{'loss': 0.1177, 'learning_rate': 1.0371439218675671e-07, 'epoch': 3.83}
{'loss': 0.121, 'learning_rate': 1.0069149044100502e-07, 'epoch': 3.83}
{'loss': 0.1335, 'learning_rate': 9.771307312571254e-08, 'epoch': 3.83}
{'loss': 0.1128, 'learning_rate': 9.47791536249676e-08, 'epoch': 3.83}
{'loss': 0.1182, 'learning_rate': 9.188974512289617e-08, 'epoch': 3.84}
{'loss': 0.1181, 'learning_rate': 8.90448606036054e-08, 'epoch': 3.84}
{'loss': 0.1223, 'learning_rate': 8.624451285112689e-08, 'epoch': 3.84}
{'loss': 0.1284, 'learning_rate': 8.348871444935902e-08, 'epoch': 3.84}
{'loss': 0.1234, 'learning_rate': 8.077747778200474e-08, 'epoch': 3.85}
{'loss': 0.1184, 'learning_rate': 7.811081503252382e-08, 'epoch': 3.85}
{'loss': 0.1162, 'learning_rate': 7.54887381840752e-08, 'epoch': 3.85}
{'loss': 0.1188, 'learning_rate': 7.291125901946027e-08, 'epoch': 3.85}
{'loss': 0.1242, 'learning_rate': 7.037838912107298e-08, 'epoch': 3.86}
{'loss': 0.1298, 'learning_rate': 6.789013987084647e-08, 'epoch': 3.86}
{'loss': 0.1172, 'learning_rate': 6.544652245020433e-08, 'epoch': 3.86}
{'loss': 0.1113, 'learning_rate': 6.304754784000721e-08, 'epoch': 3.86}
{'loss': 0.1147, 'learning_rate': 6.069322682050516e-08, 'epoch': 3.87}
{'loss': 0.1156, 'learning_rate': 5.838356997128869e-08, 'epoch': 3.87}
{'loss': 0.124, 'learning_rate': 5.611858767124001e-08, 'epoch': 3.87}
{'loss': 0.1228, 'learning_rate': 5.389829009849079e-08, 'epoch': 3.87}
{'loss': 0.1131, 'learning_rate': 5.1722687230369995e-08, 'epoch': 3.88}
{'loss': 0.1222, 'learning_rate': 4.959178884336502e-08, 'epoch': 3.88}
{'loss': 0.1182, 'learning_rate': 4.7505604513072845e-08, 'epoch': 3.88}
{'loss': 0.1168, 'learning_rate': 4.5464143614162294e-08, 'epoch': 3.88}
{'loss': 0.1135, 'learning_rate': 4.346741532032628e-08, 'epoch': 3.89}
{'loss': 0.1099, 'learning_rate': 4.151542860424407e-08, 'epoch': 3.89}
{'loss': 0.1169, 'learning_rate': 3.96081922375402e-08, 'epoch': 3.89}
{'loss': 0.1141, 'learning_rate': 3.7745714790745623e-08, 'epoch': 3.89}
{'loss': 0.1189, 'learning_rate': 3.592800463325663e-08, 'epoch': 3.9}
{'loss': 0.1184, 'learning_rate': 3.4155069933301535e-08, 'epoch': 3.9}
{'loss': 0.1265, 'learning_rate': 3.242691865790071e-08, 'epoch': 3.9}
{'loss': 0.1156, 'learning_rate': 3.074355857282996e-08, 'epoch': 3.91}
{'loss': 0.1207, 'learning_rate': 2.9104997242590528e-08, 'epoch': 3.91}
{'loss': 0.1205, 'learning_rate': 2.7511242030368035e-08, 'epoch': 3.91}
{'loss': 0.1271, 'learning_rate': 2.5962300098008042e-08, 'epoch': 3.91}
{'loss': 0.1281, 'learning_rate': 2.4458178405974974e-08, 'epoch': 3.92}
{'loss': 0.124, 'learning_rate': 2.2998883713326592e-08, 'epoch': 3.92}
{'loss': 0.1141, 'learning_rate': 2.1584422577682894e-08, 'epoch': 3.92}
{'loss': 0.1134, 'learning_rate': 2.0214801355192826e-08, 'epoch': 3.92}
{'loss': 0.1182, 'learning_rate': 1.889002620051206e-08, 'epoch': 3.93}
{'loss': 0.1066, 'learning_rate': 1.761010306676969e-08, 'epoch': 3.93}
{'loss': 0.1203, 'learning_rate': 1.6375037705543827e-08, 'epoch': 3.93}
{'loss': 0.1186, 'learning_rate': 1.518483566683826e-08, 'epoch': 3.93}
{'loss': 0.1259, 'learning_rate': 1.4039502299050266e-08, 'epoch': 3.94}
{'loss': 0.1161, 'learning_rate': 1.2939042748955078e-08, 'epoch': 3.94}
{'loss': 0.1165, 'learning_rate': 1.1883461961678112e-08, 'epoch': 3.94}
{'loss': 0.116, 'learning_rate': 1.0872764680671666e-08, 'epoch': 3.94}
{'loss': 0.1178, 'learning_rate': 9.906955447697153e-09, 'epoch': 3.95}
{'loss': 0.1203, 'learning_rate': 8.986038602802894e-09, 'epoch': 3.95}
{'loss': 0.1177, 'learning_rate': 8.110018284304132e-09, 'epoch': 3.95}
{'loss': 0.117, 'learning_rate': 7.278898428764169e-09, 'epoch': 3.95}
{'loss': 0.1251, 'learning_rate': 6.492682770979919e-09, 'epoch': 3.96}
{'loss': 0.1221, 'learning_rate': 5.751374843961932e-09, 'epoch': 3.96}
{'loss': 0.1227, 'learning_rate': 5.054977978916631e-09, 'epoch': 3.96}
{'loss': 0.1306, 'learning_rate': 4.403495305237426e-09, 'epoch': 3.96}
{'loss': 0.1129, 'learning_rate': 3.7969297504858445e-09, 'epoch': 3.97}
{'loss': 0.1208, 'learning_rate': 3.2352840403804264e-09, 'epoch': 3.97}
{'loss': 0.1283, 'learning_rate': 2.7185606987811808e-09, 'epoch': 3.97}
{'loss': 0.1171, 'learning_rate': 2.246762047685147e-09, 'epoch': 3.97}
{'loss': 0.1164, 'learning_rate': 1.8198902072097402e-09, 'epoch': 3.98}
{'loss': 0.1199, 'learning_rate': 1.437947095582759e-09, 'epoch': 3.98}
{'loss': 0.1165, 'learning_rate': 1.1009344291401657e-09, 'epoch': 3.98}
{'loss': 0.1148, 'learning_rate': 8.088537223116533e-10, 'epoch': 3.98}
{'loss': 0.1127, 'learning_rate': 5.617062876195345e-10, 'epoch': 3.99}
{'loss': 0.1129, 'learning_rate': 3.594932356654202e-10, 'epoch': 3.99}
{'loss': 0.1164, 'learning_rate': 2.02215475132439e-10, 'epoch': 3.99}
{'loss': 0.1217, 'learning_rate': 8.987371278079693e-11, 'epoch': 3.99}
{'loss': 0.1212, 'learning_rate': 2.246845343667481e-11, 'epoch': 4.0}
{'loss': 0.1224, 'learning_rate': 0.0, 'epoch': 4.0}
100%|██████████| 3120/3120 [4:19:47<00:00, 4.72s/it]/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
{'train_runtime': 15617.9007, 'train_samples_per_second': 12.785, 'train_steps_per_second': 0.2, 'train_loss': 0.610505388624584, 'epoch': 4.0}
100%|██████████| 3120/3120 [4:20:17<00:00, 5.01s/it]
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
[2024-04-21 04:02:35,619] [INFO] [launch.py:348:main] Process 274 exits successfully.
[2024-04-21 04:02:36,620] [INFO] [launch.py:348:main] Process 271 exits successfully.
[2024-04-21 04:02:36,620] [INFO] [launch.py:348:main] Process 268 exits successfully.
[2024-04-21 04:02:36,620] [INFO] [launch.py:348:main] Process 273 exits successfully.
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
[2024-04-21 04:02:37,621] [INFO] [launch.py:348:main] Process 270 exits successfully.
[2024-04-21 04:02:37,622] [INFO] [launch.py:348:main] Process 272 exits successfully.
[2024-04-21 04:02:37,622] [INFO] [launch.py:348:main] Process 269 exits successfully.
[2024-04-21 04:02:39,624] [INFO] [launch.py:348:main] Process 267 exits successfully.
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[2024-04-20 23:35:43,623] [WARNING] [runner.py:202:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only.
[2024-04-20 23:35:43,624] [INFO] [runner.py:568:main] cmd = /opt/conda/bin/python3.8 -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMCwgMSwgMiwgMywgNCwgNSwgNiwgN119 --master_addr=127.0.0.1 --master_port=29500 --enable_each_rank_log=None train.py --model_name_or_path /workspace/qwen2_4b/ --data_path refgpt_train.json --output_dir qwen2_4b_4epoch_refgpt --num_train_epochs 4 --model_max_length 2048 --per_device_train_batch_size 1 --per_device_eval_batch_size 2 --gradient_accumulation_steps 8 --evaluation_strategy no --save_strategy steps --save_steps 780 --save_total_limit 10 --learning_rate 2e-5 --warmup_ratio 0.05 --logging_steps 2 --lr_scheduler_type cosine --report_to tensorboard --bf16 True --deepspeed ds_config.json --use_lora False
[2024-04-20 23:35:44,554] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_TC=136
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.9.9-1+cuda11.3
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_DEV_PACKAGE_VERSION=2.9.9-1
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_MIN_NCHANNELS=4
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_NET_PLUGIN=none
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_VERSION=2.12.10
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_SOCKET_IFNAME=eth
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_DEBUG=INFO
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_HCA=mlx5
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_GID_INDEX=3
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_QPS_PER_CONNECTION=8
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_TIMEOUT=22
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NCCL_IB_SL=5
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_PACKAGE=libnccl2=2.9.9-1+cuda11.3
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_PACKAGE_NAME=libnccl2
[2024-04-20 23:35:46,123] [INFO] [launch.py:138:main] 0 NV_LIBNCCL_PACKAGE_VERSION=2.9.9-1
[2024-04-20 23:35:46,123] [INFO] [launch.py:145:main] WORLD INFO DICT: {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]}
[2024-04-20 23:35:46,123] [INFO] [launch.py:151:main] nnodes=1, num_local_procs=8, node_rank=0
[2024-04-20 23:35:46,123] [INFO] [launch.py:162:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]})
[2024-04-20 23:35:46,123] [INFO] [launch.py:163:main] dist_world_size=8
[2024-04-20 23:35:46,123] [INFO] [launch.py:165:main] Setting CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
[2024-04-20 23:35:46,128] [INFO] [launch.py:253:main] process 267 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=0', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,131] [INFO] [launch.py:253:main] process 268 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=1', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,134] [INFO] [launch.py:253:main] process 269 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=2', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,137] [INFO] [launch.py:253:main] process 270 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=3', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,140] [INFO] [launch.py:253:main] process 271 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=4', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,143] [INFO] [launch.py:253:main] process 272 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=5', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,147] [INFO] [launch.py:253:main] process 273 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=6', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:46,151] [INFO] [launch.py:253:main] process 274 spawned with command: ['/opt/conda/bin/python3.8', '-u', 'train.py', '--local_rank=7', '--model_name_or_path', '/workspace/qwen2_4b/', '--data_path', 'refgpt_train.json', '--output_dir', 'qwen2_4b_4epoch_refgpt', '--num_train_epochs', '4', '--model_max_length', '2048', '--per_device_train_batch_size', '1', '--per_device_eval_batch_size', '2', '--gradient_accumulation_steps', '8', '--evaluation_strategy', 'no', '--save_strategy', 'steps', '--save_steps', '780', '--save_total_limit', '10', '--learning_rate', '2e-5', '--warmup_ratio', '0.05', '--logging_steps', '2', '--lr_scheduler_type', 'cosine', '--report_to', 'tensorboard', '--bf16', 'True', '--deepspeed', 'ds_config.json', '--use_lora', 'False']
[2024-04-20 23:35:49,984] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:49,990] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:49,999] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,015] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,036] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,050] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,054] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-04-20 23:35:50,067] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[WARNING] using untested triton version (2.3.0), only 1.0.0 is known to be compatible
[2024-04-20 23:35:52,257] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-04-20 23:35:52,261] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-04-20 23:35:52,261] [INFO] [comm.py:668:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
[2024-04-20 23:35:52,266] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 6
[2024-04-20 23:35:52,300] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-04-20 23:35:52,301] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 4
[INFO] [2024-04-20 23:35:52] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 7
[2024-04-20 23:35:52,309] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 2
[2024-04-20 23:35:52,325] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 5
[2024-04-20 23:35:52,340] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO] [2024-04-20 23:35:52] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 1
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 3
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Added key: store_based_barrier_key:1 to store for rank: 0
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 3: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 5: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 6: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 1: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 4: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 2: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
[INFO] [2024-04-20 23:35:53] distributed_c10d.py : Rank 7: Completed store-based barrier for key:store_based_barrier_key:1 with 8 nodes.
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.24s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.43s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.39s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.35s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.13s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.22s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.20s/it]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Loading checkpoint shards: 100%|██████████| 2/2 [00:24<00:00, 12.20s/it]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
[INFO] [2024-04-20 23:36:35] train.py : Formatting inputs...
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Plugin name set by env to libnccl-net-none.so
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Using network IB
NCCL version 2.12.10+cuda11.3
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO Plugin name set by env to libnccl-net-none.so
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO Plugin name set by env to libnccl-net-none.so
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO Plugin name set by env to libnccl-net-none.so
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO Plugin name set by env to libnccl-net-none.so
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO Plugin name set by env to libnccl-net-none.so
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO Plugin name set by env to libnccl-net-none.so
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO Bootstrap : Using eth0:22.3.117.217<0>
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO Plugin name set by env to libnccl-net-none.so
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO NET/Plugin : No plugin found (libnccl-net-none.so), using internal implementation
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libsiw-rdmav25.so': libsiw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhns-rdmav25.so': libhns-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libcxgb4-rdmav25.so': libcxgb4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmthca-rdmav25.so': libmthca-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libhfi1verbs-rdmav25.so': libhfi1verbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'librxe-rdmav25.so': librxe-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libi40iw-rdmav25.so': libi40iw-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libvmw_pvrdma-rdmav25.so': libvmw_pvrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libocrdma-rdmav25.so': libocrdma-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libipathverbs-rdmav25.so': libipathverbs-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libbnxt_re-rdmav25.so': libbnxt_re-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libqedr-rdmav25.so': libqedr-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libmlx4-rdmav25.so': libmlx4-rdmav25.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'libefa-rdmav25.so': libefa-rdmav25.so: cannot open shared object file: No such file or directory
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:274:274 [7] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:272:272 [5] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:271:271 [4] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:268:268 [1] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:269:269 [2] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:270:270 [3] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE [1]mlx5_1:1/RoCE [2]mlx5_2:1/RoCE [3]mlx5_3:1/RoCE [RO]; OOB eth0:22.3.117.217<0>
dlcpq1dza0ye5yti-master-0:273:273 [6] NCCL INFO Using network IB
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Setting affinity for GPU 0 to 0fffff
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Setting affinity for GPU 2 to 0fffff
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Setting affinity for GPU 3 to 0fffff
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Setting affinity for GPU 1 to 0fffff
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO NCCL_MIN_NCHANNELS set by environment to 4.
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 00/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 01/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 02/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 03/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 04/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 05/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 06/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 07/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 08/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 09/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 10/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 11/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 12/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 13/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 14/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 15/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 16/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 17/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 18/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 19/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 20/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 21/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 22/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 23/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 00 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 00 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 00 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 00 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 00 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 00 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 01 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 01 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 01 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 01 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 01 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 01 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 02 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 02 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 02 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 02 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 02 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 00 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 02 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 03 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 03 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 03 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 03 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 01 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 03 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 00 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 04 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 04 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 04 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 04 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 02 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 04 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 01 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 05 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 05 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 05 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 05 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 03 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 05 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 06 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 02 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 06 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 06 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 03 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 04 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 06 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 06 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 07 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 07 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 07 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 04 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 05 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 03 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 07 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 07 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 08 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 08 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 05 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 08 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 04 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 06 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 08 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 08 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 09 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 09 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 09 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 06 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 05 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 07 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 09 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 10 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 09 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 10 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 10 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 07 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 08 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 06 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 10 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 11 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 10 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 11 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 11 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 08 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 09 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 07 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 11 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 11 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 12 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 12 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 09 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 12 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 10 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 08 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 12 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 13 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 12 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 13 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 13 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 09 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 10 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 11 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 14 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 14 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 13 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 14 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 10 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 11 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 15 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 14 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 15 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 11 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 12 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 16 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 15 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 16 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 12 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 13 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 17 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 16 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 17 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 14 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 13 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 18 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 17 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 18 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 15 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 14 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 19 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 18 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 19 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 16 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 20 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 13 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 15 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 15 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 19 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 17 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 20 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 21 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 12 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 14 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 16 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 18 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 21 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 20 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 22 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 17 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 19 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 21 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 22 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 23 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 18 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 20 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Channel 23 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 19 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 21 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 20 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 22 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 21 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 23 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 22 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 23 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 22 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 23 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 13 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 16 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 15 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 14 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 17 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 15 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 18 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 16 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 19 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 16 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 17 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 20 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 17 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 18 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 21 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 18 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 19 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 22 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 19 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 20 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 23 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 20 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 21 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 21 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 22 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 22 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 23 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 23 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 00 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 01 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 02 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 03 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 04 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 05 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 06 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 07 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 08 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 09 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 10 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 11 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 12 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 13 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 00 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 14 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 01 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 15 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 02 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 16 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 03 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 17 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 04 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 18 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 05 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 19 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 06 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 20 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 07 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 21 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 08 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 22 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 09 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Channel 23 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 10 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 11 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 12 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 13 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 00 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 00 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 00 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 00 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 14 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 01 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 00 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 01 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 01 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 01 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 01 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 02 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 15 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 02 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 02 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 02 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 02 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 03 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 16 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 03 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 03 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 03 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 03 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 04 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 17 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 04 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 04 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 04 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 04 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 18 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 05 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 05 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 05 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 05 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 19 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 06 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 06 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 06 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 06 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 20 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 07 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 07 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 07 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 07 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 21 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 08 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 08 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 05 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 08 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 22 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 08 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 09 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 09 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 06 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 09 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Channel 23 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 10 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 09 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 10 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 07 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 10 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 10 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 11 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 11 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 08 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 11 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 12 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 12 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 11 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 09 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 12 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 13 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 13 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 13 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 12 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 14 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 14 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 14 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 13 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 15 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 15 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 15 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 14 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 16 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 16 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 16 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 15 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 17 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 17 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 17 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 16 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 18 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 18 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 18 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 17 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 19 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 19 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 18 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 19 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 10 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 20 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 20 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 19 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 21 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 20 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 21 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 20 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 22 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 21 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 22 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 21 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Channel 23 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 22 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Channel 23 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 22 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Channel 23 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Channel 23 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 11 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 12 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 13 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 14 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 15 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 16 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 17 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 18 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 19 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 20 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 21 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 22 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Channel 23 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:272:1036 [5] NCCL INFO comm 0x7f3ffc008fb0 rank 5 nranks 8 cudaDev 5 busId 60 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:270:1040 [3] NCCL INFO comm 0x7f3114008fb0 rank 3 nranks 8 cudaDev 3 busId 40 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:268:1038 [1] NCCL INFO comm 0x7f3794008fb0 rank 1 nranks 8 cudaDev 1 busId 20 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:274:1029 [7] NCCL INFO comm 0x7f4f34008fb0 rank 7 nranks 8 cudaDev 7 busId 80 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:269:1039 [2] NCCL INFO comm 0x7f53a0008fb0 rank 2 nranks 8 cudaDev 2 busId 30 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:267:1006 [0] NCCL INFO comm 0x7fed4c008fb0 rank 0 nranks 8 cudaDev 0 busId 10 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:273:1041 [6] NCCL INFO comm 0x7fa874008fb0 rank 6 nranks 8 cudaDev 6 busId 70 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Launch mode Parallel
dlcpq1dza0ye5yti-master-0:271:1037 [4] NCCL INFO comm 0x7f4758008fb0 rank 4 nranks 8 cudaDev 4 busId 50 - Init COMPLETE
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Data Num: 49920
[INFO] [2024-04-20 23:38:55] train.py : Formatting inputs...
[INFO] [2024-04-20 23:38:55] train.py : Sample 34992 of the training set: {'input_ids': tensor([104600, 12, 44636, 102430, 100280, 102064, 11095, 15843, 71,
1174, 15459, 58463, 99449, 100280, 102064, 104995, 16, 18,
15, 86402, 102064, 107339, 72881, 70769, 3837, 17714, 58463,
99449, 33108, 109033, 104995, 23, 11, 15, 15, 15,
102266, 31838, 37029, 1773, 106676, 72881, 9370, 101468, 105570,
42140, 34204, 92894, 58463, 99449, 100280, 102064, 101468, 9370,
59743, 33108, 1773, 92894, 101194, 101468, 109824, 29524, 106676,
106758, 104995, 16, 15, 15, 102266, 105899, 101705, 72881,
9909, 39120, 644, 48272, 44636, 102430, 72881, 3837, 23,
15, 102266, 105899, 44956, 99727, 72881, 9909, 42, 84,
352, 48272, 112964, 106835, 33108, 105906, 112545, 23, 15,
102266, 105899, 104600, 72881, 1773, 18493, 104152, 104600, 12,
44636, 102430, 100280, 102064, 15946, 3837, 101043, 104600, 72881,
5373, 44636, 102430, 72881, 33108, 106676, 72881, 18493, 16,
24, 101186, 103982, 18830, 108201, 100169, 1773, 26940, 99470,
104600, 72881, 25067, 99461, 33956, 33477, 61443, 34204, 22,
101186, 3837, 20412, 101888, 58463, 99449, 113228, 105871, 82832,
3837, 103957, 112964, 17340, 103993, 109755, 9909, 73743, 99915,
11095, 25079, 33108, 1425, 292, 4378, 5942, 74276, 104600,
12, 44636, 102430, 100280, 102064, 9370, 101460, 105761, 104363,
20412, 101514, 99740, 78685, 3837, 99740, 78685, 116966, 100405,
104610, 99278, 102357, 70074, 9370, 119301, 119301, 111947, 111947,
9370, 102124, 90395, 100136, 104250, 70074, 47872, 102478, 1773,
33975, 25, 220, 108386, 37945, 56007, 107809, 104169, 99526,
99526, 104600, 12, 44636, 102430, 100280, 102064, 101037, 26850,
71703, 25, 220, 108386, 3837, 103942, 73670, 1773, 104600,
12, 44636, 102430, 100280, 102064, 101909, 67071, 104995, 16,
18, 15, 86402, 102064, 107339, 72881, 70769, 3837, 99558,
114021, 58463, 99449, 33108, 109033, 100361, 3837, 104678, 104995,
23, 15, 15, 15, 102266, 18493, 37029, 100001, 102064,
1773, 90919, 3837, 106676, 72881, 37029, 104346, 102538, 3837,
99427, 71304, 92894, 58463, 99449, 100280, 102064, 9370, 59743,
33108, 1773, 110075, 37029, 104346, 106204, 109824, 100630, 101705,
72881, 9909, 39120, 644, 3837, 94237, 16, 15, 15,
102266, 37029, 3837, 99558, 18493, 106676, 106758, 64359, 44636,
102430, 72881, 5373, 44956, 99727, 72881, 9909, 42, 84,
352, 3837, 94237, 23, 15, 102266, 37029, 7552, 101034,
104600, 72881, 9909, 18493, 112964, 106835, 33108, 105906, 112545,
18830, 23, 15, 102266, 37029, 74276, 18493, 100001, 104600,
12, 44636, 102430, 100280, 102064, 15946, 3837, 104600, 72881,
5373, 44636, 102430, 72881, 33108, 106676, 72881, 18830, 108201,
100169, 3837, 73670, 109234, 26939, 16, 24, 101186, 103982,
1773, 100654, 100146, 91111, 99470, 104600, 72881, 25067, 114292,
101888, 58463, 99449, 113228, 105871, 82832, 3837, 105985, 22,
101186, 104435, 65577, 3837, 103957, 99250, 112964, 17340, 109755,
1773, 116880, 3837, 104600, 12, 44636, 102430, 100280, 102064,
9370, 101460, 105761, 115093, 103926, 101514, 99740, 78685, 5373,
116966, 100405, 100136, 16530, 99278, 102357, 70074, 3837, 91572,
104250, 70074, 47872, 102478, 3407, 33975, 25, 49434, 239,
104958, 104600, 12, 44636, 102430, 100280, 102064, 118619, 99740,
78685, 37945, 56007, 100001, 99740, 78685, 104139, 100772, 11319,
106998, 64471, 33108, 109023, 99604, 26850, 71703, 25, 54851,
9370, 3837, 104600, 12, 44636, 102430, 100280, 102064, 104877,
109023, 99883, 3837, 100629, 33126, 104653, 99740, 78685, 1773,
100001, 99740, 78685, 9370, 116966, 115390, 3837, 102119, 16530,
111240, 102357, 70074, 3837, 105884, 116966, 13343, 102274, 99792,
103148, 3837, 106085, 109023, 101047, 111817, 78685, 3837, 29524,
2073, 89, 33590, 2073, 23815, 854, 49567, 116966, 13343,
107689, 118216, 102073, 102357, 70074, 1773, 104043, 3837, 104600,
12, 44636, 102430, 100280, 102064, 18493, 116966, 13343, 102119,
110921, 106050, 70074, 47872, 102478, 3837, 105064, 33108, 109023,
103943, 100687, 102665, 3837, 99519, 109023, 101158, 70074, 47872,
102064, 3837, 101970, 70074, 47872, 73670, 51463, 101970, 99689,
64559, 1773, 106279, 3837, 104600, 12, 44636, 102430, 100280,
102064, 9370, 99740, 78685, 100772, 33108, 109023, 101208, 3837,
99558, 20742, 99601, 100733, 33071, 5373, 116966, 9370, 100405,
33071, 101034, 104250, 70074, 47872, 102478, 17447, 1773, 151643]), 'labels': tensor([ -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
33975, 25, 220, 108386, 37945, 56007, 107809, 104169, 99526,
99526, 104600, 12, 44636, 102430, 100280, 102064, 101037, 26850,
71703, 25, 220, 108386, 3837, 103942, 73670, 1773, 104600,
12, 44636, 102430, 100280, 102064, 101909, 67071, 104995, 16,
18, 15, 86402, 102064, 107339, 72881, 70769, 3837, 99558,
114021, 58463, 99449, 33108, 109033, 100361, 3837, 104678, 104995,
23, 15, 15, 15, 102266, 18493, 37029, 100001, 102064,
1773, 90919, 3837, 106676, 72881, 37029, 104346, 102538, 3837,
99427, 71304, 92894, 58463, 99449, 100280, 102064, 9370, 59743,
33108, 1773, 110075, 37029, 104346, 106204, 109824, 100630, 101705,
72881, 9909, 39120, 644, 3837, 94237, 16, 15, 15,
102266, 37029, 3837, 99558, 18493, 106676, 106758, 64359, 44636,
102430, 72881, 5373, 44956, 99727, 72881, 9909, 42, 84,
352, 3837, 94237, 23, 15, 102266, 37029, 7552, 101034,
104600, 72881, 9909, 18493, 112964, 106835, 33108, 105906, 112545,
18830, 23, 15, 102266, 37029, 74276, 18493, 100001, 104600,
12, 44636, 102430, 100280, 102064, 15946, 3837, 104600, 72881,
5373, 44636, 102430, 72881, 33108, 106676, 72881, 18830, 108201,
100169, 3837, 73670, 109234, 26939, 16, 24, 101186, 103982,
1773, 100654, 100146, 91111, 99470, 104600, 72881, 25067, 114292,
101888, 58463, 99449, 113228, 105871, 82832, 3837, 105985, 22,
101186, 104435, 65577, 3837, 103957, 99250, 112964, 17340, 109755,
1773, 116880, 3837, 104600, 12, 44636, 102430, 100280, 102064,
9370, 101460, 105761, 115093, 103926, 101514, 99740, 78685, 5373,
116966, 100405, 100136, 16530, 99278, 102357, 70074, 3837, 91572,
104250, 70074, 47872, 102478, 3407, 33975, 25, 49434, 239,
104958, 104600, 12, 44636, 102430, 100280, 102064, 118619, 99740,
78685, 37945, 56007, 100001, 99740, 78685, 104139, 100772, 11319,
106998, 64471, 33108, 109023, 99604, 26850, 71703, 25, 54851,
9370, 3837, 104600, 12, 44636, 102430, 100280, 102064, 104877,
109023, 99883, 3837, 100629, 33126, 104653, 99740, 78685, 1773,
100001, 99740, 78685, 9370, 116966, 115390, 3837, 102119, 16530,
111240, 102357, 70074, 3837, 105884, 116966, 13343, 102274, 99792,
103148, 3837, 106085, 109023, 101047, 111817, 78685, 3837, 29524,
2073, 89, 33590, 2073, 23815, 854, 49567, 116966, 13343,
107689, 118216, 102073, 102357, 70074, 1773, 104043, 3837, 104600,
12, 44636, 102430, 100280, 102064, 18493, 116966, 13343, 102119,
110921, 106050, 70074, 47872, 102478, 3837, 105064, 33108, 109023,
103943, 100687, 102665, 3837, 99519, 109023, 101158, 70074, 47872,
102064, 3837, 101970, 70074, 47872, 73670, 51463, 101970, 99689,
64559, 1773, 106279, 3837, 104600, 12, 44636, 102430, 100280,
102064, 9370, 99740, 78685, 100772, 33108, 109023, 101208, 3837,
99558, 20742, 99601, 100733, 33071, 5373, 116966, 9370, 100405,
33071, 101034, 104250, 70074, 47872, 102478, 17447, 1773, 151643])}, torch.Size([603]).
孟-高棉诸语言Mon-Khmer languages南亚诸语言大约130种语言组成的语族，为南亚和东南亚大约8,000万人所使用。越南语的使用者远远多于其他南亚诸语言使用者的总和。其他有很多使用者的语言如越南北部大约100万人使用的芒语（Muong），高棉语，80万人使用的库阿语（Kuay），缅甸南部和泰国部分地区80万人使用的孟语。在所有的孟-高棉诸语言中，只有孟语、高棉语和越南语在19世纪以前有书写传统。《古孟语》已经确证写于7世纪，是关于南亚起源的手稿，后来缅甸人进行了改编（参阅Mon kingdom和Indic writing systems）。孟-高棉诸语言的典型语音特征是大量的母音，母音发音简单而不是带呼吸声的叽叽嘎嘎的表达，并且缺乏声调差异。Human: 你好，请问你能给我讲讲孟-高棉诸语言吗？
Assistant: 你好，当然可以。孟-高棉诸语言是一个由大约130种语言组成的语族，主要分布在南亚和东南亚地区，共有大约8000万人在使用这些语言。其中，越南语使用人数最多，远超其他南亚诸语言的总和。还有一些使用人数较多的语言包括芒语（Muong，约100万人使用，主要在越南北部）、高棉语、库阿语（Kuay，约80万人使用）以及孟语（在缅甸南部和泰国部分地区有80万人使用）。在这些孟-高棉诸语言中，孟语、高棉语和越南语有书写传统，可以追溯到19世纪以前。特别的是，《古孟语》是一部关于南亚起源的手稿，早在7世纪就有记录，后来被缅甸人改编。总的来说，孟-高棉诸语言的典型语音特点是拥有大量的母音、发音简单且不带呼吸声，同时缺乏声调差异。
Human: 我了解到孟-高棉诸语言有很多种母音，请问这些母音有什么特点？比如说是否和汉语不同？
Assistant: 是的，孟-高棉诸语言对比汉语来说，具有更丰富的母音。这些母音的发音比较简单，通常不伴随呼吸声，也就是说发音时声音比较单纯，不像汉语中的浊音，如“z”，“zh”等发音时伴随着喉咙里的呼吸声。此外，孟-高棉诸语言在发音时通常不会有明显的声调差异，这一点和汉语也有明显区别，因为汉语是一种声调语言，不同的声调可以表示不同的词义。总之，孟-高棉诸语言的母音特点和汉语相比，主要表现在丰富性、发音的简单性以及缺乏声调差异上。<|endoftext|>
[INFO] [2024-04-20 23:38:55] train.py : Sample 46308 of the training set: {'input_ids': tensor([100349, 102028, 85361, 112272, 16, 24, 17, 23, 21216,
17, 15, 15, 15, 7948, 53, 329, 318, 320,
47, 3433, 72, 1020, 1579, 859, 701, 28507, 104328,
100378, 104727, 1773, 16, 24, 19, 15, 104227, 107886,
3837, 42411, 99992, 30709, 101430, 101193, 102508, 99418, 104534,
3837, 103922, 99364, 113259, 100378, 103132, 1773, 102888, 3837,
42411, 104247, 26940, 120036, 99686, 100027, 25067, 9909, 28085,
3891, 665, 11, 220, 16, 24, 20, 18, 7552,
104195, 9370, 106575, 1773, 42411, 115999, 62926, 100656, 108598,
34187, 101251, 105651, 62945, 38035, 100378, 26940, 106020, 100211,
101989, 25067, 9909, 3036, 4264, 4290, 24511, 11, 220,
16, 24, 20, 21, 48272, 75882, 34718, 108438, 100648,
102375, 102461, 107309, 109452, 120411, 110803, 106411, 106681, 1773,
103957, 3837, 42411, 14777, 87256, 100152, 99487, 67279, 99959,
16530, 25710, 9370, 99619, 8863, 1773, 42411, 11622, 99497,
116170, 102627, 9909, 2082, 78728, 794, 3352, 7725, 7552,
100172, 26940, 104264, 100145, 25067, 7, 97215, 782, 83657,
19379, 11, 220, 16, 24, 20, 24, 69515, 11622,
100861, 102261, 93488, 100172, 26940, 108481, 82847, 113724, 25067,
9909, 3428, 54102, 35782, 11, 220, 16, 24, 21,
23, 48272, 62926, 11622, 104156, 100841, 105115, 100721, 105658,
103639, 100172, 26940, 100397, 99244, 108571, 99232, 25067, 9909,
53, 558, 323, 47775, 361, 11, 220, 16, 24,
21, 17, 48272, 24562, 105352, 100132, 100648, 102375, 3837,
100161, 101961, 104428, 100648, 103430, 1773, 33975, 25, 220,
14880, 106525, 100349, 102028, 13935, 101190, 100222, 101316, 105518,
26850, 71703, 25, 10236, 121, 245, 102028, 13935, 101190,
100222, 101316, 9909, 78425, 92112, 318, 3837, 35987, 13072,
53, 329, 318, 393, 3433, 72, 1020, 1579, 859,
7552, 109182, 104328, 100378, 104727, 3837, 100648, 111978, 114984,
17, 15, 101186, 19, 15, 104227, 1773, 102965, 3837,
42411, 107709, 102508, 99418, 17447, 104247, 104534, 3837, 101889,
106438, 100378, 100650, 1773, 100648, 104727, 103132, 104847, 100694,
19108, 3837, 104033, 16, 24, 20, 21, 104300, 100378,
26940, 106020, 100211, 101989, 25067, 9909, 3036, 4264, 4290,
24511, 48272, 99652, 108438, 42411, 102375, 102461, 107309, 13935,
109452, 120411, 99787, 110803, 106411, 106681, 1773, 104043, 3837,
106473, 115999, 106650, 92894, 102280, 104869, 90395, 57218, 108171,
100527, 114224, 71817, 38182, 99721, 3407, 33975, 25, 220,
42411, 104727, 34187, 102224, 100527, 104869, 26850, 71703, 25,
10236, 121, 245, 102028, 13935, 101190, 100222, 101316, 104727,
105593, 80942, 103215, 9370, 100378, 1773, 77557, 91111, 106020,
100211, 101989, 25067, 9909, 16, 24, 20, 21, 7948,
3837, 107119, 102461, 107309, 13935, 109452, 120411, 64359, 26940,
104264, 100145, 25067, 9909, 16, 24, 20, 24, 7948,
3837, 107119, 2082, 78728, 794, 3352, 7725, 64359, 26940,
100397, 99244, 108571, 99232, 25067, 9909, 16, 24, 21,
17, 7948, 3837, 107119, 104156, 107318, 105115, 13935, 99462,
99458, 103639, 7552, 101034, 26940, 108481, 82847, 113724, 25067,
9909, 16, 24, 21, 23, 7948, 3837, 107119, 98237,
13935, 23384, 93488, 74276, 100001, 100378, 99902, 106844, 108124,
104727, 101262, 3837, 97706, 17714, 42411, 57218, 108171, 102280,
114224, 107298, 104257, 100133, 3407, 33975, 25, 220, 42411,
99557, 103928, 101968, 102152, 107258, 105815, 100145, 11319, 4710,
71703, 25, 10236, 121, 245, 102028, 13935, 101190, 100222,
101316, 57218, 108171, 101968, 102152, 107258, 105815, 100145, 1773,
45181, 113460, 27369, 100192, 3837, 42411, 57218, 102461, 107309,
13935, 109452, 120411, 9909, 6828, 12597, 665, 63204, 354,
7552, 104388, 90395, 18493, 104727, 9370, 100378, 26940, 106020,
100211, 101989, 25067, 15946, 57218, 99517, 99721, 1773, 91572,
3837, 106473, 99798, 57218, 2082, 78728, 794, 3352, 7725,
104388, 62926, 104727, 34187, 99517, 107119, 9370, 100378, 26940,
104264, 100145, 25067, 1773, 104043, 3837, 106473, 57218, 98237,
13935, 23384, 93488, 99721, 104421, 34187, 100378, 26940, 108481,
82847, 113724, 87243, 110133, 102280, 104328, 114224, 104156, 107318,
105115, 13935, 99462, 99458, 103639, 9909, 34, 27009, 422,
1952, 56023, 7552, 107258, 103430, 100145, 3837, 111733, 26940,
100397, 99244, 108571, 99232, 25067, 9370, 104727, 3407, 33975,
25, 220, 100648, 104040, 33108, 100772, 102021, 26850, 71703,
25, 51461, 117, 16038, 113460, 27369, 3837, 100349, 102028,
13935, 101190, 100222, 101316, 9370, 104040, 33108, 100772, 101321,
42411, 106614, 109362, 101968, 57566, 33108, 102550, 3837, 104033,
101968, 9370, 110803, 105997, 1773, 100648, 100378, 100663, 101207,
23031, 101968, 100782, 63836, 62926, 101107, 104258, 9370, 101280,
33108, 110803, 1773, 77557, 91111, 106020, 100211, 101989, 25067,
108438, 102461, 107309, 13935, 109452, 120411, 110803, 106411, 106681,
24968, 92894, 100663, 29524, 26940, 104264, 100145, 25067, 9909,
107119, 2082, 78728, 794, 3352, 7725, 64359, 26940, 108481,
82847, 113724, 25067, 9909, 107119, 98237, 13935, 23384, 93488,
7552, 49567, 74763, 100629, 105790, 105117, 1773, 91572, 3837,
42411, 104026, 100137, 67279, 99959, 104339, 3837, 57218, 108171,
100527, 114224, 99721, 96050, 107222, 100378, 102707, 104847, 19108,
1773, 151643]), 'labels': tensor([ -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, 33975, 25, 220,
14880, 106525, 100349, 102028, 13935, 101190, 100222, 101316, 105518,
26850, 71703, 25, 10236, 121, 245, 102028, 13935, 101190,
100222, 101316, 9909, 78425, 92112, 318, 3837, 35987, 13072,
53, 329, 318, 393, 3433, 72, 1020, 1579, 859,
7552, 109182, 104328, 100378, 104727, 3837, 100648, 111978, 114984,
17, 15, 101186, 19, 15, 104227, 1773, 102965, 3837,
42411, 107709, 102508, 99418, 17447, 104247, 104534, 3837, 101889,
106438, 100378, 100650, 1773, 100648, 104727, 103132, 104847, 100694,
19108, 3837, 104033, 16, 24, 20, 21, 104300, 100378,
26940, 106020, 100211, 101989, 25067, 9909, 3036, 4264, 4290,
24511, 48272, 99652, 108438, 42411, 102375, 102461, 107309, 13935,
109452, 120411, 99787, 110803, 106411, 106681, 1773, 104043, 3837,
106473, 115999, 106650, 92894, 102280, 104869, 90395, 57218, 108171,
100527, 114224, 71817, 38182, 99721, 3407, 33975, 25, 220,
42411, 104727, 34187, 102224, 100527, 104869, 26850, 71703, 25,
10236, 121, 245, 102028, 13935, 101190, 100222, 101316, 104727,
105593, 80942, 103215, 9370, 100378, 1773, 77557, 91111, 106020,
100211, 101989, 25067, 9909, 16, 24, 20, 21, 7948,
3837, 107119, 102461, 107309, 13935, 109452, 120411, 64359, 26940,
104264, 100145, 25067, 9909, 16, 24, 20, 24, 7948,
3837, 107119, 2082, 78728, 794, 3352, 7725, 64359, 26940,
100397, 99244, 108571, 99232, 25067, 9909, 16, 24, 21,
17, 7948, 3837, 107119, 104156, 107318, 105115, 13935, 99462,
99458, 103639, 7552, 101034, 26940, 108481, 82847, 113724, 25067,
9909, 16, 24, 21, 23, 7948, 3837, 107119, 98237,
13935, 23384, 93488, 74276, 100001, 100378, 99902, 106844, 108124,
104727, 101262, 3837, 97706, 17714, 42411, 57218, 108171, 102280,
114224, 107298, 104257, 100133, 3407, 33975, 25, 220, 42411,
99557, 103928, 101968, 102152, 107258, 105815, 100145, 11319, 4710,
71703, 25, 10236, 121, 245, 102028, 13935, 101190, 100222,
101316, 57218, 108171, 101968, 102152, 107258, 105815, 100145, 1773,
45181, 113460, 27369, 100192, 3837, 42411, 57218, 102461, 107309,
13935, 109452, 120411, 9909, 6828, 12597, 665, 63204, 354,
7552, 104388, 90395, 18493, 104727, 9370, 100378, 26940, 106020,
100211, 101989, 25067, 15946, 57218, 99517, 99721, 1773, 91572,
3837, 106473, 99798, 57218, 2082, 78728, 794, 3352, 7725,
104388, 62926, 104727, 34187, 99517, 107119, 9370, 100378, 26940,
104264, 100145, 25067, 1773, 104043, 3837, 106473, 57218, 98237,
13935, 23384, 93488, 99721, 104421, 34187, 100378, 26940, 108481,
82847, 113724, 87243, 110133, 102280, 104328, 114224, 104156, 107318,
105115, 13935, 99462, 99458, 103639, 9909, 34, 27009, 422,
1952, 56023, 7552, 107258, 103430, 100145, 3837, 111733, 26940,
100397, 99244, 108571, 99232, 25067, 9370, 104727, 3407, 33975,
25, 220, 100648, 104040, 33108, 100772, 102021, 26850, 71703,
25, 51461, 117, 16038, 113460, 27369, 3837, 100349, 102028,
13935, 101190, 100222, 101316, 9370, 104040, 33108, 100772, 101321,
42411, 106614, 109362, 101968, 57566, 33108, 102550, 3837, 104033,
101968, 9370, 110803, 105997, 1773, 100648, 100378, 100663, 101207,
23031, 101968, 100782, 63836, 62926, 101107, 104258, 9370, 101280,
33108, 110803, 1773, 77557, 91111, 106020, 100211, 101989, 25067,
108438, 102461, 107309, 13935, 109452, 120411, 110803, 106411, 106681,
24968, 92894, 100663, 29524, 26940, 104264, 100145, 25067, 9909,
107119, 2082, 78728, 794, 3352, 7725, 64359, 26940, 108481,
82847, 113724, 25067, 9909, 107119, 98237, 13935, 23384, 93488,
7552, 49567, 74763, 100629, 105790, 105117, 1773, 91572, 3837,
42411, 104026, 100137, 67279, 99959, 104339, 3837, 57218, 108171,
100527, 114224, 99721, 96050, 107222, 100378, 102707, 104847, 19108,
1773, 151643])}, torch.Size([740]).
罗杰华汀1928～2000年Vadim (Plemiannikov), Roger法国电影导演。1940年代中期，他有一小段时间从事舞台剧演员，之后便转入电影生涯。最早，他担任《茱丽叶》（Juliette, 1953）一片的助理。他执导并共同编写了高度成功的异色电影《上帝创造女人》（And God Created Woman, 1956），该片奠定了他的妻子碧姬芭铎性感象征的地位。后来，他一再利用这个包赢不输的套数。他用史绰柏（Annette Stroyberg）拍《危险关系》(Dangerous Liaisons, 1959)，用珍芳达拍《太空英雌》（Barbarella, 1968），并用凯萨琳丹妮芙拍《乱世姐妹花》（Vice and Virtue, 1962），前两位都是他的妻子，最后一位则是他的情人。Human: 请告诉我罗杰·瓦迪姆是谁？
Assistant: 罗杰·瓦迪姆（Roger Vadim，全名Vadim Plemiannikov）是一位法国电影导演，他的职业生涯始于20世纪40年代。最初，他曾在舞台剧上担任演员，然后转向电影领域。他的导演生涯取得了许多成功，尤其是1956年的电影《上帝创造女人》（And God Created Woman），它奠定了他妻子碧姬·芭铎成为性感象征的地位。此外，他还执导了许多其他著名影片，并与多位知名女星进行过合作。
Human: 他导演了哪些知名影片？
Assistant: 罗杰·瓦迪姆导演了一些广受欢迎的电影。例如，《上帝创造女人》（1956年，主演碧姬·芭铎）、《危险关系》（1959年，主演Annette Stroyberg）、《乱世姐妹花》（1962年，主演凯瑟琳·德纳芙）以及《太空英雌》（1968年，主演简·方达）。这些电影不仅确立了他的导演地位，还为他与多位著名女星的合作提供了平台。
Human: 他跟那些女性明星有过私人关系？
Assistant: 罗杰·瓦迪姆与多位女性明星有过私人关系。从所提供的信息来看，他与碧姬·芭铎（Brigitte Bardot）结婚，并在导演的电影《上帝创造女人》中与她合作。同时，他还曾与Annette Stroyberg结婚并导演了她主演的电影《危险关系》。此外，他还与简·方达合作拍摄了电影《太空英雌》，并与著名法国女星凯瑟琳·德纳芙（Catherine Deneuve）有过情人关系，同时也是《乱世姐妹花》的导演。
Human: 他的风格和特点是什么？
Assistant: 根据所提供的信息，罗杰·瓦迪姆的风格和特点在于他善于发掘女性美和魅力，尤其是女性的性感一面。他的电影作品往往以女性为主角并表现她们的美丽和性感。例如，《上帝创造女人》奠定了碧姬·芭铎性感象征的地位；其他作品如《危险关系》（主演Annette Stroyberg）、《太空英雌》（主演简·方达）等也具有相似的特点。同时，他运用这种包赢的方法，与多位知名女星合作，在当时的电影市场上取得了成功。<|endoftext|>
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
[WARNING] [2024-04-20 23:38:55] logging.py : Detected kernel version 4.19.91, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
[INFO] [2024-04-20 23:39:01] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 0
[INFO] [2024-04-20 23:39:11] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:39:21] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:39:31] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:39:41] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:39:51] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:01] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:11] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:21] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:31] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:41] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:40:51] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:01] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:11] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead:
dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)
warnings.warn(
[INFO] [2024-04-20 23:41:21] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:31] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:41] distributed_c10d.py : Waiting in store based barrier to initialize process group for rank: 0, key: store_based_barrier_key:2 (world_size=8, worker_count=1, timeout=0:30:00)
[INFO] [2024-04-20 23:41:43] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 5
[INFO] [2024-04-20 23:41:43] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 1
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 4
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 6
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 7
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 3
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Added key: store_based_barrier_key:2 to store for rank: 2
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 2: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 0: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 5: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 4: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 1: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 3: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 6: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
[INFO] [2024-04-20 23:41:44] distributed_c10d.py : Rank 7: Completed store-based barrier for key:store_based_barrier_key:2 with 8 nodes.
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Setting affinity for GPU 3 to 0fffff
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Setting affinity for GPU 1 to 0fffff
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Setting affinity for GPU 0 to 0fffff
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Setting affinity for GPU 2 to 0fffff
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 00/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 01/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 02/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 03/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 04/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 05/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 06/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 07/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 08/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 09/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 10/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 11/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 12/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 13/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 14/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 15/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 16/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 17/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 18/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 19/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 20/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 21/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 22/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 23/24 : 0 1 2 3 4 5 6 7
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 00 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 00 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 00 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 00 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 00 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 01 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 00 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 01 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 01 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 01 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 01 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 02 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 01 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 02 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 02 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 02 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 03 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 02 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 02 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 03 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 03 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 03 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 03 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 04 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 03 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 04 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 04 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 05 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 04 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 04 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 04 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 05 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 05 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 06 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 05 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 05 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 06 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 06 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 07 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 06 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 00 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 06 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 05 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 07 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 07 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 08 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 00 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 07 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 01 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 06 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 08 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 08 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 09 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 01 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 08 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 02 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 07 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 09 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 09 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 07 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 02 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 10 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 09 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 03 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 08 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 10 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 10 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 08 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 03 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 10 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 04 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 09 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 11 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 11 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 11 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 09 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 04 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 11 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 05 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 12 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 12 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 12 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 10 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 10 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 05 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 12 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 06 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 13 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 11 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 11 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 06 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 13 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 13 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 07 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 13 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 14 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 12 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 12 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 07 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 14 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 08 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 14 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 15 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 13 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 08 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 15 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 09 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 15 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 14 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 09 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 16 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 10 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 15 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 16 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 10 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 13 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 14 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 16 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 17 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 11 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 16 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 17 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 11 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 14 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 15 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 17 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 18 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 12 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 17 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 18 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 15 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 12 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 18 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 19 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 13 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 18 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 19 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 16 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 13 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 19 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 20 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 14 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 19 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 20 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 17 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 14 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 20 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 21 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 15 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 16 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 20 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 21 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 18 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 15 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 21 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 22 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 16 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 21 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 22 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 16 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 22 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 23 : 4[50] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 22 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 17 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 23 : 2[30] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 17 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 23 : 5[60] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 18 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 19 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 20 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 21 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 22 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 17 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Channel 23 : 0[10] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 18 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 19 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 20 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 21 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 22 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 23 : 3[40] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 23 : 1[20] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 18 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 19 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 19 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 20 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 20 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 21 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 21 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 22 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 22 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 23 : 7[80] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 23 : 6[70] -> 7[80] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 00 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Connected all rings
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 01 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 02 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 03 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 04 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 05 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 06 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 00 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 00 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 00 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 00 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 07 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 01 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 01 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 01 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 01 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 08 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 02 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 09 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 02 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 03 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 10 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 03 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 11 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 04 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 04 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 12 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 05 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 05 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 02 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 00 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 13 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 06 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 06 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 03 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 14 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 01 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 07 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 07 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 04 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 15 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 02 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 08 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 08 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 05 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 16 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 03 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 09 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 09 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 02 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 06 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 17 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 04 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 10 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 10 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 03 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 07 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 18 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 05 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 11 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 11 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 04 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 19 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 08 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 06 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 12 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 12 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 05 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 09 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 20 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 07 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 13 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 13 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 06 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 10 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 08 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 14 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 14 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 07 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 11 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 09 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 15 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 15 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 08 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 12 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 10 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 16 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 16 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 09 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 13 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 21 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 11 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 17 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 17 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 10 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 14 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 22 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 18 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 18 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 11 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 15 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Channel 23 : 7[80] -> 6[70] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 19 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 00 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 20 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 19 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 12 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 16 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 01 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 21 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 17 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 13 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 20 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 02 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 12 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 22 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 21 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 18 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 13 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Channel 23 : 4[50] -> 3[40] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 22 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 14 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 03 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 19 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 14 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Channel 23 : 2[30] -> 1[20] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 20 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 15 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 21 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 16 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 15 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 22 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 16 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Channel 23 : 1[20] -> 0[10] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 17 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 04 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 18 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 05 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 19 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 17 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 06 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 20 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 18 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 07 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 21 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 19 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 08 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 22 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 20 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 09 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Channel 23 : 3[40] -> 2[30] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 21 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 10 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 22 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 11 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Channel 23 : 5[60] -> 4[50] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 12 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 13 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 14 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 15 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 16 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 17 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 18 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 19 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 20 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 21 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 22 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Channel 23 : 6[70] -> 5[60] via P2P/IPC/read
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO Connected all trees
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 8/8/512
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO 24 coll channels, 32 p2p channels, 32 p2p channels per peer
dlcpq1dza0ye5yti-master-0:274:1197 [7] NCCL INFO comm 0x7f51f8008fb0 rank 7 nranks 8 cudaDev 7 busId 80 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:271:1193 [4] NCCL INFO comm 0x7f4a10008fb0 rank 4 nranks 8 cudaDev 4 busId 50 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:269:1191 [2] NCCL INFO comm 0x7f5660008fb0 rank 2 nranks 8 cudaDev 2 busId 30 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:270:1195 [3] NCCL INFO comm 0x7f33d4008fb0 rank 3 nranks 8 cudaDev 3 busId 40 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:268:1194 [1] NCCL INFO comm 0x7f3a58008fb0 rank 1 nranks 8 cudaDev 1 busId 20 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:267:1190 [0] NCCL INFO comm 0x7ff06c008fb0 rank 0 nranks 8 cudaDev 0 busId 10 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:267:267 [0] NCCL INFO Launch mode Parallel
dlcpq1dza0ye5yti-master-0:273:1196 [6] NCCL INFO comm 0x7fab28008fb0 rank 6 nranks 8 cudaDev 6 busId 70 - Init COMPLETE
dlcpq1dza0ye5yti-master-0:272:1192 [5] NCCL INFO comm 0x7f42b0008fb0 rank 5 nranks 8 cudaDev 5 busId 60 - Init COMPLETE
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
0%| | 0/3120 [00:00<?, ?it/s]train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
{'loss': 1.3137, 'learning_rate': 2.564102564102564e-07, 'epoch': 0.0}
{'loss': 1.2524, 'learning_rate': 5.128205128205128e-07, 'epoch': 0.01}
{'loss': 1.2224, 'learning_rate': 7.692307692307694e-07, 'epoch': 0.01}
{'loss': 1.2209, 'learning_rate': 1.0256410256410257e-06, 'epoch': 0.01}
{'loss': 1.1776, 'learning_rate': 1.282051282051282e-06, 'epoch': 0.01}
{'loss': 1.124, 'learning_rate': 1.5384615384615387e-06, 'epoch': 0.02}
{'loss': 1.1545, 'learning_rate': 1.794871794871795e-06, 'epoch': 0.02}
{'loss': 1.1266, 'learning_rate': 2.0512820512820513e-06, 'epoch': 0.02}
{'loss': 1.1284, 'learning_rate': 2.307692307692308e-06, 'epoch': 0.02}
{'loss': 1.0904, 'learning_rate': 2.564102564102564e-06, 'epoch': 0.03}
{'loss': 1.0894, 'learning_rate': 2.8205128205128207e-06, 'epoch': 0.03}
{'loss': 1.0946, 'learning_rate': 3.0769230769230774e-06, 'epoch': 0.03}
{'loss': 1.1264, 'learning_rate': 3.3333333333333333e-06, 'epoch': 0.03}
{'loss': 1.0495, 'learning_rate': 3.58974358974359e-06, 'epoch': 0.04}
{'loss': 1.0859, 'learning_rate': 3.846153846153847e-06, 'epoch': 0.04}
{'loss': 1.0676, 'learning_rate': 4.102564102564103e-06, 'epoch': 0.04}
{'loss': 1.0675, 'learning_rate': 4.358974358974359e-06, 'epoch': 0.04}
{'loss': 1.0887, 'learning_rate': 4.615384615384616e-06, 'epoch': 0.05}
{'loss': 1.0623, 'learning_rate': 4.871794871794872e-06, 'epoch': 0.05}
{'loss': 1.0979, 'learning_rate': 5.128205128205128e-06, 'epoch': 0.05}
{'loss': 1.0907, 'learning_rate': 5.384615384615385e-06, 'epoch': 0.05}
{'loss': 1.1248, 'learning_rate': 5.641025641025641e-06, 'epoch': 0.06}
{'loss': 1.0717, 'learning_rate': 5.897435897435898e-06, 'epoch': 0.06}
{'loss': 1.0472, 'learning_rate': 6.153846153846155e-06, 'epoch': 0.06}
{'loss': 1.0468, 'learning_rate': 6.410256410256412e-06, 'epoch': 0.06}
{'loss': 1.1036, 'learning_rate': 6.666666666666667e-06, 'epoch': 0.07}
{'loss': 1.0443, 'learning_rate': 6.923076923076923e-06, 'epoch': 0.07}
{'loss': 1.1326, 'learning_rate': 7.17948717948718e-06, 'epoch': 0.07}
{'loss': 1.0333, 'learning_rate': 7.435897435897437e-06, 'epoch': 0.07}
{'loss': 1.0917, 'learning_rate': 7.692307692307694e-06, 'epoch': 0.08}
{'loss': 1.1045, 'learning_rate': 7.948717948717949e-06, 'epoch': 0.08}
{'loss': 1.0835, 'learning_rate': 8.205128205128205e-06, 'epoch': 0.08}
{'loss': 1.0796, 'learning_rate': 8.461538461538462e-06, 'epoch': 0.08}
{'loss': 1.1246, 'learning_rate': 8.717948717948719e-06, 'epoch': 0.09}
{'loss': 1.0481, 'learning_rate': 8.974358974358976e-06, 'epoch': 0.09}
{'loss': 1.0502, 'learning_rate': 9.230769230769232e-06, 'epoch': 0.09}
{'loss': 1.1048, 'learning_rate': 9.487179487179487e-06, 'epoch': 0.09}
{'loss': 1.1112, 'learning_rate': 9.743589743589744e-06, 'epoch': 0.1}
{'loss': 1.1319, 'learning_rate': 1e-05, 'epoch': 0.1}
{'loss': 1.095, 'learning_rate': 1.0256410256410256e-05, 'epoch': 0.1}
{'loss': 1.0906, 'learning_rate': 1.0512820512820514e-05, 'epoch': 0.11}
{'loss': 1.1147, 'learning_rate': 1.076923076923077e-05, 'epoch': 0.11}
{'loss': 1.0714, 'learning_rate': 1.1025641025641028e-05, 'epoch': 0.11}
{'loss': 1.1125, 'learning_rate': 1.1282051282051283e-05, 'epoch': 0.11}
{'loss': 1.0838, 'learning_rate': 1.1538461538461538e-05, 'epoch': 0.12}
{'loss': 1.1119, 'learning_rate': 1.1794871794871796e-05, 'epoch': 0.12}
{'loss': 1.1097, 'learning_rate': 1.2051282051282051e-05, 'epoch': 0.12}
{'loss': 1.1344, 'learning_rate': 1.230769230769231e-05, 'epoch': 0.12}
{'loss': 1.1108, 'learning_rate': 1.2564102564102565e-05, 'epoch': 0.13}
{'loss': 1.1308, 'learning_rate': 1.2820512820512823e-05, 'epoch': 0.13}
{'loss': 1.1198, 'learning_rate': 1.3076923076923078e-05, 'epoch': 0.13}
{'loss': 1.1589, 'learning_rate': 1.3333333333333333e-05, 'epoch': 0.13}
{'loss': 1.1473, 'learning_rate': 1.3589743589743592e-05, 'epoch': 0.14}
{'loss': 1.0719, 'learning_rate': 1.3846153846153847e-05, 'epoch': 0.14}
{'loss': 1.1418, 'learning_rate': 1.4102564102564105e-05, 'epoch': 0.14}
{'loss': 1.1056, 'learning_rate': 1.435897435897436e-05, 'epoch': 0.14}
{'loss': 1.1027, 'learning_rate': 1.4615384615384615e-05, 'epoch': 0.15}
{'loss': 1.1641, 'learning_rate': 1.4871794871794874e-05, 'epoch': 0.15}
{'loss': 1.094, 'learning_rate': 1.5128205128205129e-05, 'epoch': 0.15}
{'loss': 1.1528, 'learning_rate': 1.5384615384615387e-05, 'epoch': 0.15}
{'loss': 1.1395, 'learning_rate': 1.5641025641025644e-05, 'epoch': 0.16}
{'loss': 1.1329, 'learning_rate': 1.5897435897435897e-05, 'epoch': 0.16}
{'loss': 1.0875, 'learning_rate': 1.6153846153846154e-05, 'epoch': 0.16}
{'loss': 1.1137, 'learning_rate': 1.641025641025641e-05, 'epoch': 0.16}
{'loss': 1.1722, 'learning_rate': 1.6666666666666667e-05, 'epoch': 0.17}
{'loss': 1.0949, 'learning_rate': 1.6923076923076924e-05, 'epoch': 0.17}
{'loss': 1.1418, 'learning_rate': 1.717948717948718e-05, 'epoch': 0.17}
{'loss': 1.158, 'learning_rate': 1.7435897435897438e-05, 'epoch': 0.17}
{'loss': 1.1303, 'learning_rate': 1.7692307692307694e-05, 'epoch': 0.18}
{'loss': 1.189, 'learning_rate': 1.794871794871795e-05, 'epoch': 0.18}
{'loss': 1.1031, 'learning_rate': 1.8205128205128208e-05, 'epoch': 0.18}
{'loss': 1.1348, 'learning_rate': 1.8461538461538465e-05, 'epoch': 0.18}
{'loss': 1.1124, 'learning_rate': 1.8717948717948718e-05, 'epoch': 0.19}
{'loss': 1.1338, 'learning_rate': 1.8974358974358975e-05, 'epoch': 0.19}
{'loss': 1.1585, 'learning_rate': 1.923076923076923e-05, 'epoch': 0.19}
{'loss': 1.1598, 'learning_rate': 1.9487179487179488e-05, 'epoch': 0.19}
{'loss': 1.186, 'learning_rate': 1.9743589743589745e-05, 'epoch': 0.2}
{'loss': 1.1578, 'learning_rate': 2e-05, 'epoch': 0.2}
{'loss': 1.2017, 'learning_rate': 1.9999977531546567e-05, 'epoch': 0.2}
{'loss': 1.1756, 'learning_rate': 1.999991012628722e-05, 'epoch': 0.21}
{'loss': 1.1565, 'learning_rate': 1.9999797784524866e-05, 'epoch': 0.21}
{'loss': 1.1721, 'learning_rate': 1.999964050676434e-05, 'epoch': 0.21}
{'loss': 1.1388, 'learning_rate': 1.9999438293712382e-05, 'epoch': 0.21}
{'loss': 1.1761, 'learning_rate': 1.999919114627769e-05, 'epoch': 0.22}
{'loss': 1.1619, 'learning_rate': 1.999889906557086e-05, 'epoch': 0.22}
{'loss': 1.1452, 'learning_rate': 1.999856205290442e-05, 'epoch': 0.22}
{'loss': 1.1364, 'learning_rate': 1.9998180109792793e-05, 'epoch': 0.22}
{'loss': 1.1926, 'learning_rate': 1.999775323795232e-05, 'epoch': 0.23}
{'loss': 1.1646, 'learning_rate': 1.9997281439301218e-05, 'epoch': 0.23}
{'loss': 1.1484, 'learning_rate': 1.999676471595962e-05, 'epoch': 0.23}
{'loss': 1.1563, 'learning_rate': 1.9996203070249516e-05, 'epoch': 0.23}
{'loss': 1.1448, 'learning_rate': 1.9995596504694764e-05, 'epoch': 0.24}
{'loss': 1.1539, 'learning_rate': 1.9994945022021085e-05, 'epoch': 0.24}
{'loss': 1.1047, 'learning_rate': 1.999424862515604e-05, 'epoch': 0.24}
{'loss': 1.145, 'learning_rate': 1.999350731722902e-05, 'epoch': 0.24}
{'loss': 1.2109, 'learning_rate': 1.9992721101571238e-05, 'epoch': 0.25}
{'loss': 1.1766, 'learning_rate': 1.9991889981715696e-05, 'epoch': 0.25}
{'loss': 1.1013, 'learning_rate': 1.99910139613972e-05, 'epoch': 0.25}
{'loss': 1.1406, 'learning_rate': 1.9990093044552304e-05, 'epoch': 0.25}
{'loss': 1.1561, 'learning_rate': 1.998912723531933e-05, 'epoch': 0.26}
{'loss': 1.1512, 'learning_rate': 1.9988116538038326e-05, 'epoch': 0.26}
{'loss': 1.1941, 'learning_rate': 1.9987060957251047e-05, 'epoch': 0.26}
{'loss': 1.1429, 'learning_rate': 1.9985960497700953e-05, 'epoch': 0.26}
{'loss': 1.1689, 'learning_rate': 1.9984815164333163e-05, 'epoch': 0.27}
{'loss': 1.201, 'learning_rate': 1.998362496229446e-05, 'epoch': 0.27}
{'loss': 1.1526, 'learning_rate': 1.998238989693323e-05, 'epoch': 0.27}
{'loss': 1.1902, 'learning_rate': 1.998110997379949e-05, 'epoch': 0.27}
{'loss': 1.1751, 'learning_rate': 1.997978519864481e-05, 'epoch': 0.28}
{'loss': 1.1176, 'learning_rate': 1.997841557742232e-05, 'epoch': 0.28}
{'loss': 1.1814, 'learning_rate': 1.9977001116286675e-05, 'epoch': 0.28}
{'loss': 1.1603, 'learning_rate': 1.9975541821594028e-05, 'epoch': 0.28}
{'loss': 1.1679, 'learning_rate': 1.9974037699901993e-05, 'epoch': 0.29}
{'loss': 1.1865, 'learning_rate': 1.9972488757969635e-05, 'epoch': 0.29}
{'loss': 1.1097, 'learning_rate': 1.9970895002757413e-05, 'epoch': 0.29}
{'loss': 1.1651, 'learning_rate': 1.9969256441427173e-05, 'epoch': 0.29}
{'loss': 1.1874, 'learning_rate': 1.9967573081342103e-05, 'epoch': 0.3}
{'loss': 1.1893, 'learning_rate': 1.99658449300667e-05, 'epoch': 0.3}
{'loss': 1.1387, 'learning_rate': 1.9964071995366744e-05, 'epoch': 0.3}
{'loss': 1.155, 'learning_rate': 1.9962254285209256e-05, 'epoch': 0.31}
{'loss': 1.1563, 'learning_rate': 1.9960391807762462e-05, 'epoch': 0.31}
{'loss': 1.1502, 'learning_rate': 1.9958484571395756e-05, 'epoch': 0.31}
{'loss': 1.15, 'learning_rate': 1.9956532584679676e-05, 'epoch': 0.31}
{'loss': 1.2085, 'learning_rate': 1.995453585638584e-05, 'epoch': 0.32}
{'loss': 1.0918, 'learning_rate': 1.995249439548693e-05, 'epoch': 0.32}
{'loss': 1.1681, 'learning_rate': 1.9950408211156637e-05, 'epoch': 0.32}
{'loss': 1.1875, 'learning_rate': 1.994827731276963e-05, 'epoch': 0.32}
{'loss': 1.1399, 'learning_rate': 1.9946101709901513e-05, 'epoch': 0.33}
{'loss': 1.1883, 'learning_rate': 1.994388141232876e-05, 'epoch': 0.33}
{'loss': 1.1678, 'learning_rate': 1.9941616430028713e-05, 'epoch': 0.33}
{'loss': 1.2118, 'learning_rate': 1.9939306773179498e-05, 'epoch': 0.33}
{'loss': 1.1649, 'learning_rate': 1.9936952452159995e-05, 'epoch': 0.34}
{'loss': 1.1545, 'learning_rate': 1.9934553477549795e-05, 'epoch': 0.34}
{'loss': 1.1819, 'learning_rate': 1.9932109860129154e-05, 'epoch': 0.34}
{'loss': 1.1457, 'learning_rate': 1.992962161087893e-05, 'epoch': 0.34}
{'loss': 1.1614, 'learning_rate': 1.992708874098054e-05, 'epoch': 0.35}
{'loss': 1.1435, 'learning_rate': 1.9924511261815928e-05, 'epoch': 0.35}
{'loss': 1.1353, 'learning_rate': 1.992188918496748e-05, 'epoch': 0.35}
{'loss': 1.1642, 'learning_rate': 1.9919222522217998e-05, 'epoch': 0.35}
{'loss': 1.1312, 'learning_rate': 1.991651128555064e-05, 'epoch': 0.36}
{'loss': 1.1559, 'learning_rate': 1.9913755487148874e-05, 'epoch': 0.36}
{'loss': 1.1477, 'learning_rate': 1.9910955139396395e-05, 'epoch': 0.36}
{'loss': 1.1672, 'learning_rate': 1.9908110254877107e-05, 'epoch': 0.36}
{'loss': 1.1704, 'learning_rate': 1.990522084637503e-05, 'epoch': 0.37}
{'loss': 1.1242, 'learning_rate': 1.990228692687429e-05, 'epoch': 0.37}
{'loss': 1.164, 'learning_rate': 1.9899308509558998e-05, 'epoch': 0.37}
{'loss': 1.1559, 'learning_rate': 1.9896285607813245e-05, 'epoch': 0.37}
{'loss': 1.1351, 'learning_rate': 1.9893218235221016e-05, 'epoch': 0.38}
{'loss': 1.2004, 'learning_rate': 1.989010640556614e-05, 'epoch': 0.38}
{'loss': 1.2014, 'learning_rate': 1.988695013283221e-05, 'epoch': 0.38}
{'loss': 1.136, 'learning_rate': 1.988374943120254e-05, 'epoch': 0.38}
{'loss': 1.1745, 'learning_rate': 1.9880504315060096e-05, 'epoch': 0.39}
{'loss': 1.1619, 'learning_rate': 1.9877214798987428e-05, 'epoch': 0.39}
{'loss': 1.1502, 'learning_rate': 1.9873880897766597e-05, 'epoch': 0.39}
{'loss': 1.1542, 'learning_rate': 1.9870502626379127e-05, 'epoch': 0.39}
{'loss': 1.104, 'learning_rate': 1.9867080000005933e-05, 'epoch': 0.4}
{'loss': 1.1589, 'learning_rate': 1.9863613034027224e-05, 'epoch': 0.4}
{'loss': 1.1237, 'learning_rate': 1.9860101744022485e-05, 'epoch': 0.4}
{'loss': 1.1635, 'learning_rate': 1.985654614577036e-05, 'epoch': 0.41}
{'loss': 1.1501, 'learning_rate': 1.985294625524861e-05, 'epoch': 0.41}
{'loss': 1.1633, 'learning_rate': 1.9849302088634034e-05, 'epoch': 0.41}
{'loss': 1.1703, 'learning_rate': 1.9845613662302384e-05, 'epoch': 0.41}
{'loss': 1.1496, 'learning_rate': 1.9841880992828306e-05, 'epoch': 0.42}
{'loss': 1.22, 'learning_rate': 1.9838104096985266e-05, 'epoch': 0.42}
{'loss': 1.1372, 'learning_rate': 1.9834282991745465e-05, 'epoch': 0.42}
{'loss': 1.1988, 'learning_rate': 1.9830417694279766e-05, 'epoch': 0.42}
{'loss': 1.1268, 'learning_rate': 1.9826508221957624e-05, 'epoch': 0.43}
{'loss': 1.1662, 'learning_rate': 1.9822554592346996e-05, 'epoch': 0.43}
{'loss': 1.1371, 'learning_rate': 1.981855682321427e-05, 'epoch': 0.43}
{'loss': 1.1876, 'learning_rate': 1.981451493252418e-05, 'epoch': 0.43}
{'loss': 1.1379, 'learning_rate': 1.981042893843974e-05, 'epoch': 0.44}
{'loss': 1.1569, 'learning_rate': 1.9806298859322143e-05, 'epoch': 0.44}
{'loss': 1.0988, 'learning_rate': 1.980212471373068e-05, 'epoch': 0.44}
{'loss': 1.137, 'learning_rate': 1.979790652042268e-05, 'epoch': 0.44}
{'loss': 1.1034, 'learning_rate': 1.979364429835339e-05, 'epoch': 0.45}
{'loss': 1.1428, 'learning_rate': 1.9789338066675924e-05, 'epoch': 0.45}
{'loss': 1.1757, 'learning_rate': 1.978498784474115e-05, 'epoch': 0.45}
{'loss': 1.186, 'learning_rate': 1.978059365209762e-05, 'epoch': 0.45}
{'loss': 1.1064, 'learning_rate': 1.9776155508491482e-05, 'epoch': 0.46}
{'loss': 1.2007, 'learning_rate': 1.977167343386638e-05, 'epoch': 0.46}
{'loss': 1.2118, 'learning_rate': 1.9767147448363366e-05, 'epoch': 0.46}
{'loss': 1.134, 'learning_rate': 1.9762577572320826e-05, 'epoch': 0.46}
{'loss': 1.1829, 'learning_rate': 1.9757963826274357e-05, 'epoch': 0.47}
{'loss': 1.1299, 'learning_rate': 1.975330623095672e-05, 'epoch': 0.47}
{'loss': 1.198, 'learning_rate': 1.97486048072977e-05, 'epoch': 0.47}
{'loss': 1.1158, 'learning_rate': 1.9743859576424044e-05, 'epoch': 0.47}
{'loss': 1.167, 'learning_rate': 1.9739070559659347e-05, 'epoch': 0.48}
{'loss': 1.1606, 'learning_rate': 1.9734237778523977e-05, 'epoch': 0.48}
{'loss': 1.1466, 'learning_rate': 1.972936125473495e-05, 'epoch': 0.48}
{'loss': 1.1417, 'learning_rate': 1.9724441010205865e-05, 'epoch': 0.48}
{'loss': 1.192, 'learning_rate': 1.9719477067046768e-05, 'epoch': 0.49}
{'loss': 1.1691, 'learning_rate': 1.971446944756409e-05, 'epoch': 0.49}
{'loss': 1.1153, 'learning_rate': 1.9709418174260523e-05, 'epoch': 0.49}
{'loss': 1.1259, 'learning_rate': 1.9704323269834925e-05, 'epoch': 0.49}
{'loss': 1.1723, 'learning_rate': 1.9699184757182225e-05, 'epoch': 0.5}
{'loss': 1.1526, 'learning_rate': 1.9694002659393306e-05, 'epoch': 0.5}
{'loss': 1.1425, 'learning_rate': 1.9688776999754913e-05, 'epoch': 0.5}
{'loss': 1.1615, 'learning_rate': 1.9683507801749547e-05, 'epoch': 0.51}
{'loss': 1.144, 'learning_rate': 1.9678195089055347e-05, 'epoch': 0.51}
{'loss': 1.0944, 'learning_rate': 1.9672838885546007e-05, 'epoch': 0.51}
{'loss': 1.112, 'learning_rate': 1.966743921529065e-05, 'epoch': 0.51}
{'loss': 1.1535, 'learning_rate': 1.9661996102553716e-05, 'epoch': 0.52}
{'loss': 1.1526, 'learning_rate': 1.965650957179488e-05, 'epoch': 0.52}
{'loss': 1.157, 'learning_rate': 1.9650979647668908e-05, 'epoch': 0.52}
{'loss': 1.1447, 'learning_rate': 1.9645406355025565e-05, 'epoch': 0.52}
{'loss': 1.1192, 'learning_rate': 1.963978971890951e-05, 'epoch': 0.53}
{'loss': 1.19, 'learning_rate': 1.963412976456017e-05, 'epoch': 0.53}
{'loss': 1.1789, 'learning_rate': 1.9628426517411625e-05, 'epoch': 0.53}
{'loss': 1.204, 'learning_rate': 1.9622680003092503e-05, 'epoch': 0.53}
{'loss': 1.1377, 'learning_rate': 1.9616890247425867e-05, 'epoch': 0.54}
{'loss': 1.1645, 'learning_rate': 1.9611057276429085e-05, 'epoch': 0.54}
{'loss': 1.1234, 'learning_rate': 1.9605181116313725e-05, 'epoch': 0.54}
{'loss': 1.124, 'learning_rate': 1.9599261793485432e-05, 'epoch': 0.54}
{'loss': 1.1944, 'learning_rate': 1.959329933454381e-05, 'epoch': 0.55}
{'loss': 1.1968, 'learning_rate': 1.958729376628231e-05, 'epoch': 0.55}
{'loss': 1.1025, 'learning_rate': 1.9581245115688093e-05, 'epoch': 0.55}
{'loss': 1.1932, 'learning_rate': 1.957515340994193e-05, 'epoch': 0.55}
{'loss': 1.1497, 'learning_rate': 1.9569018676418053e-05, 'epoch': 0.56}
{'loss': 1.1787, 'learning_rate': 1.956284094268407e-05, 'epoch': 0.56}
{'loss': 1.1771, 'learning_rate': 1.9556620236500794e-05, 'epoch': 0.56}
{'loss': 1.1826, 'learning_rate': 1.955035658582216e-05, 'epoch': 0.56}
{'loss': 1.1726, 'learning_rate': 1.9544050018795076e-05, 'epoch': 0.57}
{'loss': 1.1397, 'learning_rate': 1.9537700563759303e-05, 'epoch': 0.57}
{'loss': 1.1658, 'learning_rate': 1.9531308249247328e-05, 'epoch': 0.57}
{'loss': 1.1805, 'learning_rate': 1.9524873103984234e-05, 'epoch': 0.57}
{'loss': 1.1383, 'learning_rate': 1.9518395156887574e-05, 'epoch': 0.58}
{'loss': 1.1558, 'learning_rate': 1.9511874437067243e-05, 'epoch': 0.58}
{'loss': 1.1584, 'learning_rate': 1.950531097382533e-05, 'epoch': 0.58}
{'loss': 1.2029, 'learning_rate': 1.949870479665602e-05, 'epoch': 0.58}
{'loss': 1.1418, 'learning_rate': 1.9492055935245418e-05, 'epoch': 0.59}
{'loss': 1.176, 'learning_rate': 1.9485364419471454e-05, 'epoch': 0.59}
{'loss': 1.1742, 'learning_rate': 1.9478630279403737e-05, 'epoch': 0.59}
{'loss': 1.0948, 'learning_rate': 1.9471853545303407e-05, 'epoch': 0.59}
{'loss': 1.1567, 'learning_rate': 1.9465034247623005e-05, 'epoch': 0.6}
{'loss': 1.1644, 'learning_rate': 1.9458172417006347e-05, 'epoch': 0.6}
{'loss': 1.1226, 'learning_rate': 1.9451268084288385e-05, 'epoch': 0.6}
{'loss': 1.0994, 'learning_rate': 1.9444321280495045e-05, 'epoch': 0.61}
{'loss': 1.1429, 'learning_rate': 1.943733203684312e-05, 'epoch': 0.61}
{'loss': 1.1411, 'learning_rate': 1.9430300384740108e-05, 'epoch': 0.61}
{'loss': 1.1425, 'learning_rate': 1.942322635578408e-05, 'epoch': 0.61}
{'loss': 1.1447, 'learning_rate': 1.9416109981763526e-05, 'epoch': 0.62}
{'loss': 1.1677, 'learning_rate': 1.940895129465724e-05, 'epoch': 0.62}
{'loss': 1.1862, 'learning_rate': 1.9401750326634144e-05, 'epoch': 0.62}
{'loss': 1.1923, 'learning_rate': 1.939450711005316e-05, 'epoch': 0.62}
{'loss': 1.202, 'learning_rate': 1.9387221677463064e-05, 'epoch': 0.63}
{'loss': 1.1419, 'learning_rate': 1.9379894061602335e-05, 'epoch': 0.63}
{'loss': 1.111, 'learning_rate': 1.9372524295399014e-05, 'epoch': 0.63}
{'loss': 1.1734, 'learning_rate': 1.936511241197055e-05, 'epoch': 0.63}
{'loss': 1.1437, 'learning_rate': 1.9357658444623655e-05, 'epoch': 0.64}
{'loss': 1.1303, 'learning_rate': 1.9350162426854152e-05, 'epoch': 0.64}
{'loss': 1.1779, 'learning_rate': 1.9342624392346826e-05, 'epoch': 0.64}
{'loss': 1.1967, 'learning_rate': 1.933504437497527e-05, 'epoch': 0.64}
{'loss': 1.1155, 'learning_rate': 1.9327422408801744e-05, 'epoch': 0.65}
{'loss': 1.1579, 'learning_rate': 1.9319758528076998e-05, 'epoch': 0.65}
{'loss': 1.144, 'learning_rate': 1.9312052767240153e-05, 'epoch': 0.65}
{'loss': 1.1733, 'learning_rate': 1.93043051609185e-05, 'epoch': 0.65}
{'loss': 1.1563, 'learning_rate': 1.92965157439274e-05, 'epoch': 0.66}
{'loss': 1.1467, 'learning_rate': 1.9288684551270072e-05, 'epoch': 0.66}
{'loss': 1.212, 'learning_rate': 1.9280811618137486e-05, 'epoch': 0.66}
{'loss': 1.1307, 'learning_rate': 1.9272896979908153e-05, 'epoch': 0.66}
{'loss': 1.1368, 'learning_rate': 1.9264940672148018e-05, 'epoch': 0.67}
{'loss': 1.1799, 'learning_rate': 1.9256942730610268e-05, 'epoch': 0.67}
{'loss': 1.1466, 'learning_rate': 1.9248903191235177e-05, 'epoch': 0.67}
{'loss': 1.2009, 'learning_rate': 1.9240822090149944e-05, 'epoch': 0.67}
{'loss': 1.1678, 'learning_rate': 1.9232699463668543e-05, 'epoch': 0.68}
{'loss': 1.1532, 'learning_rate': 1.9224535348291543e-05, 'epoch': 0.68}
{'loss': 1.1784, 'learning_rate': 1.9216329780705955e-05, 'epoch': 0.68}
{'loss': 1.1642, 'learning_rate': 1.9208082797785057e-05, 'epoch': 0.68}
{'loss': 1.1825, 'learning_rate': 1.9199794436588244e-05, 'epoch': 0.69}
{'loss': 1.1526, 'learning_rate': 1.9191464734360844e-05, 'epoch': 0.69}
{'loss': 1.1468, 'learning_rate': 1.9183093728533966e-05, 'epoch': 0.69}
{'loss': 1.1444, 'learning_rate': 1.917468145672432e-05, 'epoch': 0.69}
{'loss': 1.1357, 'learning_rate': 1.916622795673405e-05, 'epoch': 0.7}
{'loss': 1.1569, 'learning_rate': 1.9157733266550577e-05, 'epoch': 0.7}
{'loss': 1.1081, 'learning_rate': 1.9149197424346405e-05, 'epoch': 0.7}
{'loss': 1.1765, 'learning_rate': 1.9140620468478967e-05, 'epoch': 0.71}
{'loss': 1.2001, 'learning_rate': 1.913200243749046e-05, 'epoch': 0.71}
{'loss': 1.1429, 'learning_rate': 1.9123343370107636e-05, 'epoch': 0.71}
{'loss': 1.1898, 'learning_rate': 1.9114643305241678e-05, 'epoch': 0.71}
{'loss': 1.1473, 'learning_rate': 1.910590228198798e-05, 'epoch': 0.72}
{'loss': 1.1505, 'learning_rate': 1.9097120339625994e-05, 'epoch': 0.72}
{'loss': 1.1208, 'learning_rate': 1.9088297517619056e-05, 'epoch': 0.72}
{'loss': 1.1064, 'learning_rate': 1.9079433855614203e-05, 'epoch': 0.72}
{'loss': 1.1549, 'learning_rate': 1.9070529393441985e-05, 'epoch': 0.73}
{'loss': 1.1308, 'learning_rate': 1.9061584171116302e-05, 'epoch': 0.73}
{'loss': 1.1827, 'learning_rate': 1.9052598228834217e-05, 'epoch': 0.73}
{'loss': 1.1394, 'learning_rate': 1.9043571606975776e-05, 'epoch': 0.73}
{'loss': 1.1002, 'learning_rate': 1.9034504346103825e-05, 'epoch': 0.74}
{'loss': 1.1308, 'learning_rate': 1.9025396486963827e-05, 'epoch': 0.74}
{'loss': 1.1007, 'learning_rate': 1.9016248070483686e-05, 'epoch': 0.74}
{'loss': 1.1288, 'learning_rate': 1.900705913777356e-05, 'epoch': 0.74}
{'loss': 1.1672, 'learning_rate': 1.8997829730125662e-05, 'epoch': 0.75}
{'loss': 1.099, 'learning_rate': 1.89885598890141e-05, 'epoch': 0.75}
{'loss': 1.1658, 'learning_rate': 1.8979249656094672e-05, 'epoch': 0.75}
{'loss': 1.1251, 'learning_rate': 1.8969899073204687e-05, 'epoch': 0.75}
{'loss': 1.1284, 'learning_rate': 1.896050818236277e-05, 'epoch': 0.76}
{'loss': 1.1105, 'learning_rate': 1.895107702576868e-05, 'epoch': 0.76}
{'loss': 1.1285, 'learning_rate': 1.8941605645803115e-05, 'epoch': 0.76}
{'loss': 1.1648, 'learning_rate': 1.8932094085027534e-05, 'epoch': 0.76}
{'loss': 1.1517, 'learning_rate': 1.8922542386183942e-05, 'epoch': 0.77}
{'loss': 1.1652, 'learning_rate': 1.891295059219472e-05, 'epoch': 0.77}
{'loss': 1.1424, 'learning_rate': 1.8903318746162428e-05, 'epoch': 0.77}
{'loss': 1.1157, 'learning_rate': 1.88936468913696e-05, 'epoch': 0.77}
{'loss': 1.1597, 'learning_rate': 1.888393507127856e-05, 'epoch': 0.78}
{'loss': 1.1169, 'learning_rate': 1.8874183329531222e-05, 'epoch': 0.78}
{'loss': 1.1528, 'learning_rate': 1.88643917099489e-05, 'epoch': 0.78}
{'loss': 1.1508, 'learning_rate': 1.8854560256532098e-05, 'epoch': 0.78}
{'loss': 1.138, 'learning_rate': 1.8844689013460335e-05, 'epoch': 0.79}
{'loss': 1.1486, 'learning_rate': 1.883477802509192e-05, 'epoch': 0.79}
{'loss': 1.1427, 'learning_rate': 1.8824827335963767e-05, 'epoch': 0.79}
{'loss': 1.1143, 'learning_rate': 1.88148369907912e-05, 'epoch': 0.79}
{'loss': 1.1366, 'learning_rate': 1.8804807034467735e-05, 'epoch': 0.8}
{'loss': 1.1975, 'learning_rate': 1.879473751206489e-05, 'epoch': 0.8}
{'loss': 1.1549, 'learning_rate': 1.8784628468831997e-05, 'epoch': 0.8}
{'loss': 1.1532, 'learning_rate': 1.877447995019596e-05, 'epoch': 0.81}
{'loss': 1.1389, 'learning_rate': 1.876429200176108e-05, 'epoch': 0.81}
{'loss': 1.1319, 'learning_rate': 1.875406466930886e-05, 'epoch': 0.81}
{'loss': 1.1022, 'learning_rate': 1.874379799879775e-05, 'epoch': 0.81}
{'loss': 1.1313, 'learning_rate': 1.8733492036363007e-05, 'epoch': 0.82}
{'loss': 1.1646, 'learning_rate': 1.872314682831643e-05, 'epoch': 0.82}
{'loss': 1.208, 'learning_rate': 1.8712762421146185e-05, 'epoch': 0.82}
{'loss': 1.1441, 'learning_rate': 1.8702338861516587e-05, 'epoch': 0.82}
{'loss': 1.1564, 'learning_rate': 1.8691876196267892e-05, 'epoch': 0.83}
{'loss': 1.1334, 'learning_rate': 1.8681374472416075e-05, 'epoch': 0.83}
{'loss': 1.0704, 'learning_rate': 1.867083373715264e-05, 'epoch': 0.83}
{'loss': 1.1261, 'learning_rate': 1.866025403784439e-05, 'epoch': 0.83}
{'loss': 1.1484, 'learning_rate': 1.8649635422033218e-05, 'epoch': 0.84}
{'loss': 1.1134, 'learning_rate': 1.86389779374359e-05, 'epoch': 0.84}
{'loss': 1.1219, 'learning_rate': 1.862828163194388e-05, 'epoch': 0.84}
{'loss': 1.1314, 'learning_rate': 1.861754655362304e-05, 'epoch': 0.84}
{'loss': 1.1601, 'learning_rate': 1.8606772750713503e-05, 'epoch': 0.85}
{'loss': 1.168, 'learning_rate': 1.8595960271629412e-05, 'epoch': 0.85}
{'loss': 1.1485, 'learning_rate': 1.8585109164958698e-05, 'epoch': 0.85}
{'loss': 1.1637, 'learning_rate': 1.857421947946288e-05, 'epoch': 0.85}
{'loss': 1.1241, 'learning_rate': 1.8563291264076834e-05, 'epoch': 0.86}
{'loss': 1.1053, 'learning_rate': 1.8552324567908585e-05, 'epoch': 0.86}
{'loss': 1.1382, 'learning_rate': 1.8541319440239066e-05, 'epoch': 0.86}
{'loss': 1.1319, 'learning_rate': 1.8530275930521925e-05, 'epoch': 0.86}
{'loss': 1.0963, 'learning_rate': 1.851919408838327e-05, 'epoch': 0.87}
{'loss': 1.127, 'learning_rate': 1.8508073963621482e-05, 'epoch': 0.87}
{'loss': 1.1608, 'learning_rate': 1.8496915606206952e-05, 'epoch': 0.87}
{'loss': 1.158, 'learning_rate': 1.8485719066281894e-05, 'epoch': 0.87}
{'loss': 1.1696, 'learning_rate': 1.847448439416009e-05, 'epoch': 0.88}
{'loss': 1.1744, 'learning_rate': 1.8463211640326686e-05, 'epoch': 0.88}
{'loss': 1.1363, 'learning_rate': 1.845190085543795e-05, 'epoch': 0.88}
{'loss': 1.1001, 'learning_rate': 1.8440552090321047e-05, 'epoch': 0.88}
{'loss': 1.1647, 'learning_rate': 1.842916539597382e-05, 'epoch': 0.89}
{'loss': 1.119, 'learning_rate': 1.8417740823564552e-05, 'epoch': 0.89}
{'loss': 1.1718, 'learning_rate': 1.8406278424431737e-05, 'epoch': 0.89}
{'loss': 1.1101, 'learning_rate': 1.839477825008385e-05, 'epoch': 0.89}
{'loss': 1.1245, 'learning_rate': 1.8383240352199118e-05, 'epoch': 0.9}
{'loss': 1.1056, 'learning_rate': 1.8371664782625287e-05, 'epoch': 0.9}
{'loss': 1.1578, 'learning_rate': 1.8360051593379383e-05, 'epoch': 0.9}
{'loss': 1.1112, 'learning_rate': 1.8348400836647492e-05, 'epoch': 0.91}
{'loss': 1.1375, 'learning_rate': 1.8336712564784506e-05, 'epoch': 0.91}
{'loss': 1.1008, 'learning_rate': 1.83249868303139e-05, 'epoch': 0.91}
{'loss': 1.1385, 'learning_rate': 1.8313223685927507e-05, 'epoch': 0.91}
{'loss': 1.0728, 'learning_rate': 1.8301423184485253e-05, 'epoch': 0.92}
{'loss': 1.1382, 'learning_rate': 1.8289585379014942e-05, 'epoch': 0.92}
{'loss': 1.1376, 'learning_rate': 1.827771032271201e-05, 'epoch': 0.92}
{'loss': 1.2051, 'learning_rate': 1.8265798068939295e-05, 'epoch': 0.92}
{'loss': 1.1694, 'learning_rate': 1.825384867122677e-05, 'epoch': 0.93}
{'loss': 1.1092, 'learning_rate': 1.8241862183271338e-05, 'epoch': 0.93}
{'loss': 1.1343, 'learning_rate': 1.8229838658936566e-05, 'epoch': 0.93}
{'loss': 1.1233, 'learning_rate': 1.821777815225245e-05, 'epoch': 0.93}
{'loss': 1.0899, 'learning_rate': 1.8205680717415188e-05, 'epoch': 0.94}
{'loss': 1.1064, 'learning_rate': 1.81935464087869e-05, 'epoch': 0.94}
{'loss': 1.1538, 'learning_rate': 1.8181375280895417e-05, 'epoch': 0.94}
{'loss': 1.1558, 'learning_rate': 1.8169167388434024e-05, 'epoch': 0.94}
{'loss': 1.1142, 'learning_rate': 1.815692278626122e-05, 'epoch': 0.95}
{'loss': 1.1397, 'learning_rate': 1.8144641529400445e-05, 'epoch': 0.95}
{'loss': 1.1285, 'learning_rate': 1.8132323673039883e-05, 'epoch': 0.95}
{'loss': 1.1637, 'learning_rate': 1.8119969272532164e-05, 'epoch': 0.95}
{'loss': 1.1269, 'learning_rate': 1.8107578383394145e-05, 'epoch': 0.96}
{'loss': 1.1177, 'learning_rate': 1.8095151061306647e-05, 'epoch': 0.96}
{'loss': 1.1103, 'learning_rate': 1.808268736211421e-05, 'epoch': 0.96}
{'loss': 1.1156, 'learning_rate': 1.8070187341824848e-05, 'epoch': 0.96}
{'loss': 1.0974, 'learning_rate': 1.8057651056609784e-05, 'epoch': 0.97}
{'loss': 1.1858, 'learning_rate': 1.8045078562803203e-05, 'epoch': 0.97}
{'loss': 1.1328, 'learning_rate': 1.8032469916902004e-05, 'epoch': 0.97}
{'loss': 1.1207, 'learning_rate': 1.8019825175565544e-05, 'epoch': 0.97}
{'loss': 1.1782, 'learning_rate': 1.800714439561538e-05, 'epoch': 0.98}
{'loss': 1.1429, 'learning_rate': 1.7994427634035016e-05, 'epoch': 0.98}
{'loss': 1.1238, 'learning_rate': 1.7981674947969635e-05, 'epoch': 0.98}
{'loss': 1.1004, 'learning_rate': 1.7968886394725876e-05, 'epoch': 0.98}
{'loss': 1.096, 'learning_rate': 1.7956062031771533e-05, 'epoch': 0.99}
{'loss': 1.085, 'learning_rate': 1.7943201916735337e-05, 'epoch': 0.99}
{'loss': 1.1077, 'learning_rate': 1.793030610740665e-05, 'epoch': 0.99}
{'loss': 1.104, 'learning_rate': 1.791737466173527e-05, 'epoch': 0.99}
{'loss': 1.1207, 'learning_rate': 1.7904407637831097e-05, 'epoch': 1.0}
{'loss': 1.1076, 'learning_rate': 1.789140509396394e-05, 'epoch': 1.0}
25%|██▌ | 780/3120 [1:04:30<3:18:41, 5.09s/it]/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
{'loss': 0.8392, 'learning_rate': 1.7878367088563195e-05, 'epoch': 1.0}
{'loss': 0.7721, 'learning_rate': 1.7865293680217636e-05, 'epoch': 1.01}
{'loss': 0.7814, 'learning_rate': 1.7852184927675113e-05, 'epoch': 1.01}
{'loss': 0.7843, 'learning_rate': 1.7839040889842307e-05, 'epoch': 1.01}
{'loss': 0.7827, 'learning_rate': 1.7825861625784457e-05, 'epoch': 1.01}
{'loss': 0.8196, 'learning_rate': 1.7812647194725093e-05, 'epoch': 1.02}
{'loss': 0.8131, 'learning_rate': 1.779939765604579e-05, 'epoch': 1.02}
{'loss': 0.7676, 'learning_rate': 1.7786113069285877e-05, 'epoch': 1.02}
{'loss': 0.8016, 'learning_rate': 1.777279349414217e-05, 'epoch': 1.02}
{'loss': 0.7868, 'learning_rate': 1.7759438990468726e-05, 'epoch': 1.03}
{'loss': 0.8294, 'learning_rate': 1.7746049618276545e-05, 'epoch': 1.03}
{'loss': 0.7904, 'learning_rate': 1.7732625437733338e-05, 'epoch': 1.03}
{'loss': 0.7852, 'learning_rate': 1.771916650916321e-05, 'epoch': 1.03}
{'loss': 0.772, 'learning_rate': 1.7705672893046425e-05, 'epoch': 1.04}
{'loss': 0.772, 'learning_rate': 1.7692144650019125e-05, 'epoch': 1.04}
{'loss': 0.7656, 'learning_rate': 1.767858184087304e-05, 'epoch': 1.04}
{'loss': 0.7976, 'learning_rate': 1.7664984526555247e-05, 'epoch': 1.04}
{'loss': 0.8023, 'learning_rate': 1.765135276816787e-05, 'epoch': 1.05}
{'loss': 0.7723, 'learning_rate': 1.7637686626967815e-05, 'epoch': 1.05}
{'loss': 0.8008, 'learning_rate': 1.7623986164366487e-05, 'epoch': 1.05}
{'loss': 0.7881, 'learning_rate': 1.7610251441929532e-05, 'epoch': 1.05}
{'loss': 0.7723, 'learning_rate': 1.7596482521376546e-05, 'epoch': 1.06}
{'loss': 0.7637, 'learning_rate': 1.7582679464580796e-05, 'epoch': 1.06}
{'loss': 0.8181, 'learning_rate': 1.7568842333568952e-05, 'epoch': 1.06}
{'loss': 0.8291, 'learning_rate': 1.75549711905208e-05, 'epoch': 1.06}
{'loss': 0.7597, 'learning_rate': 1.7541066097768965e-05, 'epoch': 1.07}
{'loss': 0.8401, 'learning_rate': 1.7527127117798635e-05, 'epoch': 1.07}
{'loss': 0.8012, 'learning_rate': 1.7513154313247273e-05, 'epoch': 1.07}
{'loss': 0.8042, 'learning_rate': 1.7499147746904337e-05, 'epoch': 1.07}
{'loss': 0.7931, 'learning_rate': 1.7485107481711014e-05, 'epoch': 1.08}
{'loss': 0.815, 'learning_rate': 1.7471033580759903e-05, 'epoch': 1.08}
{'loss': 0.8042, 'learning_rate': 1.7456926107294765e-05, 'epoch': 1.08}
{'loss': 0.7814, 'learning_rate': 1.7442785124710227e-05, 'epoch': 1.08}
{'loss': 0.7983, 'learning_rate': 1.742861069655148e-05, 'epoch': 1.09}
{'loss': 0.8045, 'learning_rate': 1.741440288651403e-05, 'epoch': 1.09}
{'loss': 0.7718, 'learning_rate': 1.7400161758443377e-05, 'epoch': 1.09}
{'loss': 0.8006, 'learning_rate': 1.7385887376334743e-05, 'epoch': 1.09}
{'loss': 0.7937, 'learning_rate': 1.737157980433279e-05, 'epoch': 1.1}
{'loss': 0.7888, 'learning_rate': 1.735723910673132e-05, 'epoch': 1.1}
{'loss': 0.8135, 'learning_rate': 1.7342865347972987e-05, 'epoch': 1.1}
{'loss': 0.805, 'learning_rate': 1.7328458592649028e-05, 'epoch': 1.11}
{'loss': 0.7874, 'learning_rate': 1.7314018905498932e-05, 'epoch': 1.11}
{'loss': 0.773, 'learning_rate': 1.7299546351410197e-05, 'epoch': 1.11}
{'loss': 0.8026, 'learning_rate': 1.7285040995418003e-05, 'epoch': 1.11}
{'loss': 0.8232, 'learning_rate': 1.7270502902704925e-05, 'epoch': 1.12}
{'loss': 0.802, 'learning_rate': 1.7255932138600665e-05, 'epoch': 1.12}
{'loss': 0.797, 'learning_rate': 1.7241328768581725e-05, 'epoch': 1.12}
{'loss': 0.7957, 'learning_rate': 1.7226692858271133e-05, 'epoch': 1.12}
{'loss': 0.7776, 'learning_rate': 1.7212024473438145e-05, 'epoch': 1.13}
{'loss': 0.7955, 'learning_rate': 1.7197323679997943e-05, 'epoch': 1.13}
{'loss': 0.781, 'learning_rate': 1.718259054401135e-05, 'epoch': 1.13}
{'loss': 0.79, 'learning_rate': 1.7167825131684516e-05, 'epoch': 1.13}
{'loss': 0.8113, 'learning_rate': 1.7153027509368637e-05, 'epoch': 1.14}
{'loss': 0.8052, 'learning_rate': 1.7138197743559656e-05, 'epoch': 1.14}
{'loss': 0.7755, 'learning_rate': 1.7123335900897946e-05, 'epoch': 1.14}
{'loss': 0.818, 'learning_rate': 1.7108442048168038e-05, 'epoch': 1.14}
{'loss': 0.8166, 'learning_rate': 1.7093516252298296e-05, 'epoch': 1.15}
{'loss': 0.8, 'learning_rate': 1.707855858036063e-05, 'epoch': 1.15}
{'loss': 0.804, 'learning_rate': 1.7063569099570196e-05, 'epoch': 1.15}
{'loss': 0.8087, 'learning_rate': 1.7048547877285078e-05, 'epoch': 1.15}
{'loss': 0.7976, 'learning_rate': 1.7033494981006003e-05, 'epoch': 1.16}
{'loss': 0.7594, 'learning_rate': 1.7018410478376033e-05, 'epoch': 1.16}
{'loss': 0.7686, 'learning_rate': 1.7003294437180254e-05, 'epoch': 1.16}
{'loss': 0.7879, 'learning_rate': 1.6988146925345487e-05, 'epoch': 1.16}
{'loss': 0.8057, 'learning_rate': 1.6972968010939953e-05, 'epoch': 1.17}
{'loss': 0.8118, 'learning_rate': 1.695775776217301e-05, 'epoch': 1.17}
{'loss': 0.7613, 'learning_rate': 1.6942516247394806e-05, 'epoch': 1.17}
{'loss': 0.8186, 'learning_rate': 1.6927243535095995e-05, 'epoch': 1.17}
{'loss': 0.8214, 'learning_rate': 1.6911939693907422e-05, 'epoch': 1.18}
{'loss': 0.7938, 'learning_rate': 1.6896604792599813e-05, 'epoch': 1.18}
{'loss': 0.7804, 'learning_rate': 1.6881238900083472e-05, 'epoch': 1.18}
{'loss': 0.8361, 'learning_rate': 1.686584208540797e-05, 'epoch': 1.18}
{'loss': 0.7873, 'learning_rate': 1.685041441776183e-05, 'epoch': 1.19}
{'loss': 0.8094, 'learning_rate': 1.6834955966472214e-05, 'epoch': 1.19}
{'loss': 0.8054, 'learning_rate': 1.6819466801004622e-05, 'epoch': 1.19}
{'loss': 0.7698, 'learning_rate': 1.6803946990962577e-05, 'epoch': 1.19}
{'loss': 0.7648, 'learning_rate': 1.67883966060873e-05, 'epoch': 1.2}
{'loss': 0.7801, 'learning_rate': 1.6772815716257414e-05, 'epoch': 1.2}
{'loss': 0.8163, 'learning_rate': 1.6757204391488617e-05, 'epoch': 1.2}
{'loss': 0.8108, 'learning_rate': 1.6741562701933366e-05, 'epoch': 1.21}
{'loss': 0.8093, 'learning_rate': 1.672589071788059e-05, 'epoch': 1.21}
{'loss': 0.8412, 'learning_rate': 1.671018850975533e-05, 'epoch': 1.21}
{'loss': 0.7669, 'learning_rate': 1.6694456148118453e-05, 'epoch': 1.21}
{'loss': 0.7766, 'learning_rate': 1.6678693703666327e-05, 'epoch': 1.22}
{'loss': 0.8172, 'learning_rate': 1.66629012472305e-05, 'epoch': 1.22}
{'loss': 0.7943, 'learning_rate': 1.664707884977739e-05, 'epoch': 1.22}
{'loss': 0.7944, 'learning_rate': 1.6631226582407954e-05, 'epoch': 1.22}
{'loss': 0.7759, 'learning_rate': 1.661534451635738e-05, 'epoch': 1.23}
{'loss': 0.7853, 'learning_rate': 1.6599432722994754e-05, 'epoch': 1.23}
{'loss': 0.7879, 'learning_rate': 1.6583491273822763e-05, 'epoch': 1.23}
{'loss': 0.7834, 'learning_rate': 1.6567520240477344e-05, 'epoch': 1.23}
{'loss': 0.7992, 'learning_rate': 1.655151969472738e-05, 'epoch': 1.24}
{'loss': 0.801, 'learning_rate': 1.653548970847438e-05, 'epoch': 1.24}
{'loss': 0.7843, 'learning_rate': 1.6519430353752138e-05, 'epoch': 1.24}
{'loss': 0.7752, 'learning_rate': 1.6503341702726428e-05, 'epoch': 1.24}
{'loss': 0.8022, 'learning_rate': 1.6487223827694673e-05, 'epoch': 1.25}
{'loss': 0.8492, 'learning_rate': 1.6471076801085618e-05, 'epoch': 1.25}
{'loss': 0.7996, 'learning_rate': 1.6454900695459e-05, 'epoch': 1.25}
{'loss': 0.7611, 'learning_rate': 1.643869558350524e-05, 'epoch': 1.25}
{'loss': 0.7562, 'learning_rate': 1.6422461538045104e-05, 'epoch': 1.26}
{'loss': 0.7883, 'learning_rate': 1.6406198632029358e-05, 'epoch': 1.26}
{'loss': 0.7932, 'learning_rate': 1.638990693853848e-05, 'epoch': 1.26}
{'loss': 0.811, 'learning_rate': 1.63735865307823e-05, 'epoch': 1.26}
{'loss': 0.7592, 'learning_rate': 1.6357237482099682e-05, 'epoch': 1.27}
{'loss': 0.7816, 'learning_rate': 1.6340859865958193e-05, 'epoch': 1.27}
{'loss': 0.788, 'learning_rate': 1.6324453755953772e-05, 'epoch': 1.27}
{'loss': 0.8024, 'learning_rate': 1.630801922581041e-05, 'epoch': 1.27}
{'loss': 0.7902, 'learning_rate': 1.6291556349379794e-05, 'epoch': 1.28}
{'loss': 0.791, 'learning_rate': 1.6275065200641004e-05, 'epoch': 1.28}
{'loss': 0.7769, 'learning_rate': 1.6258545853700157e-05, 'epoch': 1.28}
{'loss': 0.8325, 'learning_rate': 1.6241998382790095e-05, 'epoch': 1.28}
{'loss': 0.7926, 'learning_rate': 1.622542286227003e-05, 'epoch': 1.29}
{'loss': 0.8153, 'learning_rate': 1.620881936662522e-05, 'epoch': 1.29}
{'loss': 0.7851, 'learning_rate': 1.6192187970466646e-05, 'epoch': 1.29}
{'loss': 0.7744, 'learning_rate': 1.617552874853065e-05, 'epoch': 1.29}
{'loss': 0.7978, 'learning_rate': 1.615884177567863e-05, 'epoch': 1.3}
{'loss': 0.7764, 'learning_rate': 1.6142127126896682e-05, 'epoch': 1.3}
{'loss': 0.7693, 'learning_rate': 1.6125384877295255e-05, 'epoch': 1.3}
{'loss': 0.8497, 'learning_rate': 1.6108615102108854e-05, 'epoch': 1.31}
{'loss': 0.7874, 'learning_rate': 1.6091817876695655e-05, 'epoch': 1.31}
{'loss': 0.7844, 'learning_rate': 1.6074993276537198e-05, 'epoch': 1.31}
{'loss': 0.8288, 'learning_rate': 1.6058141377238026e-05, 'epoch': 1.31}
{'loss': 0.8386, 'learning_rate': 1.6041262254525362e-05, 'epoch': 1.32}
{'loss': 0.7879, 'learning_rate': 1.602435598424877e-05, 'epoch': 1.32}
{'loss': 0.8181, 'learning_rate': 1.600742264237979e-05, 'epoch': 1.32}
{'loss': 0.7813, 'learning_rate': 1.599046230501163e-05, 'epoch': 1.32}
{'loss': 0.8164, 'learning_rate': 1.5973475048358795e-05, 'epoch': 1.33}
{'loss': 0.8212, 'learning_rate': 1.5956460948756765e-05, 'epoch': 1.33}
{'loss': 0.7928, 'learning_rate': 1.593942008266164e-05, 'epoch': 1.33}
{'loss': 0.7925, 'learning_rate': 1.5922352526649803e-05, 'epoch': 1.33}
{'loss': 0.7599, 'learning_rate': 1.5905258357417567e-05, 'epoch': 1.34}
{'loss': 0.7971, 'learning_rate': 1.5888137651780847e-05, 'epoch': 1.34}
{'loss': 0.8046, 'learning_rate': 1.5870990486674794e-05, 'epoch': 1.34}
{'loss': 0.7811, 'learning_rate': 1.585381693915346e-05, 'epoch': 1.34}
{'loss': 0.7737, 'learning_rate': 1.583661708638947e-05, 'epoch': 1.35}
{'loss': 0.7909, 'learning_rate': 1.581939100567363e-05, 'epoch': 1.35}
{'loss': 0.8154, 'learning_rate': 1.580213877441462e-05, 'epoch': 1.35}
{'loss': 0.831, 'learning_rate': 1.5784860470138633e-05, 'epoch': 1.35}
{'loss': 0.7842, 'learning_rate': 1.5767556170489027e-05, 'epoch': 1.36}
{'loss': 0.7851, 'learning_rate': 1.5750225953225968e-05, 'epoch': 1.36}
{'loss': 0.8102, 'learning_rate': 1.573286989622609e-05, 'epoch': 1.36}
{'loss': 0.842, 'learning_rate': 1.5715488077482152e-05, 'epoch': 1.36}
{'loss': 0.7943, 'learning_rate': 1.5698080575102662e-05, 'epoch': 1.37}
{'loss': 0.7573, 'learning_rate': 1.568064746731156e-05, 'epoch': 1.37}
{'loss': 0.837, 'learning_rate': 1.5663188832447833e-05, 'epoch': 1.37}
{'loss': 0.8127, 'learning_rate': 1.5645704748965193e-05, 'epoch': 1.37}
{'loss': 0.7727, 'learning_rate': 1.5628195295431696e-05, 'epoch': 1.38}
{'loss': 0.7898, 'learning_rate': 1.5610660550529413e-05, 'epoch': 1.38}
{'loss': 0.7841, 'learning_rate': 1.5593100593054063e-05, 'epoch': 1.38}
{'loss': 0.8299, 'learning_rate': 1.557551550191467e-05, 'epoch': 1.38}
{'loss': 0.8236, 'learning_rate': 1.5557905356133182e-05, 'epoch': 1.39}
{'loss': 0.8314, 'learning_rate': 1.554027023484416e-05, 'epoch': 1.39}
{'loss': 0.8049, 'learning_rate': 1.5522610217294377e-05, 'epoch': 1.39}
{'loss': 0.7687, 'learning_rate': 1.550492538284249e-05, 'epoch': 1.39}
{'loss': 0.8071, 'learning_rate': 1.5487215810958675e-05, 'epoch': 1.4}
{'loss': 0.7917, 'learning_rate': 1.5469481581224274e-05, 'epoch': 1.4}
{'loss': 0.7914, 'learning_rate': 1.545172277333142e-05, 'epoch': 1.4}
{'loss': 0.8045, 'learning_rate': 1.5433939467082713e-05, 'epoch': 1.41}
{'loss': 0.7788, 'learning_rate': 1.5416131742390827e-05, 'epoch': 1.41}
{'loss': 0.851, 'learning_rate': 1.5398299679278172e-05, 'epoch': 1.41}
{'loss': 0.7832, 'learning_rate': 1.5380443357876516e-05, 'epoch': 1.41}
{'loss': 0.7591, 'learning_rate': 1.5362562858426655e-05, 'epoch': 1.42}
{'loss': 0.767, 'learning_rate': 1.5344658261278013e-05, 'epoch': 1.42}
{'loss': 0.8645, 'learning_rate': 1.5326729646888314e-05, 'epoch': 1.42}
{'loss': 0.7996, 'learning_rate': 1.530877709582321e-05, 'epoch': 1.42}
{'loss': 0.8467, 'learning_rate': 1.5290800688755906e-05, 'epoch': 1.43}
{'loss': 0.8371, 'learning_rate': 1.527280050646682e-05, 'epoch': 1.43}
{'loss': 0.8251, 'learning_rate': 1.5254776629843204e-05, 'epoch': 1.43}
{'loss': 0.8115, 'learning_rate': 1.523672913987878e-05, 'epoch': 1.43}
{'loss': 0.8108, 'learning_rate': 1.5218658117673389e-05, 'epoch': 1.44}
{'loss': 0.7774, 'learning_rate': 1.5200563644432614e-05, 'epoch': 1.44}
{'loss': 0.7728, 'learning_rate': 1.518244580146742e-05, 'epoch': 1.44}
{'loss': 0.7847, 'learning_rate': 1.5164304670193792e-05, 'epoch': 1.44}
{'loss': 0.8291, 'learning_rate': 1.5146140332132359e-05, 'epoch': 1.45}
{'loss': 0.7675, 'learning_rate': 1.5127952868908043e-05, 'epoch': 1.45}
{'loss': 0.8281, 'learning_rate': 1.5109742362249673e-05, 'epoch': 1.45}
{'loss': 0.8432, 'learning_rate': 1.5091508893989633e-05, 'epoch': 1.45}
{'loss': 0.7984, 'learning_rate': 1.5073252546063493e-05, 'epoch': 1.46}
{'loss': 0.8033, 'learning_rate': 1.5054973400509631e-05, 'epoch': 1.46}
{'loss': 0.8412, 'learning_rate': 1.5036671539468879e-05, 'epoch': 1.46}
{'loss': 0.7544, 'learning_rate': 1.5018347045184132e-05, 'epoch': 1.46}
{'loss': 0.7962, 'learning_rate': 1.5000000000000002e-05, 'epoch': 1.47}
{'loss': 0.8351, 'learning_rate': 1.4981630486362435e-05, 'epoch': 1.47}
{'loss': 0.7913, 'learning_rate': 1.4963238586818346e-05, 'epoch': 1.47}
{'loss': 0.8417, 'learning_rate': 1.4944824384015238e-05, 'epoch': 1.47}
{'loss': 0.766, 'learning_rate': 1.4926387960700843e-05, 'epoch': 1.48}
{'loss': 0.7899, 'learning_rate': 1.4907929399722752e-05, 'epoch': 1.48}
{'loss': 0.7746, 'learning_rate': 1.488944878402802e-05, 'epoch': 1.48}
{'loss': 0.8103, 'learning_rate': 1.4870946196662822e-05, 'epoch': 1.48}
{'loss': 0.8422, 'learning_rate': 1.4852421720772064e-05, 'epoch': 1.49}
{'loss': 0.8136, 'learning_rate': 1.4833875439599004e-05, 'epoch': 1.49}
{'loss': 0.7957, 'learning_rate': 1.4815307436484898e-05, 'epoch': 1.49}
{'loss': 0.8081, 'learning_rate': 1.4796717794868606e-05, 'epoch': 1.49}
{'loss': 0.8154, 'learning_rate': 1.4778106598286235e-05, 'epoch': 1.5}
{'loss': 0.8193, 'learning_rate': 1.4759473930370738e-05, 'epoch': 1.5}
{'loss': 0.7687, 'learning_rate': 1.4740819874851562e-05, 'epoch': 1.5}
{'loss': 0.7366, 'learning_rate': 1.4722144515554263e-05, 'epoch': 1.51}
{'loss': 0.7733, 'learning_rate': 1.4703447936400135e-05, 'epoch': 1.51}
{'loss': 0.8318, 'learning_rate': 1.4684730221405814e-05, 'epoch': 1.51}
{'loss': 0.8014, 'learning_rate': 1.4665991454682924e-05, 'epoch': 1.51}
{'loss': 0.7706, 'learning_rate': 1.4647231720437687e-05, 'epoch': 1.52}
{'loss': 0.8441, 'learning_rate': 1.4628451102970546e-05, 'epoch': 1.52}
{'loss': 0.7787, 'learning_rate': 1.4609649686675786e-05, 'epoch': 1.52}
{'loss': 0.8178, 'learning_rate': 1.4590827556041158e-05, 'epoch': 1.52}
{'loss': 0.823, 'learning_rate': 1.4571984795647494e-05, 'epoch': 1.53}
{'loss': 0.8072, 'learning_rate': 1.4553121490168335e-05, 'epoch': 1.53}
{'loss': 0.8222, 'learning_rate': 1.4534237724369534e-05, 'epoch': 1.53}
{'loss': 0.776, 'learning_rate': 1.4515333583108896e-05, 'epoch': 1.53}
{'loss': 0.7757, 'learning_rate': 1.4496409151335786e-05, 'epoch': 1.54}
{'loss': 0.8204, 'learning_rate': 1.4477464514090745e-05, 'epoch': 1.54}
{'loss': 0.7843, 'learning_rate': 1.4458499756505116e-05, 'epoch': 1.54}
{'loss': 0.8112, 'learning_rate': 1.443951496380065e-05, 'epoch': 1.54}
{'loss': 0.8246, 'learning_rate': 1.4420510221289137e-05, 'epoch': 1.55}
{'loss': 0.7993, 'learning_rate': 1.4401485614372009e-05, 'epoch': 1.55}
{'loss': 0.8087, 'learning_rate': 1.4382441228539962e-05, 'epoch': 1.55}
{'loss': 0.7602, 'learning_rate': 1.4363377149372584e-05, 'epoch': 1.55}
{'loss': 0.7935, 'learning_rate': 1.434429346253794e-05, 'epoch': 1.56}
{'loss': 0.8377, 'learning_rate': 1.4325190253792222e-05, 'epoch': 1.56}
{'loss': 0.8346, 'learning_rate': 1.430606760897934e-05, 'epoch': 1.56}
{'loss': 0.7851, 'learning_rate': 1.4286925614030542e-05, 'epoch': 1.56}
{'loss': 0.787, 'learning_rate': 1.4267764354964038e-05, 'epoch': 1.57}
{'loss': 0.8389, 'learning_rate': 1.4248583917884595e-05, 'epoch': 1.57}
{'loss': 0.7794, 'learning_rate': 1.4229384388983167e-05, 'epoch': 1.57}
{'loss': 0.8116, 'learning_rate': 1.4210165854536495e-05, 'epoch': 1.57}
{'loss': 0.8146, 'learning_rate': 1.4190928400906731e-05, 'epoch': 1.58}
{'loss': 0.8344, 'learning_rate': 1.4171672114541042e-05, 'epoch': 1.58}
{'loss': 0.8259, 'learning_rate': 1.415239708197122e-05, 'epoch': 1.58}
{'loss': 0.7878, 'learning_rate': 1.4133103389813302e-05, 'epoch': 1.58}
{'loss': 0.7825, 'learning_rate': 1.411379112476717e-05, 'epoch': 1.59}
{'loss': 0.7542, 'learning_rate': 1.409446037361617e-05, 'epoch': 1.59}
{'loss': 0.8001, 'learning_rate': 1.4075111223226721e-05, 'epoch': 1.59}
{'loss': 0.8038, 'learning_rate': 1.4055743760547918e-05, 'epoch': 1.59}
{'loss': 0.7916, 'learning_rate': 1.4036358072611147e-05, 'epoch': 1.6}
{'loss': 0.8181, 'learning_rate': 1.4016954246529697e-05, 'epoch': 1.6}
{'loss': 0.8473, 'learning_rate': 1.3997532369498357e-05, 'epoch': 1.6}
{'loss': 0.7634, 'learning_rate': 1.3978092528793032e-05, 'epoch': 1.61}
{'loss': 0.8079, 'learning_rate': 1.3958634811770361e-05, 'epoch': 1.61}
{'loss': 0.8156, 'learning_rate': 1.39391593058673e-05, 'epoch': 1.61}
{'loss': 0.7625, 'learning_rate': 1.3919666098600753e-05, 'epoch': 1.61}
{'loss': 0.803, 'learning_rate': 1.3900155277567157e-05, 'epoch': 1.62}
{'loss': 0.8364, 'learning_rate': 1.3880626930442115e-05, 'epoch': 1.62}
{'loss': 0.7946, 'learning_rate': 1.3861081144979975e-05, 'epoch': 1.62}
{'loss': 0.8737, 'learning_rate': 1.3841518009013446e-05, 'epoch': 1.62}
{'loss': 0.8138, 'learning_rate': 1.382193761045322e-05, 'epoch': 1.63}
{'loss': 0.7597, 'learning_rate': 1.380234003728754e-05, 'epoch': 1.63}
{'loss': 0.8276, 'learning_rate': 1.378272537758185e-05, 'epoch': 1.63}
{'loss': 0.8108, 'learning_rate': 1.3763093719478357e-05, 'epoch': 1.63}
{'loss': 0.8137, 'learning_rate': 1.3743445151195658e-05, 'epoch': 1.64}
{'loss': 0.8055, 'learning_rate': 1.3723779761028349e-05, 'epoch': 1.64}
{'loss': 0.7624, 'learning_rate': 1.37040976373466e-05, 'epoch': 1.64}
{'loss': 0.862, 'learning_rate': 1.36843988685958e-05, 'epoch': 1.64}
{'loss': 0.7821, 'learning_rate': 1.3664683543296114e-05, 'epoch': 1.65}
{'loss': 0.758, 'learning_rate': 1.3644951750042115e-05, 'epoch': 1.65}
{'loss': 0.8178, 'learning_rate': 1.3625203577502384e-05, 'epoch': 1.65}
{'loss': 0.8348, 'learning_rate': 1.3605439114419095e-05, 'epoch': 1.65}
{'loss': 0.8298, 'learning_rate': 1.3585658449607632e-05, 'epoch': 1.66}
{'loss': 0.8194, 'learning_rate': 1.3565861671956189e-05, 'epoch': 1.66}
{'loss': 0.8231, 'learning_rate': 1.3546048870425356e-05, 'epoch': 1.66}
{'loss': 0.8115, 'learning_rate': 1.352622013404774e-05, 'epoch': 1.66}
{'loss': 0.7935, 'learning_rate': 1.3506375551927546e-05, 'epoch': 1.67}
{'loss': 0.7573, 'learning_rate': 1.3486515213240188e-05, 'epoch': 1.67}
{'loss': 0.819, 'learning_rate': 1.3466639207231882e-05, 'epoch': 1.67}
{'loss': 0.8032, 'learning_rate': 1.3446747623219256e-05, 'epoch': 1.67}
{'loss': 0.7717, 'learning_rate': 1.3426840550588933e-05, 'epoch': 1.68}
{'loss': 0.7919, 'learning_rate': 1.3406918078797142e-05, 'epoch': 1.68}
{'loss': 0.7638, 'learning_rate': 1.3386980297369308e-05, 'epoch': 1.68}
{'loss': 0.805, 'learning_rate': 1.3367027295899652e-05, 'epoch': 1.68}
{'loss': 0.8378, 'learning_rate': 1.3347059164050796e-05, 'epoch': 1.69}
{'loss': 0.8601, 'learning_rate': 1.332707599155334e-05, 'epoch': 1.69}
{'loss': 0.827, 'learning_rate': 1.3307077868205487e-05, 'epoch': 1.69}
{'loss': 0.8289, 'learning_rate': 1.3287064883872614e-05, 'epoch': 1.69}
{'loss': 0.8072, 'learning_rate': 1.3267037128486883e-05, 'epoch': 1.7}
{'loss': 0.7881, 'learning_rate': 1.3246994692046837e-05, 'epoch': 1.7}
{'loss': 0.7684, 'learning_rate': 1.3226937664616977e-05, 'epoch': 1.7}
{'loss': 0.7952, 'learning_rate': 1.3206866136327387e-05, 'epoch': 1.71}
{'loss': 0.8064, 'learning_rate': 1.3186780197373306e-05, 'epoch': 1.71}
{'loss': 0.8248, 'learning_rate': 1.3166679938014728e-05, 'epoch': 1.71}
{'loss': 0.7767, 'learning_rate': 1.3146565448576002e-05, 'epoch': 1.71}
{'loss': 0.7899, 'learning_rate': 1.3126436819445423e-05, 'epoch': 1.72}
{'loss': 0.7839, 'learning_rate': 1.3106294141074825e-05, 'epoch': 1.72}
{'loss': 0.7953, 'learning_rate': 1.308613750397917e-05, 'epoch': 1.72}
{'loss': 0.7935, 'learning_rate': 1.3065966998736155e-05, 'epoch': 1.72}
{'loss': 0.8131, 'learning_rate': 1.304578271598579e-05, 'epoch': 1.73}
{'loss': 0.7642, 'learning_rate': 1.302558474643e-05, 'epoch': 1.73}
{'loss': 0.7821, 'learning_rate': 1.300537318083221e-05, 'epoch': 1.73}
{'loss': 0.8327, 'learning_rate': 1.2985148110016947e-05, 'epoch': 1.73}
{'loss': 0.7973, 'learning_rate': 1.2964909624869422e-05, 'epoch': 1.74}
{'loss': 0.7698, 'learning_rate': 1.2944657816335124e-05, 'epoch': 1.74}
{'loss': 0.8402, 'learning_rate': 1.2924392775419421e-05, 'epoch': 1.74}
{'loss': 0.7686, 'learning_rate': 1.2904114593187136e-05, 'epoch': 1.74}
{'loss': 0.8225, 'learning_rate': 1.2883823360762149e-05, 'epoch': 1.75}
{'loss': 0.7842, 'learning_rate': 1.2863519169326984e-05, 'epoch': 1.75}
{'loss': 0.8096, 'learning_rate': 1.284320211012239e-05, 'epoch': 1.75}
{'loss': 0.7605, 'learning_rate': 1.2822872274446958e-05, 'epoch': 1.75}
{'loss': 0.7653, 'learning_rate': 1.280252975365667e-05, 'epoch': 1.76}
{'loss': 0.8186, 'learning_rate': 1.2782174639164528e-05, 'epoch': 1.76}
{'loss': 0.8052, 'learning_rate': 1.276180702244012e-05, 'epoch': 1.76}
{'loss': 0.8106, 'learning_rate': 1.2741426995009214e-05, 'epoch': 1.76}
{'loss': 0.8028, 'learning_rate': 1.2721034648453353e-05, 'epoch': 1.77}
{'loss': 0.7903, 'learning_rate': 1.2700630074409427e-05, 'epoch': 1.77}
{'loss': 0.7951, 'learning_rate': 1.268021336456929e-05, 'epoch': 1.77}
{'loss': 0.7863, 'learning_rate': 1.2659784610679318e-05, 'epoch': 1.77}
{'loss': 0.8114, 'learning_rate': 1.2639343904540008e-05, 'epoch': 1.78}
{'loss': 0.7654, 'learning_rate': 1.2618891338005574e-05, 'epoch': 1.78}
{'loss': 0.8028, 'learning_rate': 1.2598427002983523e-05, 'epoch': 1.78}
{'loss': 0.8131, 'learning_rate': 1.2577950991434249e-05, 'epoch': 1.78}
{'loss': 0.8281, 'learning_rate': 1.2557463395370609e-05, 'epoch': 1.79}
{'loss': 0.779, 'learning_rate': 1.2536964306857526e-05, 'epoch': 1.79}
{'loss': 0.7945, 'learning_rate': 1.2516453818011567e-05, 'epoch': 1.79}
{'loss': 0.7965, 'learning_rate': 1.2495932021000516e-05, 'epoch': 1.79}
{'loss': 0.7763, 'learning_rate': 1.247539900804299e-05, 'epoch': 1.8}
{'loss': 0.8241, 'learning_rate': 1.2454854871407993e-05, 'epoch': 1.8}
{'loss': 0.8058, 'learning_rate': 1.2434299703414523e-05, 'epoch': 1.8}
{'loss': 0.8203, 'learning_rate': 1.2413733596431141e-05, 'epoch': 1.81}
{'loss': 0.7938, 'learning_rate': 1.2393156642875579e-05, 'epoch': 1.81}
{'loss': 0.7953, 'learning_rate': 1.2372568935214298e-05, 'epoch': 1.81}
{'loss': 0.782, 'learning_rate': 1.2351970565962083e-05, 'epoch': 1.81}
{'loss': 0.8193, 'learning_rate': 1.2331361627681645e-05, 'epoch': 1.82}
{'loss': 0.8184, 'learning_rate': 1.2310742212983167e-05, 'epoch': 1.82}
{'loss': 0.8209, 'learning_rate': 1.2290112414523927e-05, 'epoch': 1.82}
{'loss': 0.7982, 'learning_rate': 1.2269472325007858e-05, 'epoch': 1.82}
{'loss': 0.779, 'learning_rate': 1.2248822037185137e-05, 'epoch': 1.83}
{'loss': 0.7905, 'learning_rate': 1.222816164385177e-05, 'epoch': 1.83}
{'loss': 0.743, 'learning_rate': 1.2207491237849174e-05, 'epoch': 1.83}
{'loss': 0.8098, 'learning_rate': 1.218681091206376e-05, 'epoch': 1.83}
{'loss': 0.7813, 'learning_rate': 1.2166120759426515e-05, 'epoch': 1.84}
{'loss': 0.7944, 'learning_rate': 1.2145420872912586e-05, 'epoch': 1.84}
{'loss': 0.8159, 'learning_rate': 1.2124711345540861e-05, 'epoch': 1.84}
{'loss': 0.7834, 'learning_rate': 1.2103992270373548e-05, 'epoch': 1.84}
{'loss': 0.8172, 'learning_rate': 1.2083263740515764e-05, 'epoch': 1.85}
{'loss': 0.7996, 'learning_rate': 1.2062525849115107e-05, 'epoch': 1.85}
{'loss': 0.8369, 'learning_rate': 1.2041778689361254e-05, 'epoch': 1.85}
{'loss': 0.767, 'learning_rate': 1.2021022354485514e-05, 'epoch': 1.85}
{'loss': 0.8078, 'learning_rate': 1.2000256937760446e-05, 'epoch': 1.86}
{'loss': 0.7994, 'learning_rate': 1.1979482532499402e-05, 'epoch': 1.86}
{'loss': 0.7912, 'learning_rate': 1.1958699232056135e-05, 'epoch': 1.86}
{'loss': 0.81, 'learning_rate': 1.1937907129824371e-05, 'epoch': 1.86}
{'loss': 0.8002, 'learning_rate': 1.1917106319237386e-05, 'epoch': 1.87}
{'loss': 0.7966, 'learning_rate': 1.1896296893767588e-05, 'epoch': 1.87}
{'loss': 0.8234, 'learning_rate': 1.1875478946926094e-05, 'epoch': 1.87}
{'loss': 0.7378, 'learning_rate': 1.1854652572262324e-05, 'epoch': 1.87}
{'loss': 0.7878, 'learning_rate': 1.1833817863363563e-05, 'epoch': 1.88}
{'loss': 0.7829, 'learning_rate': 1.1812974913854545e-05, 'epoch': 1.88}
{'loss': 0.7498, 'learning_rate': 1.1792123817397041e-05, 'epoch': 1.88}
{'loss': 0.7399, 'learning_rate': 1.1771264667689428e-05, 'epoch': 1.88}
{'loss': 0.8065, 'learning_rate': 1.1750397558466273e-05, 'epoch': 1.89}
{'loss': 0.7602, 'learning_rate': 1.1729522583497913e-05, 'epoch': 1.89}
{'loss': 0.7641, 'learning_rate': 1.1708639836590024e-05, 'epoch': 1.89}
{'loss': 0.8267, 'learning_rate': 1.1687749411583213e-05, 'epoch': 1.89}
{'loss': 0.7885, 'learning_rate': 1.1666851402352587e-05, 'epoch': 1.9}
{'loss': 0.7623, 'learning_rate': 1.164594590280734e-05, 'epoch': 1.9}
{'loss': 0.7529, 'learning_rate': 1.1625033006890316e-05, 'epoch': 1.9}
{'loss': 0.7962, 'learning_rate': 1.1604112808577603e-05, 'epoch': 1.91}
{'loss': 0.7621, 'learning_rate': 1.15831854018781e-05, 'epoch': 1.91}
{'loss': 0.7941, 'learning_rate': 1.1562250880833103e-05, 'epoch': 1.91}
{'loss': 0.8247, 'learning_rate': 1.154130933951587e-05, 'epoch': 1.91}
{'loss': 0.8127, 'learning_rate': 1.1520360872031208e-05, 'epoch': 1.92}
{'loss': 0.7675, 'learning_rate': 1.1499405572515059e-05, 'epoch': 1.92}
{'loss': 0.7718, 'learning_rate': 1.147844353513405e-05, 'epoch': 1.92}
{'loss': 0.839, 'learning_rate': 1.1457474854085095e-05, 'epoch': 1.92}
{'loss': 0.7915, 'learning_rate': 1.1436499623594963e-05, 'epoch': 1.93}
{'loss': 0.8186, 'learning_rate': 1.1415517937919846e-05, 'epoch': 1.93}
{'loss': 0.8163, 'learning_rate': 1.139452989134496e-05, 'epoch': 1.93}
{'loss': 0.8156, 'learning_rate': 1.1373535578184083e-05, 'epoch': 1.93}
{'loss': 0.8194, 'learning_rate': 1.1352535092779174e-05, 'epoch': 1.94}
{'loss': 0.7787, 'learning_rate': 1.1331528529499909e-05, 'epoch': 1.94}
{'loss': 0.7798, 'learning_rate': 1.1310515982743293e-05, 'epoch': 1.94}
{'loss': 0.7784, 'learning_rate': 1.1289497546933212e-05, 'epoch': 1.94}
{'loss': 0.7827, 'learning_rate': 1.1268473316520007e-05, 'epoch': 1.95}
{'loss': 0.7978, 'learning_rate': 1.124744338598008e-05, 'epoch': 1.95}
{'loss': 0.7453, 'learning_rate': 1.1226407849815422e-05, 'epoch': 1.95}
{'loss': 0.8078, 'learning_rate': 1.1205366802553231e-05, 'epoch': 1.95}
{'loss': 0.7637, 'learning_rate': 1.1184320338745467e-05, 'epoch': 1.96}
{'loss': 0.7807, 'learning_rate': 1.1163268552968422e-05, 'epoch': 1.96}
{'loss': 0.8054, 'learning_rate': 1.1142211539822318e-05, 'epoch': 1.96}
{'loss': 0.826, 'learning_rate': 1.112114939393085e-05, 'epoch': 1.96}
{'loss': 0.8095, 'learning_rate': 1.1100082209940795e-05, 'epoch': 1.97}
{'loss': 0.7496, 'learning_rate': 1.1079010082521557e-05, 'epoch': 1.97}
{'loss': 0.8398, 'learning_rate': 1.1057933106364758e-05, 'epoch': 1.97}
{'loss': 0.7913, 'learning_rate': 1.1036851376183812e-05, 'epoch': 1.97}
{'loss': 0.8017, 'learning_rate': 1.101576498671349e-05, 'epoch': 1.98}
{'loss': 0.7392, 'learning_rate': 1.0994674032709514e-05, 'epoch': 1.98}
{'loss': 0.7795, 'learning_rate': 1.0973578608948094e-05, 'epoch': 1.98}
{'loss': 0.7834, 'learning_rate': 1.095247881022555e-05, 'epoch': 1.98}
{'loss': 0.7992, 'learning_rate': 1.0931374731357843e-05, 'epoch': 1.99}
{'loss': 0.8043, 'learning_rate': 1.091026646718018e-05, 'epoch': 1.99}
{'loss': 0.7868, 'learning_rate': 1.0889154112546569e-05, 'epoch': 1.99}
{'loss': 0.8144, 'learning_rate': 1.0868037762329405e-05, 'epoch': 1.99}
{'loss': 0.7629, 'learning_rate': 1.0846917511419031e-05, 'epoch': 2.0}
{'loss': 0.8088, 'learning_rate': 1.0825793454723325e-05, 'epoch': 2.0}
50%|█████ | 1560/3120 [2:09:35<2:10:47, 5.03s/it]/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
{'loss': 0.4308, 'learning_rate': 1.0804665687167262e-05, 'epoch': 2.0}
{'loss': 0.4056, 'learning_rate': 1.0783534303692493e-05, 'epoch': 2.01}
{'loss': 0.4122, 'learning_rate': 1.0762399399256917e-05, 'epoch': 2.01}
{'loss': 0.4125, 'learning_rate': 1.0741261068834266e-05, 'epoch': 2.01}
{'loss': 0.3983, 'learning_rate': 1.0720119407413647e-05, 'epoch': 2.01}
{'loss': 0.3843, 'learning_rate': 1.0698974509999159e-05, 'epoch': 2.02}
{'loss': 0.4024, 'learning_rate': 1.0677826471609423e-05, 'epoch': 2.02}
{'loss': 0.3865, 'learning_rate': 1.0656675387277183e-05, 'epoch': 2.02}
{'loss': 0.3953, 'learning_rate': 1.0635521352048873e-05, 'epoch': 2.02}
{'loss': 0.3734, 'learning_rate': 1.0614364460984178e-05, 'epoch': 2.03}
{'loss': 0.3556, 'learning_rate': 1.0593204809155628e-05, 'epoch': 2.03}
{'loss': 0.3826, 'learning_rate': 1.057204249164815e-05, 'epoch': 2.03}
{'loss': 0.3836, 'learning_rate': 1.0550877603558656e-05, 'epoch': 2.03}
{'loss': 0.4046, 'learning_rate': 1.0529710239995606e-05, 'epoch': 2.04}
{'loss': 0.3772, 'learning_rate': 1.0508540496078582e-05, 'epoch': 2.04}
{'loss': 0.3808, 'learning_rate': 1.0487368466937866e-05, 'epoch': 2.04}
{'loss': 0.367, 'learning_rate': 1.0466194247714008e-05, 'epoch': 2.04}
{'loss': 0.3947, 'learning_rate': 1.0445017933557404e-05, 'epoch': 2.05}
{'loss': 0.3663, 'learning_rate': 1.0423839619627854e-05, 'epoch': 2.05}
{'loss': 0.4022, 'learning_rate': 1.0402659401094154e-05, 'epoch': 2.05}
{'loss': 0.3824, 'learning_rate': 1.0381477373133652e-05, 'epoch': 2.05}
{'loss': 0.3954, 'learning_rate': 1.036029363093183e-05, 'epoch': 2.06}
{'loss': 0.4046, 'learning_rate': 1.0339108269681874e-05, 'epoch': 2.06}
{'loss': 0.4107, 'learning_rate': 1.0317921384584245e-05, 'epoch': 2.06}
{'loss': 0.3856, 'learning_rate': 1.0296733070846252e-05, 'epoch': 2.06}
{'loss': 0.375, 'learning_rate': 1.0275543423681622e-05, 'epoch': 2.07}
{'loss': 0.4128, 'learning_rate': 1.0254352538310075e-05, 'epoch': 2.07}
{'loss': 0.3887, 'learning_rate': 1.0233160509956893e-05, 'epoch': 2.07}
{'loss': 0.3704, 'learning_rate': 1.0211967433852501e-05, 'epoch': 2.07}
{'loss': 0.382, 'learning_rate': 1.0190773405232024e-05, 'epoch': 2.08}
{'loss': 0.3955, 'learning_rate': 1.0169578519334873e-05, 'epoch': 2.08}
{'loss': 0.3985, 'learning_rate': 1.014838287140431e-05, 'epoch': 2.08}
{'loss': 0.3869, 'learning_rate': 1.012718655668702e-05, 'epoch': 2.08}
{'loss': 0.393, 'learning_rate': 1.010598967043268e-05, 'epoch': 2.09}
{'loss': 0.3714, 'learning_rate': 1.0084792307893551e-05, 'epoch': 2.09}
{'loss': 0.3715, 'learning_rate': 1.0063594564324014e-05, 'epoch': 2.09}
{'loss': 0.3817, 'learning_rate': 1.0042396534980176e-05, 'epoch': 2.09}
{'loss': 0.394, 'learning_rate': 1.0021198315119426e-05, 'epoch': 2.1}
{'loss': 0.3823, 'learning_rate': 1e-05, 'epoch': 2.1}
{'loss': 0.3811, 'learning_rate': 9.97880168488058e-06, 'epoch': 2.1}
{'loss': 0.4121, 'learning_rate': 9.957603465019827e-06, 'epoch': 2.11}
{'loss': 0.3859, 'learning_rate': 9.936405435675991e-06, 'epoch': 2.11}
{'loss': 0.3806, 'learning_rate': 9.91520769210645e-06, 'epoch': 2.11}
{'loss': 0.385, 'learning_rate': 9.894010329567322e-06, 'epoch': 2.11}
{'loss': 0.3755, 'learning_rate': 9.872813443312984e-06, 'epoch': 2.12}
{'loss': 0.4136, 'learning_rate': 9.851617128595694e-06, 'epoch': 2.12}
{'loss': 0.3898, 'learning_rate': 9.830421480665128e-06, 'epoch': 2.12}
{'loss': 0.4021, 'learning_rate': 9.809226594767979e-06, 'epoch': 2.12}
{'loss': 0.3928, 'learning_rate': 9.788032566147506e-06, 'epoch': 2.13}
{'loss': 0.3639, 'learning_rate': 9.766839490043108e-06, 'epoch': 2.13}
{'loss': 0.3843, 'learning_rate': 9.745647461689932e-06, 'epoch': 2.13}
{'loss': 0.3798, 'learning_rate': 9.724456576318383e-06, 'epoch': 2.13}
{'loss': 0.3915, 'learning_rate': 9.703266929153751e-06, 'epoch': 2.14}
{'loss': 0.3685, 'learning_rate': 9.682078615415755e-06, 'epoch': 2.14}
{'loss': 0.4122, 'learning_rate': 9.660891730318129e-06, 'epoch': 2.14}
{'loss': 0.3752, 'learning_rate': 9.63970636906817e-06, 'epoch': 2.14}
{'loss': 0.3927, 'learning_rate': 9.618522626866351e-06, 'epoch': 2.15}
{'loss': 0.4052, 'learning_rate': 9.597340598905851e-06, 'epoch': 2.15}
{'loss': 0.3777, 'learning_rate': 9.57616038037215e-06, 'epoch': 2.15}
{'loss': 0.4091, 'learning_rate': 9.554982066442601e-06, 'epoch': 2.15}
{'loss': 0.3733, 'learning_rate': 9.533805752285992e-06, 'epoch': 2.16}
{'loss': 0.3721, 'learning_rate': 9.512631533062138e-06, 'epoch': 2.16}
{'loss': 0.3936, 'learning_rate': 9.49145950392142e-06, 'epoch': 2.16}
{'loss': 0.3851, 'learning_rate': 9.470289760004398e-06, 'epoch': 2.16}
{'loss': 0.3796, 'learning_rate': 9.449122396441344e-06, 'epoch': 2.17}
{'loss': 0.3848, 'learning_rate': 9.427957508351852e-06, 'epoch': 2.17}
{'loss': 0.3836, 'learning_rate': 9.406795190844376e-06, 'epoch': 2.17}
{'loss': 0.3959, 'learning_rate': 9.385635539015824e-06, 'epoch': 2.17}
{'loss': 0.4006, 'learning_rate': 9.364478647951132e-06, 'epoch': 2.18}
{'loss': 0.3844, 'learning_rate': 9.343324612722819e-06, 'epoch': 2.18}
{'loss': 0.4128, 'learning_rate': 9.32217352839058e-06, 'epoch': 2.18}
{'loss': 0.4061, 'learning_rate': 9.301025490000843e-06, 'epoch': 2.18}
{'loss': 0.3827, 'learning_rate': 9.279880592586355e-06, 'epoch': 2.19}
{'loss': 0.4085, 'learning_rate': 9.25873893116574e-06, 'epoch': 2.19}
{'loss': 0.3793, 'learning_rate': 9.237600600743086e-06, 'epoch': 2.19}
{'loss': 0.3724, 'learning_rate': 9.216465696307513e-06, 'epoch': 2.19}
{'loss': 0.3611, 'learning_rate': 9.195334312832742e-06, 'epoch': 2.2}
{'loss': 0.3698, 'learning_rate': 9.174206545276678e-06, 'epoch': 2.2}
{'loss': 0.3819, 'learning_rate': 9.153082488580969e-06, 'epoch': 2.2}
{'loss': 0.3903, 'learning_rate': 9.131962237670599e-06, 'epoch': 2.21}
{'loss': 0.3894, 'learning_rate': 9.11084588745343e-06, 'epoch': 2.21}
{'loss': 0.3942, 'learning_rate': 9.089733532819825e-06, 'epoch': 2.21}
{'loss': 0.3799, 'learning_rate': 9.068625268642162e-06, 'epoch': 2.21}
{'loss': 0.3966, 'learning_rate': 9.047521189774456e-06, 'epoch': 2.22}
{'loss': 0.3808, 'learning_rate': 9.026421391051908e-06, 'epoch': 2.22}
{'loss': 0.3848, 'learning_rate': 9.005325967290489e-06, 'epoch': 2.22}
{'loss': 0.4027, 'learning_rate': 8.984235013286512e-06, 'epoch': 2.22}
{'loss': 0.3943, 'learning_rate': 8.963148623816191e-06, 'epoch': 2.23}
{'loss': 0.3921, 'learning_rate': 8.942066893635245e-06, 'epoch': 2.23}
{'loss': 0.3841, 'learning_rate': 8.920989917478446e-06, 'epoch': 2.23}
{'loss': 0.3639, 'learning_rate': 8.899917790059208e-06, 'epoch': 2.23}
{'loss': 0.3924, 'learning_rate': 8.878850606069152e-06, 'epoch': 2.24}
{'loss': 0.4187, 'learning_rate': 8.857788460177685e-06, 'epoch': 2.24}
{'loss': 0.3605, 'learning_rate': 8.836731447031581e-06, 'epoch': 2.24}
{'loss': 0.3867, 'learning_rate': 8.815679661254538e-06, 'epoch': 2.24}
{'loss': 0.391, 'learning_rate': 8.79463319744677e-06, 'epoch': 2.25}
{'loss': 0.3901, 'learning_rate': 8.77359215018458e-06, 'epoch': 2.25}
{'loss': 0.3709, 'learning_rate': 8.752556614019924e-06, 'epoch': 2.25}
{'loss': 0.4087, 'learning_rate': 8.731526683479991e-06, 'epoch': 2.25}
{'loss': 0.3871, 'learning_rate': 8.710502453066791e-06, 'epoch': 2.26}
{'loss': 0.4008, 'learning_rate': 8.689484017256712e-06, 'epoch': 2.26}
{'loss': 0.3916, 'learning_rate': 8.668471470500094e-06, 'epoch': 2.26}
{'loss': 0.3945, 'learning_rate': 8.647464907220833e-06, 'epoch': 2.26}
{'loss': 0.3975, 'learning_rate': 8.626464421815919e-06, 'epoch': 2.27}
{'loss': 0.3856, 'learning_rate': 8.605470108655046e-06, 'epoch': 2.27}
{'loss': 0.3875, 'learning_rate': 8.584482062080154e-06, 'epoch': 2.27}
{'loss': 0.3793, 'learning_rate': 8.56350037640504e-06, 'epoch': 2.27}
{'loss': 0.3688, 'learning_rate': 8.542525145914907e-06, 'epoch': 2.28}
{'loss': 0.3756, 'learning_rate': 8.521556464865955e-06, 'epoch': 2.28}
{'loss': 0.3982, 'learning_rate': 8.500594427484946e-06, 'epoch': 2.28}
{'loss': 0.3684, 'learning_rate': 8.479639127968793e-06, 'epoch': 2.28}
{'loss': 0.3618, 'learning_rate': 8.458690660484134e-06, 'epoch': 2.29}
{'loss': 0.3696, 'learning_rate': 8.4377491191669e-06, 'epoch': 2.29}
{'loss': 0.3897, 'learning_rate': 8.416814598121901e-06, 'epoch': 2.29}
{'loss': 0.3597, 'learning_rate': 8.395887191422397e-06, 'epoch': 2.29}
{'loss': 0.3656, 'learning_rate': 8.374966993109689e-06, 'epoch': 2.3}
{'loss': 0.3702, 'learning_rate': 8.35405409719266e-06, 'epoch': 2.3}
{'loss': 0.3971, 'learning_rate': 8.333148597647414e-06, 'epoch': 2.3}
{'loss': 0.3746, 'learning_rate': 8.312250588416792e-06, 'epoch': 2.31}
{'loss': 0.3684, 'learning_rate': 8.291360163409978e-06, 'epoch': 2.31}
{'loss': 0.3889, 'learning_rate': 8.27047741650209e-06, 'epoch': 2.31}
{'loss': 0.4051, 'learning_rate': 8.249602441533727e-06, 'epoch': 2.31}
{'loss': 0.39, 'learning_rate': 8.228735332310575e-06, 'epoch': 2.32}
{'loss': 0.3554, 'learning_rate': 8.207876182602959e-06, 'epoch': 2.32}
{'loss': 0.3847, 'learning_rate': 8.187025086145458e-06, 'epoch': 2.32}
{'loss': 0.3708, 'learning_rate': 8.16618213663644e-06, 'epoch': 2.32}
{'loss': 0.3767, 'learning_rate': 8.14534742773768e-06, 'epoch': 2.33}
{'loss': 0.3775, 'learning_rate': 8.12452105307391e-06, 'epoch': 2.33}
{'loss': 0.4184, 'learning_rate': 8.103703106232416e-06, 'epoch': 2.33}
{'loss': 0.3827, 'learning_rate': 8.082893680762619e-06, 'epoch': 2.33}
{'loss': 0.3907, 'learning_rate': 8.062092870175629e-06, 'epoch': 2.34}
{'loss': 0.3817, 'learning_rate': 8.041300767943867e-06, 'epoch': 2.34}
{'loss': 0.3809, 'learning_rate': 8.0205174675006e-06, 'epoch': 2.34}
{'loss': 0.3952, 'learning_rate': 7.999743062239557e-06, 'epoch': 2.34}
{'loss': 0.3844, 'learning_rate': 7.978977645514488e-06, 'epoch': 2.35}
{'loss': 0.385, 'learning_rate': 7.958221310638749e-06, 'epoch': 2.35}
{'loss': 0.3926, 'learning_rate': 7.937474150884898e-06, 'epoch': 2.35}
{'loss': 0.3856, 'learning_rate': 7.916736259484239e-06, 'epoch': 2.35}
{'loss': 0.3861, 'learning_rate': 7.896007729626457e-06, 'epoch': 2.36}
{'loss': 0.3853, 'learning_rate': 7.875288654459144e-06, 'epoch': 2.36}
{'loss': 0.3897, 'learning_rate': 7.854579127087418e-06, 'epoch': 2.36}
{'loss': 0.3908, 'learning_rate': 7.833879240573487e-06, 'epoch': 2.36}
{'loss': 0.4056, 'learning_rate': 7.813189087936243e-06, 'epoch': 2.37}
{'loss': 0.378, 'learning_rate': 7.792508762150833e-06, 'epoch': 2.37}
{'loss': 0.3971, 'learning_rate': 7.771838356148231e-06, 'epoch': 2.37}
{'loss': 0.3823, 'learning_rate': 7.751177962814867e-06, 'epoch': 2.37}
{'loss': 0.3894, 'learning_rate': 7.730527674992143e-06, 'epoch': 2.38}
{'loss': 0.3829, 'learning_rate': 7.709887585476075e-06, 'epoch': 2.38}
{'loss': 0.3749, 'learning_rate': 7.689257787016834e-06, 'epoch': 2.38}
{'loss': 0.3979, 'learning_rate': 7.668638372318359e-06, 'epoch': 2.38}
{'loss': 0.3682, 'learning_rate': 7.648029434037915e-06, 'epoch': 2.39}
{'loss': 0.3408, 'learning_rate': 7.627431064785705e-06, 'epoch': 2.39}
{'loss': 0.3749, 'learning_rate': 7.606843357124426e-06, 'epoch': 2.39}
{'loss': 0.3857, 'learning_rate': 7.5862664035688604e-06, 'epoch': 2.39}
{'loss': 0.391, 'learning_rate': 7.565700296585483e-06, 'epoch': 2.4}
{'loss': 0.3709, 'learning_rate': 7.545145128592009e-06, 'epoch': 2.4}
{'loss': 0.3874, 'learning_rate': 7.524600991957013e-06, 'epoch': 2.4}
{'loss': 0.3881, 'learning_rate': 7.504067978999484e-06, 'epoch': 2.41}
{'loss': 0.3869, 'learning_rate': 7.483546181988437e-06, 'epoch': 2.41}
{'loss': 0.3638, 'learning_rate': 7.463035693142473e-06, 'epoch': 2.41}
{'loss': 0.3865, 'learning_rate': 7.442536604629396e-06, 'epoch': 2.41}
{'loss': 0.4102, 'learning_rate': 7.422049008565757e-06, 'epoch': 2.42}
{'loss': 0.4113, 'learning_rate': 7.40157299701648e-06, 'epoch': 2.42}
{'loss': 0.4165, 'learning_rate': 7.38110866199443e-06, 'epoch': 2.42}
{'loss': 0.3775, 'learning_rate': 7.360656095459995e-06, 'epoch': 2.42}
{'loss': 0.3756, 'learning_rate': 7.340215389320686e-06, 'epoch': 2.43}
{'loss': 0.3758, 'learning_rate': 7.31978663543071e-06, 'epoch': 2.43}
{'loss': 0.3839, 'learning_rate': 7.299369925590575e-06, 'epoch': 2.43}
{'loss': 0.3921, 'learning_rate': 7.278965351546648e-06, 'epoch': 2.43}
{'loss': 0.4005, 'learning_rate': 7.258573004990789e-06, 'epoch': 2.44}
{'loss': 0.3916, 'learning_rate': 7.2381929775598835e-06, 'epoch': 2.44}
{'loss': 0.3971, 'learning_rate': 7.217825360835475e-06, 'epoch': 2.44}
{'loss': 0.3681, 'learning_rate': 7.197470246343333e-06, 'epoch': 2.44}
{'loss': 0.3765, 'learning_rate': 7.1771277255530456e-06, 'epoch': 2.45}
{'loss': 0.381, 'learning_rate': 7.156797889877613e-06, 'epoch': 2.45}
{'loss': 0.3638, 'learning_rate': 7.136480830673018e-06, 'epoch': 2.45}
{'loss': 0.3783, 'learning_rate': 7.116176639237853e-06, 'epoch': 2.45}
{'loss': 0.3704, 'learning_rate': 7.095885406812866e-06, 'epoch': 2.46}
{'loss': 0.3908, 'learning_rate': 7.075607224580581e-06, 'epoch': 2.46}
{'loss': 0.3999, 'learning_rate': 7.05534218366488e-06, 'epoch': 2.46}
{'loss': 0.3768, 'learning_rate': 7.035090375130581e-06, 'epoch': 2.46}
{'loss': 0.3601, 'learning_rate': 7.014851889983058e-06, 'epoch': 2.47}
{'loss': 0.3643, 'learning_rate': 6.994626819167789e-06, 'epoch': 2.47}
{'loss': 0.3719, 'learning_rate': 6.974415253570003e-06, 'epoch': 2.47}
{'loss': 0.3831, 'learning_rate': 6.9542172840142105e-06, 'epoch': 2.47}
{'loss': 0.402, 'learning_rate': 6.934033001263847e-06, 'epoch': 2.48}
{'loss': 0.3967, 'learning_rate': 6.913862496020831e-06, 'epoch': 2.48}
{'loss': 0.3879, 'learning_rate': 6.893705858925179e-06, 'epoch': 2.48}
{'loss': 0.4073, 'learning_rate': 6.873563180554583e-06, 'epoch': 2.48}
{'loss': 0.3656, 'learning_rate': 6.853434551424001e-06, 'epoch': 2.49}
{'loss': 0.3959, 'learning_rate': 6.833320061985278e-06, 'epoch': 2.49}
{'loss': 0.3569, 'learning_rate': 6.813219802626698e-06, 'epoch': 2.49}
{'loss': 0.3647, 'learning_rate': 6.793133863672616e-06, 'epoch': 2.49}
{'loss': 0.3795, 'learning_rate': 6.773062335383024e-06, 'epoch': 2.5}
{'loss': 0.3523, 'learning_rate': 6.7530053079531664e-06, 'epoch': 2.5}
{'loss': 0.3683, 'learning_rate': 6.73296287151312e-06, 'epoch': 2.5}
{'loss': 0.3993, 'learning_rate': 6.712935116127389e-06, 'epoch': 2.51}
{'loss': 0.3766, 'learning_rate': 6.692922131794517e-06, 'epoch': 2.51}
{'loss': 0.3865, 'learning_rate': 6.672924008446663e-06, 'epoch': 2.51}
{'loss': 0.3695, 'learning_rate': 6.652940835949208e-06, 'epoch': 2.51}
{'loss': 0.3803, 'learning_rate': 6.632972704100349e-06, 'epoch': 2.52}
{'loss': 0.3885, 'learning_rate': 6.6130197026306945e-06, 'epoch': 2.52}
{'loss': 0.38, 'learning_rate': 6.59308192120286e-06, 'epoch': 2.52}
{'loss': 0.3645, 'learning_rate': 6.573159449411071e-06, 'epoch': 2.52}
{'loss': 0.3649, 'learning_rate': 6.553252376780749e-06, 'epoch': 2.53}
{'loss': 0.3767, 'learning_rate': 6.533360792768122e-06, 'epoch': 2.53}
{'loss': 0.3762, 'learning_rate': 6.513484786759818e-06, 'epoch': 2.53}
{'loss': 0.3885, 'learning_rate': 6.4936244480724575e-06, 'epoch': 2.53}
{'loss': 0.3901, 'learning_rate': 6.473779865952263e-06, 'epoch': 2.54}
{'loss': 0.3846, 'learning_rate': 6.453951129574644e-06, 'epoch': 2.54}
{'loss': 0.3772, 'learning_rate': 6.434138328043815e-06, 'epoch': 2.54}
{'loss': 0.3927, 'learning_rate': 6.4143415503923676e-06, 'epoch': 2.54}
{'loss': 0.3657, 'learning_rate': 6.39456088558091e-06, 'epoch': 2.55}
{'loss': 0.377, 'learning_rate': 6.374796422497622e-06, 'epoch': 2.55}
{'loss': 0.3522, 'learning_rate': 6.355048249957887e-06, 'epoch': 2.55}
{'loss': 0.378, 'learning_rate': 6.335316456703891e-06, 'epoch': 2.55}
{'loss': 0.386, 'learning_rate': 6.315601131404201e-06, 'epoch': 2.56}
{'loss': 0.3726, 'learning_rate': 6.295902362653401e-06, 'epoch': 2.56}
{'loss': 0.385, 'learning_rate': 6.276220238971653e-06, 'epoch': 2.56}
{'loss': 0.36, 'learning_rate': 6.256554848804343e-06, 'epoch': 2.56}
{'loss': 0.3847, 'learning_rate': 6.236906280521646e-06, 'epoch': 2.57}
{'loss': 0.3653, 'learning_rate': 6.2172746224181524e-06, 'epoch': 2.57}
{'loss': 0.3703, 'learning_rate': 6.197659962712461e-06, 'epoch': 2.57}
{'loss': 0.3693, 'learning_rate': 6.178062389546784e-06, 'epoch': 2.57}
{'loss': 0.3625, 'learning_rate': 6.158481990986558e-06, 'epoch': 2.58}
{'loss': 0.3673, 'learning_rate': 6.138918855020028e-06, 'epoch': 2.58}
{'loss': 0.3656, 'learning_rate': 6.119373069557889e-06, 'epoch': 2.58}
{'loss': 0.3595, 'learning_rate': 6.099844722432844e-06, 'epoch': 2.58}
{'loss': 0.3788, 'learning_rate': 6.080333901399252e-06, 'epoch': 2.59}
{'loss': 0.3646, 'learning_rate': 6.060840694132701e-06, 'epoch': 2.59}
{'loss': 0.3802, 'learning_rate': 6.041365188229641e-06, 'epoch': 2.59}
{'loss': 0.3416, 'learning_rate': 6.021907471206971e-06, 'epoch': 2.59}
{'loss': 0.3886, 'learning_rate': 6.002467630501646e-06, 'epoch': 2.6}
{'loss': 0.3918, 'learning_rate': 5.983045753470308e-06, 'epoch': 2.6}
{'loss': 0.3844, 'learning_rate': 5.963641927388854e-06, 'epoch': 2.6}
{'loss': 0.3844, 'learning_rate': 5.944256239452085e-06, 'epoch': 2.61}
{'loss': 0.3726, 'learning_rate': 5.924888776773281e-06, 'epoch': 2.61}
{'loss': 0.3803, 'learning_rate': 5.905539626383831e-06, 'epoch': 2.61}
{'loss': 0.3422, 'learning_rate': 5.886208875232833e-06, 'epoch': 2.61}
{'loss': 0.3681, 'learning_rate': 5.866896610186701e-06, 'epoch': 2.62}
{'loss': 0.3803, 'learning_rate': 5.847602918028785e-06, 'epoch': 2.62}
{'loss': 0.366, 'learning_rate': 5.82832788545896e-06, 'epoch': 2.62}
{'loss': 0.3866, 'learning_rate': 5.809071599093272e-06, 'epoch': 2.62}
{'loss': 0.3796, 'learning_rate': 5.789834145463506e-06, 'epoch': 2.63}
{'loss': 0.4013, 'learning_rate': 5.7706156110168384e-06, 'epoch': 2.63}
{'loss': 0.3908, 'learning_rate': 5.7514160821154085e-06, 'epoch': 2.63}
{'loss': 0.3768, 'learning_rate': 5.732235645035964e-06, 'epoch': 2.63}
{'loss': 0.3917, 'learning_rate': 5.713074385969457e-06, 'epoch': 2.64}
{'loss': 0.3823, 'learning_rate': 5.693932391020664e-06, 'epoch': 2.64}
{'loss': 0.3765, 'learning_rate': 5.67480974620778e-06, 'epoch': 2.64}
{'loss': 0.3858, 'learning_rate': 5.65570653746206e-06, 'epoch': 2.64}
{'loss': 0.3673, 'learning_rate': 5.63662285062742e-06, 'epoch': 2.65}
{'loss': 0.3838, 'learning_rate': 5.617558771460039e-06, 'epoch': 2.65}
{'loss': 0.3876, 'learning_rate': 5.598514385627997e-06, 'epoch': 2.65}
{'loss': 0.3825, 'learning_rate': 5.579489778710867e-06, 'epoch': 2.65}
{'loss': 0.3756, 'learning_rate': 5.56048503619935e-06, 'epoch': 2.66}
{'loss': 0.3673, 'learning_rate': 5.541500243494889e-06, 'epoch': 2.66}
{'loss': 0.3816, 'learning_rate': 5.522535485909258e-06, 'epoch': 2.66}
{'loss': 0.3637, 'learning_rate': 5.5035908486642195e-06, 'epoch': 2.66}
{'loss': 0.3577, 'learning_rate': 5.484666416891109e-06, 'epoch': 2.67}
{'loss': 0.3672, 'learning_rate': 5.465762275630471e-06, 'epoch': 2.67}
{'loss': 0.3735, 'learning_rate': 5.446878509831668e-06, 'epoch': 2.67}
{'loss': 0.3645, 'learning_rate': 5.428015204352508e-06, 'epoch': 2.67}
{'loss': 0.3621, 'learning_rate': 5.409172443958844e-06, 'epoch': 2.68}
{'loss': 0.3423, 'learning_rate': 5.390350313324214e-06, 'epoch': 2.68}
{'loss': 0.3821, 'learning_rate': 5.371548897029457e-06, 'epoch': 2.68}
{'loss': 0.353, 'learning_rate': 5.352768279562315e-06, 'epoch': 2.68}
{'loss': 0.378, 'learning_rate': 5.334008545317082e-06, 'epoch': 2.69}
{'loss': 0.352, 'learning_rate': 5.315269778594189e-06, 'epoch': 2.69}
{'loss': 0.3678, 'learning_rate': 5.2965520635998676e-06, 'epoch': 2.69}
{'loss': 0.3559, 'learning_rate': 5.277855484445736e-06, 'epoch': 2.69}
{'loss': 0.3828, 'learning_rate': 5.259180125148442e-06, 'epoch': 2.7}
{'loss': 0.3932, 'learning_rate': 5.240526069629265e-06, 'epoch': 2.7}
{'loss': 0.3874, 'learning_rate': 5.22189340171377e-06, 'epoch': 2.7}
{'loss': 0.3908, 'learning_rate': 5.203282205131395e-06, 'epoch': 2.71}
{'loss': 0.3731, 'learning_rate': 5.184692563515104e-06, 'epoch': 2.71}
{'loss': 0.3504, 'learning_rate': 5.166124560401002e-06, 'epoch': 2.71}
{'loss': 0.3724, 'learning_rate': 5.147578279227943e-06, 'epoch': 2.71}
{'loss': 0.3595, 'learning_rate': 5.129053803337181e-06, 'epoch': 2.72}
{'loss': 0.3831, 'learning_rate': 5.110551215971981e-06, 'epoch': 2.72}
{'loss': 0.3842, 'learning_rate': 5.0920706002772525e-06, 'epoch': 2.72}
{'loss': 0.3923, 'learning_rate': 5.073612039299157e-06, 'epoch': 2.72}
{'loss': 0.3539, 'learning_rate': 5.055175615984766e-06, 'epoch': 2.73}
{'loss': 0.3891, 'learning_rate': 5.036761413181659e-06, 'epoch': 2.73}
{'loss': 0.3639, 'learning_rate': 5.018369513637567e-06, 'epoch': 2.73}
{'loss': 0.3498, 'learning_rate': 5.000000000000003e-06, 'epoch': 2.73}
{'loss': 0.3427, 'learning_rate': 4.981652954815871e-06, 'epoch': 2.74}
{'loss': 0.3843, 'learning_rate': 4.963328460531127e-06, 'epoch': 2.74}
{'loss': 0.3662, 'learning_rate': 4.9450265994903666e-06, 'epoch': 2.74}
{'loss': 0.3609, 'learning_rate': 4.926747453936509e-06, 'epoch': 2.74}
{'loss': 0.3881, 'learning_rate': 4.908491106010368e-06, 'epoch': 2.75}
{'loss': 0.3757, 'learning_rate': 4.890257637750332e-06, 'epoch': 2.75}
{'loss': 0.3864, 'learning_rate': 4.872047131091961e-06, 'epoch': 2.75}
{'loss': 0.3691, 'learning_rate': 4.853859667867641e-06, 'epoch': 2.75}
{'loss': 0.3943, 'learning_rate': 4.8356953298062115e-06, 'epoch': 2.76}
{'loss': 0.3739, 'learning_rate': 4.817554198532582e-06, 'epoch': 2.76}
{'loss': 0.3871, 'learning_rate': 4.799436355567391e-06, 'epoch': 2.76}
{'loss': 0.3774, 'learning_rate': 4.781341882326615e-06, 'epoch': 2.76}
{'loss': 0.3881, 'learning_rate': 4.763270860121222e-06, 'epoch': 2.77}
{'loss': 0.3652, 'learning_rate': 4.745223370156797e-06, 'epoch': 2.77}
{'loss': 0.3647, 'learning_rate': 4.727199493533181e-06, 'epoch': 2.77}
{'loss': 0.3818, 'learning_rate': 4.709199311244098e-06, 'epoch': 2.77}
{'loss': 0.3594, 'learning_rate': 4.691222904176791e-06, 'epoch': 2.78}
{'loss': 0.3448, 'learning_rate': 4.673270353111687e-06, 'epoch': 2.78}
{'loss': 0.353, 'learning_rate': 4.655341738721989e-06, 'epoch': 2.78}
{'loss': 0.37, 'learning_rate': 4.63743714157335e-06, 'epoch': 2.78}
{'loss': 0.3812, 'learning_rate': 4.619556642123484e-06, 'epoch': 2.79}
{'loss': 0.3642, 'learning_rate': 4.6017003207218294e-06, 'epoch': 2.79}
{'loss': 0.3725, 'learning_rate': 4.583868257609171e-06, 'epoch': 2.79}
{'loss': 0.3473, 'learning_rate': 4.566060532917288e-06, 'epoch': 2.79}
{'loss': 0.3708, 'learning_rate': 4.548277226668585e-06, 'epoch': 2.8}
{'loss': 0.3881, 'learning_rate': 4.530518418775734e-06, 'epoch': 2.8}
{'loss': 0.379, 'learning_rate': 4.512784189041328e-06, 'epoch': 2.8}
{'loss': 0.4005, 'learning_rate': 4.4950746171575135e-06, 'epoch': 2.81}
{'loss': 0.3562, 'learning_rate': 4.477389782705628e-06, 'epoch': 2.81}
{'loss': 0.364, 'learning_rate': 4.459729765155842e-06, 'epoch': 2.81}
{'loss': 0.3626, 'learning_rate': 4.442094643866817e-06, 'epoch': 2.81}
{'loss': 0.3813, 'learning_rate': 4.424484498085335e-06, 'epoch': 2.82}
{'loss': 0.3717, 'learning_rate': 4.406899406945938e-06, 'epoch': 2.82}
{'loss': 0.3593, 'learning_rate': 4.389339449470592e-06, 'epoch': 2.82}
{'loss': 0.3905, 'learning_rate': 4.371804704568309e-06, 'epoch': 2.82}
{'loss': 0.3682, 'learning_rate': 4.354295251034811e-06, 'epoch': 2.83}
{'loss': 0.3833, 'learning_rate': 4.336811167552167e-06, 'epoch': 2.83}
{'loss': 0.3714, 'learning_rate': 4.319352532688444e-06, 'epoch': 2.83}
{'loss': 0.3757, 'learning_rate': 4.301919424897339e-06, 'epoch': 2.83}
{'loss': 0.3832, 'learning_rate': 4.284511922517853e-06, 'epoch': 2.84}
{'loss': 0.355, 'learning_rate': 4.267130103773911e-06, 'epoch': 2.84}
{'loss': 0.3941, 'learning_rate': 4.249774046774034e-06, 'epoch': 2.84}
{'loss': 0.3579, 'learning_rate': 4.232443829510977e-06, 'epoch': 2.84}
{'loss': 0.3482, 'learning_rate': 4.2151395298613675e-06, 'epoch': 2.85}
{'loss': 0.3803, 'learning_rate': 4.19786122558538e-06, 'epoch': 2.85}
{'loss': 0.3312, 'learning_rate': 4.180608994326371e-06, 'epoch': 2.85}
{'loss': 0.3837, 'learning_rate': 4.163382913610533e-06, 'epoch': 2.85}
{'loss': 0.3622, 'learning_rate': 4.1461830608465385e-06, 'epoch': 2.86}
{'loss': 0.3751, 'learning_rate': 4.129009513325211e-06, 'epoch': 2.86}
{'loss': 0.3627, 'learning_rate': 4.111862348219158e-06, 'epoch': 2.86}
{'loss': 0.3627, 'learning_rate': 4.0947416425824334e-06, 'epoch': 2.86}
{'loss': 0.3708, 'learning_rate': 4.077647473350201e-06, 'epoch': 2.87}
{'loss': 0.3605, 'learning_rate': 4.060579917338362e-06, 'epoch': 2.87}
{'loss': 0.3913, 'learning_rate': 4.04353905124324e-06, 'epoch': 2.87}
{'loss': 0.3695, 'learning_rate': 4.026524951641204e-06, 'epoch': 2.87}
{'loss': 0.3571, 'learning_rate': 4.009537694988372e-06, 'epoch': 2.88}
{'loss': 0.3624, 'learning_rate': 3.99257735762021e-06, 'epoch': 2.88}
{'loss': 0.3622, 'learning_rate': 3.975644015751234e-06, 'epoch': 2.88}
{'loss': 0.3679, 'learning_rate': 3.958737745474638e-06, 'epoch': 2.88}
{'loss': 0.3724, 'learning_rate': 3.941858622761975e-06, 'epoch': 2.89}
{'loss': 0.3856, 'learning_rate': 3.925006723462806e-06, 'epoch': 2.89}
{'loss': 0.3827, 'learning_rate': 3.908182123304344e-06, 'epoch': 2.89}
{'loss': 0.3512, 'learning_rate': 3.891384897891148e-06, 'epoch': 2.89}
{'loss': 0.3472, 'learning_rate': 3.8746151227047455e-06, 'epoch': 2.9}
{'loss': 0.3541, 'learning_rate': 3.857872873103322e-06, 'epoch': 2.9}
{'loss': 0.352, 'learning_rate': 3.84115822432137e-06, 'epoch': 2.9}
{'loss': 0.3821, 'learning_rate': 3.824471251469352e-06, 'epoch': 2.91}
{'loss': 0.358, 'learning_rate': 3.807812029533362e-06, 'epoch': 2.91}
{'loss': 0.3785, 'learning_rate': 3.7911806333747847e-06, 'epoch': 2.91}
{'loss': 0.3556, 'learning_rate': 3.7745771377299758e-06, 'epoch': 2.91}
{'loss': 0.3449, 'learning_rate': 3.7580016172099067e-06, 'epoch': 2.92}
{'loss': 0.3615, 'learning_rate': 3.7414541462998446e-06, 'epoch': 2.92}
{'loss': 0.3758, 'learning_rate': 3.724934799358998e-06, 'epoch': 2.92}
{'loss': 0.3621, 'learning_rate': 3.708443650620206e-06, 'epoch': 2.92}
{'loss': 0.3675, 'learning_rate': 3.691980774189591e-06, 'epoch': 2.93}
{'loss': 0.3816, 'learning_rate': 3.6755462440462288e-06, 'epoch': 2.93}
{'loss': 0.3575, 'learning_rate': 3.659140134041812e-06, 'epoch': 2.93}
{'loss': 0.3613, 'learning_rate': 3.6427625179003223e-06, 'epoch': 2.93}
{'loss': 0.3693, 'learning_rate': 3.626413469217702e-06, 'epoch': 2.94}
{'loss': 0.3807, 'learning_rate': 3.6100930614615204e-06, 'epoch': 2.94}
{'loss': 0.3767, 'learning_rate': 3.5938013679706453e-06, 'epoch': 2.94}
{'loss': 0.3436, 'learning_rate': 3.5775384619549e-06, 'epoch': 2.94}
{'loss': 0.3562, 'learning_rate': 3.5613044164947617e-06, 'epoch': 2.95}
{'loss': 0.3736, 'learning_rate': 3.5450993045409997e-06, 'epoch': 2.95}
{'loss': 0.3543, 'learning_rate': 3.5289231989143868e-06, 'epoch': 2.95}
{'loss': 0.3391, 'learning_rate': 3.5127761723053313e-06, 'epoch': 2.95}
{'loss': 0.3482, 'learning_rate': 3.496658297273574e-06, 'epoch': 2.96}
{'loss': 0.4039, 'learning_rate': 3.4805696462478634e-06, 'epoch': 2.96}
{'loss': 0.3547, 'learning_rate': 3.46451029152562e-06, 'epoch': 2.96}
{'loss': 0.3706, 'learning_rate': 3.448480305272619e-06, 'epoch': 2.96}
{'loss': 0.3797, 'learning_rate': 3.4324797595226567e-06, 'epoch': 2.97}
{'loss': 0.374, 'learning_rate': 3.41650872617724e-06, 'epoch': 2.97}
{'loss': 0.3481, 'learning_rate': 3.400567277005247e-06, 'epoch': 2.97}
{'loss': 0.3659, 'learning_rate': 3.384655483642624e-06, 'epoch': 2.97}
{'loss': 0.3454, 'learning_rate': 3.3687734175920505e-06, 'epoch': 2.98}
{'loss': 0.3827, 'learning_rate': 3.352921150222612e-06, 'epoch': 2.98}
{'loss': 0.3635, 'learning_rate': 3.337098752769503e-06, 'epoch': 2.98}
{'loss': 0.3556, 'learning_rate': 3.321306296333673e-06, 'epoch': 2.98}
{'loss': 0.355, 'learning_rate': 3.3055438518815486e-06, 'epoch': 2.99}
{'loss': 0.358, 'learning_rate': 3.2898114902446708e-06, 'epoch': 2.99}
{'loss': 0.3402, 'learning_rate': 3.274109282119413e-06, 'epoch': 2.99}
{'loss': 0.3406, 'learning_rate': 3.2584372980666344e-06, 'epoch': 2.99}
{'loss': 0.3545, 'learning_rate': 3.2427956085113875e-06, 'epoch': 3.0}
{'loss': 0.3491, 'learning_rate': 3.2271842837425917e-06, 'epoch': 3.0}
75%|███████▌ | 2340/3120 [3:14:45<1:02:44, 4.83s/it]/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
train.py:238: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
input_ids = [torch.tensor(x) for x in input_ids]
train.py:242: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
labels = [torch.tensor(x) for x in labels]
{'loss': 0.1364, 'learning_rate': 3.2116033939127024e-06, 'epoch': 3.0}
{'loss': 0.1197, 'learning_rate': 3.1960530090374277e-06, 'epoch': 3.01}
{'loss': 0.1254, 'learning_rate': 3.180533198995379e-06, 'epoch': 3.01}
{'loss': 0.1163, 'learning_rate': 3.165044033527789e-06, 'epoch': 3.01}
{'loss': 0.1316, 'learning_rate': 3.1495855822381716e-06, 'epoch': 3.01}
{'loss': 0.119, 'learning_rate': 3.134157914592032e-06, 'epoch': 3.02}
{'loss': 0.1231, 'learning_rate': 3.1187610999165317e-06, 'epoch': 3.02}
{'loss': 0.1362, 'learning_rate': 3.1033952074001882e-06, 'epoch': 3.02}
{'loss': 0.1191, 'learning_rate': 3.088060306092582e-06, 'epoch': 3.02}
{'loss': 0.126, 'learning_rate': 3.0727564649040066e-06, 'epoch': 3.03}
{'loss': 0.1242, 'learning_rate': 3.057483752605196e-06, 'epoch': 3.03}
{'loss': 0.1365, 'learning_rate': 3.042242237826991e-06, 'epoch': 3.03}
{'loss': 0.1274, 'learning_rate': 3.0270319890600465e-06, 'epoch': 3.03}
{'loss': 0.1334, 'learning_rate': 3.011853074654515e-06, 'epoch': 3.04}
{'loss': 0.1344, 'learning_rate': 2.996705562819747e-06, 'epoch': 3.04}
{'loss': 0.1246, 'learning_rate': 2.981589521623973e-06, 'epoch': 3.04}
{'loss': 0.1302, 'learning_rate': 2.9665050189940013e-06, 'epoch': 3.04}
{'loss': 0.1188, 'learning_rate': 2.951452122714926e-06, 'epoch': 3.05}
{'loss': 0.132, 'learning_rate': 2.936430900429805e-06, 'epoch': 3.05}
{'loss': 0.1304, 'learning_rate': 2.9214414196393702e-06, 'epoch': 3.05}
{'loss': 0.1438, 'learning_rate': 2.906483747701705e-06, 'epoch': 3.05}
{'loss': 0.1328, 'learning_rate': 2.8915579518319626e-06, 'epoch': 3.06}
{'loss': 0.1251, 'learning_rate': 2.8766640991020533e-06, 'epoch': 3.06}
{'loss': 0.1324, 'learning_rate': 2.861802256440348e-06, 'epoch': 3.06}
{'loss': 0.1249, 'learning_rate': 2.8469724906313677e-06, 'epoch': 3.06}
{'loss': 0.123, 'learning_rate': 2.8321748683154893e-06, 'epoch': 3.07}
{'loss': 0.1236, 'learning_rate': 2.8174094559886535e-06, 'epoch': 3.07}
{'loss': 0.1277, 'learning_rate': 2.8026763200020557e-06, 'epoch': 3.07}
{'loss': 0.1212, 'learning_rate': 2.7879755265618558e-06, 'epoch': 3.07}
{'loss': 0.1219, 'learning_rate': 2.773307141728867e-06, 'epoch': 3.08}
{'loss': 0.1199, 'learning_rate': 2.7586712314182775e-06, 'epoch': 3.08}
{'loss': 0.134, 'learning_rate': 2.744067861399333e-06, 'epoch': 3.08}
{'loss': 0.1293, 'learning_rate': 2.729497097295075e-06, 'epoch': 3.08}
{'loss': 0.1308, 'learning_rate': 2.714959004582003e-06, 'epoch': 3.09}
{'loss': 0.1208, 'learning_rate': 2.700453648589805e-06, 'epoch': 3.09}
{'loss': 0.1265, 'learning_rate': 2.6859810945010687e-06, 'epoch': 3.09}
{'loss': 0.1356, 'learning_rate': 2.671541407350975e-06, 'epoch': 3.09}
{'loss': 0.1307, 'learning_rate': 2.6571346520270147e-06, 'epoch': 3.1}
{'loss': 0.1225, 'learning_rate': 2.642760893268684e-06, 'epoch': 3.1}
{'loss': 0.128, 'learning_rate': 2.628420195667214e-06, 'epoch': 3.1}
{'loss': 0.1261, 'learning_rate': 2.614112623665259e-06, 'epoch': 3.11}
{'loss': 0.1141, 'learning_rate': 2.5998382415566258e-06, 'epoch': 3.11}
{'loss': 0.1226, 'learning_rate': 2.5855971134859736e-06, 'epoch': 3.11}
{'loss': 0.122, 'learning_rate': 2.5713893034485216e-06, 'epoch': 3.11}
{'loss': 0.1346, 'learning_rate': 2.55721487528978e-06, 'epoch': 3.12}
{'loss': 0.1393, 'learning_rate': 2.5430738927052346e-06, 'epoch': 3.12}
{'loss': 0.1241, 'learning_rate': 2.5289664192400996e-06, 'epoch': 3.12}
{'loss': 0.1317, 'learning_rate': 2.514892518288988e-06, 'epoch': 3.12}
{'loss': 0.1216, 'learning_rate': 2.5008522530956647e-06, 'epoch': 3.13}
{'loss': 0.1126, 'learning_rate': 2.4868456867527315e-06, 'epoch': 3.13}
{'loss': 0.1228, 'learning_rate': 2.4728728822013683e-06, 'epoch': 3.13}
{'loss': 0.1331, 'learning_rate': 2.4589339022310386e-06, 'epoch': 3.13}
{'loss': 0.1179, 'learning_rate': 2.445028809479203e-06, 'epoch': 3.14}
{'loss': 0.1303, 'learning_rate': 2.431157666431052e-06, 'epoch': 3.14}
{'loss': 0.1261, 'learning_rate': 2.4173205354192065e-06, 'epoch': 3.14}
{'loss': 0.1229, 'learning_rate': 2.403517478623456e-06, 'epoch': 3.14}
{'loss': 0.1267, 'learning_rate': 2.3897485580704684e-06, 'epoch': 3.15}
{'loss': 0.1336, 'learning_rate': 2.3760138356335172e-06, 'epoch': 3.15}
{'loss': 0.1331, 'learning_rate': 2.3623133730321923e-06, 'epoch': 3.15}
{'loss': 0.1206, 'learning_rate': 2.348647231832131e-06, 'epoch': 3.15}
{'loss': 0.1218, 'learning_rate': 2.335015473444754e-06, 'epoch': 3.16}
{'loss': 0.1079, 'learning_rate': 2.3214181591269603e-06, 'epoch': 3.16}
{'loss': 0.1292, 'learning_rate': 2.30785534998088e-06, 'epoch': 3.16}
{'loss': 0.1182, 'learning_rate': 2.2943271069535754e-06, 'epoch': 3.16}
{'loss': 0.1204, 'learning_rate': 2.2808334908367914e-06, 'epoch': 3.17}
{'loss': 0.1376, 'learning_rate': 2.267374562266662e-06, 'epoch': 3.17}
{'loss': 0.1348, 'learning_rate': 2.2539503817234553e-06, 'epoch': 3.17}
{'loss': 0.1212, 'learning_rate': 2.240561009531281e-06, 'epoch': 3.17}
{'loss': 0.1257, 'learning_rate': 2.227206505857834e-06, 'epoch': 3.18}
{'loss': 0.1218, 'learning_rate': 2.2138869307141266e-06, 'epoch': 3.18}
{'loss': 0.1254, 'learning_rate': 2.200602343954209e-06, 'epoch': 3.18}
{'loss': 0.1212, 'learning_rate': 2.1873528052749094e-06, 'epoch': 3.18}
{'loss': 0.1241, 'learning_rate': 2.1741383742155476e-06, 'epoch': 3.19}
{'loss': 0.1176, 'learning_rate': 2.1609591101576945e-06, 'epoch': 3.19}
{'loss': 0.1265, 'learning_rate': 2.147815072324886e-06, 'epoch': 3.19}
{'loss': 0.1251, 'learning_rate': 2.1347063197823648e-06, 'epoch': 3.19}
{'loss': 0.1243, 'learning_rate': 2.1216329114368083e-06, 'epoch': 3.2}
{'loss': 0.136, 'learning_rate': 2.1085949060360654e-06, 'epoch': 3.2}
{'loss': 0.1278, 'learning_rate': 2.0955923621689034e-06, 'epoch': 3.2}
{'loss': 0.1275, 'learning_rate': 2.0826253382647334e-06, 'epoch': 3.21}
{'loss': 0.136, 'learning_rate': 2.0696938925933505e-06, 'epoch': 3.21}
{'loss': 0.1118, 'learning_rate': 2.056798083264667e-06, 'epoch': 3.21}
{'loss': 0.1247, 'learning_rate': 2.0439379682284686e-06, 'epoch': 3.21}
{'loss': 0.1344, 'learning_rate': 2.0311136052741274e-06, 'epoch': 3.22}
{'loss': 0.123, 'learning_rate': 2.0183250520303653e-06, 'epoch': 3.22}
{'loss': 0.113, 'learning_rate': 2.0055723659649907e-06, 'epoch': 3.22}
{'loss': 0.1266, 'learning_rate': 1.9928556043846215e-06, 'epoch': 3.22}
{'loss': 0.1277, 'learning_rate': 1.9801748244344587e-06, 'epoch': 3.23}
{'loss': 0.1283, 'learning_rate': 1.967530083097996e-06, 'epoch': 3.23}
{'loss': 0.1212, 'learning_rate': 1.9549214371968008e-06, 'epoch': 3.23}
{'loss': 0.1186, 'learning_rate': 1.9423489433902186e-06, 'epoch': 3.23}
{'loss': 0.118, 'learning_rate': 1.9298126581751542e-06, 'epoch': 3.24}
{'loss': 0.1188, 'learning_rate': 1.917312637885791e-06, 'epoch': 3.24}
{'loss': 0.1348, 'learning_rate': 1.9048489386933545e-06, 'epoch': 3.24}
{'loss': 0.1427, 'learning_rate': 1.8924216166058574e-06, 'epoch': 3.24}
{'loss': 0.1329, 'learning_rate': 1.8800307274678364e-06, 'epoch': 3.25}
{'loss': 0.125, 'learning_rate': 1.8676763269601195e-06, 'epoch': 3.25}
{'loss': 0.1309, 'learning_rate': 1.8553584705995564e-06, 'epoch': 3.25}
{'loss': 0.129, 'learning_rate': 1.8430772137387853e-06, 'epoch': 3.25}
{'loss': 0.1235, 'learning_rate': 1.8308326115659757e-06, 'epoch': 3.26}
{'loss': 0.1207, 'learning_rate': 1.8186247191045857e-06, 'epoch': 3.26}
{'loss': 0.1282, 'learning_rate': 1.8064535912131032e-06, 'epoch': 3.26}
{'loss': 0.1158, 'learning_rate': 1.7943192825848133e-06, 'epoch': 3.26}
{'loss': 0.1288, 'learning_rate': 1.7822218477475496e-06, 'epoch': 3.27}
{'loss': 0.1245, 'learning_rate': 1.7701613410634367e-06, 'epoch': 3.27}
{'loss': 0.1259, 'learning_rate': 1.7581378167286655e-06, 'epoch': 3.27}
{'loss': 0.1144, 'learning_rate': 1.7461513287732313e-06, 'epoch': 3.27}
{'loss': 0.1242, 'learning_rate': 1.7342019310607062e-06, 'epoch': 3.28}
{'loss': 0.1128, 'learning_rate': 1.7222896772879871e-06, 'epoch': 3.28}
{'loss': 0.1197, 'learning_rate': 1.7104146209850591e-06, 'epoch': 3.28}
{'loss': 0.1195, 'learning_rate': 1.6985768155147498e-06, 'epoch': 3.28}
{'loss': 0.1182, 'learning_rate': 1.6867763140724969e-06, 'epoch': 3.29}
{'loss': 0.1248, 'learning_rate': 1.6750131696861016e-06, 'epoch': 3.29}
{'loss': 0.1273, 'learning_rate': 1.6632874352154982e-06, 'epoch': 3.29}
{'loss': 0.1213, 'learning_rate': 1.6515991633525118e-06, 'epoch': 3.29}
{'loss': 0.1178, 'learning_rate': 1.6399484066206183e-06, 'epoch': 3.3}
{'loss': 0.1275, 'learning_rate': 1.6283352173747148e-06, 'epoch': 3.3}
{'loss': 0.136, 'learning_rate': 1.6167596478008817e-06, 'epoch': 3.3}
{'loss': 0.1204, 'learning_rate': 1.6052217499161516e-06, 'epoch': 3.31}
{'loss': 0.1268, 'learning_rate': 1.5937215755682667e-06, 'epoch': 3.31}
{'loss': 0.1272, 'learning_rate': 1.5822591764354512e-06, 'epoch': 3.31}
{'loss': 0.1187, 'learning_rate': 1.5708346040261812e-06, 'epoch': 3.31}
{'loss': 0.1303, 'learning_rate': 1.559447909678954e-06, 'epoch': 3.32}
{'loss': 0.1261, 'learning_rate': 1.5480991445620541e-06, 'epoch': 3.32}
{'loss': 0.1215, 'learning_rate': 1.5367883596733157e-06, 'epoch': 3.32}
{'loss': 0.1355, 'learning_rate': 1.5255156058399124e-06, 'epoch': 3.32}
{'loss': 0.13, 'learning_rate': 1.5142809337181063e-06, 'epoch': 3.33}
{'loss': 0.1094, 'learning_rate': 1.5030843937930485e-06, 'epoch': 3.33}
{'loss': 0.1251, 'learning_rate': 1.4919260363785215e-06, 'epoch': 3.33}
{'loss': 0.1266, 'learning_rate': 1.4808059116167306e-06, 'epoch': 3.33}
{'loss': 0.1176, 'learning_rate': 1.4697240694780769e-06, 'epoch': 3.34}
{'loss': 0.1426, 'learning_rate': 1.4586805597609333e-06, 'epoch': 3.34}
{'loss': 0.1218, 'learning_rate': 1.4476754320914187e-06, 'epoch': 3.34}
{'loss': 0.1146, 'learning_rate': 1.4367087359231668e-06, 'epoch': 3.34}
{'loss': 0.1221, 'learning_rate': 1.4257805205371233e-06, 'epoch': 3.35}
{'loss': 0.1175, 'learning_rate': 1.4148908350413048e-06, 'epoch': 3.35}
{'loss': 0.1182, 'learning_rate': 1.4040397283705898e-06, 'epoch': 3.35}
{'loss': 0.1177, 'learning_rate': 1.3932272492864984e-06, 'epoch': 3.35}
{'loss': 0.1311, 'learning_rate': 1.3824534463769633e-06, 'epoch': 3.36}
{'loss': 0.1197, 'learning_rate': 1.3717183680561253e-06, 'epoch': 3.36}
{'loss': 0.1288, 'learning_rate': 1.3610220625641002e-06, 'epoch': 3.36}
{'loss': 0.1307, 'learning_rate': 1.3503645779667852e-06, 'epoch': 3.36}
{'loss': 0.1291, 'learning_rate': 1.339745962155613e-06, 'epoch': 3.37}
{'loss': 0.13, 'learning_rate': 1.3291662628473634e-06, 'epoch': 3.37}
{'loss': 0.1334, 'learning_rate': 1.3186255275839266e-06, 'epoch': 3.37}
{'loss': 0.1305, 'learning_rate': 1.308123803732111e-06, 'epoch': 3.37}
{'loss': 0.1264, 'learning_rate': 1.2976611384834148e-06, 'epoch': 3.38}
{'loss': 0.1207, 'learning_rate': 1.2872375788538171e-06, 'epoch': 3.38}
{'loss': 0.1173, 'learning_rate': 1.2768531716835754e-06, 'epoch': 3.38}
{'loss': 0.1223, 'learning_rate': 1.266507963636997e-06, 'epoch': 3.38}
{'loss': 0.1222, 'learning_rate': 1.2562020012022512e-06, 'epoch': 3.39}
{'loss': 0.1214, 'learning_rate': 1.2459353306911438e-06, 'epoch': 3.39}
{'loss': 0.1133, 'learning_rate': 1.2357079982389197e-06, 'epoch': 3.39}
{'loss': 0.1208, 'learning_rate': 1.2255200498040432e-06, 'epoch': 3.39}
{'loss': 0.1127, 'learning_rate': 1.2153715311680048e-06, 'epoch': 3.4}
{'loss': 0.1262, 'learning_rate': 1.2052624879351105e-06, 'epoch': 3.4}
{'loss': 0.1257, 'learning_rate': 1.1951929655322692e-06, 'epoch': 3.4}
{'loss': 0.1334, 'learning_rate': 1.1851630092088051e-06, 'epoch': 3.41}
{'loss': 0.1286, 'learning_rate': 1.175172664036235e-06, 'epoch': 3.41}
{'loss': 0.1316, 'learning_rate': 1.1652219749080817e-06, 'epoch': 3.41}
{'loss': 0.1305, 'learning_rate': 1.155310986539665e-06, 'epoch': 3.41}
{'loss': 0.1179, 'learning_rate': 1.1454397434679022e-06, 'epoch': 3.42}
{'loss': 0.1289, 'learning_rate': 1.1356082900511023e-06, 'epoch': 3.42}
{'loss': 0.1197, 'learning_rate': 1.12581667046878e-06, 'epoch': 3.42}
{'loss': 0.1154, 'learning_rate': 1.116064928721442e-06, 'epoch': 3.42}
{'loss': 0.1292, 'learning_rate': 1.1063531086304003e-06, 'epoch': 3.43}
{'loss': 0.1225, 'learning_rate': 1.0966812538375738e-06, 'epoch': 3.43}
{'loss': 0.1136, 'learning_rate': 1.0870494078052796e-06, 'epoch': 3.43}
{'loss': 0.1207, 'learning_rate': 1.0774576138160596e-06, 'epoch': 3.43}
{'loss': 0.1334, 'learning_rate': 1.067905914972468e-06, 'epoch': 3.44}
{'loss': 0.136, 'learning_rate': 1.0583943541968856e-06, 'epoch': 3.44}
{'loss': 0.1279, 'learning_rate': 1.0489229742313223e-06, 'epoch': 3.44}
{'loss': 0.1199, 'learning_rate': 1.0394918176372327e-06, 'epoch': 3.44}
{'loss': 0.1309, 'learning_rate': 1.0301009267953145e-06, 'epoch': 3.45}
{'loss': 0.1151, 'learning_rate': 1.0207503439053279e-06, 'epoch': 3.45}
{'loss': 0.1052, 'learning_rate': 1.0114401109859019e-06, 'epoch': 3.45}
{'loss': 0.1137, 'learning_rate': 1.0021702698743408e-06, 'epoch': 3.45}
{'loss': 0.1258, 'learning_rate': 9.929408622264448e-07, 'epoch': 3.46}
{'loss': 0.1157, 'learning_rate': 9.837519295163155e-07, 'epoch': 3.46}
{'loss': 0.1131, 'learning_rate': 9.746035130361741e-07, 'epoch': 3.46}
{'loss': 0.1176, 'learning_rate': 9.65495653896179e-07, 'epoch': 3.46}
{'loss': 0.1165, 'learning_rate': 9.564283930242258e-07, 'epoch': 3.47}
{'loss': 0.1168, 'learning_rate': 9.474017711657835e-07, 'epoch': 3.47}
{'loss': 0.1243, 'learning_rate': 9.384158288836987e-07, 'epoch': 3.47}
{'loss': 0.1134, 'learning_rate': 9.294706065580172e-07, 'epoch': 3.47}
{'loss': 0.1404, 'learning_rate': 9.205661443857994e-07, 'epoch': 3.48}
{'loss': 0.1233, 'learning_rate': 9.117024823809462e-07, 'epoch': 3.48}
{'loss': 0.1295, 'learning_rate': 9.028796603740097e-07, 'epoch': 3.48}
{'loss': 0.1089, 'learning_rate': 8.940977180120247e-07, 'epoch': 3.48}
{'loss': 0.1252, 'learning_rate': 8.853566947583259e-07, 'epoch': 3.49}
{'loss': 0.1215, 'learning_rate': 8.766566298923651e-07, 'epoch': 3.49}
{'loss': 0.1299, 'learning_rate': 8.67997562509546e-07, 'epoch': 3.49}
{'loss': 0.1278, 'learning_rate': 8.593795315210341e-07, 'epoch': 3.49}
{'loss': 0.1151, 'learning_rate': 8.508025756535987e-07, 'epoch': 3.5}
{'loss': 0.1218, 'learning_rate': 8.42266733449425e-07, 'epoch': 3.5}
{'loss': 0.1245, 'learning_rate': 8.337720432659513e-07, 'epoch': 3.5}
{'loss': 0.1219, 'learning_rate': 8.253185432756816e-07, 'epoch': 3.51}
{'loss': 0.1172, 'learning_rate': 8.169062714660347e-07, 'epoch': 3.51}
{'loss': 0.1283, 'learning_rate': 8.085352656391576e-07, 'epoch': 3.51}
{'loss': 0.1163, 'learning_rate': 8.002055634117578e-07, 'epoch': 3.51}
{'loss': 0.1344, 'learning_rate': 7.919172022149458e-07, 'epoch': 3.52}
{'loss': 0.1333, 'learning_rate': 7.836702192940493e-07, 'epoch': 3.52}
{'loss': 0.1251, 'learning_rate': 7.754646517084585e-07, 'epoch': 3.52}
{'loss': 0.1139, 'learning_rate': 7.673005363314578e-07, 'epoch': 3.52}
{'loss': 0.1332, 'learning_rate': 7.591779098500584e-07, 'epoch': 3.53}
{'loss': 0.1195, 'learning_rate': 7.510968087648262e-07, 'epoch': 3.53}
{'loss': 0.1266, 'learning_rate': 7.430572693897342e-07, 'epoch': 3.53}
{'loss': 0.1197, 'learning_rate': 7.350593278519824e-07, 'epoch': 3.53}
{'loss': 0.1265, 'learning_rate': 7.271030200918483e-07, 'epoch': 3.54}
{'loss': 0.1323, 'learning_rate': 7.19188381862519e-07, 'epoch': 3.54}
{'loss': 0.1295, 'learning_rate': 7.113154487299267e-07, 'epoch': 3.54}
{'loss': 0.1257, 'learning_rate': 7.034842560726008e-07, 'epoch': 3.54}
{'loss': 0.1181, 'learning_rate': 6.956948390814977e-07, 'epoch': 3.55}
{'loss': 0.1172, 'learning_rate': 6.879472327598502e-07, 'epoch': 3.55}
{'loss': 0.1159, 'learning_rate': 6.802414719230021e-07, 'epoch': 3.55}
{'loss': 0.1198, 'learning_rate': 6.725775911982602e-07, 'epoch': 3.55}
{'loss': 0.1183, 'learning_rate': 6.649556250247325e-07, 'epoch': 3.56}
{'loss': 0.123, 'learning_rate': 6.573756076531779e-07, 'epoch': 3.56}
{'loss': 0.1302, 'learning_rate': 6.498375731458529e-07, 'epoch': 3.56}
{'loss': 0.1213, 'learning_rate': 6.423415553763479e-07, 'epoch': 3.56}
{'loss': 0.1143, 'learning_rate': 6.348875880294536e-07, 'epoch': 3.57}
{'loss': 0.127, 'learning_rate': 6.274757046009871e-07, 'epoch': 3.57}
{'loss': 0.1211, 'learning_rate': 6.201059383976671e-07, 'epoch': 3.57}
{'loss': 0.123, 'learning_rate': 6.127783225369377e-07, 'epoch': 3.57}
{'loss': 0.1254, 'learning_rate': 6.054928899468427e-07, 'epoch': 3.58}
{'loss': 0.1124, 'learning_rate': 5.982496733658582e-07, 'epoch': 3.58}
{'loss': 0.1232, 'learning_rate': 5.910487053427616e-07, 'epoch': 3.58}
{'loss': 0.123, 'learning_rate': 5.83890018236476e-07, 'epoch': 3.58}
{'loss': 0.1266, 'learning_rate': 5.767736442159256e-07, 'epoch': 3.59}
{'loss': 0.1382, 'learning_rate': 5.696996152598966e-07, 'epoch': 3.59}
{'loss': 0.1271, 'learning_rate': 5.626679631568832e-07, 'epoch': 3.59}
{'loss': 0.1204, 'learning_rate': 5.556787195049573e-07, 'epoch': 3.59}
{'loss': 0.1105, 'learning_rate': 5.487319157116178e-07, 'epoch': 3.6}
{'loss': 0.1189, 'learning_rate': 5.418275829936537e-07, 'epoch': 3.6}
{'loss': 0.1171, 'learning_rate': 5.349657523769991e-07, 'epoch': 3.6}
{'loss': 0.128, 'learning_rate': 5.281464546965953e-07, 'epoch': 3.61}
{'loss': 0.1279, 'learning_rate': 5.213697205962631e-07, 'epoch': 3.61}
{'loss': 0.1265, 'learning_rate': 5.146355805285452e-07, 'epoch': 3.61}
{'loss': 0.1182, 'learning_rate': 5.079440647545852e-07, 'epoch': 3.61}
{'loss': 0.1316, 'learning_rate': 5.012952033439844e-07, 'epoch': 3.62}
{'loss': 0.1137, 'learning_rate': 4.946890261746706e-07, 'epoch': 3.62}
{'loss': 0.1154, 'learning_rate': 4.881255629327608e-07, 'epoch': 3.62}
{'loss': 0.1171, 'learning_rate': 4.816048431124265e-07, 'epoch': 3.62}
{'loss': 0.1264, 'learning_rate': 4.7512689601576843e-07, 'epoch': 3.63}
{'loss': 0.1184, 'learning_rate': 4.686917507526756e-07, 'epoch': 3.63}
{'loss': 0.1295, 'learning_rate': 4.6229943624069963e-07, 'epoch': 3.63}
{'loss': 0.1232, 'learning_rate': 4.5594998120492505e-07, 'epoch': 3.63}
{'loss': 0.1172, 'learning_rate': 4.4964341417784165e-07, 'epoch': 3.64}
{'loss': 0.1111, 'learning_rate': 4.4337976349920763e-07, 'epoch': 3.64}
{'loss': 0.1286, 'learning_rate': 4.3715905731593233e-07, 'epoch': 3.64}
{'loss': 0.1141, 'learning_rate': 4.309813235819471e-07, 'epoch': 3.64}
{'loss': 0.1203, 'learning_rate': 4.248465900580734e-07, 'epoch': 3.65}
{'loss': 0.117, 'learning_rate': 4.1875488431190934e-07, 'epoch': 3.65}
{'loss': 0.1156, 'learning_rate': 4.127062337176935e-07, 'epoch': 3.65}
{'loss': 0.1264, 'learning_rate': 4.0670066545619224e-07, 'epoch': 3.65}
{'loss': 0.1208, 'learning_rate': 4.0073820651457043e-07, 'epoch': 3.66}
{'loss': 0.1171, 'learning_rate': 3.9481888368627764e-07, 'epoch': 3.66}
{'loss': 0.118, 'learning_rate': 3.889427235709153e-07, 'epoch': 3.66}
{'loss': 0.1444, 'learning_rate': 3.8310975257413343e-07, 'epoch': 3.66}
{'loss': 0.1221, 'learning_rate': 3.773199969074959e-07, 'epoch': 3.67}
{'loss': 0.1296, 'learning_rate': 3.715734825883766e-07, 'epoch': 3.67}
{'loss': 0.109, 'learning_rate': 3.658702354398325e-07, 'epoch': 3.67}
{'loss': 0.1116, 'learning_rate': 3.6021028109049063e-07, 'epoch': 3.67}
{'loss': 0.1114, 'learning_rate': 3.5459364497443696e-07, 'epoch': 3.68}
{'loss': 0.1193, 'learning_rate': 3.4902035233109644e-07, 'epoch': 3.68}
{'loss': 0.1159, 'learning_rate': 3.4349042820512325e-07, 'epoch': 3.68}
{'loss': 0.1081, 'learning_rate': 3.38003897446284e-07, 'epoch': 3.68}
{'loss': 0.1281, 'learning_rate': 3.325607847093537e-07, 'epoch': 3.69}
{'loss': 0.1353, 'learning_rate': 3.2716111445399324e-07, 'epoch': 3.69}
{'loss': 0.121, 'learning_rate': 3.2180491094465414e-07, 'epoch': 3.69}
{'loss': 0.126, 'learning_rate': 3.164921982504565e-07, 'epoch': 3.69}
{'loss': 0.126, 'learning_rate': 3.112230002450889e-07, 'epoch': 3.7}
{'loss': 0.1314, 'learning_rate': 3.059973406066963e-07, 'epoch': 3.7}
{'loss': 0.1231, 'learning_rate': 3.0081524281777687e-07, 'epoch': 3.7}
{'loss': 0.1258, 'learning_rate': 2.9567673016507646e-07, 'epoch': 3.71}
{'loss': 0.1364, 'learning_rate': 2.905818257394799e-07, 'epoch': 3.71}
{'loss': 0.1183, 'learning_rate': 2.855305524359131e-07, 'epoch': 3.71}
{'loss': 0.1205, 'learning_rate': 2.805229329532344e-07, 'epoch': 3.71}
{'loss': 0.1203, 'learning_rate': 2.7555898979413796e-07, 'epoch': 3.72}
{'loss': 0.1065, 'learning_rate': 2.706387452650494e-07, 'epoch': 3.72}
{'loss': 0.1149, 'learning_rate': 2.657622214760247e-07, 'epoch': 3.72}
{'loss': 0.1283, 'learning_rate': 2.609294403406537e-07, 'epoch': 3.72}
{'loss': 0.1139, 'learning_rate': 2.56140423575959e-07, 'epoch': 3.73}
{'loss': 0.1173, 'learning_rate': 2.513951927023017e-07, 'epoch': 3.73}
{'loss': 0.1178, 'learning_rate': 2.4669376904328244e-07, 'epoch': 3.73}
{'loss': 0.1232, 'learning_rate': 2.420361737256438e-07, 'epoch': 3.73}
{'loss': 0.1231, 'learning_rate': 2.3742242767917921e-07, 'epoch': 3.74}
{'loss': 0.1208, 'learning_rate': 2.3285255163663535e-07, 'epoch': 3.74}
{'loss': 0.1088, 'learning_rate': 2.2832656613362092e-07, 'epoch': 3.74}
{'loss': 0.1195, 'learning_rate': 2.2384449150851695e-07, 'epoch': 3.74}
{'loss': 0.1203, 'learning_rate': 2.1940634790238003e-07, 'epoch': 3.75}
{'loss': 0.1131, 'learning_rate': 2.1501215525885245e-07, 'epoch': 3.75}
{'loss': 0.1247, 'learning_rate': 2.1066193332407891e-07, 'epoch': 3.75}
{'loss': 0.1202, 'learning_rate': 2.063557016466111e-07, 'epoch': 3.75}
{'loss': 0.1264, 'learning_rate': 2.0209347957732328e-07, 'epoch': 3.76}
{'loss': 0.1156, 'learning_rate': 1.978752862693212e-07, 'epoch': 3.76}
{'loss': 0.1261, 'learning_rate': 1.9370114067785995e-07, 'epoch': 3.76}
{'loss': 0.1253, 'learning_rate': 1.8957106156026084e-07, 'epoch': 3.76}
{'loss': 0.1213, 'learning_rate': 1.854850674758213e-07, 'epoch': 3.77}
{'loss': 0.1227, 'learning_rate': 1.8144317678573497e-07, 'epoch': 3.77}
{'loss': 0.1287, 'learning_rate': 1.7744540765300743e-07, 'epoch': 3.77}
{'loss': 0.1139, 'learning_rate': 1.7349177804237837e-07, 'epoch': 3.77}
{'loss': 0.126, 'learning_rate': 1.6958230572023504e-07, 'epoch': 3.78}
{'loss': 0.1169, 'learning_rate': 1.6571700825453674e-07, 'epoch': 3.78}
{'loss': 0.1208, 'learning_rate': 1.618959030147349e-07, 'epoch': 3.78}
{'loss': 0.1237, 'learning_rate': 1.5811900717169537e-07, 'epoch': 3.78}
{'loss': 0.1337, 'learning_rate': 1.5438633769761845e-07, 'epoch': 3.79}
{'loss': 0.1102, 'learning_rate': 1.506979113659679e-07, 'epoch': 3.79}
{'loss': 0.1215, 'learning_rate': 1.4705374475138978e-07, 'epoch': 3.79}
{'loss': 0.1297, 'learning_rate': 1.4345385422964043e-07, 'epoch': 3.79}
{'loss': 0.1351, 'learning_rate': 1.398982559775175e-07, 'epoch': 3.8}
{'loss': 0.1203, 'learning_rate': 1.3638696597277678e-07, 'epoch': 3.8}
{'loss': 0.117, 'learning_rate': 1.3291999999407e-07, 'epoch': 3.8}
{'loss': 0.1234, 'learning_rate': 1.2949737362087156e-07, 'epoch': 3.81}
{'loss': 0.1173, 'learning_rate': 1.2611910223340408e-07, 'epoch': 3.81}
{'loss': 0.1239, 'learning_rate': 1.227852010125752e-07, 'epoch': 3.81}
{'loss': 0.1202, 'learning_rate': 1.1949568493990428e-07, 'epoch': 3.81}
{'loss': 0.1185, 'learning_rate': 1.1625056879746133e-07, 'epoch': 3.82}
{'loss': 0.1214, 'learning_rate': 1.1304986716779265e-07, 'epoch': 3.82}
{'loss': 0.1281, 'learning_rate': 1.0989359443386305e-07, 'epoch': 3.82}
{'loss': 0.1163, 'learning_rate': 1.0678176477898372e-07, 'epoch': 3.82}
{'loss': 0.1177, 'learning_rate': 1.0371439218675671e-07, 'epoch': 3.83}
{'loss': 0.121, 'learning_rate': 1.0069149044100502e-07, 'epoch': 3.83}
{'loss': 0.1335, 'learning_rate': 9.771307312571254e-08, 'epoch': 3.83}
{'loss': 0.1128, 'learning_rate': 9.47791536249676e-08, 'epoch': 3.83}
{'loss': 0.1182, 'learning_rate': 9.188974512289617e-08, 'epoch': 3.84}
{'loss': 0.1181, 'learning_rate': 8.90448606036054e-08, 'epoch': 3.84}
{'loss': 0.1223, 'learning_rate': 8.624451285112689e-08, 'epoch': 3.84}
{'loss': 0.1284, 'learning_rate': 8.348871444935902e-08, 'epoch': 3.84}
{'loss': 0.1234, 'learning_rate': 8.077747778200474e-08, 'epoch': 3.85}
{'loss': 0.1184, 'learning_rate': 7.811081503252382e-08, 'epoch': 3.85}
{'loss': 0.1162, 'learning_rate': 7.54887381840752e-08, 'epoch': 3.85}
{'loss': 0.1188, 'learning_rate': 7.291125901946027e-08, 'epoch': 3.85}
{'loss': 0.1242, 'learning_rate': 7.037838912107298e-08, 'epoch': 3.86}
{'loss': 0.1298, 'learning_rate': 6.789013987084647e-08, 'epoch': 3.86}
{'loss': 0.1172, 'learning_rate': 6.544652245020433e-08, 'epoch': 3.86}
{'loss': 0.1113, 'learning_rate': 6.304754784000721e-08, 'epoch': 3.86}
{'loss': 0.1147, 'learning_rate': 6.069322682050516e-08, 'epoch': 3.87}
{'loss': 0.1156, 'learning_rate': 5.838356997128869e-08, 'epoch': 3.87}
{'loss': 0.124, 'learning_rate': 5.611858767124001e-08, 'epoch': 3.87}
{'loss': 0.1228, 'learning_rate': 5.389829009849079e-08, 'epoch': 3.87}
{'loss': 0.1131, 'learning_rate': 5.1722687230369995e-08, 'epoch': 3.88}
{'loss': 0.1222, 'learning_rate': 4.959178884336502e-08, 'epoch': 3.88}
{'loss': 0.1182, 'learning_rate': 4.7505604513072845e-08, 'epoch': 3.88}
{'loss': 0.1168, 'learning_rate': 4.5464143614162294e-08, 'epoch': 3.88}
{'loss': 0.1135, 'learning_rate': 4.346741532032628e-08, 'epoch': 3.89}
{'loss': 0.1099, 'learning_rate': 4.151542860424407e-08, 'epoch': 3.89}
{'loss': 0.1169, 'learning_rate': 3.96081922375402e-08, 'epoch': 3.89}
{'loss': 0.1141, 'learning_rate': 3.7745714790745623e-08, 'epoch': 3.89}
{'loss': 0.1189, 'learning_rate': 3.592800463325663e-08, 'epoch': 3.9}
{'loss': 0.1184, 'learning_rate': 3.4155069933301535e-08, 'epoch': 3.9}
{'loss': 0.1265, 'learning_rate': 3.242691865790071e-08, 'epoch': 3.9}
{'loss': 0.1156, 'learning_rate': 3.074355857282996e-08, 'epoch': 3.91}
{'loss': 0.1207, 'learning_rate': 2.9104997242590528e-08, 'epoch': 3.91}
{'loss': 0.1205, 'learning_rate': 2.7511242030368035e-08, 'epoch': 3.91}
{'loss': 0.1271, 'learning_rate': 2.5962300098008042e-08, 'epoch': 3.91}
{'loss': 0.1281, 'learning_rate': 2.4458178405974974e-08, 'epoch': 3.92}
{'loss': 0.124, 'learning_rate': 2.2998883713326592e-08, 'epoch': 3.92}
{'loss': 0.1141, 'learning_rate': 2.1584422577682894e-08, 'epoch': 3.92}
{'loss': 0.1134, 'learning_rate': 2.0214801355192826e-08, 'epoch': 3.92}
{'loss': 0.1182, 'learning_rate': 1.889002620051206e-08, 'epoch': 3.93}
{'loss': 0.1066, 'learning_rate': 1.761010306676969e-08, 'epoch': 3.93}
{'loss': 0.1203, 'learning_rate': 1.6375037705543827e-08, 'epoch': 3.93}
{'loss': 0.1186, 'learning_rate': 1.518483566683826e-08, 'epoch': 3.93}
{'loss': 0.1259, 'learning_rate': 1.4039502299050266e-08, 'epoch': 3.94}
{'loss': 0.1161, 'learning_rate': 1.2939042748955078e-08, 'epoch': 3.94}
{'loss': 0.1165, 'learning_rate': 1.1883461961678112e-08, 'epoch': 3.94}
{'loss': 0.116, 'learning_rate': 1.0872764680671666e-08, 'epoch': 3.94}
{'loss': 0.1178, 'learning_rate': 9.906955447697153e-09, 'epoch': 3.95}
{'loss': 0.1203, 'learning_rate': 8.986038602802894e-09, 'epoch': 3.95}
{'loss': 0.1177, 'learning_rate': 8.110018284304132e-09, 'epoch': 3.95}
{'loss': 0.117, 'learning_rate': 7.278898428764169e-09, 'epoch': 3.95}
{'loss': 0.1251, 'learning_rate': 6.492682770979919e-09, 'epoch': 3.96}
{'loss': 0.1221, 'learning_rate': 5.751374843961932e-09, 'epoch': 3.96}
{'loss': 0.1227, 'learning_rate': 5.054977978916631e-09, 'epoch': 3.96}
{'loss': 0.1306, 'learning_rate': 4.403495305237426e-09, 'epoch': 3.96}
{'loss': 0.1129, 'learning_rate': 3.7969297504858445e-09, 'epoch': 3.97}
{'loss': 0.1208, 'learning_rate': 3.2352840403804264e-09, 'epoch': 3.97}
{'loss': 0.1283, 'learning_rate': 2.7185606987811808e-09, 'epoch': 3.97}
{'loss': 0.1171, 'learning_rate': 2.246762047685147e-09, 'epoch': 3.97}
{'loss': 0.1164, 'learning_rate': 1.8198902072097402e-09, 'epoch': 3.98}
{'loss': 0.1199, 'learning_rate': 1.437947095582759e-09, 'epoch': 3.98}
{'loss': 0.1165, 'learning_rate': 1.1009344291401657e-09, 'epoch': 3.98}
{'loss': 0.1148, 'learning_rate': 8.088537223116533e-10, 'epoch': 3.98}
{'loss': 0.1127, 'learning_rate': 5.617062876195345e-10, 'epoch': 3.99}
{'loss': 0.1129, 'learning_rate': 3.594932356654202e-10, 'epoch': 3.99}
{'loss': 0.1164, 'learning_rate': 2.02215475132439e-10, 'epoch': 3.99}
{'loss': 0.1217, 'learning_rate': 8.987371278079693e-11, 'epoch': 3.99}
{'loss': 0.1212, 'learning_rate': 2.246845343667481e-11, 'epoch': 4.0}
{'loss': 0.1224, 'learning_rate': 0.0, 'epoch': 4.0}
100%|██████████| 3120/3120 [4:19:47<00:00, 4.72s/it]/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py:1365: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
warnings.warn(
{'train_runtime': 15617.9007, 'train_samples_per_second': 12.785, 'train_steps_per_second': 0.2, 'train_loss': 0.610505388624584, 'epoch': 4.0}
100%|██████████| 3120/3120 [4:20:17<00:00, 5.01s/it]
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
[2024-04-21 04:02:35,619] [INFO] [launch.py:348:main] Process 274 exits successfully.
[2024-04-21 04:02:36,620] [INFO] [launch.py:348:main] Process 271 exits successfully.
[2024-04-21 04:02:36,620] [INFO] [launch.py:348:main] Process 268 exits successfully.
[2024-04-21 04:02:36,620] [INFO] [launch.py:348:main] Process 273 exits successfully.
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
[2024-04-21 04:02:37,621] [INFO] [launch.py:348:main] Process 270 exits successfully.
[2024-04-21 04:02:37,622] [INFO] [launch.py:348:main] Process 272 exits successfully.
[2024-04-21 04:02:37,622] [INFO] [launch.py:348:main] Process 269 exits successfully.
[2024-04-21 04:02:39,624] [INFO] [launch.py:348:main] Process 267 exits successfully.