==== STARTING EXPERIMENT: qwen3_1.7b_coding_fft ==== Log File: saves/qwen3_1.7b/coding/fft/qwen3_1.7b_coding_fft_20260429_170521.log HF Hub: https://huggingface.co/KKHYA/qwen3-1.7b-fft-coding Timestamp: 2026-04-29 17:05:21 ===================================== [INFO|2026-04-29 17:05:36] llamafactory.launcher:144 >> Initializing 8 distributed tasks at: 127.0.0.1:42957 W0429 17:05:37.808000 243330 site-packages/torch/distributed/run.py:852] W0429 17:05:37.808000 243330 site-packages/torch/distributed/run.py:852] ***************************************** W0429 17:05:37.808000 243330 site-packages/torch/distributed/run.py:852] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. W0429 17:05:37.808000 243330 site-packages/torch/distributed/run.py:852] ***************************************** warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead. warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead. warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead. warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead. warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead. warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead. warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead. warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead. [W429 17:05:52.621446681 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator()) [W429 17:05:52.624434293 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator()) [W429 17:05:52.640682928 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator()) ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Bootstrap: Using eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO cudaDriverVersion 13000 [W429 17:05:52.660057146 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator()) ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO NCCL version 2.28.9+cuda13.0 [W429 17:05:52.673190851 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator()) [W429 17:05:52.676049525 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator()) [W429 17:05:52.699482425 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator()) [W429 17:05:52.738945900 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator()) ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO cudaDriverVersion 13000 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Bootstrap: Using eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO NCCL version 2.28.9+cuda13.0 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Comm config Blocking set to 1 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO cudaDriverVersion 13000 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Bootstrap: Using eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO NCCL version 2.28.9+cuda13.0 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO cudaDriverVersion 13000 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Bootstrap: Using eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO NCCL version 2.28.9+cuda13.0 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Comm config Blocking set to 1 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Comm config Blocking set to 1 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO cudaDriverVersion 13000 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO cudaDriverVersion 13000 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO cudaDriverVersion 13000 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Bootstrap: Using eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO NCCL version 2.28.9+cuda13.0 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Bootstrap: Using eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO NCCL version 2.28.9+cuda13.0 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Bootstrap: Using eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO NCCL version 2.28.9+cuda13.0 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Comm config Blocking set to 1 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Comm config Blocking set to 1 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Comm config Blocking set to 1 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Comm config Blocking set to 1 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO cudaDriverVersion 13000 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Bootstrap: Using eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO NCCL version 2.28.9+cuda13.0 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Comm config Blocking set to 1 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10) ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO NET/Socket : Using [0]eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10) ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO NET/Socket : Using [0]eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10) ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO NET/Socket : Using [0]eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10) ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO NET/Socket : Using [0]eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10) ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO NET/Socket : Using [0]eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10) ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO NET/Socket : Using [0]eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10) ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO NET/Socket : Using [0]eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10) ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO NET/Socket : Using [0]eth0:10.200.170.96<0> ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO ncclCommInitRankConfig comm 0x55ae88a9dd90 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 commId 0xa7453980088d05e1 - Init START ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO ncclCommInitRankConfig comm 0x561d46f046a0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 commId 0xa7453980088d05e1 - Init START ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO ncclCommInitRankConfig comm 0x55b7d7b1b9d0 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 commId 0xa7453980088d05e1 - Init START ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO ncclCommInitRankConfig comm 0x558ed75c7da0 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 commId 0xa7453980088d05e1 - Init START ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO RAS client listening socket at ::1<28028> ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO RAS client listening socket at ::1<28028> ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO ncclCommInitRankConfig comm 0x55b4f5018ae0 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 commId 0xa7453980088d05e1 - Init START ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO ncclCommInitRankConfig comm 0x558fd1beff10 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 commId 0xa7453980088d05e1 - Init START ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO RAS client listening socket at ::1<28028> ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO RAS client listening socket at ::1<28028> ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO ncclCommInitRankConfig comm 0x56416eeca870 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 commId 0xa7453980088d05e1 - Init START ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO RAS client listening socket at ::1<28028> ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO ncclCommInitRankConfig comm 0x55ab97689c00 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 commId 0xa7453980088d05e1 - Init START ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO RAS client listening socket at ::1<28028> ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO RAS client listening socket at ::1<28028> ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO RAS client listening socket at ::1<28028> ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Bootstrap timings total 0.001876 (create 0.000033, send 0.000066, recv 0.000124, ring 0.000111, delay 0.000001) ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Bootstrap timings total 0.000689 (create 0.000033, send 0.000071, recv 0.000113, ring 0.000103, delay 0.000001) ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Bootstrap timings total 0.003558 (create 0.000034, send 0.000066, recv 0.000186, ring 0.002903, delay 0.000001) ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Bootstrap timings total 0.018024 (create 0.000041, send 0.000080, recv 0.014577, ring 0.002899, delay 0.000001) ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Bootstrap timings total 0.006715 (create 0.000036, send 0.000073, recv 0.006098, ring 0.000133, delay 0.000001) ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Bootstrap timings total 0.017532 (create 0.000035, send 0.000062, recv 0.000076, ring 0.001294, delay 0.000001) ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Bootstrap timings total 0.034926 (create 0.000040, send 0.000182, recv 0.016987, ring 0.017193, delay 0.000001) ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Bootstrap timings total 0.017917 (create 0.000036, send 0.000064, recv 0.000159, ring 0.016940, delay 0.000001) ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 7 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO NVLS multicast support is not available on dev 7 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO NVLS multicast support is not available on dev 0 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 1 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO NVLS multicast support is not available on dev 1 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 6 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO NVLS multicast support is not available on dev 6 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 2 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO NVLS multicast support is not available on dev 2 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 5 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO NVLS multicast support is not available on dev 5 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 3 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO NVLS multicast support is not available on dev 3 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 4 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO NVLS multicast support is not available on dev 4 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO comm 0x55b4f5018ae0 rank 6 nRanks 8 nNodes 1 localRanks 8 localRank 6 MNNVL 0 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO comm 0x558ed75c7da0 rank 1 nRanks 8 nNodes 1 localRanks 8 localRank 1 MNNVL 0 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO comm 0x558fd1beff10 rank 5 nRanks 8 nNodes 1 localRanks 8 localRank 5 MNNVL 0 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO comm 0x55ab97689c00 rank 7 nRanks 8 nNodes 1 localRanks 8 localRank 7 MNNVL 0 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO comm 0x561d46f046a0 rank 4 nRanks 8 nNodes 1 localRanks 8 localRank 4 MNNVL 0 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO comm 0x55ae88a9dd90 rank 3 nRanks 8 nNodes 1 localRanks 8 localRank 3 MNNVL 0 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO comm 0x55b7d7b1b9d0 rank 2 nRanks 8 nNodes 1 localRanks 8 localRank 2 MNNVL 0 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO comm 0x56416eeca870 rank 0 nRanks 8 nNodes 1 localRanks 8 localRank 0 MNNVL 0 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 00/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 01/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 02/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 03/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 04/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 05/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 06/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 07/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 08/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 09/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 10/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 11/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 12/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 13/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 14/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 15/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 16/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 17/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 18/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 19/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 20/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 21/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 22/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 23/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243416:243804 [0] NCCL INFO [Proxy Service] Device 5 CPU core 77 ywang29-p4d-debug-1-worker-0:243416:243805 [0] NCCL INFO [Proxy Service UDS] Device 5 CPU core 87 ywang29-p4d-debug-1-worker-0:243414:243806 [0] NCCL INFO [Proxy Service] Device 3 CPU core 56 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so ywang29-p4d-debug-1-worker-0:243414:243807 [0] NCCL INFO [Proxy Service UDS] Device 3 CPU core 9 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243412:243808 [0] NCCL INFO [Proxy Service] Device 1 CPU core 3 ywang29-p4d-debug-1-worker-0:243412:243809 [0] NCCL INFO [Proxy Service UDS] Device 1 CPU core 4 ywang29-p4d-debug-1-worker-0:243415:243810 [0] NCCL INFO [Proxy Service] Device 4 CPU core 43 ywang29-p4d-debug-1-worker-0:243415:243811 [0] NCCL INFO [Proxy Service UDS] Device 4 CPU core 47 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so ywang29-p4d-debug-1-worker-0:243418:243812 [0] NCCL INFO [Proxy Service] Device 7 CPU core 93 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243418:243813 [0] NCCL INFO [Proxy Service UDS] Device 7 CPU core 46 ywang29-p4d-debug-1-worker-0:243411:243814 [0] NCCL INFO [Proxy Service] Device 0 CPU core 23 ywang29-p4d-debug-1-worker-0:243411:243815 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 5 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243417:243817 [0] NCCL INFO [Proxy Service UDS] Device 6 CPU core 42 ywang29-p4d-debug-1-worker-0:243417:243816 [0] NCCL INFO [Proxy Service] Device 6 CPU core 89 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243413:243818 [0] NCCL INFO [Proxy Service] Device 2 CPU core 6 ywang29-p4d-debug-1-worker-0:243413:243819 [0] NCCL INFO [Proxy Service UDS] Device 2 CPU core 59 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3) ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3) ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3) ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3) ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3) ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3) ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO CC Off, workFifoBytes 1048576 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3) ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3) ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO ncclCommInitRankConfig comm 0x55b7d7b1b9d0 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 commId 0xa7453980088d05e1 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO ncclCommInitRankConfig comm 0x561d46f046a0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 commId 0xa7453980088d05e1 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO ncclCommInitRankConfig comm 0x55b4f5018ae0 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 commId 0xa7453980088d05e1 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Init timings - ncclCommInitRankConfig: rank 4 nranks 8 total 0.50 (kernels 0.18, alloc 0.18, bootstrap 0.02, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.06, rest 0.02) ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Init timings - ncclCommInitRankConfig: rank 6 nranks 8 total 0.49 (kernels 0.18, alloc 0.19, bootstrap 0.01, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.05, rest 0.03) ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Init timings - ncclCommInitRankConfig: rank 2 nranks 8 total 0.51 (kernels 0.18, alloc 0.18, bootstrap 0.02, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.06, rest 0.02) ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO ncclCommInitRankConfig comm 0x56416eeca870 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 commId 0xa7453980088d05e1 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Init timings - ncclCommInitRankConfig: rank 0 nranks 8 total 0.49 (kernels 0.19, alloc 0.18, bootstrap 0.00, allgathers 0.01, topo 0.03, graphs 0.00, connections 0.06, rest 0.02) ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO ncclCommInitRankConfig comm 0x55ae88a9dd90 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 commId 0xa7453980088d05e1 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO ncclCommInitRankConfig comm 0x558fd1beff10 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 commId 0xa7453980088d05e1 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Init timings - ncclCommInitRankConfig: rank 5 nranks 8 total 0.49 (kernels 0.18, alloc 0.19, bootstrap 0.00, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.05, rest 0.03) ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO ncclCommInitRankConfig comm 0x55ab97689c00 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 commId 0xa7453980088d05e1 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Init timings - ncclCommInitRankConfig: rank 7 nranks 8 total 0.49 (kernels 0.19, alloc 0.17, bootstrap 0.00, allgathers 0.01, topo 0.03, graphs 0.00, connections 0.06, rest 0.02) ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Init timings - ncclCommInitRankConfig: rank 3 nranks 8 total 0.51 (kernels 0.19, alloc 0.17, bootstrap 0.03, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.06, rest 0.02) ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO ncclCommInitRankConfig comm 0x558ed75c7da0 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 commId 0xa7453980088d05e1 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Init timings - ncclCommInitRankConfig: rank 1 nranks 8 total 0.49 (kernels 0.18, alloc 0.18, bootstrap 0.02, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.06, rest 0.02) [INFO|2026-04-29 17:05:53] llamafactory.hparams.parser:505 >> Process rank: 7, world size: 8, device: cuda:7, distributed training: True, compute dtype: torch.bfloat16 [INFO|2026-04-29 17:05:53] llamafactory.hparams.parser:505 >> Process rank: 6, world size: 8, device: cuda:6, distributed training: True, compute dtype: torch.bfloat16 [INFO|2026-04-29 17:05:53] llamafactory.hparams.parser:505 >> Process rank: 4, world size: 8, device: cuda:4, distributed training: True, compute dtype: torch.bfloat16 [INFO|2026-04-29 17:05:53] llamafactory.hparams.parser:505 >> Process rank: 5, world size: 8, device: cuda:5, distributed training: True, compute dtype: torch.bfloat16 [INFO|2026-04-29 17:05:53] llamafactory.hparams.parser:505 >> Process rank: 3, world size: 8, device: cuda:3, distributed training: True, compute dtype: torch.bfloat16 [INFO|2026-04-29 17:05:53] llamafactory.hparams.parser:505 >> Process rank: 1, world size: 8, device: cuda:1, distributed training: True, compute dtype: torch.bfloat16 [INFO|2026-04-29 17:05:53] llamafactory.hparams.parser:505 >> Process rank: 2, world size: 8, device: cuda:2, distributed training: True, compute dtype: torch.bfloat16 [INFO|2026-04-29 17:05:53] llamafactory.hparams.parser:505 >> Process rank: 0, world size: 8, device: cuda:0, distributed training: True, compute dtype: torch.bfloat16 [INFO|configuration_utils.py:670] 2026-04-29 17:05:53,802 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/config.json [INFO|configuration_utils.py:742] 2026-04-29 17:05:53,805 >> Model config Qwen3Config { "architectures": [ "Qwen3ForCausalLM" ], "attention_bias": false, "attention_dropout": 0.0, "bos_token_id": 151643, "dtype": "bfloat16", "eos_token_id": 151645, "head_dim": 128, "hidden_act": "silu", "hidden_size": 2048, "initializer_range": 0.02, "intermediate_size": 6144, "layer_types": [ "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention" ], "masked_layers": null, "max_position_embeddings": 40960, "max_window_layers": 28, "model_type": "qwen3", "num_attention_heads": 16, "num_hidden_layers": 28, "num_key_value_heads": 8, "pad_token_id": null, "rms_norm_eps": 1e-06, "rope_parameters": { "rope_theta": 1000000, "rope_type": "default" }, "sliding_window": null, "sparsity_attn": null, "sparsity_mlp": null, "subnet_mode": null, "subnet_type": null, "threshold_attn": null, "threshold_mlp": null, "tie_word_embeddings": true, "transformers_version": "5.2.0", "use_cache": true, "use_sliding_window": false, "vocab_size": 151936 } [INFO|configuration_utils.py:670] 2026-04-29 17:05:56,244 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/config.json [INFO|configuration_utils.py:742] 2026-04-29 17:05:56,245 >> Model config Qwen3Config { "architectures": [ "Qwen3ForCausalLM" ], "attention_bias": false, "attention_dropout": 0.0, "bos_token_id": 151643, "dtype": "bfloat16", "eos_token_id": 151645, "head_dim": 128, "hidden_act": "silu", "hidden_size": 2048, "initializer_range": 0.02, "intermediate_size": 6144, "layer_types": [ "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention" ], "masked_layers": null, "max_position_embeddings": 40960, "max_window_layers": 28, "model_type": "qwen3", "num_attention_heads": 16, "num_hidden_layers": 28, "num_key_value_heads": 8, "pad_token_id": null, "rms_norm_eps": 1e-06, "rope_parameters": { "rope_theta": 1000000, "rope_type": "default" }, "sliding_window": null, "sparsity_attn": null, "sparsity_mlp": null, "subnet_mode": null, "subnet_type": null, "threshold_attn": null, "threshold_mlp": null, "tie_word_embeddings": true, "transformers_version": "5.2.0", "use_cache": true, "use_sliding_window": false, "vocab_size": 151936 } [INFO|configuration_utils.py:670] 2026-04-29 17:05:56,358 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/config.json [INFO|configuration_utils.py:742] 2026-04-29 17:05:56,359 >> Model config Qwen3Config { "architectures": [ "Qwen3ForCausalLM" ], "attention_bias": false, "attention_dropout": 0.0, "bos_token_id": 151643, "dtype": "bfloat16", "eos_token_id": 151645, "head_dim": 128, "hidden_act": "silu", "hidden_size": 2048, "initializer_range": 0.02, "intermediate_size": 6144, "layer_types": [ "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention" ], "masked_layers": null, "max_position_embeddings": 40960, "max_window_layers": 28, "model_type": "qwen3", "num_attention_heads": 16, "num_hidden_layers": 28, "num_key_value_heads": 8, "pad_token_id": null, "rms_norm_eps": 1e-06, "rope_parameters": { "rope_theta": 1000000, "rope_type": "default" }, "sliding_window": null, "sparsity_attn": null, "sparsity_mlp": null, "subnet_mode": null, "subnet_type": null, "threshold_attn": null, "threshold_mlp": null, "tie_word_embeddings": true, "transformers_version": "5.2.0", "use_cache": true, "use_sliding_window": false, "vocab_size": 151936 } ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 16/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 17/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 18/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 19/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 20/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 21/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 22/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 23/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 00/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 01/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 02/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 03/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 04/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 05/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 06/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 07/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 08/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 09/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 10/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 11/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 12/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 13/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 14/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 15/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 16/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 17/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 18/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 19/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 20/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 21/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 22/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 23/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 16/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 17/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 18/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 19/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 20/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 21/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 22/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 23/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 16/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 17/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 18/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 19/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 16/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 20/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 17/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 21/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 18/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 22/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 19/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 23/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 20/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 21/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 22/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 23/0 : 5[5] -> 6[6] via P2P/CUMEM/read [INFO|2026-04-29 17:05:57] llamafactory.data.loader:144 >> Loading dataset allenai/tulu-3-sft-personas-code... ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 16/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 17/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 18/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 19/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 20/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 21/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 22/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 23/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 00/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 01/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 02/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 03/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 04/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 05/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 06/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 07/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 08/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 09/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 10/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 11/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 12/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 13/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 14/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 15/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 16/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 17/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 18/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 19/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 20/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 21/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 22/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 23/0 : 7[7] -> 0[0] via P2P/CUMEM/read [INFO|2026-04-29 17:05:58] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset allenai/tulu-3-sft-personas-code. [INFO|2026-04-29 17:05:59] llamafactory.data.loader:144 >> Loading dataset KKHYA/evol_codealpaca_converted... Repo card metadata block was not found. Setting CardData to empty. [INFO|2026-04-29 17:06:00] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset KKHYA/evol_codealpaca_converted. [INFO|2026-04-29 17:06:00] llamafactory.data.loader:144 >> Loading dataset KKHYA/codefeedback_filtered_instructions_converted... Repo card metadata block was not found. Setting CardData to empty. [INFO|2026-04-29 17:06:01] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset KKHYA/codefeedback_filtered_instructions_converted. ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 16/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 17/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 18/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 19/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 20/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 21/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 22/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 23/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 training example: input_ids: [151644, 872, 198, 7985, 264, 10135, 729, 311, 11047, 279, 2790, 1372, 315, 8845, 16548, 553, 1674, 3767, 300, 4308, 304, 264, 2661, 3200, 504, 264, 1140, 315, 2432, 3059, 13, 8886, 2432, 1102, 374, 15251, 438, 264, 10997, 448, 6894, 330, 5117, 26532, 497, 330, 13757, 26532, 497, 330, 5117, 96244, 497, 323, 330, 13757, 96244, 3263, 1674, 3767, 300, 4308, 1410, 387, 2987, 279, 2114, 476, 3123, 2083, 304, 894, 2432, 13, 576, 1946, 374, 264, 1140, 315, 1741, 2432, 1102, 57514, 11, 323, 279, 2550, 1265, 387, 458, 7546, 14064, 279, 2790, 1372, 315, 8845, 16548, 553, 1674, 3767, 300, 4308, 382, 2505, 510, 12, 1565, 6347, 13576, 44622, 362, 1140, 315, 57514, 11, 1380, 1817, 10997, 5610, 510, 220, 481, 330, 5117, 26532, 1, 320, 917, 1648, 576, 829, 315, 279, 2114, 2083, 624, 220, 481, 330, 13757, 26532, 1, 320, 917, 1648, 576, 829, 315, 279, 3123, 2083, 624, 220, 481, 330, 5117, 96244, 1, 320, 396, 1648, 576, 1372, 315, 8845, 16548, 553, 279, 2114, 2083, 624, 220, 481, 330, 13757, 96244, 1, 320, 396, 1648, 576, 1372, 315, 8845, 16548, 553, 279, 3123, 2083, 382, 5097, 510, 12, 1527, 7546, 14064, 279, 2790, 1372, 315, 8845, 16548, 553, 1674, 3767, 300, 4308, 382, 13314, 510, 73594, 12669, 198, 6347, 13576, 284, 2278, 262, 5212, 5117, 26532, 788, 330, 2101, 3767, 300, 4308, 497, 330, 13757, 26532, 788, 330, 14597, 32, 497, 330, 5117, 96244, 788, 220, 17, 11, 330, 13757, 96244, 788, 220, 16, 1583, 262, 5212, 5117, 26532, 788, 330, 14597, 33, 497, 330, 13757, 26532, 788, 330, 2101, 3767, 300, 4308, 497, 330, 5117, 96244, 788, 220, 18, 11, 330, 13757, 96244, 788, 220, 17, 1583, 262, 5212, 5117, 26532, 788, 330, 2101, 3767, 300, 4308, 497, 330, 13757, 26532, 788, 330, 14597, 34, 497, 330, 5117, 96244, 788, 220, 16, 11, 330, 13757, 96244, 788, 220, 16, 1583, 262, 5212, 5117, 26532, 788, 330, 14597, 35, 497, 330, 13757, 26532, 788, 330, 2101, 3767, 300, 4308, 497, 330, 5117, 96244, 788, 220, 15, 11, 330, 13757, 96244, 788, 220, 18, 532, 921, 2, 31021, 9258, 25, 220, 23, 198, 73594, 151645, 198, 151644, 77091, 198, 750, 11047, 8418, 3767, 300, 4308, 96244, 25401, 13576, 982, 262, 2790, 96244, 284, 220, 15, 198, 262, 369, 2432, 304, 2432, 13576, 510, 286, 421, 2432, 1183, 5117, 26532, 1341, 621, 330, 2101, 3767, 300, 4308, 4660, 310, 2790, 96244, 1421, 2432, 1183, 5117, 96244, 7026, 286, 4409, 2432, 1183, 13757, 26532, 1341, 621, 330, 2101, 3767, 300, 4308, 4660, 310, 2790, 96244, 1421, 2432, 1183, 13757, 96244, 7026, 262, 470, 2790, 96244, 151645, 198] inputs: <|im_start|>user Write a python function to calculate the total number of goals scored by Alanyaspor in a given season from a list of match results. Each match result is represented as a dictionary with keys "home_team", "away_team", "home_goals", and "away_goals". Alanyaspor could be either the home or away team in any match. The input is a list of such match result dictionaries, and the output should be an integer representing the total number of goals scored by Alanyaspor. Input: - `match_results`: A list of dictionaries, where each dictionary contains: - "home_team" (string): The name of the home team. - "away_team" (string): The name of the away team. - "home_goals" (int): The number of goals scored by the home team. - "away_goals" (int): The number of goals scored by the away team. Output: - An integer representing the total number of goals scored by Alanyaspor. Example: ```python match_results = [ {"home_team": "Alanyaspor", "away_team": "TeamA", "home_goals": 2, "away_goals": 1}, {"home_team": "TeamB", "away_team": "Alanyaspor", "home_goals": 3, "away_goals": 2}, {"home_team": "Alanyaspor", "away_team": "TeamC", "home_goals": 1, "away_goals": 1}, {"home_team": "TeamD", "away_team": "Alanyaspor", "home_goals": 0, "away_goals": 3} ] # Expected Output: 8 ```<|im_end|> <|im_start|>assistant def calculate_alanyaspor_goals(match_results): total_goals = 0 for match in match_results: if match["home_team"] == "Alanyaspor": total_goals += match["home_goals"] elif match["away_team"] == "Alanyaspor": total_goals += match["away_goals"] return total_goals<|im_end|> label_ids: [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 750, 11047, 8418, 3767, 300, 4308, 96244, 25401, 13576, 982, 262, 2790, 96244, 284, 220, 15, 198, 262, 369, 2432, 304, 2432, 13576, 510, 286, 421, 2432, 1183, 5117, 26532, 1341, 621, 330, 2101, 3767, 300, 4308, 4660, 310, 2790, 96244, 1421, 2432, 1183, 5117, 96244, 7026, 286, 4409, 2432, 1183, 13757, 26532, 1341, 621, 330, 2101, 3767, 300, 4308, 4660, 310, 2790, 96244, 1421, 2432, 1183, 13757, 96244, 7026, 262, 470, 2790, 96244, 151645, 198] labels: def calculate_alanyaspor_goals(match_results): total_goals = 0 for match in match_results: if match["home_team"] == "Alanyaspor": total_goals += match["home_goals"] elif match["away_team"] == "Alanyaspor": total_goals += match["away_goals"] return total_goals<|im_end|> Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. Repo card metadata block was not found. Setting CardData to empty. [INFO|configuration_utils.py:670] 2026-04-29 17:06:05,518 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/config.json [INFO|configuration_utils.py:742] 2026-04-29 17:06:05,518 >> Model config Qwen3Config { "architectures": [ "Qwen3ForCausalLM" ], "attention_bias": false, "attention_dropout": 0.0, "bos_token_id": 151643, "dtype": "bfloat16", "eos_token_id": 151645, "head_dim": 128, "hidden_act": "silu", "hidden_size": 2048, "initializer_range": 0.02, "intermediate_size": 6144, "layer_types": [ "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention", "full_attention" ], "masked_layers": null, "max_position_embeddings": 40960, "max_window_layers": 28, "model_type": "qwen3", "num_attention_heads": 16, "num_hidden_layers": 28, "num_key_value_heads": 8, "pad_token_id": null, "rms_norm_eps": 1e-06, "rope_parameters": { "rope_theta": 1000000, "rope_type": "default" }, "sliding_window": null, "sparsity_attn": null, "sparsity_mlp": null, "subnet_mode": null, "subnet_type": null, "threshold_attn": null, "threshold_mlp": null, "tie_word_embeddings": true, "transformers_version": "5.2.0", "use_cache": true, "use_sliding_window": false, "vocab_size": 151936 } [INFO|2026-04-29 17:06:05] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training. [INFO|modeling_utils.py:710] 2026-04-29 17:06:06,257 >> loading weights file model.safetensors from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/model.safetensors.index.json [INFO|modeling_utils.py:779] 2026-04-29 17:06:06,257 >> Will use dtype=torch.bfloat16 as defined in model's config object [INFO|configuration_utils.py:1014] 2026-04-29 17:06:06,258 >> Generate config GenerationConfig { "bos_token_id": 151643, "eos_token_id": 151645, "output_attentions": false, "output_hidden_states": false, "use_cache": false } Loading weights: 0%| | 0/311 [00:00> The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning Loading weights: 1%| | 2/311 [00:00<00:24, 12.67it/s, Materializing param=lm_head.weight] Loading weights: 1%| | 2/311 [00:00<00:24, 12.67it/s, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 2/311 [00:00<00:24, 12.67it/s, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 2/311 [00:00<00:42, 7.33it/s, Materializing param=lm_head.weight] Loading weights: 1%| | 2/311 [00:00<00:42, 7.33it/s, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 2/311 [00:00<00:42, 7.33it/s, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 3/311 [00:00<00:42, 7.33it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/311 [00:00<00:42, 7.33it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%| | 2/311 [00:00<00:43, 7.14it/s, Materializing param=lm_head.weight] Loading weights: 1%| | 2/311 [00:00<00:43, 7.14it/s, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 2/311 [00:00<00:43, 7.14it/s, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 3/311 [00:00<00:43, 7.14it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/311 [00:00<00:43, 7.14it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:43, 7.14it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:43, 7.14it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:41, 7.33it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 1%| | 3/311 [00:00<00:18, 16.73it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/311 [00:00<00:18, 16.73it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:22, 13.86it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:22, 13.86it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:22, 13.86it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:42, 7.14it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:22, 13.86it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:22, 13.86it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 1%| | 2/311 [00:00<00:40, 7.59it/s, Materializing param=lm_head.weight] Loading weights: 1%| | 2/311 [00:00<00:40, 7.59it/s, Materializing param=model.embed_tokens.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 1%| | 2/311 [00:00<00:40, 7.59it/s, Materializing param=model.embed_tokens.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:40, 7.33it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 1%| | 3/311 [00:00<00:40, 7.59it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/311 [00:00<00:40, 7.59it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:40, 7.59it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:40, 7.59it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:22, 13.86it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:22, 13.86it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:40, 7.59it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:40, 7.59it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 1%| | 2/311 [00:00<00:49, 6.28it/s, Materializing param=lm_head.weight] Loading weights: 1%| | 2/311 [00:00<00:49, 6.28it/s, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 2/311 [00:00<00:49, 6.28it/s, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 3/311 [00:00<00:49, 6.28it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/311 [00:00<00:49, 6.28it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:39, 7.33it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:48, 6.28it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:48, 6.28it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:48, 6.28it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:48, 6.28it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:40, 7.59it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:40, 7.59it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:40, 7.59it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:40, 7.59it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:41, 7.14it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:48, 6.28it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:48, 6.28it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:40, 7.14it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:39, 7.59it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:48, 6.28it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:48, 6.28it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:48, 6.28it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:48, 6.28it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:48, 6.28it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:48, 6.28it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:47, 6.28it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:47, 6.28it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:21, 13.86it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:47, 6.28it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:47, 6.28it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 10%|█ | 32/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 10%|█ | 32/311 [00:00<00:38, 7.33it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:47, 6.28it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:47, 6.28it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:47, 6.28it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:47, 6.28it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:47, 6.28it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 11%|█ | 33/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:47, 6.28it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 11%|█ | 33/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:47, 6.28it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:47, 6.28it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 11%|█ | 34/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 11%|█ | 34/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 10%|█ | 32/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 10%|█ | 32/311 [00:00<00:39, 7.14it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:38, 7.59it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 11%|█ | 33/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 11%|█ | 33/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 11%|█ | 34/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 11%|█ | 34/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:37, 7.33it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 10%|█ | 32/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 10%|█ | 32/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.self_attn.k_proj.weight][INFO|configuration_utils.py:967] 2026-04-29 17:06:07,901 >> loading configuration file generation_config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/generation_config.json Loading weights: 11%|█ | 33/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 11%|█ | 33/311 [00:00<00:20, 13.86it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 11%|█ | 34/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.2.self_attn.q_proj.weight][INFO|configuration_utils.py:1014] 2026-04-29 17:06:07,901 >> Generate config GenerationConfig { "bos_token_id": 151643, "do_sample": true, "eos_token_id": [ 151645, 151643 ], "pad_token_id": 151643, "temperature": 0.6, "top_k": 20, "top_p": 0.95 } Loading weights: 11%|█ | 34/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:36, 7.33it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 1%| | 3/311 [00:00<00:23, 13.12it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/311 [00:00<00:23, 13.12it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:46, 6.28it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:45, 6.28it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:45, 6.28it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:45, 6.28it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:45, 6.28it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:38, 7.14it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:45, 6.28it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:45, 6.28it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:45, 6.28it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:45, 6.28it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:37, 7.59it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 10%|█ | 32/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 10%|█ | 32/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 11%|█ | 33/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 11%|█ | 33/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 11%|█ | 34/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 11%|█ | 34/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:36, 7.59it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:19, 13.86it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:18, 13.86it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:18, 13.86it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:45, 6.28it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:45, 6.28it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:45, 6.28it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:45, 6.28it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 132.29it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:18, 13.86it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:18, 13.86it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:44, 6.28it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:44, 6.28it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:44, 6.28it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:44, 6.28it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:44, 6.28it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:44, 6.28it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 10%|█ | 32/311 [00:00<00:44, 6.28it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 10%|█ | 32/311 [00:00<00:44, 6.28it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:35, 7.59it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 11%|█ | 33/311 [00:00<00:44, 6.28it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 11%|█ | 33/311 [00:00<00:44, 6.28it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 11%|█ | 34/311 [00:00<00:44, 6.28it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 11%|█ | 34/311 [00:00<00:44, 6.28it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:43, 6.28it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:43, 6.28it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:43, 6.28it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:43, 6.28it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:43, 6.28it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:43, 6.28it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 20%|██ | 63/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 20%|██ | 63/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 21%|██ | 64/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 21%|██ | 64/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 21%|██ | 65/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 21%|██ | 65/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 21%|██ | 66/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 21%|██ | 66/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 20%|██ | 63/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 20%|██ | 63/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 21%|██ | 64/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 21%|██ | 64/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 21%|██ | 65/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 21%|██ | 65/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 21%|██ | 66/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 21%|██ | 66/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 20%|██ | 63/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 20%|██ | 63/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 21%|██ | 64/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 21%|██ | 64/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 21%|██ | 65/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 21%|██ | 65/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 21%|██ | 66/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 21%|██ | 66/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 20%|██ | 63/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 20%|██ | 63/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 21%|██ | 64/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 21%|██ | 64/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 21%|██ | 65/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 21%|██ | 65/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 1%| | 3/311 [00:00<00:24, 12.67it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 1%| | 3/311 [00:00<00:24, 12.67it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 21%|██ | 66/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 21%|██ | 66/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 10%|█ | 32/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 10%|█ | 32/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█ | 33/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 11%|█ | 33/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 11%|█ | 34/311 [00:00<00:23, 11.58it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 11%|█ | 34/311 [00:00<00:23, 11.58it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:23, 11.58it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:23, 11.58it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 2%|▏ | 5/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:23, 11.58it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:23, 11.58it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:23, 11.58it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:23, 11.58it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 2%|▏ | 6/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:23, 11.58it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:23, 11.58it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 158.89it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.mlp.up_proj.weight][INFO|dynamic_module_utils.py:406] 2026-04-29 17:06:08,005 >> Could not locate the custom_generate/generate.py inside Qwen/Qwen3-1.7B. Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.self_attn.o_proj.weight][INFO|2026-04-29 17:06:08] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled. [INFO|2026-04-29 17:06:08] llamafactory.model.model_utils.attention:144 >> Using torch SDPA for faster training and inference. [INFO|2026-04-29 17:06:08] llamafactory.model.adapter:144 >> Upcasting trainable params to float32. [INFO|2026-04-29 17:06:08] llamafactory.model.adapter:144 >> Fine-tuning method: Full Loading weights: 19%|█▊ | 58/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 2%|▏ | 7/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 3%|▎ | 8/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 3%|▎ | 9/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 3%|▎ | 10/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 30%|███ | 94/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 30%|███ | 94/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 132.29it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 4%|▎ | 11/311 [00:00<00:26, 11.53it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 4%|▍ | 12/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 4%|▍ | 13/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 5%|▍ | 14/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 5%|▍ | 15/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 30%|███ | 94/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 30%|███ | 94/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 31%|███ | 95/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 31%|███ | 95/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 31%|███ | 96/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 31%|███ | 96/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 31%|███ | 97/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 31%|███ | 97/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 5%|▌ | 16/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 31%|███ | 95/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 31%|███ | 95/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 31%|███ | 96/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 31%|███ | 96/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 31%|███ | 97/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 31%|███ | 97/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 20%|██ | 63/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 5%|▌ | 17/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 20%|██ | 63/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 21%|██ | 64/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 21%|██ | 64/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 21%|██ | 65/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 21%|██ | 65/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:01, 175.84it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 21%|██ | 66/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 21%|██ | 66/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 6%|▌ | 18/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 6%|▌ | 19/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 6%|▋ | 20/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 7%|▋ | 21/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 30%|███ | 94/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 30%|███ | 94/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 7%|▋ | 22/311 [00:00<00:25, 11.53it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 7%|▋ | 23/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 155.68it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 8%|▊ | 24/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 8%|▊ | 25/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 8%|▊ | 26/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 31%|███ | 95/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:02, 114.67it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 31%|███ | 95/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 31%|███ | 96/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 31%|███ | 96/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 31%|███ | 97/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 31%|███ | 97/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 9%|▊ | 27/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 30%|███ | 94/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 30%|███ | 94/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 9%|▉ | 28/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.mlp.gate_proj.weight][INFO|2026-04-29 17:06:08] llamafactory.model.loader:144 >> trainable params: 2,031,739,904 || all params: 2,031,739,904 || trainable%: 100.0000 Loading weights: 31%|███ | 95/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 31%|███ | 95/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 31%|███ | 96/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 31%|███ | 96/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 31%|███ | 97/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 31%|███ | 97/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 9%|▉ | 29/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 10%|▉ | 30/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 10%|▉ | 31/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 10%|█ | 32/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 10%|█ | 32/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 11%|█ | 33/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 11%|█ | 33/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 11%|█ | 34/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 11%|█ | 34/311 [00:00<00:24, 11.53it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:23, 11.53it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 11%|█▏ | 35/311 [00:00<00:23, 11.53it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:23, 11.53it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 12%|█▏ | 36/311 [00:00<00:23, 11.53it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:23, 11.53it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 12%|█▏ | 37/311 [00:00<00:23, 11.53it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 20%|██ | 63/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 20%|██ | 63/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 21%|██ | 64/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 21%|██ | 64/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 21%|██ | 65/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 21%|██ | 65/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 21%|██ | 66/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 21%|██ | 66/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:23, 11.53it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 12%|█▏ | 38/311 [00:00<00:23, 11.53it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:02, 120.27it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 120.27it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 120.27it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 120.27it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 120.27it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 47/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 15%|█▌ | 48/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 254.36it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 120.27it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 120.27it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 120.27it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 120.27it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 120.27it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 120.27it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 120.27it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 120.27it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 49/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 30%|███ | 94/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 30%|███ | 94/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 16%|█▌ | 50/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 228.68it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 31%|███ | 95/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 31%|███ | 95/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 31%|███ | 96/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 31%|███ | 96/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 31%|███ | 97/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 31%|███ | 97/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 51/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 17%|█▋ | 52/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 17%|█▋ | 53/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 54/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 55/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 56/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 18%|█▊ | 57/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 19%|█▊ | 58/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 297.27it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 19%|█▉ | 59/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 19%|█▉ | 60/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 259.35it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 20%|█▉ | 61/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 20%|█▉ | 62/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:01, 182.18it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 20%|██ | 63/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 20%|██ | 63/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 21%|██ | 64/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 21%|██ | 64/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 21%|██ | 65/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 21%|██ | 65/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 21%|██ | 66/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 21%|██ | 66/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 30%|███ | 94/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 30%|███ | 94/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 67/311 [00:00<00:02, 121.83it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 31%|███ | 95/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 31%|███ | 95/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 31%|███ | 96/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 31%|███ | 96/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 31%|███ | 97/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 31%|███ | 97/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 121.83it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:01, 200.31it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 200.31it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 200.31it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 200.31it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 200.31it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 200.31it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 200.31it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 200.31it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 200.31it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 200.31it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 200.31it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 313.51it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 30%|██▉ | 93/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 290.52it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 30%|███ | 94/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 30%|███ | 94/311 [00:00<00:01, 216.05it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.10.self_attn.v_proj.weight][WARNING|2026-04-29 17:06:08] llamafactory.train.callbacks:155 >> Previous trainer log in this folder will be deleted. Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.mlp.up_proj.weight][WARNING|trainer_utils.py:1234] 2026-04-29 17:06:08,234 >> The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}. Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 31%|███ | 95/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 31%|███ | 95/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 31%|███ | 96/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 31%|███ | 96/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 31%|███ | 97/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 31%|███ | 97/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 345.22it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 315.95it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 240.68it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 216.05it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 40%|████ | 125/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 41%|████ | 126/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 348.61it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 255.40it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 41%|████ | 127/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 41%|████ | 128/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 331.54it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 372.07it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 349.86it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 45%|████▍ | 139/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 45%|████▌ | 140/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 45%|████▌ | 141/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 46%|████▌ | 142/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 46%|████▌ | 143/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 46%|████▋ | 144/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 47%|████▋ | 145/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 47%|████▋ | 146/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 277.44it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 47%|████▋ | 147/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 80%|███████▉ | 248/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 80%|███████▉ | 248/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 80%|███████▉ | 248/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 248/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 48%|████▊ | 148/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 80%|████████ | 249/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 80%|████████ | 249/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 80%|████████ | 249/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 80%|████████ | 250/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 80%|████████ | 250/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 80%|████████ | 249/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 80%|████████ | 250/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 80%|████████ | 250/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 81%|████████ | 251/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 81%|████████ | 251/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 81%|████████ | 251/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 81%|████████ | 251/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 81%|████████ | 252/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 81%|████████ | 252/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 81%|████████ | 252/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 81%|████████ | 252/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 81%|████████▏ | 253/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 81%|████████▏ | 253/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 254/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 254/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 149/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 81%|████████▏ | 253/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 81%|████████▏ | 253/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 254/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 254/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 256/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 82%|████████▏ | 256/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 82%|████████▏ | 255/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 83%|████████▎ | 257/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 257/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 82%|████████▏ | 256/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 82%|████████▏ | 256/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 83%|████████▎ | 257/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 257/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 258/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 258/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 48%|████▊ | 150/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 49%|████▊ | 151/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 152/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 153/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 154/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 83%|████████▎ | 259/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 259/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 50%|████▉ | 155/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 50%|█████ | 156/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 50%|█████ | 157/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 51%|█████ | 158/311 [00:00<00:00, 256.55it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 83%|████████▎ | 258/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 84%|████████▎ | 260/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 260/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 83%|████████▎ | 258/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 84%|████████▍ | 261/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 83%|████████▎ | 259/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 84%|████████▍ | 261/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 83%|████████▎ | 259/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 84%|████████▎ | 260/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 260/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 84%|████████▍ | 262/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 261/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 262/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 261/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 262/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 84%|████████▍ | 262/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 85%|████████▍ | 263/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 85%|████████▍ | 263/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 85%|████████▍ | 263/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 85%|████████▍ | 263/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 85%|████████▍ | 264/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 264/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 264/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 264/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 265/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 85%|████████▌ | 265/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 51%|█████ | 159/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 86%|████████▌ | 266/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 86%|████████▌ | 266/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 86%|████████▌ | 267/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 267/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 85%|████████▌ | 265/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 268/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 265/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 86%|████████▌ | 268/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 266/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 86%|████████▌ | 266/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 86%|████████▌ | 267/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 86%|████████▌ | 267/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 86%|████████▌ | 268/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 268/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 80%|███████▉ | 248/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 80%|███████▉ | 248/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 51%|█████▏ | 160/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 80%|████████ | 249/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 80%|████████ | 249/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 80%|████████ | 250/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 80%|████████ | 250/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 81%|████████ | 251/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 81%|████████ | 251/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 81%|████████ | 252/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 81%|████████ | 252/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 161/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 81%|████████▏ | 253/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 253/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 162/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 254/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 163/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 254/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 53%|█████▎ | 164/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 256/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 82%|████████▏ | 256/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 83%|████████▎ | 257/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 257/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 53%|█████▎ | 165/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 166/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 248/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 80%|███████▉ | 248/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 54%|█████▎ | 167/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 271/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 87%|████████▋ | 271/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 87%|████████▋ | 271/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 54%|█████▍ | 168/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 87%|████████▋ | 271/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 87%|████████▋ | 272/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 87%|████████▋ | 272/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 87%|████████▋ | 272/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 87%|████████▋ | 272/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 169/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 273/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 88%|████████▊ | 273/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 88%|████████▊ | 273/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 88%|████████▊ | 273/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:00<00:00, 375.64it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 292.69it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 83%|████████▎ | 258/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 275/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 83%|████████▎ | 258/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 275/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 275/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 88%|████████▊ | 275/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 88%|████████▊ | 275/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 89%|████████▊ | 276/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 89%|████████▊ | 276/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 89%|████████▊ | 276/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 89%|████████▊ | 276/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 80%|████████ | 249/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 80%|████████ | 249/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 80%|████████ | 250/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 80%|████████ | 250/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 81%|████████ | 251/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 81%|████████ | 251/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 277/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 81%|████████ | 252/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 81%|████████ | 252/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 278/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 89%|████████▉ | 278/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 90%|████████▉ | 279/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 90%|████████▉ | 279/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 89%|████████▉ | 278/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 89%|████████▉ | 278/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 90%|████████▉ | 279/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 90%|████████▉ | 279/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 55%|█████▍ | 170/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 81%|████████▏ | 253/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 81%|████████▏ | 253/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 259/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 259/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 82%|████████▏ | 254/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 254/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 90%|█████████ | 280/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 90%|█████████ | 280/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 82%|████████▏ | 256/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 82%|████████▏ | 256/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 83%|████████▎ | 257/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 257/311 [00:00<00:00, 357.93it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 55%|█████▍ | 171/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 90%|█████████ | 280/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 84%|████████▎ | 260/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 90%|█████████ | 280/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 84%|████████▎ | 260/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 84%|████████▍ | 261/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 84%|████████▍ | 261/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 262/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 84%|████████▍ | 262/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 85%|████████▍ | 263/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 85%|████████▍ | 263/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 85%|████████▍ | 264/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 264/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 85%|████████▌ | 265/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 85%|████████▌ | 265/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 86%|████████▌ | 266/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 266/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 172/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 282/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 282/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 56%|█████▌ | 173/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 56%|█████▌ | 174/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 86%|████████▌ | 267/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 267/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 91%|█████████ | 283/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 91%|█████████ | 283/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 282/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 268/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 91%|█████████ | 282/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 268/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 91%|█████████▏| 284/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 91%|█████████ | 283/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 56%|█████▋ | 175/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 91%|█████████▏| 284/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 91%|█████████ | 283/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 91%|█████████▏| 284/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 284/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 92%|█████████▏| 286/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 286/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 286/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 286/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 287/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 287/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 287/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 287/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 83%|████████▎ | 258/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 258/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 258/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 259/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 259/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 176/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 93%|█████████▎| 288/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 177/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 57%|█████▋ | 178/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 289/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 93%|█████████▎| 289/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 93%|█████████▎| 290/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 290/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 289/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 93%|█████████▎| 289/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 93%|█████████▎| 290/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 179/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 93%|█████████▎| 290/311 [00:00<00:00, 403.91it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 180/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 84%|████████▎ | 260/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 260/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 84%|████████▍ | 261/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 261/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 262/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 262/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 263/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 263/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 264/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 264/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 181/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 85%|████████▌ | 265/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 85%|████████▌ | 265/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 86%|████████▌ | 266/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 86%|████████▌ | 266/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 267/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 267/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 86%|████████▌ | 268/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 268/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 87%|████████▋ | 271/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 87%|████████▋ | 271/311 [00:00<00:00, 379.16it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 87%|████████▋ | 272/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 87%|████████▋ | 272/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 87%|████████▋ | 272/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 88%|████████▊ | 273/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 88%|████████▊ | 273/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 292/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 94%|█████████▍| 292/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 94%|█████████▍| 292/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 94%|█████████▍| 292/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 182/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 275/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 88%|████████▊ | 275/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 89%|████████▊ | 276/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 89%|████████▊ | 276/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 309.59it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 94%|█████████▍| 293/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 94%|█████████▍| 293/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 89%|████████▉ | 278/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 95%|█████████▍| 294/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 95%|█████████▍| 294/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 89%|████████▉ | 278/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 95%|█████████▍| 295/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 95%|█████████▍| 295/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 95%|█████████▌| 296/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 95%|█████████▌| 296/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 293/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 94%|█████████▍| 293/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 95%|█████████▍| 294/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 95%|█████████▍| 294/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 95%|█████████▍| 295/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 95%|█████████▍| 295/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 95%|█████████▌| 296/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 95%|█████████▌| 296/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 183/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 184/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 185/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 95%|█████████▌| 297/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 95%|█████████▌| 297/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 96%|█████████▌| 298/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 186/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 96%|█████████▌| 298/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 96%|█████████▋| 300/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 87%|████████▋ | 270/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 96%|█████████▋| 300/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 90%|████████▉ | 279/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 90%|████████▉ | 279/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 60%|██████ | 187/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 95%|█████████▌| 297/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 95%|█████████▌| 297/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 90%|█████████ | 280/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 60%|██████ | 188/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 90%|█████████ | 280/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 96%|█████████▌| 298/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 96%|█████████▌| 298/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 96%|█████████▋| 300/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 96%|█████████▋| 300/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 97%|█████████▋| 301/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 301/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 61%|██████ | 189/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 61%|██████ | 190/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 61%|██████▏ | 191/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 97%|█████████▋| 301/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 301/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 97%|█████████▋| 302/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 87%|████████▋ | 271/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 271/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 272/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 87%|████████▋ | 272/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 88%|████████▊ | 273/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 88%|████████▊ | 273/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 62%|██████▏ | 192/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 275/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 88%|████████▊ | 275/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 89%|████████▊ | 276/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 89%|████████▊ | 276/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 303/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 97%|█████████▋| 303/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 303/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 97%|█████████▋| 303/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 91%|█████████ | 282/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 91%|█████████ | 282/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 91%|█████████ | 283/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 283/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 91%|█████████▏| 284/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 284/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 278/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 89%|████████▉ | 278/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 90%|████████▉ | 279/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 90%|████████▉ | 279/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 92%|█████████▏| 286/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 286/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 287/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 287/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 62%|██████▏ | 193/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 289/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 93%|█████████▎| 289/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 93%|█████████▎| 290/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 304/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 304/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 93%|█████████▎| 290/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 304/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 304/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 305/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 305/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 306/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 306/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 305/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 305/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 306/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 306/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 90%|█████████ | 280/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 90%|█████████ | 280/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 99%|█████████▉| 308/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 308/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 308/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 308/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 62%|██████▏ | 194/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 99%|█████████▉| 309/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 99%|█████████▉| 309/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 99%|█████████▉| 309/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 99%|█████████▉| 309/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 100%|█████████▉| 310/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 195/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 100%|█████████▉| 310/311 [00:00<00:00, 424.06it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 100%|█████████▉| 310/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 310/311 [00:00<00:00, 406.45it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 63%|██████▎ | 196/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 100%|██████████| 311/311 [00:00<00:00, 406.45it/s, Materializing param=model.norm.weight] Loading weights: 63%|██████▎ | 197/311 [00:00<00:00, 306.36it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 100%|██████████| 311/311 [00:00<00:00, 424.06it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:00<00:00, 406.45it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:00<00:00, 424.06it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:00<00:00, 322.60it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:00<00:00, 329.56it/s, Materializing param=model.norm.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▎ | 198/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 64%|██████▍ | 199/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 94%|█████████▍| 292/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 94%|█████████▍| 292/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 64%|██████▍ | 200/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 65%|██████▍ | 201/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 65%|██████▍ | 202/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 91%|█████████ | 282/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 282/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 283/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 283/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 91%|█████████▏| 284/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 284/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 293/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 94%|█████████▍| 293/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 95%|█████████▍| 294/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 95%|█████████▍| 294/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 95%|█████████▍| 295/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 95%|█████████▍| 295/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 95%|█████████▌| 296/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 296/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 286/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 286/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 287/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 287/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 65%|██████▌ | 203/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 95%|█████████▌| 297/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 95%|█████████▌| 297/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 96%|█████████▌| 298/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 96%|█████████▌| 298/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 289/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 93%|█████████▎| 289/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 96%|█████████▌| 299/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 290/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 290/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 96%|█████████▋| 300/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 96%|█████████▋| 300/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 97%|█████████▋| 301/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 301/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 204/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 315.03it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 80%|███████▉ | 248/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 80%|███████▉ | 248/311 [00:00<00:00, 327.90it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 205/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 66%|██████▌ | 206/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 207/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 208/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 303/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 303/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 94%|█████████▍| 292/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 94%|█████████▍| 292/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 209/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 68%|██████▊ | 210/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 211/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 68%|██████▊ | 212/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 80%|████████ | 249/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 68%|██████▊ | 213/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 80%|████████ | 249/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 80%|████████ | 250/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 80%|████████ | 250/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 304/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 304/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 81%|████████ | 251/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 81%|████████ | 251/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 305/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 305/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 81%|████████ | 252/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 81%|████████ | 252/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 98%|█████████▊| 306/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 306/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 293/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 94%|█████████▍| 293/311 [00:00<00:00, 369.38it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 308/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 308/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 309/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 99%|█████████▉| 309/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 69%|██████▉ | 214/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 100%|█████████▉| 310/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 310/311 [00:00<00:00, 400.33it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 100%|██████████| 311/311 [00:00<00:00, 400.33it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:00<00:00, 400.33it/s, Materializing param=model.norm.weight] Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 100%|██████████| 311/311 [00:00<00:00, 323.93it/s, Materializing param=model.norm.weight] Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 369.38it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 69%|██████▉ | 215/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 216/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 70%|██████▉ | 217/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 70%|███████ | 218/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 70%|███████ | 219/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 71%|███████ | 220/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 71%|███████ | 221/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 71%|███████▏ | 222/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 72%|███████▏ | 223/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 72%|███████▏ | 224/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 72%|███████▏ | 225/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 334.15it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 327.90it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 383.44it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning Loading weights: 100%|██████████| 311/311 [00:01<00:00, 383.44it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:01<00:00, 383.44it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:01<00:00, 300.46it/s, Materializing param=model.norm.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 226/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.mlp.up_proj.weight]The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 73%|███████▎ | 227/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 73%|███████▎ | 228/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 74%|███████▎ | 229/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 230/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 80%|████████ | 249/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 80%|████████ | 249/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 80%|████████ | 250/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 80%|████████ | 250/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 81%|████████ | 251/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 81%|████████ | 251/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 231/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 81%|████████ | 252/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 75%|███████▍ | 232/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 81%|████████ | 252/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 233/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 75%|███████▌ | 234/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 235/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 236/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 237/311 [00:00<00:00, 329.16it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 77%|███████▋ | 238/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 239/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 240/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 241/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 78%|███████▊ | 242/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 78%|███████▊ | 243/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 244/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 79%|███████▉ | 245/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 79%|███████▉ | 246/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 79%|███████▉ | 247/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 334.15it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 80%|███████▉ | 248/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 80%|███████▉ | 248/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 80%|████████ | 249/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 80%|████████ | 249/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 80%|████████ | 250/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 80%|████████ | 250/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 81%|████████ | 251/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 81%|████████ | 251/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 81%|████████ | 252/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 81%|████████ | 252/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 81%|████████▏ | 253/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 81%|████████▏ | 253/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning Loading weights: 82%|████████▏ | 254/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 254/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 255/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 82%|████████▏ | 256/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 82%|████████▏ | 256/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 83%|████████▎ | 257/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 257/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 83%|████████▎ | 258/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 258/311 [00:00<00:00, 345.75it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Comm config Blocking set to 1 Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 347.39it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 100%|██████████| 311/311 [00:01<00:00, 347.39it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:01<00:00, 347.39it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:01<00:00, 272.27it/s, Materializing param=model.norm.weight] Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 345.75it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 348.48it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 100%|██████████| 311/311 [00:01<00:00, 348.48it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:01<00:00, 348.48it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:01<00:00, 268.26it/s, Materializing param=model.norm.weight] Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 369.53it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 100%|██████████| 311/311 [00:01<00:00, 369.53it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:01<00:00, 369.53it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 311/311 [00:01<00:00, 277.81it/s, Materializing param=model.norm.weight] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}. The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}. The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}. The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}. The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}. The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Comm config Blocking set to 1 The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Comm config Blocking set to 1 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Comm config Blocking set to 1 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Comm config Blocking set to 1 The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}. ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Comm config Blocking set to 1 The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}. ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Comm config Blocking set to 1 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Comm config Blocking set to 1 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Initialized NET plugin Socket ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Assigned NET plugin Socket to comm ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Using network Socket ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO ncclCommSplit comm 0x55aba4786d20 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 parent 0x55ab97689c00 splitCount 1 color 1266629538 key 7- Init START ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO ncclCommSplit comm 0x55b7e396d9c0 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 parent 0x55b7d7b1b9d0 splitCount 1 color 1266629538 key 2- Init START ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO ncclCommSplit comm 0x558ee45ba3d0 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 parent 0x558ed75c7da0 splitCount 1 color 1266629538 key 1- Init START ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO ncclCommSplit comm 0x561d4c169060 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 parent 0x561d46f046a0 splitCount 1 color 1266629538 key 4- Init START ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO ncclCommSplit comm 0x55ae95eb9e30 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 parent 0x55ae88a9dd90 splitCount 1 color 1266629538 key 3- Init START ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO ncclCommSplit comm 0x55b500e80270 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 parent 0x55b4f5018ae0 splitCount 1 color 1266629538 key 6- Init START ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO ncclCommSplit comm 0x558fdd7d54e0 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 parent 0x558fd1beff10 splitCount 1 color 1266629538 key 5- Init START ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO ncclCommSplit comm 0x56417bb8e370 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 parent 0x56416eeca870 splitCount 1 color 1266629538 key 0- Init START ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 4 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO NVLS multicast support is not available on dev 4 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 7 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO NVLS multicast support is not available on dev 7 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO NVLS multicast support is not available on dev 0 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 5 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO NVLS multicast support is not available on dev 5 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 2 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO NVLS multicast support is not available on dev 2 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 3 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO NVLS multicast support is not available on dev 3 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 1 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO NVLS multicast support is not available on dev 1 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 6 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95). ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO NVLS multicast support is not available on dev 6 (NVLS_NCHANNELS 0) ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO comm 0x558fdd7d54e0 rank 5 nRanks 8 nNodes 1 localRanks 8 localRank 5 MNNVL 0 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO comm 0x55ae95eb9e30 rank 3 nRanks 8 nNodes 1 localRanks 8 localRank 3 MNNVL 0 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO comm 0x55b500e80270 rank 6 nRanks 8 nNodes 1 localRanks 8 localRank 6 MNNVL 0 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO comm 0x561d4c169060 rank 4 nRanks 8 nNodes 1 localRanks 8 localRank 4 MNNVL 0 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO comm 0x558ee45ba3d0 rank 1 nRanks 8 nNodes 1 localRanks 8 localRank 1 MNNVL 0 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO comm 0x55b7e396d9c0 rank 2 nRanks 8 nNodes 1 localRanks 8 localRank 2 MNNVL 0 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO comm 0x55aba4786d20 rank 7 nRanks 8 nNodes 1 localRanks 8 localRank 7 MNNVL 0 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO comm 0x56417bb8e370 rank 0 nRanks 8 nNodes 1 localRanks 8 localRank 0 MNNVL 0 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 00/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 01/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 02/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 03/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 04/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 05/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 06/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 07/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 08/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 09/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 10/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 11/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 12/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 13/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 14/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 15/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 16/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 17/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 18/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 19/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 20/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 21/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 22/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 23/24 : 0 1 2 3 4 5 6 7 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO P2P Chunksize set to 524288 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243412:244109 [0] NCCL INFO [Proxy Service] Device 1 CPU core 0 ywang29-p4d-debug-1-worker-0:243412:244110 [0] NCCL INFO [Proxy Service UDS] Device 1 CPU core 49 ywang29-p4d-debug-1-worker-0:243416:244111 [0] NCCL INFO [Proxy Service] Device 5 CPU core 30 ywang29-p4d-debug-1-worker-0:243416:244112 [0] NCCL INFO [Proxy Service UDS] Device 5 CPU core 31 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243418:244113 [0] NCCL INFO [Proxy Service] Device 7 CPU core 80 ywang29-p4d-debug-1-worker-0:243418:244114 [0] NCCL INFO [Proxy Service UDS] Device 7 CPU core 33 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243413:244115 [0] NCCL INFO [Proxy Service] Device 2 CPU core 52 ywang29-p4d-debug-1-worker-0:243413:244116 [0] NCCL INFO [Proxy Service UDS] Device 2 CPU core 8 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243417:244117 [0] NCCL INFO [Proxy Service] Device 6 CPU core 87 ywang29-p4d-debug-1-worker-0:243417:244118 [0] NCCL INFO [Proxy Service UDS] Device 6 CPU core 43 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243414:244119 [0] NCCL INFO [Proxy Service] Device 3 CPU core 65 ywang29-p4d-debug-1-worker-0:243414:244120 [0] NCCL INFO [Proxy Service UDS] Device 3 CPU core 66 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243415:244122 [0] NCCL INFO [Proxy Service UDS] Device 4 CPU core 82 ywang29-p4d-debug-1-worker-0:243415:244121 [0] NCCL INFO [Proxy Service] Device 4 CPU core 37 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1 ywang29-p4d-debug-1-worker-0:243411:244123 [0] NCCL INFO [Proxy Service] Device 0 CPU core 11 ywang29-p4d-debug-1-worker-0:243411:244124 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 61 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO CC Off, workFifoBytes 1048576 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO ncclCommSplit comm 0x55aba4786d20 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 parent 0x55ab97689c00 splitCount 1 color 1266629538 key 7 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO ncclCommSplit comm 0x55ae95eb9e30 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 parent 0x55ae88a9dd90 splitCount 1 color 1266629538 key 3 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO ncclCommSplit comm 0x558ee45ba3d0 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 parent 0x558ed75c7da0 splitCount 1 color 1266629538 key 1 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO ncclCommSplit comm 0x558fdd7d54e0 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 parent 0x558fd1beff10 splitCount 1 color 1266629538 key 5 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Init timings - ncclCommSplit: rank 7 nranks 8 total 0.17 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.03, graphs 0.00, connections 0.06, rest 0.07) ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Init timings - ncclCommSplit: rank 1 nranks 8 total 0.89 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.00, topo 0.03, graphs 0.00, connections 0.06, rest 0.79) ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Init timings - ncclCommSplit: rank 3 nranks 8 total 0.58 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.03, graphs 0.00, connections 0.06, rest 0.48) ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Init timings - ncclCommSplit: rank 5 nranks 8 total 0.78 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.03, graphs 0.00, connections 0.06, rest 0.68) ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO ncclCommSplit comm 0x56417bb8e370 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 parent 0x56416eeca870 splitCount 1 color 1266629538 key 0 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO ncclCommSplit comm 0x561d4c169060 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 parent 0x561d46f046a0 splitCount 1 color 1266629538 key 4 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Init timings - ncclCommSplit: rank 0 nranks 8 total 1.50 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.03, graphs 0.00, connections 0.06, rest 1.41) ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO ncclCommSplit comm 0x55b7e396d9c0 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 parent 0x55b7d7b1b9d0 splitCount 1 color 1266629538 key 2 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO ncclCommSplit comm 0x55b500e80270 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 parent 0x55b4f5018ae0 splitCount 1 color 1266629538 key 6 - Init COMPLETE ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Init timings - ncclCommSplit: rank 4 nranks 8 total 0.65 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.03, graphs 0.00, connections 0.06, rest 0.55) ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Init timings - ncclCommSplit: rank 2 nranks 8 total 0.66 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.03, graphs 0.00, connections 0.06, rest 0.56) ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Init timings - ncclCommSplit: rank 6 nranks 8 total 0.12 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.06, rest 0.02) ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 00/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 01/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 00/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 02/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 03/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 01/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 04/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 02/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 03/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 05/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 06/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 04/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 05/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 06/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 07/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 08/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 09/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 07/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 08/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 10/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 11/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 16/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 09/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 12/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 13/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 14/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 10/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 17/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 15/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 16/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 18/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 17/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 19/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 11/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 20/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 12/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 16/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 21/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 13/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 18/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 19/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 16/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 17/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 14/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 22/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 16/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 15/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 20/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 18/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 17/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 21/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 19/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Channel 23/0 : 3[3] -> 4[4] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 16/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 16/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 17/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 16/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 18/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 17/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 17/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 22/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 19/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 18/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 18/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 18/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 19/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 17/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 19/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 19/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 20/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 20/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Channel 23/0 : 2[2] -> 3[3] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 21/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 20/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 21/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 20/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 21/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 21/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 22/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 20/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 22/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 18/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 21/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Channel 23/0 : 6[6] -> 7[7] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Channel 23/0 : 5[5] -> 6[6] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 22/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Channel 23/0 : 1[1] -> 2[2] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 22/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 19/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 22/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Channel 23/0 : 7[7] -> 0[0] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 20/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Channel 23/0 : 4[4] -> 5[5] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 21/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 22/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Channel 23/0 : 0[0] -> 1[1] via P2P/CUMEM/read ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243412:243412 [1] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243418:243418 [7] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 ywang29-p4d-debug-1-worker-0:243411:243411 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 1 Before initializing optimizer states MA 4.73 GB Max_MA 4.73 GB CA 4.77 GB Max_CA 5 GB CPU Virtual Memory: used = 57.11 GB, percent = 5.1% After initializing optimizer states MA 4.73 GB Max_MA 5.68 GB CA 5.71 GB Max_CA 6 GB CPU Virtual Memory: used = 57.11 GB, percent = 5.1% After initializing ZeRO optimizer MA 4.73 GB Max_MA 4.73 GB CA 5.71 GB Max_CA 6 GB CPU Virtual Memory: used = 57.11 GB, percent = 5.1% [INFO|trainer.py:1587] 2026-04-29 17:06:18,875 >> ***** Running training ***** [INFO|trainer.py:1588] 2026-04-29 17:06:18,875 >> Num examples = 30,000 [INFO|trainer.py:1589] 2026-04-29 17:06:18,875 >> Num Epochs = 4 [INFO|trainer.py:1590] 2026-04-29 17:06:18,875 >> Instantaneous batch size per device = 2 [INFO|trainer.py:1593] 2026-04-29 17:06:18,875 >> Total train batch size (w. parallel, distributed & accumulation) = 128 [INFO|trainer.py:1594] 2026-04-29 17:06:18,875 >> Gradient Accumulation steps = 8 [INFO|trainer.py:1595] 2026-04-29 17:06:18,875 >> Total optimization steps = 940 [INFO|trainer.py:1596] 2026-04-29 17:06:18,876 >> Number of trainable parameters = 2,031,739,904 wandb: [wandb.login()] Loaded credentials for https://api.wandb.ai from WANDB_API_KEY. wandb: Currently logged in as: kkhya (maskmoe) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin wandb: setting up run 152xmw6f wandb: Tracking run with wandb version 0.26.1 wandb: Run data is saved locally in /nfs/ywang29/lm-factory/wandb/run-20260429_170619-152xmw6f wandb: Run `wandb offline` to turn off syncing. wandb: Syncing run qwen3_1.7b_coding_fft wandb: ⭐️ View project at https://wandb.ai/maskmoe/MFT-LM wandb: 🚀 View run at https://wandb.ai/maskmoe/MFT-LM/runs/152xmw6f 0%| | 0/940 [00:00> Saving model checkpoint to saves/qwen3_1.7b/coding/fft/checkpoint-500 [INFO|configuration_utils.py:432] 2026-04-29 17:34:16,788 >> Configuration saved in saves/qwen3_1.7b/coding/fft/checkpoint-500/config.json [INFO|configuration_utils.py:803] 2026-04-29 17:34:16,791 >> Configuration saved in saves/qwen3_1.7b/coding/fft/checkpoint-500/generation_config.json Writing model shards: 0%| | 0/1 [00:00> Model weights saved in saves/qwen3_1.7b/coding/fft/checkpoint-500/model.safetensors [INFO|tokenization_utils_base.py:3224] 2026-04-29 17:34:23,025 >> chat template saved in saves/qwen3_1.7b/coding/fft/checkpoint-500/chat_template.jinja [INFO|tokenization_utils_base.py:2078] 2026-04-29 17:34:23,028 >> tokenizer config file saved in saves/qwen3_1.7b/coding/fft/checkpoint-500/tokenizer_config.json 53%|█████▎ | 501/940 [28:06<45:20, 6.20s/it] 53%|█████▎ | 502/940 [28:08<37:22, 5.12s/it] 54%|█████▎ | 503/940 [28:12<34:07, 4.69s/it] 54%|█████▎ | 504/940 [28:15<30:03, 4.14s/it] 54%|█████▎ | 505/940 [28:19<29:04, 4.01s/it] 54%|█████▍ | 506/940 [28:22<27:51, 3.85s/it] 54%|█████▍ | 507/940 [28:26<28:28, 3.95s/it] 54%|█████▍ | 508/940 [28:30<27:05, 3.76s/it] 54%|█████▍ | 509/940 [28:33<26:34, 3.70s/it] 54%|█████▍ | 510/940 [28:36<24:28, 3.41s/it] {'loss': '0.6106', 'grad_norm': '0.9619', 'learning_rate': '5.149e-06', 'epoch': '2.171'} 54%|█████▍ | 510/940 [28:36<24:28, 3.41s/it] 54%|█████▍ | 511/940 [28:39<22:53, 3.20s/it] 54%|█████▍ | 512/940 [28:43<25:11, 3.53s/it] 55%|█████▍ | 513/940 [28:46<24:09, 3.40s/it] 55%|█████▍ | 514/940 [28:49<23:16, 3.28s/it] 55%|█████▍ | 515/940 [28:53<23:52, 3.37s/it] 55%|█████▍ | 516/940 [28:56<23:45, 3.36s/it] 55%|█████▌ | 517/940 [28:59<23:12, 3.29s/it] 55%|█████▌ | 518/940 [29:02<23:10, 3.30s/it] 55%|█████▌ | 519/940 [29:05<21:42, 3.09s/it] 55%|█████▌ | 520/940 [29:09<22:45, 3.25s/it] {'loss': '0.6105', 'grad_norm': '0.9641', 'learning_rate': '4.963e-06', 'epoch': '2.213'} 55%|█████▌ | 520/940 [29:09<22:45, 3.25s/it] 55%|█████▌ | 521/940 [29:12<22:41, 3.25s/it] 56%|█████▌ | 522/940 [29:15<21:56, 3.15s/it] 56%|█████▌ | 523/940 [29:17<20:54, 3.01s/it] 56%|█████▌ | 524/940 [29:21<21:55, 3.16s/it] 56%|█████▌ | 525/940 [29:24<22:03, 3.19s/it] 56%|█████▌ | 526/940 [29:28<24:15, 3.52s/it] 56%|█████▌ | 527/940 [29:32<23:38, 3.44s/it] 56%|█████▌ | 528/940 [29:34<22:08, 3.22s/it] 56%|█████▋ | 529/940 [29:39<24:13, 3.54s/it] 56%|█████▋ | 530/940 [29:42<24:24, 3.57s/it] {'loss': '0.5798', 'grad_norm': '0.9159', 'learning_rate': '4.777e-06', 'epoch': '2.256'} 56%|█████▋ | 530/940 [29:42<24:24, 3.57s/it] 56%|█████▋ | 531/940 [29:47<26:59, 3.96s/it] 57%|█████▋ | 532/940 [29:50<24:51, 3.66s/it] 57%|█████▋ | 533/940 [29:54<25:00, 3.69s/it] 57%|█████▋ | 534/940 [29:57<23:54, 3.53s/it] 57%|█████▋ | 535/940 [30:01<23:50, 3.53s/it] 57%|█████▋ | 536/940 [30:04<22:46, 3.38s/it] 57%|█████▋ | 537/940 [30:07<21:51, 3.25s/it] 57%|█████▋ | 538/940 [30:10<22:31, 3.36s/it] 57%|█████▋ | 539/940 [30:13<22:02, 3.30s/it] 57%|█████▋ | 540/940 [30:17<22:52, 3.43s/it] {'loss': '0.6197', 'grad_norm': '0.9406', 'learning_rate': '4.592e-06', 'epoch': '2.299'} 57%|█████▋ | 540/940 [30:17<22:52, 3.43s/it] 58%|█████▊ | 541/940 [30:20<21:40, 3.26s/it] 58%|█████▊ | 542/940 [30:23<21:08, 3.19s/it] 58%|█████▊ | 543/940 [30:26<21:00, 3.18s/it] 58%|█████▊ | 544/940 [30:31<23:43, 3.59s/it] 58%|█████▊ | 545/940 [30:34<22:20, 3.39s/it] 58%|█████▊ | 546/940 [30:38<23:14, 3.54s/it] 58%|█████▊ | 547/940 [30:41<23:06, 3.53s/it] 58%|█████▊ | 548/940 [30:44<22:22, 3.42s/it] 58%|█████▊ | 549/940 [30:47<21:32, 3.31s/it] 59%|█████▊ | 550/940 [30:50<21:17, 3.28s/it] {'loss': '0.5832', 'grad_norm': '1.024', 'learning_rate': '4.407e-06', 'epoch': '2.341'} 59%|█████▊ | 550/940 [30:50<21:17, 3.28s/it] 59%|█████▊ | 551/940 [30:53<20:25, 3.15s/it] 59%|█████▊ | 552/940 [30:56<19:29, 3.01s/it] 59%|█████▉ | 553/940 [30:59<19:46, 3.07s/it] 59%|█████▉ | 554/940 [31:03<21:38, 3.36s/it] 59%|█████▉ | 555/940 [31:06<21:03, 3.28s/it] 59%|█████▉ | 556/940 [31:09<20:28, 3.20s/it] 59%|█████▉ | 557/940 [31:12<19:50, 3.11s/it] 59%|█████▉ | 558/940 [31:17<23:44, 3.73s/it] 59%|█████▉ | 559/940 [31:21<23:26, 3.69s/it] 60%|█████▉ | 560/940 [31:24<22:30, 3.55s/it] {'loss': '0.5983', 'grad_norm': '0.9565', 'learning_rate': '4.223e-06', 'epoch': '2.384'} 60%|█████▉ | 560/940 [31:24<22:30, 3.55s/it] 60%|█████▉ | 561/940 [31:28<23:22, 3.70s/it] 60%|█████▉ | 562/940 [31:32<22:59, 3.65s/it] 60%|█████▉ | 563/940 [31:36<23:16, 3.71s/it] 60%|██████ | 564/940 [31:39<21:53, 3.49s/it] 60%|██████ | 565/940 [31:42<20:53, 3.34s/it] 60%|██████ | 566/940 [31:45<21:24, 3.43s/it] 60%|██████ | 567/940 [31:50<23:30, 3.78s/it] 60%|██████ | 568/940 [31:53<21:26, 3.46s/it] 61%|██████ | 569/940 [31:55<20:09, 3.26s/it] 61%|██████ | 570/940 [32:00<21:57, 3.56s/it] {'loss': '0.5884', 'grad_norm': '0.9118', 'learning_rate': '4.04e-06', 'epoch': '2.427'} 61%|██████ | 570/940 [32:00<21:57, 3.56s/it] 61%|██████ | 571/940 [32:03<21:48, 3.55s/it] 61%|██████ | 572/940 [32:07<22:46, 3.71s/it] 61%|██████ | 573/940 [32:11<23:02, 3.77s/it] 61%|██████ | 574/940 [32:14<20:56, 3.43s/it] 61%|██████ | 575/940 [32:18<22:58, 3.78s/it] 61%|██████▏ | 576/940 [32:22<23:20, 3.85s/it] 61%|██████▏ | 577/940 [32:25<21:37, 3.57s/it] 61%|██████▏ | 578/940 [32:28<20:15, 3.36s/it] 62%|██████▏ | 579/940 [32:31<19:02, 3.17s/it] 62%|██████▏ | 580/940 [32:34<19:33, 3.26s/it] {'loss': '0.5839', 'grad_norm': '0.9704', 'learning_rate': '3.859e-06', 'epoch': '2.469'} 62%|██████▏ | 580/940 [32:34<19:33, 3.26s/it] 62%|██████▏ | 581/940 [32:37<19:06, 3.19s/it] 62%|██████▏ | 582/940 [32:41<19:42, 3.30s/it] 62%|██████▏ | 583/940 [32:44<19:04, 3.21s/it] 62%|██████▏ | 584/940 [32:47<17:49, 3.01s/it] 62%|██████▏ | 585/940 [32:49<17:31, 2.96s/it] 62%|██████▏ | 586/940 [32:53<18:49, 3.19s/it] 62%|██████▏ | 587/940 [32:57<20:17, 3.45s/it] 63%|██████▎ | 588/940 [33:01<20:14, 3.45s/it] 63%|██████▎ | 589/940 [33:04<20:29, 3.50s/it] 63%|██████▎ | 590/940 [33:07<18:45, 3.22s/it] {'loss': '0.6247', 'grad_norm': '0.9491', 'learning_rate': '3.679e-06', 'epoch': '2.512'} 63%|██████▎ | 590/940 [33:07<18:45, 3.22s/it] 63%|██████▎ | 591/940 [33:12<22:45, 3.91s/it] 63%|██████▎ | 592/940 [33:16<21:35, 3.72s/it] 63%|██████▎ | 593/940 [33:20<22:16, 3.85s/it] 63%|██████▎ | 594/940 [33:26<25:41, 4.46s/it] 63%|██████▎ | 595/940 [33:29<23:34, 4.10s/it] 63%|██████▎ | 596/940 [33:32<22:36, 3.94s/it] 64%|██████▎ | 597/940 [33:36<22:19, 3.90s/it] 64%|██████▎ | 598/940 [33:41<22:57, 4.03s/it] 64%|██████▎ | 599/940 [33:45<23:02, 4.05s/it] 64%|██████▍ | 600/940 [33:48<21:18, 3.76s/it] {'loss': '0.6042', 'grad_norm': '0.9721', 'learning_rate': '3.501e-06', 'epoch': '2.555'} 64%|██████▍ | 600/940 [33:48<21:18, 3.76s/it] 64%|██████▍ | 601/940 [33:50<19:29, 3.45s/it] 64%|██████▍ | 602/940 [33:55<20:54, 3.71s/it] 64%|██████▍ | 603/940 [33:58<20:40, 3.68s/it] 64%|██████▍ | 604/940 [34:01<18:44, 3.35s/it] 64%|██████▍ | 605/940 [34:04<18:38, 3.34s/it] 64%|██████▍ | 606/940 [34:09<20:07, 3.61s/it] 65%|██████▍ | 607/940 [34:11<18:16, 3.29s/it] 65%|██████▍ | 608/940 [34:14<18:02, 3.26s/it] 65%|██████▍ | 609/940 [34:18<18:09, 3.29s/it] 65%|██████▍ | 610/940 [34:21<18:50, 3.43s/it] {'loss': '0.6286', 'grad_norm': '1.117', 'learning_rate': '3.325e-06', 'epoch': '2.597'} 65%|██████▍ | 610/940 [34:21<18:50, 3.43s/it] 65%|██████▌ | 611/940 [34:24<17:48, 3.25s/it] 65%|██████▌ | 612/940 [34:27<17:10, 3.14s/it] 65%|██████▌ | 613/940 [34:31<17:49, 3.27s/it] 65%|██████▌ | 614/940 [34:34<17:10, 3.16s/it] 65%|██████▌ | 615/940 [34:37<17:44, 3.28s/it] 66%|██████▌ | 616/940 [34:40<17:30, 3.24s/it] 66%|██████▌ | 617/940 [34:44<18:06, 3.36s/it] 66%|██████▌ | 618/940 [34:48<18:20, 3.42s/it] 66%|██████▌ | 619/940 [34:52<20:11, 3.77s/it] 66%|██████▌ | 620/940 [34:55<18:19, 3.44s/it] {'loss': '0.6007', 'grad_norm': '1.024', 'learning_rate': '3.151e-06', 'epoch': '2.64'} 66%|██████▌ | 620/940 [34:55<18:19, 3.44s/it] 66%|██████▌ | 621/940 [34:58<17:11, 3.23s/it] 66%|██████▌ | 622/940 [35:00<15:39, 2.95s/it] 66%|██████▋ | 623/940 [35:03<15:47, 2.99s/it] 66%|██████▋ | 624/940 [35:07<17:35, 3.34s/it] 66%|██████▋ | 625/940 [35:11<18:00, 3.43s/it] 67%|██████▋ | 626/940 [35:14<17:10, 3.28s/it] 67%|██████▋ | 627/940 [35:17<17:12, 3.30s/it] 67%|██████▋ | 628/940 [35:20<16:09, 3.11s/it] 67%|██████▋ | 629/940 [35:22<15:13, 2.94s/it] 67%|██████▋ | 630/940 [35:25<14:30, 2.81s/it] {'loss': '0.5929', 'grad_norm': '1.057', 'learning_rate': '2.98e-06', 'epoch': '2.683'} 67%|██████▋ | 630/940 [35:25<14:30, 2.81s/it] 67%|██████▋ | 631/940 [35:28<14:30, 2.82s/it] 67%|██████▋ | 632/940 [35:31<14:55, 2.91s/it] 67%|██████▋ | 633/940 [35:34<16:20, 3.19s/it] 67%|██████▋ | 634/940 [35:37<15:46, 3.09s/it] 68%|██████▊ | 635/940 [35:41<16:01, 3.15s/it] 68%|██████▊ | 636/940 [35:44<15:37, 3.08s/it] 68%|██████▊ | 637/940 [35:47<16:47, 3.32s/it] 68%|██████▊ | 638/940 [35:52<18:33, 3.69s/it] 68%|██████▊ | 639/940 [35:55<16:52, 3.36s/it] 68%|██████▊ | 640/940 [35:58<17:24, 3.48s/it] {'loss': '0.6104', 'grad_norm': '1.038', 'learning_rate': '2.811e-06', 'epoch': '2.725'} 68%|██████▊ | 640/940 [35:58<17:24, 3.48s/it] 68%|██████▊ | 641/940 [36:01<15:41, 3.15s/it] 68%|██████▊ | 642/940 [36:04<15:55, 3.21s/it] 68%|██████▊ | 643/940 [36:08<17:19, 3.50s/it] 69%|██████▊ | 644/940 [36:11<16:19, 3.31s/it] 69%|██████▊ | 645/940 [36:14<15:47, 3.21s/it] 69%|██████▊ | 646/940 [36:17<15:16, 3.12s/it] 69%|██████▉ | 647/940 [36:21<16:14, 3.33s/it] 69%|██████▉ | 648/940 [36:24<16:04, 3.30s/it] 69%|██████▉ | 649/940 [36:28<16:49, 3.47s/it] 69%|██████▉ | 650/940 [36:31<16:48, 3.48s/it] {'loss': '0.5875', 'grad_norm': '0.9824', 'learning_rate': '2.646e-06', 'epoch': '2.768'} 69%|██████▉ | 650/940 [36:31<16:48, 3.48s/it] 69%|██████▉ | 651/940 [36:34<15:08, 3.14s/it] 69%|██████▉ | 652/940 [36:37<14:42, 3.06s/it] 69%|██████▉ | 653/940 [36:40<15:18, 3.20s/it] 70%|██████▉ | 654/940 [36:43<15:01, 3.15s/it] 70%|██████▉ | 655/940 [36:46<14:03, 2.96s/it] 70%|██████▉ | 656/940 [36:49<14:31, 3.07s/it] 70%|██████▉ | 657/940 [36:51<13:29, 2.86s/it] 70%|███████ | 658/940 [36:55<13:48, 2.94s/it] 70%|███████ | 659/940 [36:59<15:39, 3.34s/it] 70%|███████ | 660/940 [37:04<18:00, 3.86s/it] {'loss': '0.5813', 'grad_norm': '1.017', 'learning_rate': '2.484e-06', 'epoch': '2.811'} 70%|███████ | 660/940 [37:04<18:00, 3.86s/it] 70%|███████ | 661/940 [37:07<16:38, 3.58s/it] 70%|███████ | 662/940 [37:11<17:01, 3.67s/it] 71%|███████ | 663/940 [37:14<16:43, 3.62s/it] 71%|███████ | 664/940 [37:17<15:16, 3.32s/it] 71%|███████ | 665/940 [37:20<15:33, 3.39s/it] 71%|███████ | 666/940 [37:23<14:32, 3.19s/it] 71%|███████ | 667/940 [37:26<14:46, 3.25s/it] 71%|███████ | 668/940 [37:32<17:21, 3.83s/it] 71%|███████ | 669/940 [37:34<15:20, 3.40s/it] 71%|███████▏ | 670/940 [37:37<14:18, 3.18s/it] {'loss': '0.61', 'grad_norm': '1.189', 'learning_rate': '2.325e-06', 'epoch': '2.853'} 71%|███████▏ | 670/940 [37:37<14:18, 3.18s/it] 71%|███████▏ | 671/940 [37:40<13:47, 3.08s/it] 71%|███████▏ | 672/940 [37:42<12:56, 2.90s/it] 72%|███████▏ | 673/940 [37:45<13:26, 3.02s/it] 72%|███████▏ | 674/940 [37:48<12:22, 2.79s/it] 72%|███████▏ | 675/940 [37:50<12:06, 2.74s/it] 72%|███████▏ | 676/940 [37:53<11:50, 2.69s/it] 72%|███████▏ | 677/940 [37:55<11:46, 2.69s/it] 72%|███████▏ | 678/940 [37:59<12:46, 2.93s/it] 72%|███████▏ | 679/940 [38:03<14:17, 3.28s/it] 72%|███████▏ | 680/940 [38:08<16:08, 3.72s/it] {'loss': '0.6121', 'grad_norm': '0.9742', 'learning_rate': '2.17e-06', 'epoch': '2.896'} 72%|███████▏ | 680/940 [38:08<16:08, 3.72s/it] 72%|███████▏ | 681/940 [38:12<16:20, 3.79s/it] 73%|███████▎ | 682/940 [38:15<15:04, 3.51s/it] 73%|███████▎ | 683/940 [38:17<14:12, 3.32s/it] 73%|███████▎ | 684/940 [38:22<15:02, 3.52s/it] 73%|███████▎ | 685/940 [38:25<15:00, 3.53s/it] 73%|███████▎ | 686/940 [38:27<13:21, 3.16s/it] 73%|███████▎ | 687/940 [38:32<14:35, 3.46s/it] 73%|███████▎ | 688/940 [38:35<14:27, 3.44s/it] 73%|███████▎ | 689/940 [38:38<14:07, 3.38s/it] 73%|███████▎ | 690/940 [38:41<13:59, 3.36s/it] {'loss': '0.6172', 'grad_norm': '0.9061', 'learning_rate': '2.019e-06', 'epoch': '2.939'} 73%|███████▎ | 690/940 [38:41<13:59, 3.36s/it] 74%|███████▎ | 691/940 [38:45<13:52, 3.35s/it] 74%|███████▎ | 692/940 [38:48<14:06, 3.41s/it] 74%|███████▎ | 693/940 [38:51<12:50, 3.12s/it] 74%|███████▍ | 694/940 [38:53<12:08, 2.96s/it] 74%|███████▍ | 695/940 [38:57<12:20, 3.02s/it] 74%|███████▍ | 696/940 [39:02<14:41, 3.61s/it] 74%|███████▍ | 697/940 [39:05<14:02, 3.47s/it] 74%|███████▍ | 698/940 [39:09<15:22, 3.81s/it] 74%|███████▍ | 699/940 [39:12<13:52, 3.45s/it] 74%|███████▍ | 700/940 [39:14<12:30, 3.13s/it] {'loss': '0.588', 'grad_norm': '1.029', 'learning_rate': '1.872e-06', 'epoch': '2.981'} 74%|███████▍ | 700/940 [39:14<12:30, 3.13s/it] 75%|███████▍ | 701/940 [39:17<12:34, 3.16s/it] 75%|███████▍ | 702/940 [39:20<11:43, 2.95s/it] 75%|███████▍ | 703/940 [39:23<12:18, 3.12s/it] 75%|███████▍ | 704/940 [39:26<11:59, 3.05s/it] 75%|███████▌ | 705/940 [39:27<09:38, 2.46s/it] 75%|███████▌ | 706/940 [39:32<12:00, 3.08s/it] 75%|███████▌ | 707/940 [39:35<12:14, 3.15s/it] 75%|███████▌ | 708/940 [39:39<12:43, 3.29s/it] 75%|███████▌ | 709/940 [39:43<14:08, 3.67s/it] 76%|███████▌ | 710/940 [39:46<13:10, 3.44s/it] {'loss': '0.5795', 'grad_norm': '1.006', 'learning_rate': '1.73e-06', 'epoch': '3.021'} 76%|███████▌ | 710/940 [39:46<13:10, 3.44s/it] 76%|███████▌ | 711/940 [39:49<12:12, 3.20s/it] 76%|███████▌ | 712/940 [39:53<12:56, 3.41s/it] 76%|███████▌ | 713/940 [39:56<12:16, 3.24s/it] 76%|███████▌ | 714/940 [39:58<11:34, 3.07s/it] 76%|███████▌ | 715/940 [40:01<11:00, 2.94s/it] 76%|███████▌ | 716/940 [40:05<12:03, 3.23s/it] 76%|███████▋ | 717/940 [40:09<12:26, 3.35s/it] 76%|███████▋ | 718/940 [40:12<12:37, 3.41s/it] 76%|███████▋ | 719/940 [40:15<12:11, 3.31s/it] 77%|███████▋ | 720/940 [40:18<11:50, 3.23s/it] {'loss': '0.5703', 'grad_norm': '0.975', 'learning_rate': '1.591e-06', 'epoch': '3.064'} 77%|███████▋ | 720/940 [40:18<11:50, 3.23s/it] 77%|███████▋ | 721/940 [40:21<11:10, 3.06s/it] 77%|███████▋ | 722/940 [40:24<10:44, 2.96s/it] 77%|███████▋ | 723/940 [40:28<12:08, 3.36s/it] 77%|███████▋ | 724/940 [40:31<12:05, 3.36s/it] 77%|███████▋ | 725/940 [40:35<12:48, 3.57s/it] 77%|███████▋ | 726/940 [40:38<11:37, 3.26s/it] 77%|███████▋ | 727/940 [40:42<12:25, 3.50s/it] 77%|███████▋ | 728/940 [40:45<11:24, 3.23s/it] 78%|███████▊ | 729/940 [40:47<10:49, 3.08s/it] 78%|███████▊ | 730/940 [40:51<11:44, 3.35s/it] {'loss': '0.584', 'grad_norm': '0.9798', 'learning_rate': '1.458e-06', 'epoch': '3.107'} 78%|███████▊ | 730/940 [40:51<11:44, 3.35s/it] 78%|███████▊ | 731/940 [40:54<11:08, 3.20s/it] 78%|███████▊ | 732/940 [40:57<10:17, 2.97s/it] 78%|███████▊ | 733/940 [41:00<10:43, 3.11s/it] 78%|███████▊ | 734/940 [41:04<11:40, 3.40s/it] 78%|███████▊ | 735/940 [41:08<11:49, 3.46s/it] 78%|███████▊ | 736/940 [41:10<11:00, 3.24s/it] 78%|███████▊ | 737/940 [41:14<11:18, 3.34s/it] 79%|███████▊ | 738/940 [41:17<11:26, 3.40s/it] 79%|███████▊ | 739/940 [41:21<11:23, 3.40s/it] 79%|███████▊ | 740/940 [41:24<11:28, 3.44s/it] {'loss': '0.5282', 'grad_norm': '1.004', 'learning_rate': '1.329e-06', 'epoch': '3.149'} 79%|███████▊ | 740/940 [41:24<11:28, 3.44s/it] 79%|███████▉ | 741/940 [41:28<11:14, 3.39s/it] 79%|███████▉ | 742/940 [41:31<11:07, 3.37s/it] 79%|███████▉ | 743/940 [41:35<11:32, 3.52s/it] 79%|███████▉ | 744/940 [41:39<12:04, 3.70s/it] 79%|███████▉ | 745/940 [41:43<12:13, 3.76s/it] 79%|███████▉ | 746/940 [41:45<10:57, 3.39s/it] 79%|███████▉ | 747/940 [41:49<10:49, 3.37s/it] 80%|███████▉ | 748/940 [41:51<10:08, 3.17s/it] 80%|███████▉ | 749/940 [41:55<10:39, 3.35s/it] 80%|███████▉ | 750/940 [41:59<10:35, 3.34s/it] {'loss': '0.5751', 'grad_norm': '0.9754', 'learning_rate': '1.206e-06', 'epoch': '3.192'} 80%|███████▉ | 750/940 [41:59<10:35, 3.34s/it] 80%|███████▉ | 751/940 [42:01<09:37, 3.06s/it] 80%|████████ | 752/940 [42:04<09:53, 3.16s/it] 80%|████████ | 753/940 [42:08<10:39, 3.42s/it] 80%|████████ | 754/940 [42:11<10:11, 3.29s/it] 80%|████████ | 755/940 [42:14<09:36, 3.12s/it] 80%|████████ | 756/940 [42:17<09:49, 3.20s/it] 81%|████████ | 757/940 [42:22<11:09, 3.66s/it] 81%|████████ | 758/940 [42:25<10:19, 3.41s/it] 81%|████████ | 759/940 [42:28<09:37, 3.19s/it] 81%|████████ | 760/940 [42:31<09:25, 3.14s/it] {'loss': '0.5495', 'grad_norm': '0.9909', 'learning_rate': '1.088e-06', 'epoch': '3.235'} 81%|████████ | 760/940 [42:31<09:25, 3.14s/it] 81%|████████ | 761/940 [42:34<09:29, 3.18s/it] 81%|████████ | 762/940 [42:38<09:59, 3.37s/it] 81%|████████ | 763/940 [42:41<09:40, 3.28s/it] 81%|████████▏ | 764/940 [42:44<09:07, 3.11s/it] 81%|████████▏ | 765/940 [42:46<08:46, 3.01s/it] 81%|████████▏ | 766/940 [42:50<09:38, 3.32s/it] 82%|████████▏ | 767/940 [42:56<11:12, 3.89s/it] 82%|████████▏ | 768/940 [42:59<10:47, 3.77s/it] 82%|████████▏ | 769/940 [43:02<10:09, 3.56s/it] 82%|████████▏ | 770/940 [43:06<10:08, 3.58s/it] {'loss': '0.5537', 'grad_norm': '0.9958', 'learning_rate': '9.746e-07', 'epoch': '3.277'} 82%|████████▏ | 770/940 [43:06<10:08, 3.58s/it] 82%|████████▏ | 771/940 [43:09<09:24, 3.34s/it] 82%|████████▏ | 772/940 [43:12<09:21, 3.34s/it] 82%|████████▏ | 773/940 [43:15<09:17, 3.34s/it] 82%|████████▏ | 774/940 [43:19<09:18, 3.36s/it] 82%|████████▏ | 775/940 [43:22<08:56, 3.25s/it] 83%|████████▎ | 776/940 [43:25<08:39, 3.17s/it] 83%|████████▎ | 777/940 [43:28<08:53, 3.27s/it] 83%|████████▎ | 778/940 [43:33<09:48, 3.63s/it] 83%|████████▎ | 779/940 [43:37<10:05, 3.76s/it] 83%|████████▎ | 780/940 [43:40<09:22, 3.51s/it] {'loss': '0.5587', 'grad_norm': '1.047', 'learning_rate': '8.673e-07', 'epoch': '3.32'} 83%|████████▎ | 780/940 [43:40<09:22, 3.51s/it] 83%|████████▎ | 781/940 [43:42<08:33, 3.23s/it] 83%|████████▎ | 782/940 [43:45<08:23, 3.19s/it] 83%|████████▎ | 783/940 [43:48<08:00, 3.06s/it] 83%|████████▎ | 784/940 [43:50<07:25, 2.86s/it] 84%|████████▎ | 785/940 [43:53<07:16, 2.81s/it] 84%|████████▎ | 786/940 [43:56<07:11, 2.80s/it] 84%|████████▎ | 787/940 [44:02<09:37, 3.78s/it] 84%|████████▍ | 788/940 [44:04<08:35, 3.39s/it] 84%|████████▍ | 789/940 [44:08<08:36, 3.42s/it] 84%|████████▍ | 790/940 [44:11<08:17, 3.32s/it] {'loss': '0.5653', 'grad_norm': '0.9495', 'learning_rate': '7.657e-07', 'epoch': '3.363'} 84%|████████▍ | 790/940 [44:11<08:17, 3.32s/it] 84%|████████▍ | 791/940 [44:15<08:25, 3.39s/it] 84%|████████▍ | 792/940 [44:19<08:49, 3.58s/it] 84%|████████▍ | 793/940 [44:21<08:02, 3.28s/it] 84%|████████▍ | 794/940 [44:24<07:38, 3.14s/it] 85%|████████▍ | 795/940 [44:29<08:50, 3.66s/it] 85%|████████▍ | 796/940 [44:33<09:23, 3.91s/it] 85%|████████▍ | 797/940 [44:37<09:12, 3.86s/it] 85%|████████▍ | 798/940 [44:40<08:10, 3.46s/it] 85%|████████▌ | 799/940 [44:44<08:41, 3.70s/it] 85%|████████▌ | 800/940 [44:48<08:48, 3.77s/it] {'loss': '0.5531', 'grad_norm': '1.351', 'learning_rate': '6.699e-07', 'epoch': '3.405'} 85%|████████▌ | 800/940 [44:48<08:48, 3.77s/it] 85%|████████▌ | 801/940 [44:50<07:52, 3.40s/it] 85%|████████▌ | 802/940 [44:53<07:33, 3.29s/it] 85%|████████▌ | 803/940 [44:59<08:58, 3.93s/it] 86%|████████▌ | 804/940 [45:03<08:56, 3.94s/it] 86%|████████▌ | 805/940 [45:06<08:33, 3.80s/it] 86%|████████▌ | 806/940 [45:09<08:02, 3.60s/it] 86%|████████▌ | 807/940 [45:12<07:31, 3.39s/it] 86%|████████▌ | 808/940 [45:15<07:03, 3.21s/it] 86%|████████▌ | 809/940 [45:19<07:17, 3.34s/it] 86%|████████▌ | 810/940 [45:24<08:26, 3.90s/it] {'loss': '0.5778', 'grad_norm': '1.008', 'learning_rate': '5.8e-07', 'epoch': '3.448'} 86%|████████▌ | 810/940 [45:24<08:26, 3.90s/it] 86%|████████▋ | 811/940 [45:31<10:07, 4.71s/it] 86%|████████▋ | 812/940 [45:35<09:49, 4.61s/it] 86%|████████▋ | 813/940 [45:38<08:57, 4.23s/it] 87%|████████▋ | 814/940 [45:42<08:44, 4.16s/it] 87%|████████▋ | 815/940 [45:45<07:57, 3.82s/it] 87%|████████▋ | 816/940 [45:49<07:43, 3.74s/it] 87%|████████▋ | 817/940 [45:51<06:44, 3.29s/it] 87%|████████▋ | 818/940 [45:55<07:05, 3.48s/it] 87%|████████▋ | 819/940 [45:57<06:20, 3.15s/it] 87%|████████▋ | 820/940 [46:01<06:34, 3.29s/it] {'loss': '0.5518', 'grad_norm': '0.9772', 'learning_rate': '4.963e-07', 'epoch': '3.491'} 87%|████████▋ | 820/940 [46:01<06:34, 3.29s/it] 87%|████████▋ | 821/940 [46:04<06:27, 3.26s/it] 87%|████████▋ | 822/940 [46:07<06:20, 3.22s/it] 88%|████████▊ | 823/940 [46:10<05:44, 2.94s/it] 88%|████████▊ | 824/940 [46:15<06:57, 3.60s/it] 88%|████████▊ | 825/940 [46:17<06:20, 3.31s/it] 88%|████████▊ | 826/940 [46:20<05:59, 3.15s/it] 88%|████████▊ | 827/940 [46:23<05:35, 2.97s/it] 88%|████████▊ | 828/940 [46:26<05:37, 3.02s/it] 88%|████████▊ | 829/940 [46:28<05:17, 2.86s/it] 88%|████████▊ | 830/940 [46:31<04:59, 2.72s/it] {'loss': '0.5774', 'grad_norm': '1.047', 'learning_rate': '4.188e-07', 'epoch': '3.533'} 88%|████████▊ | 830/940 [46:31<04:59, 2.72s/it] 88%|████████▊ | 831/940 [46:35<05:43, 3.15s/it] 89%|████████▊ | 832/940 [46:38<05:41, 3.16s/it] 89%|████████▊ | 833/940 [46:41<05:27, 3.07s/it] 89%|████████▊ | 834/940 [46:45<06:01, 3.41s/it] 89%|████████▉ | 835/940 [46:48<05:30, 3.15s/it] 89%|████████▉ | 836/940 [46:51<05:44, 3.32s/it] 89%|████████▉ | 837/940 [46:54<05:20, 3.11s/it] 89%|████████▉ | 838/940 [46:56<04:57, 2.92s/it] 89%|████████▉ | 839/940 [46:59<04:52, 2.89s/it] 89%|████████▉ | 840/940 [47:03<05:07, 3.07s/it] {'loss': '0.5664', 'grad_norm': '0.9108', 'learning_rate': '3.476e-07', 'epoch': '3.576'} 89%|████████▉ | 840/940 [47:03<05:07, 3.07s/it] 89%|████████▉ | 841/940 [47:07<05:38, 3.42s/it] 90%|████████▉ | 842/940 [47:10<05:07, 3.13s/it] 90%|████████▉ | 843/940 [47:13<05:07, 3.17s/it] 90%|████████▉ | 844/940 [47:15<04:43, 2.95s/it] 90%|████████▉ | 845/940 [47:18<04:38, 2.93s/it] 90%|█████████ | 846/940 [47:21<04:42, 3.00s/it] 90%|█████████ | 847/940 [47:25<04:59, 3.22s/it] 90%|█████████ | 848/940 [47:29<05:06, 3.33s/it] 90%|█████████ | 849/940 [47:32<05:16, 3.48s/it] 90%|█████████ | 850/940 [47:36<05:14, 3.50s/it] {'loss': '0.5665', 'grad_norm': '0.987', 'learning_rate': '2.828e-07', 'epoch': '3.619'} 90%|█████████ | 850/940 [47:36<05:14, 3.50s/it] 91%|█████████ | 851/940 [47:40<05:19, 3.59s/it] 91%|█████████ | 852/940 [47:42<04:51, 3.31s/it] 91%|█████████ | 853/940 [47:45<04:20, 2.99s/it] 91%|█████████ | 854/940 [47:47<04:06, 2.87s/it] 91%|█████████ | 855/940 [47:50<04:00, 2.83s/it] 91%|█████████ | 856/940 [47:53<03:52, 2.77s/it] 91%|█████████ | 857/940 [47:55<03:41, 2.67s/it] 91%|█████████▏| 858/940 [47:57<03:29, 2.56s/it] 91%|█████████▏| 859/940 [48:00<03:30, 2.60s/it] 91%|█████████▏| 860/940 [48:03<03:33, 2.67s/it] {'loss': '0.5847', 'grad_norm': '1.056', 'learning_rate': '2.245e-07', 'epoch': '3.661'} 91%|█████████▏| 860/940 [48:03<03:33, 2.67s/it] 92%|█████████▏| 861/940 [48:06<03:30, 2.67s/it] 92%|█████████▏| 862/940 [48:09<03:36, 2.78s/it] 92%|█████████▏| 863/940 [48:12<03:42, 2.89s/it] 92%|█████████▏| 864/940 [48:14<03:34, 2.82s/it] 92%|█████████▏| 865/940 [48:17<03:32, 2.84s/it] 92%|█████████▏| 866/940 [48:20<03:25, 2.78s/it] 92%|█████████▏| 867/940 [48:23<03:25, 2.81s/it] 92%|█████████▏| 868/940 [48:26<03:32, 2.95s/it] 92%|█████████▏| 869/940 [48:29<03:38, 3.08s/it] 93%|█████████▎| 870/940 [48:33<03:36, 3.10s/it] {'loss': '0.5574', 'grad_norm': '1.035', 'learning_rate': '1.728e-07', 'epoch': '3.704'} 93%|█████████▎| 870/940 [48:33<03:36, 3.10s/it] 93%|█████████▎| 871/940 [48:36<03:30, 3.05s/it] 93%|█████████▎| 872/940 [48:39<03:36, 3.19s/it] 93%|█████████▎| 873/940 [48:43<03:59, 3.57s/it] 93%|█████████▎| 874/940 [48:46<03:40, 3.34s/it] 93%|█████████▎| 875/940 [48:50<03:39, 3.38s/it] 93%|█████████▎| 876/940 [48:53<03:35, 3.36s/it] 93%|█████████▎| 877/940 [48:56<03:18, 3.14s/it] 93%|█████████▎| 878/940 [48:59<03:18, 3.21s/it] 94%|█████████▎| 879/940 [49:03<03:28, 3.41s/it] 94%|█████████▎| 880/940 [49:06<03:21, 3.36s/it] {'loss': '0.5604', 'grad_norm': '1.031', 'learning_rate': '1.277e-07', 'epoch': '3.747'} 94%|█████████▎| 880/940 [49:06<03:21, 3.36s/it] 94%|█████████▎| 881/940 [49:09<03:03, 3.11s/it] 94%|█████████▍| 882/940 [49:12<02:59, 3.10s/it] 94%|█████████▍| 883/940 [49:14<02:48, 2.95s/it] 94%|█████████▍| 884/940 [49:19<03:05, 3.31s/it] 94%|█████████▍| 885/940 [49:21<02:47, 3.04s/it] 94%|█████████▍| 886/940 [49:25<02:56, 3.28s/it] 94%|█████████▍| 887/940 [49:28<02:54, 3.29s/it] 94%|█████████▍| 888/940 [49:33<03:13, 3.73s/it] 95%|█████████▍| 889/940 [49:36<03:02, 3.59s/it] 95%|█████████▍| 890/940 [49:39<02:43, 3.27s/it] {'loss': '0.5812', 'grad_norm': '0.9566', 'learning_rate': '8.94e-08', 'epoch': '3.789'} 95%|█████████▍| 890/940 [49:39<02:43, 3.27s/it] 95%|█████████▍| 891/940 [49:41<02:31, 3.09s/it] 95%|█████████▍| 892/940 [49:44<02:27, 3.08s/it] 95%|█████████▌| 893/940 [49:48<02:25, 3.09s/it] 95%|█████████▌| 894/940 [49:51<02:34, 3.36s/it] 95%|█████████▌| 895/940 [49:55<02:36, 3.49s/it] 95%|█████████▌| 896/940 [49:58<02:25, 3.30s/it] 95%|█████████▌| 897/940 [50:01<02:16, 3.17s/it] 96%|█████████▌| 898/940 [50:05<02:19, 3.32s/it] 96%|█████████▌| 899/940 [50:08<02:22, 3.46s/it] 96%|█████████▌| 900/940 [50:13<02:25, 3.64s/it] {'loss': '0.5552', 'grad_norm': '0.9202', 'learning_rate': '5.784e-08', 'epoch': '3.832'} 96%|█████████▌| 900/940 [50:13<02:25, 3.64s/it] 96%|█████████▌| 901/940 [50:17<02:29, 3.84s/it] 96%|█████████▌| 902/940 [50:21<02:32, 4.02s/it] 96%|█████████▌| 903/940 [50:24<02:18, 3.74s/it] 96%|█████████▌| 904/940 [50:27<02:02, 3.40s/it] 96%|█████████▋| 905/940 [50:32<02:11, 3.74s/it] 96%|█████████▋| 906/940 [50:36<02:10, 3.84s/it] 96%|█████████▋| 907/940 [50:40<02:09, 3.93s/it] 97%|█████████▋| 908/940 [50:42<01:52, 3.51s/it] 97%|█████████▋| 909/940 [50:45<01:41, 3.29s/it] 97%|█████████▋| 910/940 [50:49<01:45, 3.53s/it] {'loss': '0.5619', 'grad_norm': '0.9611', 'learning_rate': '3.309e-08', 'epoch': '3.875'} 97%|█████████▋| 910/940 [50:49<01:45, 3.53s/it] 97%|█████████▋| 911/940 [50:53<01:49, 3.77s/it] 97%|█████████▋| 912/940 [50:57<01:47, 3.85s/it] 97%|█████████▋| 913/940 [51:01<01:42, 3.79s/it] 97%|█████████▋| 914/940 [51:04<01:30, 3.48s/it] 97%|█████████▋| 915/940 [51:08<01:29, 3.58s/it] 97%|█████████▋| 916/940 [51:12<01:29, 3.72s/it] 98%|█████████▊| 917/940 [51:15<01:19, 3.46s/it] 98%|█████████▊| 918/940 [51:18<01:16, 3.49s/it] 98%|█████████▊| 919/940 [51:21<01:06, 3.19s/it] 98%|█████████▊| 920/940 [51:24<01:04, 3.21s/it] {'loss': '0.5391', 'grad_norm': '0.9684', 'learning_rate': '1.52e-08', 'epoch': '3.917'} 98%|█████████▊| 920/940 [51:24<01:04, 3.21s/it] 98%|█████████▊| 921/940 [51:27<01:00, 3.18s/it] 98%|█████████▊| 922/940 [51:30<00:53, 2.99s/it] 98%|█████████▊| 923/940 [51:32<00:48, 2.86s/it] 98%|█████████▊| 924/940 [51:36<00:51, 3.19s/it] 98%|█████████▊| 925/940 [51:39<00:44, 3.00s/it] 99%|█████████▊| 926/940 [51:42<00:43, 3.08s/it] 99%|█████████▊| 927/940 [51:45<00:39, 3.03s/it] 99%|█████████▊| 928/940 [51:48<00:36, 3.08s/it] 99%|█████████▉| 929/940 [51:51<00:33, 3.07s/it] 99%|█████████▉| 930/940 [51:54<00:29, 2.94s/it] {'loss': '0.5671', 'grad_norm': '0.9603', 'learning_rate': '4.171e-09', 'epoch': '3.96'} 99%|█████████▉| 930/940 [51:54<00:29, 2.94s/it] 99%|█████████▉| 931/940 [51:56<00:25, 2.83s/it] 99%|█████████▉| 932/940 [51:59<00:21, 2.70s/it] 99%|█████████▉| 933/940 [52:02<00:20, 2.98s/it] 99%|█████████▉| 934/940 [52:05<00:17, 2.83s/it] 99%|█████████▉| 935/940 [52:08<00:14, 2.82s/it] 100%|█████████▉| 936/940 [52:12<00:13, 3.30s/it] 100%|█████████▉| 937/940 [52:15<00:09, 3.13s/it] 100%|█████████▉| 938/940 [52:20<00:07, 3.88s/it] 100%|█████████▉| 939/940 [52:23<00:03, 3.50s/it] 100%|██████████| 940/940 [52:24<00:00, 2.76s/it] {'loss': '0.521', 'grad_norm': '1.651', 'learning_rate': '3.447e-11', 'epoch': '4'} 100%|██████████| 940/940 [52:24<00:00, 2.76s/it][INFO|trainer.py:3797] 2026-04-29 17:58:48,007 >> Saving model checkpoint to saves/qwen3_1.7b/coding/fft/checkpoint-940 [INFO|configuration_utils.py:432] 2026-04-29 17:58:48,014 >> Configuration saved in saves/qwen3_1.7b/coding/fft/checkpoint-940/config.json [INFO|configuration_utils.py:803] 2026-04-29 17:58:48,018 >> Configuration saved in saves/qwen3_1.7b/coding/fft/checkpoint-940/generation_config.json Writing model shards: 0%| | 0/1 [00:00> Model weights saved in saves/qwen3_1.7b/coding/fft/checkpoint-940/model.safetensors [INFO|tokenization_utils_base.py:3224] 2026-04-29 17:58:54,112 >> chat template saved in saves/qwen3_1.7b/coding/fft/checkpoint-940/chat_template.jinja [INFO|tokenization_utils_base.py:2078] 2026-04-29 17:58:54,115 >> tokenizer config file saved in saves/qwen3_1.7b/coding/fft/checkpoint-940/tokenizer_config.json [INFO|trainer.py:1863] 2026-04-29 17:58:54,274 >> Training completed. Do not forget to share your model on huggingface.co/models =) {'train_runtime': '3155', 'train_samples_per_second': '38.03', 'train_steps_per_second': '0.298', 'train_loss': '0.6604', 'epoch': '4'} 100%|██████████| 940/940 [52:33<00:00, 2.76s/it] 100%|██████████| 940/940 [52:33<00:00, 3.36s/it] [INFO|trainer.py:3797] 2026-04-29 17:58:55,513 >> Saving model checkpoint to saves/qwen3_1.7b/coding/fft [INFO|configuration_utils.py:432] 2026-04-29 17:58:55,520 >> Configuration saved in saves/qwen3_1.7b/coding/fft/config.json [INFO|configuration_utils.py:803] 2026-04-29 17:58:55,525 >> Configuration saved in saves/qwen3_1.7b/coding/fft/generation_config.json Writing model shards: 0%| | 0/1 [00:00> Model weights saved in saves/qwen3_1.7b/coding/fft/model.safetensors [INFO|tokenization_utils_base.py:3224] 2026-04-29 17:59:01,701 >> chat template saved in saves/qwen3_1.7b/coding/fft/chat_template.jinja [INFO|tokenization_utils_base.py:2078] 2026-04-29 17:59:01,704 >> tokenizer config file saved in saves/qwen3_1.7b/coding/fft/tokenizer_config.json ywang29-p4d-debug-1-worker-0:243413:243413 [2] NCCL INFO comm 0x55b7e396d9c0 rank 2 nranks 8 cudaDev 2 busId 201c0 - Destroy COMPLETE ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO comm 0x55ae95eb9e30 rank 3 nranks 8 cudaDev 3 busId 201d0 - Destroy COMPLETE ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO comm 0x558fdd7d54e0 rank 5 nranks 8 cudaDev 5 busId 901d0 - Destroy COMPLETE ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO comm 0x561d4c169060 rank 4 nranks 8 cudaDev 4 busId 901c0 - Destroy COMPLETE ywang29-p4d-debug-1-worker-0:243417:243417 [6] NCCL INFO comm 0x55b500e80270 rank 6 nranks 8 cudaDev 6 busId a01c0 - Destroy COMPLETE ywang29-p4d-debug-1-worker-0:243416:243416 [5] NCCL INFO comm 0x558fd1beff10 rank 5 nranks 8 cudaDev 5 busId 901d0 - Destroy COMPLETE ywang29-p4d-debug-1-worker-0:243415:243415 [4] NCCL INFO comm 0x561d46f046a0 rank 4 nranks 8 cudaDev 4 busId 901c0 - Destroy COMPLETE ywang29-p4d-debug-1-worker-0:243414:243414 [3] NCCL INFO comm 0x55ae88a9dd90 rank 3 nranks 8 cudaDev 3 busId 201d0 - Destroy COMPLETE [INFO|trainer.py:3797] 2026-04-29 17:59:04,829 >> Saving model checkpoint to saves/qwen3_1.7b/coding/fft [INFO|configuration_utils.py:432] 2026-04-29 17:59:04,837 >> Configuration saved in saves/qwen3_1.7b/coding/fft/config.json [INFO|configuration_utils.py:803] 2026-04-29 17:59:04,840 >> Configuration saved in saves/qwen3_1.7b/coding/fft/generation_config.json Writing model shards: 0%| | 0/1 [00:00> Model weights saved in saves/qwen3_1.7b/coding/fft/model.safetensors [INFO|tokenization_utils_base.py:3224] 2026-04-29 17:59:11,820 >> chat template saved in saves/qwen3_1.7b/coding/fft/chat_template.jinja [INFO|tokenization_utils_base.py:2078] 2026-04-29 17:59:11,823 >> tokenizer config file saved in saves/qwen3_1.7b/coding/fft/tokenizer_config.json [INFO|modelcard.py:266] 2026-04-29 17:59:12,253 >> Dropping the following result as it does not have all the necessary fields: {'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}} Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 0%| | 29.6kB / 4.06GB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 0%| | 29.6kB / 4.06GB  Processing Files (1 / 2) : 0%| | 11.5MB / 4.07GB, 57.1MB/s ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 0%| | 29.6kB / 4.06GB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 0%| | 29.6kB / 4.06GB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 0%| | 29.6kB / 4.06GB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 0%| | 636kB / 4.06GB  Processing Files (1 / 2) : 0%| | 12.1MB / 4.07GB, 12.1MB/s New Data Upload : 0%| | 607kB / 134MB, 606kB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 0%| | 1.24MB / 4.06GB  Processing Files (1 / 2) : 0%| | 12.7MB / 4.07GB, 10.6MB/s New Data Upload : 1%| | 1.21MB / 134MB, 1.01MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 0%| | 8.52MB / 4.06GB  Processing Files (1 / 2) : 0%| | 19.9MB / 4.07GB, 14.2MB/s New Data Upload : 6%|▋ | 8.49MB / 134MB, 6.06MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 1%| | 21.3MB / 4.06GB  Processing Files (1 / 2) : 1%| | 32.7MB / 4.07GB, 20.4MB/s New Data Upload : 16%|█▌ | 21.2MB / 134MB, 13.3MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 1%| | 35.8MB / 4.06GB  Processing Files (1 / 2) : 1%| | 47.2MB / 4.07GB, 26.2MB/s New Data Upload : 27%|██▋ | 35.8MB / 134MB, 19.9MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 1%| | 44.9MB / 4.06GB  Processing Files (1 / 2) : 1%|▏ | 56.3MB / 4.07GB, 28.2MB/s New Data Upload : 33%|███▎ | 44.9MB / 134MB, 22.4MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 1%|▏ | 58.9MB / 4.06GB  Processing Files (1 / 2) : 2%|▏ | 70.3MB / 4.07GB, 31.9MB/s New Data Upload : 44%|████▍ | 58.8MB / 134MB, 26.7MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB  Processing Files (1 / 2) : 2%|▏ | 78.2MB / 4.07GB, 32.6MB/s New Data Upload : 50%|████▉ | 66.7MB / 134MB, 27.8MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 2%|▏ | 71.3MB / 4.06GB  Processing Files (1 / 2) : 2%|▏ | 82.7MB / 4.07GB, 24.3MB/s New Data Upload : 35%|███▌ | 71.2MB / 201MB, 20.9MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 2%|▏ | 84.0MB / 4.06GB  Processing Files (1 / 2) : 2%|▏ | 95.4MB / 4.07GB, 26.5MB/s New Data Upload : 42%|████▏ | 83.9MB / 201MB, 23.3MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 2%|▏ | 98.5MB / 4.06GB  Processing Files (1 / 2) : 3%|▎ | 110MB / 4.07GB, 28.9MB/s New Data Upload : 37%|███▋ | 98.5MB / 268MB, 25.9MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 3%|▎ | 109MB / 4.06GB  Processing Files (1 / 2) : 3%|▎ | 120MB / 4.07GB, 30.1MB/s New Data Upload : 41%|████ | 109MB / 268MB, 27.2MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 3%|▎ | 134MB / 4.06GB  Processing Files (1 / 2) : 4%|▎ | 145MB / 4.07GB, 34.5MB/s New Data Upload : 50%|████▉ | 134MB / 268MB, 31.8MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 4%|▍ | 160MB / 4.06GB  Processing Files (1 / 2) : 4%|▍ | 172MB / 4.07GB, 39.0MB/s New Data Upload : 48%|████▊ | 160MB / 335MB, 36.4MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 4%|▍ | 177MB / 4.06GB  Processing Files (1 / 2) : 5%|▍ | 188MB / 4.07GB, 40.9MB/s New Data Upload : 44%|████▍ | 177MB / 402MB, 38.4MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 5%|▍ | 199MB / 4.06GB  Processing Files (1 / 2) : 5%|▌ | 211MB / 4.07GB, 43.9MB/s New Data Upload : 49%|████▉ | 199MB / 402MB, 41.5MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 6%|▌ | 227MB / 4.06GB  Processing Files (1 / 2) : 6%|▌ | 238MB / 4.07GB, 47.7MB/s New Data Upload : 48%|████▊ | 227MB / 469MB, 45.4MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 6%|▌ | 246MB / 4.06GB  Processing Files (1 / 2) : 6%|▋ | 258MB / 4.07GB, 49.6MB/s New Data Upload : 52%|█████▏ | 246MB / 469MB, 47.4MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 7%|▋ | 268MB / 4.06GB  Processing Files (1 / 2) : 7%|▋ | 280MB / 4.07GB, 51.8MB/s New Data Upload : 57%|█████▋ | 268MB / 469MB, 49.7MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 7%|▋ | 294MB / 4.06GB  Processing Files (1 / 2) : 7%|▋ | 305MB / 4.07GB, 54.5MB/s New Data Upload : 63%|██████▎ | 294MB / 469MB, 52.4MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 8%|▊ | 321MB / 4.06GB  Processing Files (1 / 2) : 8%|▊ | 332MB / 4.07GB, 57.3MB/s New Data Upload : 60%|█████▉ | 321MB / 536MB, 55.3MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 9%|▊ | 349MB / 4.06GB  Processing Files (1 / 2) : 9%|▉ | 360MB / 4.07GB, 60.1MB/s New Data Upload : 65%|██████▌ | 349MB / 536MB, 58.2MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 9%|▉ | 381MB / 4.06GB  Processing Files (1 / 2) : 10%|▉ | 392MB / 4.07GB, 63.2MB/s New Data Upload : 63%|██████▎ | 380MB / 603MB, 61.4MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 10%|█ | 416MB / 4.06GB  Processing Files (1 / 2) : 10%|█ | 428MB / 4.07GB, 66.8MB/s New Data Upload : 69%|██████▉ | 416MB / 603MB, 65.1MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 11%|█ | 447MB / 4.06GB  Processing Files (1 / 2) : 11%|█ | 458MB / 4.07GB, 69.4MB/s New Data Upload : 74%|███████▍ | 447MB / 603MB, 67.7MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 12%|█▏ | 476MB / 4.06GB  Processing Files (1 / 2) : 12%|█▏ | 488MB / 4.07GB, 71.7MB/s New Data Upload : 71%|███████ | 476MB / 671MB, 70.0MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 13%|█▎ | 510MB / 4.06GB  Processing Files (1 / 2) : 13%|█▎ | 521MB / 4.07GB, 74.4MB/s New Data Upload : 76%|███████▌ | 509MB / 671MB, 72.8MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 13%|█▎ | 542MB / 4.06GB  Processing Files (1 / 2) : 14%|█▎ | 554MB / 4.07GB, 76.9MB/s New Data Upload : 81%|████████ | 542MB / 671MB, 75.3MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 14%|█▍ | 573MB / 4.06GB  Processing Files (1 / 2) : 14%|█▍ | 584MB / 4.07GB, 79.0MB/s New Data Upload : 85%|████████▌ | 573MB / 671MB, 77.4MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 15%|█▍ | 590MB / 4.06GB  Processing Files (1 / 2) : 15%|█▍ | 602MB / 4.07GB, 79.2MB/s New Data Upload : 80%|███████▉ | 590MB / 738MB, 77.6MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 15%|█▌ | 614MB / 4.06GB  Processing Files (1 / 2) : 15%|█▌ | 626MB / 4.07GB, 80.2MB/s New Data Upload : 76%|███████▋ | 614MB / 805MB, 78.7MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 16%|█▌ | 637MB / 4.06GB  Processing Files (1 / 2) : 16%|█▌ | 649MB / 4.07GB, 81.1MB/s New Data Upload : 79%|███████▉ | 637MB / 805MB, 79.7MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 17%|█▋ | 676MB / 4.06GB  Processing Files (1 / 2) : 17%|█▋ | 688MB / 4.07GB, 83.8MB/s New Data Upload : 84%|████████▍ | 676MB / 805MB, 82.4MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 17%|█▋ | 705MB / 4.06GB  Processing Files (1 / 2) : 18%|█▊ | 717MB / 4.07GB, 85.3MB/s New Data Upload : 81%|████████ | 705MB / 872MB, 84.0MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 18%|█▊ | 738MB / 4.06GB  Processing Files (1 / 2) : 18%|█▊ | 749MB / 4.07GB, 87.1MB/s New Data Upload : 79%|███████▊ | 737MB / 939MB, 85.8MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 19%|█▉ | 770MB / 4.06GB  Processing Files (1 / 2) : 19%|█▉ | 781MB / 4.07GB, 88.8MB/s New Data Upload : 82%|████████▏ | 770MB / 939MB, 87.4MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 20%|█▉ | 805MB / 4.06GB  Processing Files (1 / 2) : 20%|██ | 816MB / 4.07GB, 90.6MB/s New Data Upload : 86%|████████▌ | 804MB / 939MB, 89.3MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 20%|██ | 826MB / 4.06GB  Processing Files (1 / 2) : 21%|██ | 837MB / 4.07GB, 91.0MB/s New Data Upload : 82%|████████▏ | 825MB / 1.01GB, 89.7MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 22%|██▏ | 902MB / 4.06GB  Processing Files (1 / 2) : 22%|██▏ | 914MB / 4.07GB, 97.2MB/s New Data Upload : 86%|████████▌ | 860MB / 1.01GB, 91.5MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 24%|██▍ | 974MB / 4.06GB  Processing Files (1 / 2) : 24%|██▍ | 986MB / 4.07GB, 103MB/s New Data Upload : 84%|████████▍ | 901MB / 1.07GB, 93.9MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 26%|██▌ | 1.06GB / 4.06GB  Processing Files (1 / 2) : 26%|██▋ | 1.07GB / 4.07GB, 109MB/s New Data Upload : 87%|████████▋ | 933MB / 1.07GB, 95.2MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 27%|██▋ | 1.11GB / 4.06GB  Processing Files (1 / 2) : 27%|██▋ | 1.12GB / 4.07GB, 112MB/s New Data Upload : 85%|████████▍ | 967MB / 1.14GB, 96.7MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 28%|██▊ | 1.15GB / 4.06GB  Processing Files (1 / 2) : 28%|██▊ | 1.16GB / 4.07GB, 114MB/s New Data Upload : 88%|████████▊ | 1.01GB / 1.14GB, 98.5MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 29%|██▉ | 1.18GB / 4.06GB  Processing Files (1 / 2) : 29%|██▉ | 1.19GB / 4.07GB, 116MB/s New Data Upload : 86%|████████▌ | 1.04GB / 1.21GB, 102MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 30%|██▉ | 1.20GB / 4.06GB  Processing Files (1 / 2) : 30%|██▉ | 1.21GB / 4.07GB, 118MB/s New Data Upload : 83%|████████▎ | 1.06GB / 1.27GB, 104MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 30%|███ | 1.23GB / 4.06GB  Processing Files (1 / 2) : 30%|███ | 1.24GB / 4.07GB, 120MB/s New Data Upload : 85%|████████▌ | 1.08GB / 1.27GB, 106MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 31%|███ | 1.25GB / 4.06GB  Processing Files (1 / 2) : 31%|███ | 1.27GB / 4.07GB, 123MB/s New Data Upload : 83%|████████▎ | 1.11GB / 1.34GB, 109MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 31%|███▏ | 1.27GB / 4.06GB  Processing Files (1 / 2) : 31%|███▏ | 1.28GB / 4.07GB, 125MB/s New Data Upload : 84%|████████▍ | 1.13GB / 1.34GB, 111MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 32%|███▏ | 1.30GB / 4.06GB  Processing Files (1 / 2) : 32%|███▏ | 1.32GB / 4.07GB, 128MB/s New Data Upload : 83%|████████▎ | 1.16GB / 1.41GB, 114MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 33%|███▎ | 1.35GB / 4.06GB  Processing Files (1 / 2) : 33%|███▎ | 1.36GB / 4.07GB, 131MB/s New Data Upload : 86%|████████▌ | 1.21GB / 1.41GB, 118MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 34%|███▍ | 1.39GB / 4.06GB  Processing Files (1 / 2) : 34%|███▍ | 1.40GB / 4.07GB, 134MB/s New Data Upload : 85%|████████▍ | 1.25GB / 1.48GB, 121MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 35%|███▌ | 1.44GB / 4.06GB  Processing Files (1 / 2) : 36%|███▌ | 1.45GB / 4.07GB, 138MB/s New Data Upload : 88%|████████▊ | 1.30GB / 1.48GB, 124MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 36%|███▋ | 1.47GB / 4.06GB  Processing Files (1 / 2) : 36%|███▋ | 1.48GB / 4.07GB, 140MB/s New Data Upload : 86%|████████▋ | 1.33GB / 1.54GB, 126MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 37%|███▋ | 1.50GB / 4.06GB  Processing Files (1 / 2) : 37%|███▋ | 1.51GB / 4.07GB, 141MB/s New Data Upload : 88%|████████▊ | 1.36GB / 1.54GB, 128MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 38%|███▊ | 1.54GB / 4.06GB  Processing Files (1 / 2) : 38%|███▊ | 1.55GB / 4.07GB, 144MB/s New Data Upload : 87%|████████▋ | 1.39GB / 1.61GB, 130MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 39%|███▉ | 1.58GB / 4.06GB  Processing Files (1 / 2) : 39%|███▉ | 1.59GB / 4.07GB, 148MB/s New Data Upload : 89%|████████▉ | 1.43GB / 1.61GB, 134MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 40%|███▉ | 1.61GB / 4.06GB  Processing Files (1 / 2) : 40%|███▉ | 1.62GB / 4.07GB, 151MB/s New Data Upload : 88%|████████▊ | 1.47GB / 1.68GB, 137MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 41%|████ | 1.66GB / 4.06GB  Processing Files (1 / 2) : 41%|████ | 1.67GB / 4.07GB, 156MB/s New Data Upload : 90%|█████████ | 1.52GB / 1.68GB, 142MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 42%|████▏ | 1.69GB / 4.06GB  Processing Files (1 / 2) : 42%|████▏ | 1.70GB / 4.07GB, 159MB/s New Data Upload : 89%|████████▊ | 1.55GB / 1.74GB, 145MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 42%|████▏ | 1.72GB / 4.06GB  Processing Files (1 / 2) : 42%|████▏ | 1.73GB / 4.07GB, 162MB/s New Data Upload : 91%|█████████ | 1.58GB / 1.74GB, 148MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 43%|████▎ | 1.75GB / 4.06GB  Processing Files (1 / 2) : 43%|████▎ | 1.76GB / 4.07GB, 164MB/s New Data Upload : 92%|█████████▏| 1.61GB / 1.74GB, 150MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 44%|████▍ | 1.79GB / 4.06GB  Processing Files (1 / 2) : 44%|████▍ | 1.80GB / 4.07GB, 166MB/s New Data Upload : 91%|█████████ | 1.65GB / 1.81GB, 152MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 45%|████▍ | 1.82GB / 4.06GB  Processing Files (1 / 2) : 45%|████▍ | 1.83GB / 4.07GB, 168MB/s New Data Upload : 89%|████████▉ | 1.68GB / 1.88GB, 154MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 46%|████▌ | 1.85GB / 4.06GB  Processing Files (1 / 2) : 46%|████▌ | 1.86GB / 4.07GB, 169MB/s New Data Upload : 91%|█████████ | 1.71GB / 1.88GB, 155MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 47%|████▋ | 1.90GB / 4.06GB  Processing Files (1 / 2) : 47%|████▋ | 1.91GB / 4.07GB, 170MB/s New Data Upload : 93%|█████████▎| 1.76GB / 1.88GB, 156MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 47%|████▋ | 1.92GB / 4.06GB  Processing Files (1 / 2) : 47%|████▋ | 1.93GB / 4.07GB, 171MB/s New Data Upload : 91%|█████████▏| 1.78GB / 1.94GB, 157MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 48%|████▊ | 1.94GB / 4.06GB  Processing Files (1 / 2) : 48%|████▊ | 1.95GB / 4.07GB, 171MB/s New Data Upload : 93%|█████████▎| 1.80GB / 1.94GB, 157MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 49%|████▊ | 1.97GB / 4.06GB  Processing Files (1 / 2) : 49%|████▊ | 1.98GB / 4.07GB, 171MB/s New Data Upload : 91%|█████████ | 1.83GB / 2.01GB, 157MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 49%|████▉ | 2.00GB / 4.06GB  Processing Files (1 / 2) : 49%|████▉ | 2.01GB / 4.07GB, 172MB/s New Data Upload : 92%|█████████▏| 1.86GB / 2.01GB, 158MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 50%|████▉ | 2.02GB / 4.06GB  Processing Files (1 / 2) : 50%|████▉ | 2.03GB / 4.07GB, 172MB/s New Data Upload : 90%|█████████ | 1.88GB / 2.08GB, 158MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 51%|█████ | 2.06GB / 4.06GB  Processing Files (1 / 2) : 51%|█████ | 2.07GB / 4.07GB, 173MB/s New Data Upload : 89%|████████▉ | 1.91GB / 2.15GB, 159MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 51%|█████▏ | 2.09GB / 4.06GB  Processing Files (1 / 2) : 52%|█████▏ | 2.10GB / 4.07GB, 174MB/s New Data Upload : 91%|█████████ | 1.95GB / 2.15GB, 160MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 53%|█████▎ | 2.14GB / 4.06GB  Processing Files (1 / 2) : 53%|█████▎ | 2.15GB / 4.07GB, 175MB/s New Data Upload : 90%|█████████ | 2.00GB / 2.21GB, 161MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 54%|█████▎ | 2.18GB / 4.06GB  Processing Files (1 / 2) : 54%|█████▍ | 2.19GB / 4.07GB, 176MB/s New Data Upload : 89%|████████▉ | 2.04GB / 2.28GB, 163MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 54%|█████▍ | 2.21GB / 4.06GB  Processing Files (1 / 2) : 55%|█████▍ | 2.23GB / 4.07GB, 176MB/s New Data Upload : 91%|█████████ | 2.07GB / 2.28GB, 162MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 55%|█████▌ | 2.25GB / 4.06GB  Processing Files (1 / 2) : 56%|█████▌ | 2.26GB / 4.07GB, 177MB/s New Data Upload : 90%|████████▉ | 2.11GB / 2.35GB, 163MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 56%|█████▌ | 2.28GB / 4.06GB  Processing Files (1 / 2) : 56%|█████▌ | 2.29GB / 4.07GB, 176MB/s New Data Upload : 91%|█████████ | 2.13GB / 2.35GB, 163MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 57%|█████▋ | 2.31GB / 4.06GB  Processing Files (1 / 2) : 57%|█████▋ | 2.32GB / 4.07GB, 176MB/s New Data Upload : 90%|████████▉ | 2.17GB / 2.41GB, 162MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 58%|█████▊ | 2.34GB / 4.06GB  Processing Files (1 / 2) : 58%|█████▊ | 2.35GB / 4.07GB, 176MB/s New Data Upload : 91%|█████████ | 2.20GB / 2.41GB, 162MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 58%|█████▊ | 2.38GB / 4.06GB  Processing Files (1 / 2) : 59%|█████▊ | 2.39GB / 4.07GB, 177MB/s New Data Upload : 90%|█████████ | 2.24GB / 2.48GB, 163MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 59%|█████▉ | 2.41GB / 4.06GB  Processing Files (1 / 2) : 59%|█████▉ | 2.42GB / 4.07GB, 178MB/s New Data Upload : 91%|█████████▏| 2.27GB / 2.48GB, 165MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 60%|██████ | 2.45GB / 4.06GB  Processing Files (1 / 2) : 60%|██████ | 2.46GB / 4.07GB, 180MB/s New Data Upload : 93%|█████████▎| 2.30GB / 2.48GB, 166MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 61%|██████ | 2.48GB / 4.06GB  Processing Files (1 / 2) : 61%|██████ | 2.49GB / 4.07GB, 180MB/s New Data Upload : 92%|█████████▏| 2.33GB / 2.55GB, 166MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 62%|██████▏ | 2.52GB / 4.06GB  Processing Files (1 / 2) : 62%|██████▏ | 2.53GB / 4.07GB, 180MB/s New Data Upload : 91%|█████████ | 2.37GB / 2.62GB, 166MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 63%|██████▎ | 2.56GB / 4.06GB  Processing Files (1 / 2) : 63%|██████▎ | 2.57GB / 4.07GB, 182MB/s New Data Upload : 92%|█████████▏| 2.42GB / 2.62GB, 168MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 64%|██████▍ | 2.59GB / 4.06GB  Processing Files (1 / 2) : 64%|██████▍ | 2.61GB / 4.07GB, 182MB/s New Data Upload : 91%|█████████▏| 2.45GB / 2.68GB, 168MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 65%|██████▍ | 2.64GB / 4.06GB  Processing Files (1 / 2) : 65%|██████▌ | 2.65GB / 4.07GB, 183MB/s New Data Upload : 91%|█████████ | 2.50GB / 2.75GB, 169MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 66%|██████▌ | 2.67GB / 4.06GB  Processing Files (1 / 2) : 66%|██████▌ | 2.68GB / 4.07GB, 183MB/s New Data Upload : 92%|█████████▏| 2.53GB / 2.75GB, 169MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 67%|██████▋ | 2.72GB / 4.06GB  Processing Files (1 / 2) : 67%|██████▋ | 2.73GB / 4.07GB, 185MB/s New Data Upload : 91%|█████████▏| 2.57GB / 2.82GB, 172MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 68%|██████▊ | 2.75GB / 4.06GB  Processing Files (1 / 2) : 68%|██████▊ | 2.76GB / 4.07GB, 181MB/s New Data Upload : 93%|█████████▎| 2.61GB / 2.82GB, 171MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 68%|██████▊ | 2.78GB / 4.06GB  Processing Files (1 / 2) : 68%|██████▊ | 2.79GB / 4.07GB, 177MB/s New Data Upload : 91%|█████████▏| 2.63GB / 2.88GB, 170MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 69%|██████▉ | 2.80GB / 4.06GB  Processing Files (1 / 2) : 69%|██████▉ | 2.82GB / 4.07GB, 171MB/s New Data Upload : 92%|█████████▏| 2.66GB / 2.88GB, 170MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 70%|███████ | 2.85GB / 4.06GB  Processing Files (1 / 2) : 70%|███████ | 2.86GB / 4.07GB, 171MB/s New Data Upload : 92%|█████████▏| 2.71GB / 2.95GB, 171MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 71%|███████▏ | 2.90GB / 4.06GB  Processing Files (1 / 2) : 71%|███████▏ | 2.91GB / 4.07GB, 172MB/s New Data Upload : 91%|█████████▏| 2.75GB / 3.02GB, 172MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 72%|███████▏ | 2.93GB / 4.06GB  Processing Files (1 / 2) : 72%|███████▏ | 2.94GB / 4.07GB, 171MB/s New Data Upload : 92%|█████████▏| 2.79GB / 3.02GB, 171MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 73%|███████▎ | 2.98GB / 4.06GB  Processing Files (1 / 2) : 73%|███████▎ | 2.99GB / 4.07GB, 174MB/s New Data Upload : 92%|█████████▏| 2.84GB / 3.09GB, 174MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 74%|███████▍ | 3.02GB / 4.06GB  Processing Files (1 / 2) : 74%|███████▍ | 3.03GB / 4.07GB, 176MB/s New Data Upload : 93%|█████████▎| 2.88GB / 3.09GB, 176MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 75%|███████▌ | 3.07GB / 4.06GB  Processing Files (1 / 2) : 76%|███████▌ | 3.08GB / 4.07GB, 178MB/s New Data Upload : 93%|█████████▎| 2.93GB / 3.15GB, 178MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 76%|███████▋ | 3.10GB / 4.06GB  Processing Files (1 / 2) : 76%|███████▋ | 3.12GB / 4.07GB, 180MB/s New Data Upload : 92%|█████████▏| 2.96GB / 3.22GB, 180MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 77%|███████▋ | 3.13GB / 4.06GB  Processing Files (1 / 2) : 77%|███████▋ | 3.14GB / 4.07GB, 179MB/s New Data Upload : 93%|█████████▎| 2.99GB / 3.22GB, 179MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 78%|███████▊ | 3.17GB / 4.06GB  Processing Files (1 / 2) : 78%|███████▊ | 3.18GB / 4.07GB, 178MB/s New Data Upload : 92%|█████████▏| 3.03GB / 3.29GB, 178MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 79%|███████▉ | 3.20GB / 4.06GB  Processing Files (1 / 2) : 79%|███████▉ | 3.21GB / 4.07GB, 177MB/s New Data Upload : 93%|█████████▎| 3.06GB / 3.29GB, 177MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 80%|███████▉ | 3.24GB / 4.06GB  Processing Files (1 / 2) : 80%|███████▉ | 3.26GB / 4.07GB, 177MB/s New Data Upload : 93%|█████████▎| 3.10GB / 3.35GB, 177MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 81%|████████ | 3.29GB / 4.06GB  Processing Files (1 / 2) : 81%|████████ | 3.30GB / 4.07GB, 178MB/s New Data Upload : 94%|█████████▍| 3.15GB / 3.35GB, 178MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 82%|████████▏ | 3.32GB / 4.06GB  Processing Files (1 / 2) : 82%|████████▏ | 3.33GB / 4.07GB, 178MB/s New Data Upload : 93%|█████████▎| 3.18GB / 3.42GB, 178MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 83%|████████▎ | 3.36GB / 4.06GB  Processing Files (1 / 2) : 83%|████████▎ | 3.37GB / 4.07GB, 179MB/s New Data Upload : 92%|█████████▏| 3.22GB / 3.49GB, 179MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 84%|████████▍ | 3.41GB / 4.06GB  Processing Files (1 / 2) : 84%|████████▍ | 3.42GB / 4.07GB, 180MB/s New Data Upload : 94%|█████████▎| 3.27GB / 3.49GB, 180MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 85%|████████▌ | 3.46GB / 4.06GB  Processing Files (1 / 2) : 85%|████████▌ | 3.47GB / 4.07GB, 181MB/s New Data Upload : 95%|█████████▌| 3.32GB / 3.49GB, 181MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 86%|████████▌ | 3.50GB / 4.06GB  Processing Files (1 / 2) : 86%|████████▌ | 3.51GB / 4.07GB, 181MB/s New Data Upload : 94%|█████████▍| 3.36GB / 3.55GB, 181MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 87%|████████▋ | 3.55GB / 4.06GB  Processing Files (1 / 2) : 87%|████████▋ | 3.56GB / 4.07GB, 182MB/s New Data Upload : 94%|█████████▍| 3.40GB / 3.62GB, 182MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 88%|████████▊ | 3.58GB / 4.06GB  Processing Files (1 / 2) : 88%|████████▊ | 3.59GB / 4.07GB, 183MB/s New Data Upload : 95%|█████████▌| 3.44GB / 3.62GB, 183MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 89%|████████▉ | 3.62GB / 4.06GB  Processing Files (1 / 2) : 89%|████████▉ | 3.63GB / 4.07GB, 183MB/s New Data Upload : 94%|█████████▍| 3.48GB / 3.69GB, 183MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 90%|████████▉ | 3.65GB / 4.06GB  Processing Files (1 / 2) : 90%|████████▉ | 3.66GB / 4.07GB, 182MB/s New Data Upload : 95%|█████████▌| 3.51GB / 3.69GB, 182MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 91%|█████████ | 3.68GB / 4.06GB  Processing Files (1 / 2) : 91%|█████████ | 3.69GB / 4.07GB, 182MB/s New Data Upload : 94%|█████████▍| 3.54GB / 3.76GB, 182MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 91%|█████████▏| 3.71GB / 4.06GB  Processing Files (1 / 2) : 91%|█████████▏| 3.72GB / 4.07GB, 182MB/s New Data Upload : 95%|█████████▍| 3.57GB / 3.76GB, 182MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 92%|█████████▏| 3.74GB / 4.06GB  Processing Files (1 / 2) : 92%|█████████▏| 3.75GB / 4.07GB, 181MB/s New Data Upload : 94%|█████████▍| 3.60GB / 3.82GB, 181MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 93%|█████████▎| 3.78GB / 4.06GB  Processing Files (1 / 2) : 93%|█████████▎| 3.79GB / 4.07GB, 182MB/s New Data Upload : 95%|█████████▌| 3.64GB / 3.82GB, 182MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 94%|█████████▎| 3.81GB / 4.06GB  Processing Files (1 / 2) : 94%|█████████▍| 3.82GB / 4.07GB, 183MB/s New Data Upload : 94%|█████████▍| 3.67GB / 3.89GB, 183MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 95%|█████████▍| 3.86GB / 4.06GB  Processing Files (1 / 2) : 95%|█████████▍| 3.87GB / 4.07GB, 185MB/s New Data Upload : 96%|█████████▌| 3.72GB / 3.89GB, 185MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 96%|█████████▌| 3.91GB / 4.06GB  Processing Files (1 / 2) : 96%|█████████▌| 3.92GB / 4.07GB, 187MB/s New Data Upload : 97%|█████████▋| 3.76GB / 3.89GB, 187MB/s  ...ing/fft/training_args.bin: 13%|█▎ | 937B / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 97%|█████████▋| 3.94GB / 4.06GB  ...ing/fft/training_args.bin: 13%|█▎ | 937B / 7.18kB  Processing Files (1 / 3) : 97%|█████████▋| 3.96GB / 4.07GB, 189MB/s New Data Upload : 97%|█████████▋| 3.80GB / 3.92GB, 189MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 98%|█████████▊| 3.96GB / 4.06GB  ...ing/fft/training_args.bin: 19%|█▊ | 1.34kB / 7.18kB  Processing Files (1 / 3) : 98%|█████████▊| 3.97GB / 4.07GB, 187MB/s New Data Upload : 97%|█████████▋| 3.82GB / 3.92GB, 187MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 98%|█████████▊| 3.99GB / 4.06GB  ...ing/fft/training_args.bin: 39%|███▉ | 2.81kB / 7.18kB  Processing Files (1 / 3) : 98%|█████████▊| 4.00GB / 4.07GB, 186MB/s New Data Upload : 98%|█████████▊| 3.85GB / 3.92GB, 186MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 99%|█████████▊| 4.01GB / 4.06GB  ...ing/fft/training_args.bin: 58%|█████▊ | 4.15kB / 7.18kB  Processing Files (1 / 3) : 99%|█████████▊| 4.02GB / 4.07GB, 183MB/s New Data Upload : 99%|█████████▊| 3.87GB / 3.92GB, 183MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 99%|█████████▉| 4.02GB / 4.06GB  ...ing/fft/training_args.bin: 78%|███████▊ | 5.62kB / 7.18kB  Processing Files (1 / 3) : 99%|█████████▉| 4.03GB / 4.07GB, 181MB/s New Data Upload : 99%|█████████▉| 3.88GB / 3.92GB, 181MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 99%|█████████▉| 4.04GB / 4.06GB  ...ing/fft/training_args.bin: 99%|█████████▊| 7.09kB / 7.18kB  Processing Files (1 / 3) : 99%|█████████▉| 4.05GB / 4.07GB, 179MB/s New Data Upload : 99%|█████████▉| 3.90GB / 3.92GB, 179MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.04GB / 4.06GB  ...ing/fft/training_args.bin: 99%|█████████▊| 7.09kB / 7.18kB  Processing Files (1 / 3) : 100%|█████████▉| 4.06GB / 4.07GB, 176MB/s New Data Upload : 100%|█████████▉| 3.90GB / 3.92GB, 176MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.05GB / 4.06GB  ...ing/fft/training_args.bin: 99%|█████████▊| 7.09kB / 7.18kB  Processing Files (1 / 3) : 100%|█████████▉| 4.06GB / 4.07GB, 174MB/s New Data Upload : 100%|█████████▉| 3.91GB / 3.92GB, 174MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  Processing Files (2 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 172MB/s New Data Upload : 100%|█████████▉| 3.92GB / 3.92GB, 172MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  Processing Files (2 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 169MB/s New Data Upload : 100%|█████████▉| 3.92GB / 3.92GB, 169MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  Processing Files (2 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 156MB/s New Data Upload : 100%|█████████▉| 3.92GB / 3.92GB, 156MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  Processing Files (2 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 144MB/s New Data Upload : 100%|██████████| 3.92GB / 3.92GB, 144MB/s  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  Processing Files (2 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 122MB/s New Data Upload : 100%|██████████| 3.92GB / 3.92GB, 122MB/s ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB ***** train metrics ***** epoch = 4.0 total_flos = 742719798GF train_loss = 0.6604 train_runtime = 0:52:35.39 train_samples_per_second = 38.03 train_steps_per_second = 0.298 Figure saved at: saves/qwen3_1.7b/coding/fft/training_loss.png [WARNING|2026-04-29 17:59:55] llamafactory.extras.ploting:149 >> No metric eval_loss to plot. [WARNING|2026-04-29 17:59:55] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot. [INFO|trainer.py:3797] 2026-04-29 17:59:57,214 >> Saving model checkpoint to saves/qwen3_1.7b/coding/fft [INFO|configuration_utils.py:432] 2026-04-29 17:59:57,221 >> Configuration saved in saves/qwen3_1.7b/coding/fft/config.json [INFO|configuration_utils.py:803] 2026-04-29 17:59:57,226 >> Configuration saved in saves/qwen3_1.7b/coding/fft/generation_config.json Writing model shards: 0%| | 0/1 [00:00> Model weights saved in saves/qwen3_1.7b/coding/fft/model.safetensors [INFO|tokenization_utils_base.py:3224] 2026-04-29 18:00:04,084 >> chat template saved in saves/qwen3_1.7b/coding/fft/chat_template.jinja [INFO|tokenization_utils_base.py:2078] 2026-04-29 18:00:04,090 >> tokenizer config file saved in saves/qwen3_1.7b/coding/fft/tokenizer_config.json [INFO|modelcard.py:266] 2026-04-29 18:00:04,389 >> Dropping the following result as it does not have all the necessary fields: {'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}} Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 2%|▏ | 72.0MB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 2%|▏ | 72.0MB / 4.06GB  Processing Files (2 / 3) : 2%|▏ | 83.4MB / 4.07GB, ???B/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 5%|▍ | 184MB / 4.06GB  Processing Files (2 / 3) : 5%|▍ | 195MB / 4.07GB, 560MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 8%|▊ | 336MB / 4.06GB  Processing Files (2 / 3) : 9%|▊ | 347MB / 4.07GB, 661MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 12%|█▏ | 496MB / 4.06GB  Processing Files (2 / 3) : 12%|█▏ | 507MB / 4.07GB, 707MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 16%|█▌ | 656MB / 4.06GB  Processing Files (2 / 3) : 16%|█▋ | 667MB / 4.07GB, 731MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 20%|██ | 824MB / 4.06GB  Processing Files (2 / 3) : 21%|██ | 835MB / 4.07GB, 753MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 24%|██▍ | 992MB / 4.06GB  Processing Files (2 / 3) : 25%|██▍ | 1.00GB / 4.07GB, 767MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 28%|██▊ | 1.14GB / 4.06GB  Processing Files (2 / 3) : 28%|██▊ | 1.15GB / 4.07GB, 762MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 31%|███▏ | 1.28GB / 4.06GB  Processing Files (2 / 3) : 32%|███▏ | 1.29GB / 4.07GB, 754MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 36%|███▌ | 1.45GB / 4.06GB  Processing Files (2 / 3) : 36%|███▌ | 1.46GB / 4.07GB, 763MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 40%|███▉ | 1.61GB / 4.06GB  Processing Files (2 / 3) : 40%|███▉ | 1.63GB / 4.07GB, 771MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 44%|████▍ | 1.78GB / 4.06GB  Processing Files (2 / 3) : 44%|████▍ | 1.79GB / 4.07GB, 778MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 48%|████▊ | 1.95GB / 4.06GB  Processing Files (2 / 3) : 48%|████▊ | 1.96GB / 4.07GB, 783MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 52%|█████▏ | 2.12GB / 4.06GB  Processing Files (2 / 3) : 52%|█████▏ | 2.13GB / 4.07GB, 787MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 56%|█████▋ | 2.29GB / 4.06GB  Processing Files (2 / 3) : 56%|█████▋ | 2.30GB / 4.07GB, 791MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 60%|██████ | 2.45GB / 4.06GB  Processing Files (2 / 3) : 61%|██████ | 2.47GB / 4.07GB, 794MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 65%|██████▍ | 2.62GB / 4.06GB  Processing Files (2 / 3) : 65%|██████▍ | 2.63GB / 4.07GB, 797MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 68%|██████▊ | 2.78GB / 4.06GB  Processing Files (2 / 3) : 69%|██████▊ | 2.79GB / 4.07GB, 797MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 73%|███████▎ | 2.96GB / 4.06GB  Processing Files (2 / 3) : 73%|███████▎ | 2.97GB / 4.07GB, 802MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 77%|███████▋ | 3.13GB / 4.06GB  Processing Files (2 / 3) : 77%|███████▋ | 3.14GB / 4.07GB, 804MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 81%|████████ | 3.29GB / 4.06GB  Processing Files (2 / 3) : 81%|████████ | 3.31GB / 4.07GB, 806MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 85%|████████▌ | 3.46GB / 4.06GB  Processing Files (2 / 3) : 85%|████████▌ | 3.47GB / 4.07GB, 807MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 89%|████████▉ | 3.63GB / 4.06GB  Processing Files (2 / 3) : 89%|████████▉ | 3.64GB / 4.07GB, 809MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 93%|█████████▎| 3.80GB / 4.06GB  Processing Files (2 / 3) : 93%|█████████▎| 3.81GB / 4.07GB, 810MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 98%|█████████▊| 3.97GB / 4.06GB  Processing Files (2 / 3) : 98%|█████████▊| 3.98GB / 4.07GB, 811MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  Processing Files (2 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 797MB/s New Data Upload : 43%|████▎ | 4.32MB / 10.1MB, 864kB/s  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  Processing Files (2 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 713MB/s New Data Upload : 98%|█████████▊| 9.87MB / 10.1MB, 1.76MB/s  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB  Processing Files (3 / 3) : 100%|██████████| 4.07GB / 4.07GB, 665MB/s New Data Upload : 100%|██████████| 10.1MB / 10.1MB, 1.68MB/s  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB  ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB  ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB  ...ing/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB  Processing Files (3 / 3) : 100%|██████████| 4.07GB / 4.07GB, 570MB/s New Data Upload : 100%|██████████| 10.1MB / 10.1MB, 1.44MB/s ...ing/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB ...coding/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB ...ing/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB