KKHYA/qwen3-1.7b-fft-math
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
634c2e8d5761d82673df1aa258830d9f194759a2
qwen3-1.7b-fft-math/qwen3_1.7b_math_fft_20260429_154203.log
ModelHub XC 634c2e8d57 初始化项目，由ModelHub XC社区提供模型 
Model: KKHYA/qwen3-1.7b-fft-math
Source: Original Platform
2026-06-16 05:05:17 +08:00

2844 lines
979 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

==== STARTING EXPERIMENT: qwen3_1.7b_math_fft ====
Log File: saves/qwen3_1.7b/math/fft/qwen3_1.7b_math_fft_20260429_154203.log
HF Hub: https://huggingface.co/KKHYA/qwen3-1.7b-fft-math
Timestamp: 2026-04-29 15:42:03
=====================================
[INFO|2026-04-29 15:42:18] llamafactory.launcher:144 >> Initializing 8 distributed tasks at: 127.0.0.1:49311
W0429 15:42:19.791000 2470856 site-packages/torch/distributed/run.py:852]
W0429 15:42:19.791000 2470856 site-packages/torch/distributed/run.py:852] *****************************************
W0429 15:42:19.791000 2470856 site-packages/torch/distributed/run.py:852] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
W0429 15:42:19.791000 2470856 site-packages/torch/distributed/run.py:852] *****************************************
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
df: /root/.triton/autotune: No such file or directory
df: df: /root/.triton/autotune/root/.triton/autotunedf: /root/.triton/autotunedf: df: /root/.triton/autotune/root/.triton/autotunedf: /root/.triton/autotune: No such file or directory
: No such file or directory
: No such file or directory
: No such file or directory
: No such file or directory
: No such file or directory
[W429 15:42:34.641839785 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W429 15:42:34.641839403 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W429 15:42:34.648204081 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W429 15:42:34.656834411 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W429 15:42:34.704861810 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W429 15:42:34.738333250 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W429 15:42:34.757560566 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[W429 15:42:34.774695972 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NCCL version 2.28.9+cuda13.0
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NCCL version 2.28.9+cuda13.0
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NCCL version 2.28.9+cuda13.0
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NCCL version 2.28.9+cuda13.0
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NCCL version 2.28.9+cuda13.0
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NCCL version 2.28.9+cuda13.0
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NCCL version 2.28.9+cuda13.0
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NCCL version 2.28.9+cuda13.0
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ncclCommInitRankConfig comm 0x55e36d363080 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 commId 0xbdc52b61bae6206d - Init START
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ncclCommInitRankConfig comm 0x56071a5c6320 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 commId 0xbdc52b61bae6206d - Init START
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ncclCommInitRankConfig comm 0x558cf7486a10 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 commId 0xbdc52b61bae6206d - Init START
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ncclCommInitRankConfig comm 0x55ffbe0f7c80 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 commId 0xbdc52b61bae6206d - Init START
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ncclCommInitRankConfig comm 0x5647078795e0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 commId 0xbdc52b61bae6206d - Init START
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ncclCommInitRankConfig comm 0x5612d38dcbf0 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 commId 0xbdc52b61bae6206d - Init START
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ncclCommInitRankConfig comm 0x5602809d0750 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 commId 0xbdc52b61bae6206d - Init START
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ncclCommInitRankConfig comm 0x55c0201d0a60 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 commId 0xbdc52b61bae6206d - Init START
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Bootstrap timings total 0.008871 (create 0.000085, send 0.000076, recv 0.007974, ring 0.000343, delay 0.000001)
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Bootstrap timings total 0.001031 (create 0.000050, send 0.000072, recv 0.000209, ring 0.000343, delay 0.000001)
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Bootstrap timings total 0.001899 (create 0.000034, send 0.000071, recv 0.000096, ring 0.000324, delay 0.000001)
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Bootstrap timings total 0.005427 (create 0.000034, send 0.000064, recv 0.000224, ring 0.004700, delay 0.000001)
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Bootstrap timings total 0.002049 (create 0.000036, send 0.000068, recv 0.001328, ring 0.000239, delay 0.000001)
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Bootstrap timings total 0.029934 (create 0.000045, send 0.000090, recv 0.024643, ring 0.004705, delay 0.000001)
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Bootstrap timings total 0.000897 (create 0.000036, send 0.000064, recv 0.000252, ring 0.000131, delay 0.000001)
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Bootstrap timings total 0.010645 (create 0.000037, send 0.000077, recv 0.000107, ring 0.000122, delay 0.000001)
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 3 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NVLS multicast support is not available on dev 3 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 4 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NVLS multicast support is not available on dev 4 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 2 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NVLS multicast support is not available on dev 2 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 1 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NVLS multicast support is not available on dev 1 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 5 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NVLS multicast support is not available on dev 5 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 6 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NVLS multicast support is not available on dev 6 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NVLS multicast support is not available on dev 0 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 7 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NVLS multicast support is not available on dev 7 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO comm 0x5612d38dcbf0 rank 3 nRanks 8 nNodes 1 localRanks 8 localRank 3 MNNVL 0
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO comm 0x55c0201d0a60 rank 5 nRanks 8 nNodes 1 localRanks 8 localRank 5 MNNVL 0
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO comm 0x5647078795e0 rank 4 nRanks 8 nNodes 1 localRanks 8 localRank 4 MNNVL 0
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO comm 0x56071a5c6320 rank 6 nRanks 8 nNodes 1 localRanks 8 localRank 6 MNNVL 0
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO comm 0x558cf7486a10 rank 1 nRanks 8 nNodes 1 localRanks 8 localRank 1 MNNVL 0
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO comm 0x55e36d363080 rank 7 nRanks 8 nNodes 1 localRanks 8 localRank 7 MNNVL 0
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO comm 0x5602809d0750 rank 2 nRanks 8 nNodes 1 localRanks 8 localRank 2 MNNVL 0
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO comm 0x55ffbe0f7c80 rank 0 nRanks 8 nNodes 1 localRanks 8 localRank 0 MNNVL 0
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 00/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 01/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 02/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 03/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 04/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 05/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 06/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 07/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 08/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 09/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 10/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 11/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 12/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 13/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 14/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 15/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 16/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 17/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 18/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 19/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 20/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 21/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 22/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 23/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470948:2471411 [0] NCCL INFO [Proxy Service UDS] Device 1 CPU core 49
ywang29-p4d-debug-worker-0:2470948:2471410 [0] NCCL INFO [Proxy Service] Device 1 CPU core 23
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470953:2471412 [0] NCCL INFO [Proxy Service] Device 5 CPU core 80
ywang29-p4d-debug-worker-0:2470953:2471413 [0] NCCL INFO [Proxy Service UDS] Device 5 CPU core 86
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470947:2471414 [0] NCCL INFO [Proxy Service] Device 0 CPU core 15
ywang29-p4d-debug-worker-0:2470947:2471415 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 65
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470949:2471416 [0] NCCL INFO [Proxy Service] Device 2 CPU core 9
ywang29-p4d-debug-worker-0:2470949:2471417 [0] NCCL INFO [Proxy Service UDS] Device 2 CPU core 9
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470952:2471418 [0] NCCL INFO [Proxy Service] Device 4 CPU core 38
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
ywang29-p4d-debug-worker-0:2470952:2471419 [0] NCCL INFO [Proxy Service UDS] Device 4 CPU core 39
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470955:2471421 [0] NCCL INFO [Proxy Service UDS] Device 7 CPU core 34
ywang29-p4d-debug-worker-0:2470955:2471420 [0] NCCL INFO [Proxy Service] Device 7 CPU core 82
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470954:2471422 [0] NCCL INFO [Proxy Service] Device 6 CPU core 31
ywang29-p4d-debug-worker-0:2470954:2471423 [0] NCCL INFO [Proxy Service UDS] Device 6 CPU core 31
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470951:2471424 [0] NCCL INFO [Proxy Service] Device 3 CPU core 4
ywang29-p4d-debug-worker-0:2470951:2471425 [0] NCCL INFO [Proxy Service UDS] Device 3 CPU core 3
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO CC Off, workFifoBytes 1048576
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ncclCommInitRankConfig comm 0x5602809d0750 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 commId 0xbdc52b61bae6206d - Init COMPLETE
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ncclCommInitRankConfig comm 0x56071a5c6320 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 commId 0xbdc52b61bae6206d - Init COMPLETE
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ncclCommInitRankConfig comm 0x5647078795e0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 commId 0xbdc52b61bae6206d - Init COMPLETE
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ncclCommInitRankConfig comm 0x55ffbe0f7c80 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 commId 0xbdc52b61bae6206d - Init COMPLETE
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Init timings - ncclCommInitRankConfig: rank 2 nranks 8 total 0.52 (kernels 0.17, alloc 0.22, bootstrap 0.00, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.06, rest 0.04)
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Init timings - ncclCommInitRankConfig: rank 4 nranks 8 total 0.53 (kernels 0.18, alloc 0.22, bootstrap 0.00, allgathers 0.00, topo 0.03, graphs 0.00, connections 0.06, rest 0.04)
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Init timings - ncclCommInitRankConfig: rank 6 nranks 8 total 0.53 (kernels 0.18, alloc 0.22, bootstrap 0.01, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.05, rest 0.04)
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Init timings - ncclCommInitRankConfig: rank 0 nranks 8 total 0.53 (kernels 0.18, alloc 0.22, bootstrap 0.01, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.06, rest 0.03)
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ncclCommInitRankConfig comm 0x558cf7486a10 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 commId 0xbdc52b61bae6206d - Init COMPLETE
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ncclCommInitRankConfig comm 0x55e36d363080 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 commId 0xbdc52b61bae6206d - Init COMPLETE
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ncclCommInitRankConfig comm 0x55c0201d0a60 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 commId 0xbdc52b61bae6206d - Init COMPLETE
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Init timings - ncclCommInitRankConfig: rank 1 nranks 8 total 0.53 (kernels 0.18, alloc 0.22, bootstrap 0.01, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.08, rest 0.02)
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Init timings - ncclCommInitRankConfig: rank 7 nranks 8 total 0.53 (kernels 0.18, alloc 0.20, bootstrap 0.03, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.06, rest 0.04)
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Init timings - ncclCommInitRankConfig: rank 5 nranks 8 total 0.53 (kernels 0.18, alloc 0.22, bootstrap 0.00, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.08, rest 0.02)
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ncclCommInitRankConfig comm 0x5612d38dcbf0 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 commId 0xbdc52b61bae6206d - Init COMPLETE
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Init timings - ncclCommInitRankConfig: rank 3 nranks 8 total 0.53 (kernels 0.18, alloc 0.22, bootstrap 0.00, allgathers 0.00, topo 0.03, graphs 0.00, connections 0.06, rest 0.04)
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 5, world size: 8, device: cuda:5, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 4, world size: 8, device: cuda:4, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 6, world size: 8, device: cuda:6, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 7, world size: 8, device: cuda:7, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 1, world size: 8, device: cuda:1, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 2, world size: 8, device: cuda:2, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 3, world size: 8, device: cuda:3, distributed training: True, compute dtype: torch.bfloat16
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 0, world size: 8, device: cuda:0, distributed training: True, compute dtype: torch.bfloat16
[INFO|configuration_utils.py:670] 2026-04-29 15:42:35,891 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/config.json
[INFO|configuration_utils.py:742] 2026-04-29 15:42:35,894 >> Model config Qwen3Config {
"architectures": [
"Qwen3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 151643,
"dtype": "bfloat16",
"eos_token_id": 151645,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 2048,
"initializer_range": 0.02,
"intermediate_size": 6144,
"layer_types": [
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention"
],
"masked_layers": null,
"max_position_embeddings": 40960,
"max_window_layers": 28,
"model_type": "qwen3",
"num_attention_heads": 16,
"num_hidden_layers": 28,
"num_key_value_heads": 8,
"pad_token_id": null,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"rope_theta": 1000000,
"rope_type": "default"
},
"sliding_window": null,
"sparsity_attn": null,
"sparsity_mlp": null,
"subnet_mode": null,
"subnet_type": null,
"threshold_attn": null,
"threshold_mlp": null,
"tie_word_embeddings": true,
"transformers_version": "5.2.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 151936
}
[INFO|configuration_utils.py:670] 2026-04-29 15:42:39,796 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/config.json
[INFO|configuration_utils.py:742] 2026-04-29 15:42:39,797 >> Model config Qwen3Config {
"architectures": [
"Qwen3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 151643,
"dtype": "bfloat16",
"eos_token_id": 151645,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 2048,
"initializer_range": 0.02,
"intermediate_size": 6144,
"layer_types": [
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention"
],
"masked_layers": null,
"max_position_embeddings": 40960,
"max_window_layers": 28,
"model_type": "qwen3",
"num_attention_heads": 16,
"num_hidden_layers": 28,
"num_key_value_heads": 8,
"pad_token_id": null,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"rope_theta": 1000000,
"rope_type": "default"
},
"sliding_window": null,
"sparsity_attn": null,
"sparsity_mlp": null,
"subnet_mode": null,
"subnet_type": null,
"threshold_attn": null,
"threshold_mlp": null,
"tie_word_embeddings": true,
"transformers_version": "5.2.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 151936
}
[INFO|configuration_utils.py:670] 2026-04-29 15:42:39,906 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/config.json
[INFO|configuration_utils.py:742] 2026-04-29 15:42:39,907 >> Model config Qwen3Config {
"architectures": [
"Qwen3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 151643,
"dtype": "bfloat16",
"eos_token_id": 151645,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 2048,
"initializer_range": 0.02,
"intermediate_size": 6144,
"layer_types": [
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention"
],
"masked_layers": null,
"max_position_embeddings": 40960,
"max_window_layers": 28,
"model_type": "qwen3",
"num_attention_heads": 16,
"num_hidden_layers": 28,
"num_key_value_heads": 8,
"pad_token_id": null,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"rope_theta": 1000000,
"rope_type": "default"
},
"sliding_window": null,
"sparsity_attn": null,
"sparsity_mlp": null,
"subnet_mode": null,
"subnet_type": null,
"threshold_attn": null,
"threshold_mlp": null,
"tie_word_embeddings": true,
"transformers_version": "5.2.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 151936
}
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 16/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 17/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 18/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 19/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 20/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 21/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 22/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 23/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 00/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 01/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 02/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 03/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 04/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 05/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 06/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 07/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 08/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 09/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 10/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 11/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 12/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 13/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 14/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 15/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 16/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 17/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 18/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 19/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 20/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 21/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 22/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 23/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 16/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 17/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 18/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 19/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 20/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 21/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 22/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 23/0 : 1[1] -> 2[2] via P2P/CUMEM/read
[INFO|2026-04-29 15:42:41] llamafactory.data.loader:144 >> Loading dataset KKHYA/metamath_converted...
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 16/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 17/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 18/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 19/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 20/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 21/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 22/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 23/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 00/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 01/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 02/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 03/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 04/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 05/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 06/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 07/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 08/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 09/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 10/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 11/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 12/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 13/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 14/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 15/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 16/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 17/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 18/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 19/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 20/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 21/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 22/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 23/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 16/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 17/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 18/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 19/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 20/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 21/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 22/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 23/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 16/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 17/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 18/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 19/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 20/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 21/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 22/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 23/0 : 2[2] -> 3[3] via P2P/CUMEM/read
Repo card metadata block was not found. Setting CardData to empty.
Setting num_proc from 16 to 2 for the train split as it only contains 2 shards.
Generating train split: 0%| | 0/394996 [00:00<?, ? examples/s]
Generating train split: 1%| | 3000/394996 [00:00<00:47, 8307.48 examples/s]
Generating train split: 12%|█▏ | 47000/394996 [00:00<00:02, 129958.07 examples/s]
Generating train split: 24%|██▎ | 93000/394996 [00:00<00:01, 215888.84 examples/s]
Generating train split: 37%|███▋ | 147000/394996 [00:00<00:00, 305992.13 examples/s]
Generating train split: 50%|█████ | 199000/394996 [00:00<00:00, 362914.23 examples/s]
Generating train split: 64%|██████▍ | 252000/394996 [00:00<00:00, 402970.92 examples/s]
Generating train split: 78%|███████▊ | 309000/394996 [00:00<00:00, 446553.16 examples/s]
Generating train split: 92%|█████████▏| 365000/394996 [00:01<00:00, 465307.36 examples/s]
Generating train split: 100%|██████████| 394996/394996 [00:01<00:00, 315529.42 examples/s]
[INFO|2026-04-29 15:42:50] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset KKHYA/metamath_converted.
Converting format of dataset (num_proc=16): 0%| | 0/10000 [00:00<?, ? examples/s]
Converting format of dataset (num_proc=16): 5%|▌ | 527/10000 [00:00<00:08, 1118.11 examples/s]
Converting format of dataset (num_proc=16): 100%|██████████| 10000/10000 [00:00<00:00, 14140.08 examples/s]
[INFO|2026-04-29 15:42:50] llamafactory.data.loader:144 >> Loading dataset KKHYA/numinamath_tir_converted...
Repo card metadata block was not found. Setting CardData to empty.
Setting num_proc from 16 back to 1 for the train split to disable multiprocessing as it only contains one shard.
Generating train split: 0%| | 0/72441 [00:00<?, ? examples/s]
Generating train split: 8%|▊ | 6000/72441 [00:00<00:01, 52916.63 examples/s]
Generating train split: 22%|██▏ | 16000/72441 [00:00<00:00, 75880.62 examples/s]
Generating train split: 39%|███▊ | 28000/72441 [00:00<00:00, 93879.61 examples/s]
Generating train split: 54%|█████▍ | 39000/72441 [00:00<00:00, 95367.14 examples/s]
Generating train split: 68%|██████▊ | 49000/72441 [00:00<00:00, 95909.35 examples/s]
Generating train split: 84%|████████▍ | 61000/72441 [00:00<00:00, 102761.15 examples/s]
Generating train split: 100%|██████████| 72441/72441 [00:00<00:00, 103411.87 examples/s]
Generating train split: 100%|██████████| 72441/72441 [00:00<00:00, 96274.33 examples/s]
Setting num_proc from 16 back to 1 for the test split to disable multiprocessing as it only contains one shard.
Generating test split: 0%| | 0/99 [00:00<?, ? examples/s]
Generating test split: 100%|██████████| 99/99 [00:00<00:00, 29883.85 examples/s]
[INFO|2026-04-29 15:43:05] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset KKHYA/numinamath_tir_converted.
Converting format of dataset (num_proc=16): 0%| | 0/10000 [00:00<?, ? examples/s]
Converting format of dataset (num_proc=16): 4%|▍ | 440/10000 [00:00<00:10, 920.19 examples/s]
Converting format of dataset (num_proc=16): 100%|██████████| 10000/10000 [00:00<00:00, 14552.33 examples/s]
[INFO|2026-04-29 15:43:05] llamafactory.data.loader:144 >> Loading dataset KKHYA/numinamath_cot_converted...
Repo card metadata block was not found. Setting CardData to empty.
Setting num_proc from 16 to 5 for the train split as it only contains 5 shards.
Generating train split: 0%| | 0/859493 [00:00<?, ? examples/s]
Generating train split: 0%| | 1000/859493 [00:00<03:57, 3608.37 examples/s]
Generating train split: 6%|▌ | 48000/859493 [00:00<00:05, 159542.65 examples/s]
Generating train split: 14%|█▍ | 124000/859493 [00:00<00:02, 332328.64 examples/s]
Generating train split: 24%|██▍ | 205000/859493 [00:00<00:01, 464546.22 examples/s]
Generating train split: 33%|███▎ | 285000/859493 [00:00<00:01, 554177.55 examples/s]
Generating train split: 42%|████▏ | 359000/859493 [00:00<00:00, 576642.19 examples/s]
Generating train split: 52%|█████▏ | 443000/859493 [00:00<00:00, 637290.61 examples/s]
Generating train split: 60%|██████ | 519000/859493 [00:01<00:00, 661365.99 examples/s]
Generating train split: 69%|██████▉ | 592000/859493 [00:01<00:00, 655939.40 examples/s]
Generating train split: 79%|███████▊ | 675000/859493 [00:01<00:00, 669802.57 examples/s]
Generating train split: 87%|████████▋ | 751000/859493 [00:01<00:00, 682799.05 examples/s]
Generating train split: 96%|█████████▋| 827898/859493 [00:01<00:00, 705069.90 examples/s]
Generating train split: 100%|██████████| 859493/859493 [00:01<00:00, 532761.42 examples/s]
Setting num_proc from 16 back to 1 for the test split to disable multiprocessing as it only contains one shard.
Generating test split: 0%| | 0/100 [00:00<?, ? examples/s]
Generating test split: 100%|██████████| 100/100 [00:00<00:00, 11119.28 examples/s]
[INFO|2026-04-29 15:43:22] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset KKHYA/numinamath_cot_converted.
Converting format of dataset (num_proc=16): 0%| | 0/10000 [00:00<?, ? examples/s]
Converting format of dataset (num_proc=16): 5%|▌ | 520/10000 [00:00<00:10, 869.91 examples/s]
Converting format of dataset (num_proc=16): 100%|██████████| 10000/10000 [00:00<00:00, 12240.88 examples/s]
[INFO|2026-04-29 15:43:23] llamafactory.data.loader:144 >> Loading dataset allenai/tulu-3-sft-personas-math...
Setting num_proc from 16 to 2 for the train split as it only contains 2 shards.
Generating train split: 0%| | 0/149960 [00:00<?, ? examples/s]
Generating train split: 1%| | 1000/149960 [00:00<00:22, 6515.30 examples/s]
Generating train split: 11%|█▏ | 17000/149960 [00:00<00:01, 76378.12 examples/s]
Generating train split: 22%|██▏ | 33000/149960 [00:00<00:01, 107818.99 examples/s]
Generating train split: 34%|███▍ | 51000/149960 [00:00<00:00, 129688.58 examples/s]
Generating train split: 44%|████▍ | 66000/149960 [00:00<00:00, 133412.83 examples/s]
Generating train split: 56%|█████▌ | 84000/149960 [00:00<00:00, 139172.97 examples/s]
Generating train split: 69%|██████▉ | 104000/149960 [00:00<00:00, 154551.66 examples/s]
Generating train split: 81%|████████ | 121000/149960 [00:00<00:00, 157547.03 examples/s]
Generating train split: 93%|█████████▎| 140000/149960 [00:01<00:00, 162863.39 examples/s]
Generating train split: 100%|██████████| 149960/149960 [00:01<00:00, 126921.10 examples/s]
[INFO|2026-04-29 15:43:31] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset allenai/tulu-3-sft-personas-math.
Converting format of dataset (num_proc=16): 0%| | 0/10000 [00:00<?, ? examples/s]
Converting format of dataset (num_proc=16): 5%|▌ | 529/10000 [00:00<00:10, 911.77 examples/s]
Converting format of dataset (num_proc=16): 100%|██████████| 10000/10000 [00:00<00:00, 12526.85 examples/s]
[INFO|2026-04-29 15:43:32] llamafactory.data.loader:144 >> Loading dataset allenai/tulu-3-sft-personas-math-grade...
Setting num_proc from 16 back to 1 for the train split to disable multiprocessing as it only contains one shard.
Generating train split: 0%| | 0/49980 [00:00<?, ? examples/s]
Generating train split: 30%|███ | 15000/49980 [00:00<00:00, 140747.51 examples/s]
Generating train split: 72%|███████▏ | 36000/49980 [00:00<00:00, 175603.03 examples/s]
Generating train split: 100%|██████████| 49980/49980 [00:00<00:00, 182907.76 examples/s]
[INFO|2026-04-29 15:43:36] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset allenai/tulu-3-sft-personas-math-grade.
Converting format of dataset (num_proc=16): 0%| | 0/10000 [00:00<?, ? examples/s]
Converting format of dataset (num_proc=16): 5%|▍ | 459/10000 [00:00<00:12, 751.02 examples/s]
Converting format of dataset (num_proc=16): 100%|██████████| 10000/10000 [00:00<00:00, 12057.75 examples/s]
[INFO|2026-04-29 15:43:37] llamafactory.data.loader:144 >> Loading dataset allenai/tulu-3-sft-personas-algebra...
Setting num_proc from 16 back to 1 for the train split to disable multiprocessing as it only contains one shard.
Generating train split: 0%| | 0/20000 [00:00<?, ? examples/s]
Generating train split: 40%|████ | 8000/20000 [00:00<00:00, 74061.29 examples/s]
Generating train split: 100%|██████████| 20000/20000 [00:00<00:00, 100731.63 examples/s]
[INFO|2026-04-29 15:43:41] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset allenai/tulu-3-sft-personas-algebra.
Converting format of dataset (num_proc=16): 0%| | 0/10000 [00:00<?, ? examples/s]
Converting format of dataset (num_proc=16): 6%|▌ | 567/10000 [00:00<00:09, 973.12 examples/s]
Converting format of dataset (num_proc=16): 100%|██████████| 10000/10000 [00:00<00:00, 12467.30 examples/s]
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 16/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 17/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 18/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 19/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 20/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 21/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 22/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 23/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Running tokenizer on dataset (num_proc=16): 0%| | 0/60000 [00:00<?, ? examples/s]Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Repo card metadata block was not found. Setting CardData to empty.
Running tokenizer on dataset (num_proc=16): 2%|▏ | 1000/60000 [00:03<02:59, 329.09 examples/s]
Running tokenizer on dataset (num_proc=16): 3%|▎ | 2000/60000 [00:03<01:37, 597.68 examples/s]
Running tokenizer on dataset (num_proc=16): 5%|▌ | 3000/60000 [00:04<01:09, 818.81 examples/s]
Running tokenizer on dataset (num_proc=16): 8%|▊ | 4750/60000 [00:04<00:39, 1387.60 examples/s]
Running tokenizer on dataset (num_proc=16): 10%|▉ | 5750/60000 [00:05<00:28, 1874.67 examples/s]
Running tokenizer on dataset (num_proc=16): 11%|█▏ | 6750/60000 [00:05<00:30, 1722.68 examples/s]
Running tokenizer on dataset (num_proc=16): 14%|█▍ | 8500/60000 [00:06<00:22, 2288.73 examples/s]
Running tokenizer on dataset (num_proc=16): 16%|█▌ | 9500/60000 [00:06<00:19, 2590.57 examples/s]
Running tokenizer on dataset (num_proc=16): 18%|█▊ | 10500/60000 [00:07<00:31, 1584.14 examples/s]
Running tokenizer on dataset (num_proc=16): 19%|█▉ | 11500/60000 [00:08<00:26, 1803.92 examples/s]
Running tokenizer on dataset (num_proc=16): 20%|██ | 12250/60000 [00:09<00:36, 1302.72 examples/s]
Running tokenizer on dataset (num_proc=16): 22%|██▏ | 13250/60000 [00:09<00:28, 1660.18 examples/s]
Running tokenizer on dataset (num_proc=16): 24%|██▍ | 14250/60000 [00:09<00:25, 1785.82 examples/s]
Running tokenizer on dataset (num_proc=16): 25%|██▌ | 15250/60000 [00:10<00:27, 1625.85 examples/s]
Running tokenizer on dataset (num_proc=16): 27%|██▋ | 16250/60000 [00:11<00:26, 1626.96 examples/s]
Running tokenizer on dataset (num_proc=16): 29%|██▉ | 17250/60000 [00:11<00:20, 2123.23 examples/s]
Running tokenizer on dataset (num_proc=16): 30%|███ | 18250/60000 [00:11<00:18, 2251.50 examples/s]
Running tokenizer on dataset (num_proc=16): 32%|███▏ | 19250/60000 [00:12<00:16, 2456.03 examples/s]
Running tokenizer on dataset (num_proc=16): 34%|███▍ | 20250/60000 [00:12<00:18, 2172.46 examples/s]
Running tokenizer on dataset (num_proc=16): 37%|███▋ | 22250/60000 [00:13<00:12, 2923.16 examples/s]
Running tokenizer on dataset (num_proc=16): 40%|████ | 24000/60000 [00:13<00:10, 3422.65 examples/s]
Running tokenizer on dataset (num_proc=16): 42%|████▏ | 25000/60000 [00:13<00:12, 2897.39 examples/s]
Running tokenizer on dataset (num_proc=16): 45%|████▍ | 26750/60000 [00:14<00:10, 3200.26 examples/s]
Running tokenizer on dataset (num_proc=16): 46%|████▌ | 27500/60000 [00:14<00:09, 3545.46 examples/s]
Running tokenizer on dataset (num_proc=16): 47%|████▋ | 28250/60000 [00:14<00:10, 2940.95 examples/s]
Running tokenizer on dataset (num_proc=16): 49%|████▉ | 29250/60000 [00:15<00:09, 3161.81 examples/s]
Running tokenizer on dataset (num_proc=16): 50%|█████ | 30250/60000 [00:15<00:12, 2332.51 examples/s]
Running tokenizer on dataset (num_proc=16): 52%|█████▏ | 31000/60000 [00:16<00:12, 2399.25 examples/s]
Running tokenizer on dataset (num_proc=16): 53%|█████▎ | 32000/60000 [00:17<00:16, 1698.81 examples/s]
Running tokenizer on dataset (num_proc=16): 55%|█████▌ | 33000/60000 [00:18<00:18, 1465.71 examples/s]
Running tokenizer on dataset (num_proc=16): 57%|█████▋ | 34000/60000 [00:18<00:15, 1641.74 examples/s]
Running tokenizer on dataset (num_proc=16): 58%|█████▊ | 35000/60000 [00:19<00:15, 1641.56 examples/s]
Running tokenizer on dataset (num_proc=16): 60%|██████ | 36000/60000 [00:19<00:11, 2028.31 examples/s]
Running tokenizer on dataset (num_proc=16): 63%|██████▎ | 38000/60000 [00:20<00:11, 1898.06 examples/s]
Running tokenizer on dataset (num_proc=16): 65%|██████▌ | 39000/60000 [00:20<00:08, 2370.35 examples/s]
Running tokenizer on dataset (num_proc=16): 68%|██████▊ | 41000/60000 [00:20<00:06, 3153.95 examples/s]
Running tokenizer on dataset (num_proc=16): 70%|██████▉ | 41750/60000 [00:21<00:07, 2406.48 examples/s]
Running tokenizer on dataset (num_proc=16): 71%|███████▏ | 42750/60000 [00:21<00:06, 2521.33 examples/s]
Running tokenizer on dataset (num_proc=16): 75%|███████▍ | 44750/60000 [00:22<00:05, 2998.25 examples/s]
Running tokenizer on dataset (num_proc=16): 76%|███████▌ | 45500/60000 [00:22<00:05, 2795.64 examples/s]
Running tokenizer on dataset (num_proc=16): 78%|███████▊ | 46500/60000 [00:23<00:05, 2659.38 examples/s]
Running tokenizer on dataset (num_proc=16): 79%|███████▉ | 47500/60000 [00:23<00:04, 3028.73 examples/s]
Running tokenizer on dataset (num_proc=16): 81%|████████ | 48500/60000 [00:23<00:03, 2909.60 examples/s]
Running tokenizer on dataset (num_proc=16): 82%|████████▎ | 49500/60000 [00:24<00:03, 2941.81 examples/s]
Running tokenizer on dataset (num_proc=16): 84%|████████▍ | 50250/60000 [00:24<00:04, 2075.51 examples/s]
Running tokenizer on dataset (num_proc=16): 85%|████████▌ | 51250/60000 [00:24<00:03, 2676.86 examples/s]
Running tokenizer on dataset (num_proc=16): 87%|████████▋ | 52000/60000 [00:25<00:02, 3023.88 examples/s]
Running tokenizer on dataset (num_proc=16): 88%|████████▊ | 53000/60000 [00:25<00:02, 2461.22 examples/s]
Running tokenizer on dataset (num_proc=16): 90%|█████████ | 54000/60000 [00:25<00:01, 3157.23 examples/s]
Running tokenizer on dataset (num_proc=16): 91%|█████████▏| 54750/60000 [00:26<00:01, 2711.01 examples/s]
Running tokenizer on dataset (num_proc=16): 93%|█████████▎| 55750/60000 [00:27<00:02, 1587.03 examples/s]
Running tokenizer on dataset (num_proc=16): 94%|█████████▍| 56500/60000 [00:27<00:01, 1820.15 examples/s]
Running tokenizer on dataset (num_proc=16): 96%|█████████▌| 57500/60000 [00:27<00:01, 2027.74 examples/s]
Running tokenizer on dataset (num_proc=16): 97%|█████████▋| 58250/60000 [00:29<00:01, 1051.85 examples/s]
Running tokenizer on dataset (num_proc=16): 100%|██████████| 60000/60000 [00:31<00:00, 1037.92 examples/s]
Running tokenizer on dataset (num_proc=16): 100%|██████████| 60000/60000 [00:31<00:00, 1906.46 examples/s]
training example:
input_ids:
[151644, 872, 198, 67909, 702, 2326, 3039, 438, 1753, 3220, 438, 21475, 13, 2379, 614, 264, 10856, 2790, 315, 400, 22, 21, 13, 2585, 1753, 3220, 1558, 19685, 614, 30, 151645, 198, 151644, 77091, 198, 10061, 594, 9658, 21475, 702, 856, 11192, 624, 12209, 19685, 702, 220, 18, 87, 11192, 624, 785, 2790, 3311, 315, 3220, 807, 614, 374, 856, 488, 220, 18, 87, 284, 220, 19, 87, 198, 1654, 1414, 429, 279, 2790, 3311, 315, 3220, 807, 614, 374, 400, 22, 21, 11, 773, 220, 19, 87, 284, 400, 22, 21, 198, 12509, 6577, 2176, 11067, 315, 279, 23606, 553, 220, 19, 11, 582, 633, 856, 284, 400, 16, 24, 198, 12549, 19685, 702, 2326, 3039, 438, 1753, 3220, 438, 21475, 11, 19685, 702, 220, 18, 353, 400, 16, 24, 284, 400, 20, 22, 198, 54815, 11, 19685, 702, 400, 20, 22, 198, 820, 220, 20, 22, 198, 785, 4226, 374, 25, 220, 20, 22, 151645, 198]
inputs:
<|im_start|>user
Jean has three times as much money as Jane. They have a combined total of $76. How much money does Jean have?<|im_end|>
<|im_start|>assistant
Let's assume Jane has x dollars.
Then Jean has 3x dollars.
The total amount of money they have is x + 3x = 4x
We know that the total amount of money they have is $76, so 4x = $76
Dividing both sides of the equation by 4, we get x = $19
Since Jean has three times as much money as Jane, Jean has 3 * $19 = $57
Therefore, Jean has $57
#### 57
The answer is: 57<|im_end|>
label_ids:
[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 10061, 594, 9658, 21475, 702, 856, 11192, 624, 12209, 19685, 702, 220, 18, 87, 11192, 624, 785, 2790, 3311, 315, 3220, 807, 614, 374, 856, 488, 220, 18, 87, 284, 220, 19, 87, 198, 1654, 1414, 429, 279, 2790, 3311, 315, 3220, 807, 614, 374, 400, 22, 21, 11, 773, 220, 19, 87, 284, 400, 22, 21, 198, 12509, 6577, 2176, 11067, 315, 279, 23606, 553, 220, 19, 11, 582, 633, 856, 284, 400, 16, 24, 198, 12549, 19685, 702, 2326, 3039, 438, 1753, 3220, 438, 21475, 11, 19685, 702, 220, 18, 353, 400, 16, 24, 284, 400, 20, 22, 198, 54815, 11, 19685, 702, 400, 20, 22, 198, 820, 220, 20, 22, 198, 785, 4226, 374, 25, 220, 20, 22, 151645, 198]
labels:
Let's assume Jane has x dollars.
Then Jean has 3x dollars.
The total amount of money they have is x + 3x = 4x
We know that the total amount of money they have is $76, so 4x = $76
Dividing both sides of the equation by 4, we get x = $19
Since Jean has three times as much money as Jane, Jean has 3 * $19 = $57
Therefore, Jean has $57
#### 57
The answer is: 57<|im_end|>
[INFO|configuration_utils.py:670] 2026-04-29 15:44:15,949 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/config.json
[INFO|configuration_utils.py:742] 2026-04-29 15:44:15,949 >> Model config Qwen3Config {
"architectures": [
"Qwen3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 151643,
"dtype": "bfloat16",
"eos_token_id": 151645,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 2048,
"initializer_range": 0.02,
"intermediate_size": 6144,
"layer_types": [
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention"
],
"masked_layers": null,
"max_position_embeddings": 40960,
"max_window_layers": 28,
"model_type": "qwen3",
"num_attention_heads": 16,
"num_hidden_layers": 28,
"num_key_value_heads": 8,
"pad_token_id": null,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"rope_theta": 1000000,
"rope_type": "default"
},
"sliding_window": null,
"sparsity_attn": null,
"sparsity_mlp": null,
"subnet_mode": null,
"subnet_type": null,
"threshold_attn": null,
"threshold_mlp": null,
"tie_word_embeddings": true,
"transformers_version": "5.2.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 151936
}
[INFO|2026-04-29 15:44:15] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
[INFO|modeling_utils.py:710] 2026-04-29 15:44:17,465 >> loading weights file model.safetensors from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/model.safetensors.index.json
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
Fetching 2 files: 50%|█████ | 1/2 [00:04<00:04, 4.85s/it]
Fetching 2 files: 100%|██████████| 2/2 [00:04<00:00, 2.43s/it]
[INFO|modeling_utils.py:779] 2026-04-29 15:44:22,425 >> Will use dtype=torch.bfloat16 as defined in model's config object
Fetching 2 files: 50%|█████ | 1/2 [00:04<00:04, 4.02s/it]
Fetching 2 files: 100%|██████████| 2/2 [00:04<00:00, 2.01s/it]
[INFO|configuration_utils.py:1014] 2026-04-29 15:44:22,426 >> Generate config GenerationConfig {
"bos_token_id": 151643,
"eos_token_id": 151645,
"output_attentions": false,
"output_hidden_states": false,
"use_cache": false
}
Fetching 2 files: 50%|█████ | 1/2 [00:04<00:04, 4.02s/it]
Fetching 2 files: 100%|██████████| 2/2 [00:04<00:00, 2.01s/it]
Fetching 2 files: 50%|█████ | 1/2 [00:03<00:03, 3.98s/it]
Fetching 2 files: 100%|██████████| 2/2 [00:03<00:00, 1.99s/it]
Fetching 2 files: 50%|█████ | 1/2 [00:03<00:03, 3.98s/it]
Fetching 2 files: 100%|██████████| 2/2 [00:03<00:00, 1.99s/it]
Fetching 2 files: 50%|█████ | 1/2 [00:04<00:04, 4.07s/it]
Fetching 2 files: 100%|██████████| 2/2 [00:04<00:00, 2.04s/it]
Fetching 2 files: 50%|█████ | 1/2 [00:04<00:04, 4.17s/it]
Fetching 2 files: 100%|██████████| 2/2 [00:04<00:00, 2.08s/it]
Fetching 2 files: 50%|█████ | 1/2 [00:04<00:04, 4.12s/it]
Fetching 2 files: 100%|██████████| 2/2 [00:04<00:00, 2.06s/it]
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
Loading weights: 0%| | 1/311 [00:00<00:00, 13357.66it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 1/311 [00:00<00:00, 6721.64it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
Loading weights: 0%| | 1/311 [00:00<00:00, 13025.79it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 1/311 [00:00<00:00, 6523.02it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
Loading weights: 0%| | 1/311 [00:00<00:00, 11915.64it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 1/311 [00:00<00:00, 6105.25it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
Loading weights: 0%| | 1/311 [00:00<00:00, 12985.46it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 1/311 [00:00<00:00, 6442.86it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
Loading weights: 0%| | 1/311 [00:00<00:00, 8867.45it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 1/311 [00:00<00:00, 4777.11it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
Loading weights: 0%| | 1/311 [00:00<00:00, 11459.85it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
Loading weights: 0%| | 1/311 [00:00<00:00, 6159.04it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 1/311 [00:00<00:00, 12087.33it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 1/311 [00:00<00:00, 5949.37it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
Loading weights: 0%| | 1/311 [00:00<00:00, 10538.45it/s, Materializing param=lm_head.weight]
Loading weights: 0%| | 1/311 [00:00<00:00, 5729.92it/s, Materializing param=lm_head.weight]
Loading weights: 1%| | 2/311 [00:00<00:25, 12.28it/s, Materializing param=lm_head.weight]
Loading weights: 1%| | 2/311 [00:00<00:25, 12.28it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:25, 12.28it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:19, 15.56it/s, Materializing param=lm_head.weight]
Loading weights: 1%| | 2/311 [00:00<00:19, 15.56it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:19, 15.56it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:19, 15.74it/s, Materializing param=lm_head.weight]
Loading weights: 1%| | 2/311 [00:00<00:19, 15.74it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:19, 15.74it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:20, 15.33it/s, Materializing param=lm_head.weight]
Loading weights: 1%| | 2/311 [00:00<00:23, 12.95it/s, Materializing param=lm_head.weight]
Loading weights: 1%| | 2/311 [00:00<00:20, 15.33it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:20, 15.33it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:23, 12.95it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:23, 12.95it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:33, 9.20it/s, Materializing param=lm_head.weight]
Loading weights: 1%| | 2/311 [00:00<00:33, 9.20it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:33, 9.20it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 3/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%| | 3/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 1%| | 2/311 [00:00<00:23, 13.41it/s, Materializing param=lm_head.weight]
Loading weights: 1%| | 2/311 [00:00<00:23, 13.41it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:23, 13.41it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 1%| | 2/311 [00:00<00:33, 9.34it/s, Materializing param=lm_head.weight]
Loading weights: 1%| | 2/311 [00:00<00:33, 9.34it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 2/311 [00:00<00:33, 9.34it/s, Materializing param=model.embed_tokens.weight]
Loading weights: 1%| | 3/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%| | 3/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 15%|█▌ | 48/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 15%|█▌ | 48/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 16%|█▋ | 51/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 16%|█▋ | 51/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 17%|█▋ | 52/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 17%|█▋ | 52/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 17%|█▋ | 53/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 17%|█▋ | 53/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 17%|█▋ | 54/311 [00:00<00:27, 9.20it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 17%|█▋ | 54/311 [00:00<00:27, 9.20it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 15%|█▌ | 48/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 15%|█▌ | 48/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 16%|█▋ | 51/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 16%|█▋ | 51/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 17%|█▋ | 52/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 52/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 53/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 17%|█▋ | 53/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 17%|█▋ | 54/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 17%|█▋ | 54/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 18%|█▊ | 55/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 18%|█▊ | 55/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 18%|█▊ | 56/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 56/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 20%|██ | 63/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 20%|██ | 63/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 21%|██ | 64/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 21%|██ | 64/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 21%|██ | 65/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 21%|██ | 65/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 21%|██ | 66/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 21%|██ | 66/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 1%| | 3/311 [00:00<00:20, 15.33it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%| | 3/311 [00:00<00:20, 15.33it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 1%| | 3/311 [00:00<00:19, 15.74it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%| | 3/311 [00:00<00:19, 15.56it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%| | 3/311 [00:00<00:25, 12.28it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%| | 3/311 [00:00<00:19, 15.74it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%| | 3/311 [00:00<00:19, 15.56it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%| | 3/311 [00:00<00:25, 12.28it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.29it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.29it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.29it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 1%| | 3/311 [00:00<00:23, 12.95it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%| | 3/311 [00:00<00:23, 12.95it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 20%|██ | 63/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 20%|██ | 63/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 21%|██ | 64/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 21%|██ | 64/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 21%|██ | 65/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 21%|██ | 65/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:23, 13.29it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:23, 13.29it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 21%|██ | 66/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 21%|██ | 66/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 30%|██▉ | 93/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 30%|██▉ | 93/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 30%|███ | 94/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 30%|███ | 94/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 28%|██▊ | 88/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 28%|██▊ | 88/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 31%|███ | 95/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 31%|███ | 95/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 31%|███ | 96/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 31%|███ | 96/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 31%|███ | 97/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 31%|███ | 97/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 29%|██▊ | 89/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 29%|██▊ | 89/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 29%|██▉ | 90/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 29%|██▉ | 90/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 29%|██▉ | 91/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 29%|██▉ | 91/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 30%|██▉ | 92/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 30%|██▉ | 92/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 30%|██▉ | 93/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 30%|██▉ | 93/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 30%|███ | 94/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 30%|███ | 94/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 31%|███ | 95/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 31%|███ | 95/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 31%|███ | 96/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 31%|███ | 96/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 1%| | 3/311 [00:00<00:22, 13.41it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%| | 3/311 [00:00<00:22, 13.41it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 31%|███ | 97/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 31%|███ | 97/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 2%|▏ | 5/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 2%|▏ | 6/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 9%|▉ | 28/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 2%|▏ | 7/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 9%|▉ | 29/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 3%|▎ | 8/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 10%|▉ | 30/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 3%|▎ | 9/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 10%|▉ | 31/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 3%|▎ | 10/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 10%|█ | 32/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 15%|█▌ | 48/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 15%|█▌ | 48/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 16%|█▌ | 49/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 4%|▎ | 11/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 11%|█ | 33/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 11%|█ | 34/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 4%|▍ | 12/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 4%|▍ | 13/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 5%|▍ | 14/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:23, 11.41it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:23, 11.41it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 5%|▍ | 15/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 12%|█▏ | 36/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 12%|█▏ | 37/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 40%|████ | 125/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 40%|████ | 125/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 5%|▌ | 16/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights: 41%|████ | 126/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 41%|████ | 126/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 41%|████ | 127/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 41%|████ | 127/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 5%|▌ | 17/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 41%|████ | 128/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 41%|████ | 128/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 40%|████ | 125/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 40%|████ | 125/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 6%|▌ | 18/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights: 41%|████ | 126/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 41%|████ | 126/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 6%|▌ | 19/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 6%|▋ | 20/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 7%|▋ | 21/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 15%|█▌ | 48/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 15%|█▌ | 48/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 41%|████ | 127/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 41%|████ | 127/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 22/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 7%|▋ | 23/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 8%|▊ | 24/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 8%|▊ | 25/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 8%|▊ | 26/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 41%|████ | 128/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 41%|████ | 128/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 15%|█▌ | 47/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 9%|▊ | 27/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 16%|█▌ | 50/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 18%|█▊ | 55/311 [00:01<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 15%|█▌ | 47/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 20%|█▉ | 62/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 9%|▊ | 27/311 [00:01<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 16%|█▋ | 51/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 16%|█▌ | 50/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 56/311 [00:01<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 56/311 [00:01<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 15%|█▌ | 48/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 15%|█▌ | 48/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 9%|▉ | 28/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights: 18%|█▊ | 57/311 [00:01<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 16%|█▋ | 51/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 20%|██ | 63/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 17%|█▋ | 52/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 18%|█▊ | 57/311 [00:01<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 16%|█▋ | 51/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 20%|██ | 63/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 52/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 9%|▉ | 28/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 9%|▉ | 28/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 16%|█▌ | 49/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 17%|█▋ | 52/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 53/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 17%|█▋ | 52/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 53/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 9%|▉ | 29/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 17%|█▋ | 53/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 17%|█▋ | 54/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 21%|██ | 65/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 19%|█▉ | 59/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 17%|█▋ | 53/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 9%|▉ | 29/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights: 17%|█▋ | 54/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 21%|██ | 65/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 19%|█▉ | 59/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 16%|█▋ | 51/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 17%|█▋ | 54/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 16%|█▋ | 51/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 18%|█▊ | 55/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 19%|█▉ | 60/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 21%|██ | 66/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 17%|█▋ | 54/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 21%|██ | 66/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 18%|█▊ | 55/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 10%|▉ | 30/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 52/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 18%|█▊ | 55/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 17%|█▋ | 52/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 20%|█▉ | 61/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 10%|▉ | 30/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights: 22%|██▏ | 67/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 56/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 55/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 20%|█▉ | 61/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 22%|██▏ | 67/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 56/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 17%|█▋ | 53/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 18%|█▊ | 56/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 17%|█▋ | 53/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 22%|██▏ | 68/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 18%|█▊ | 57/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 56/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 10%|▉ | 31/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 22%|██▏ | 68/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 18%|█▊ | 56/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 57/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 17%|█▋ | 54/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 10%|▉ | 31/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights: 18%|█▊ | 57/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 20%|██ | 63/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 54/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 22%|██▏ | 69/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 18%|█▊ | 57/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 20%|██ | 63/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 54/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 22%|██▏ | 69/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 18%|█▊ | 57/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 10%|█ | 32/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 23%|██▎ | 70/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 18%|█▊ | 55/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 23%|██▎ | 70/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 18%|█▊ | 55/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 10%|█ | 32/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 19%|█▉ | 59/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 19%|█▉ | 59/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 21%|██ | 65/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 23%|██▎ | 71/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 18%|█▊ | 56/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 19%|█▉ | 59/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 21%|██ | 65/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 23%|██▎ | 71/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 18%|█▊ | 56/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 11%|█ | 33/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 19%|█▉ | 59/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 21%|██ | 66/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 23%|██▎ | 72/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 11%|█ | 33/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights: 18%|█▊ | 57/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 21%|██ | 66/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 23%|██▎ | 72/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 57/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 20%|█▉ | 61/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 20%|█▉ | 61/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 19%|█▊ | 58/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 20%|█▉ | 61/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 11%|█ | 34/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 19%|█▊ | 58/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 20%|█▉ | 61/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 11%|█ | 34/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 19%|█▉ | 59/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 19%|█▉ | 59/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 24%|██▍ | 74/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 20%|██ | 63/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 24%|██▍ | 74/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 11%|█▏ | 35/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 20%|██ | 63/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 20%|██ | 63/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 19%|█▉ | 60/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 11%|█▏ | 35/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights: 20%|██ | 63/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 19%|█▉ | 60/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 24%|██▍ | 75/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 24%|██▍ | 75/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 20%|█▉ | 61/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 24%|██▍ | 76/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 20%|█▉ | 61/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 12%|█▏ | 36/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 24%|██▍ | 76/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 21%|██ | 65/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 21%|██ | 65/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 21%|██ | 65/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 12%|█▏ | 36/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 21%|██ | 65/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 25%|██▍ | 77/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 25%|██▍ | 77/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 21%|██ | 66/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 21%|██ | 66/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 20%|██ | 63/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 21%|██ | 66/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 12%|█▏ | 37/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 21%|██ | 66/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 25%|██▌ | 78/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 20%|██ | 63/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 25%|██▌ | 78/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 12%|█▏ | 37/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 25%|██▌ | 79/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 25%|██▌ | 79/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 12%|█▏ | 38/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 21%|██ | 65/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 26%|██▌ | 80/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 21%|██ | 65/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 12%|█▏ | 38/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 26%|██▌ | 80/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 21%|██ | 66/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 26%|██▌ | 81/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 21%|██ | 66/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 26%|██▌ | 81/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 13%|█▎ | 39/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 26%|██▋ | 82/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 13%|█▎ | 40/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 27%|██▋ | 83/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 27%|██▋ | 83/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 13%|█▎ | 40/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 27%|██▋ | 84/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 27%|██▋ | 84/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 13%|█▎ | 41/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 27%|██▋ | 85/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 13%|█▎ | 41/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 27%|██▋ | 85/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 28%|██▊ | 86/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 14%|█▎ | 42/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 28%|██▊ | 86/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 14%|█▎ | 42/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 14%|█▍ | 43/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 28%|██▊ | 88/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 28%|██▊ | 88/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 14%|█▍ | 43/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 29%|██▊ | 89/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 29%|██▊ | 89/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 14%|█▍ | 44/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 29%|██▉ | 90/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 14%|█▍ | 44/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 27%|██▋ | 85/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 29%|██▉ | 90/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 27%|██▋ | 85/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 29%|██▉ | 91/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 29%|██▉ | 91/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 28%|██▊ | 86/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 14%|█▍ | 45/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 28%|██▊ | 86/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 14%|█▍ | 45/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights: 30%|██▉ | 92/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 28%|██▊ | 87/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 30%|██▉ | 92/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 28%|██▊ | 87/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 15%|█▍ | 46/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 28%|██▊ | 88/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 30%|██▉ | 93/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 15%|█▍ | 46/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights: 28%|██▊ | 88/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 30%|██▉ | 93/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 29%|██▊ | 89/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 29%|██▊ | 89/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 15%|█▌ | 47/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 15%|█▌ | 47/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 29%|██▉ | 90/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 31%|███ | 95/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 29%|██▉ | 90/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 15%|█▌ | 48/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 31%|███ | 95/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 15%|█▌ | 48/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights: 29%|██▉ | 91/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 27%|██▋ | 85/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 29%|██▉ | 91/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 31%|███ | 96/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 27%|██▋ | 85/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 16%|█▌ | 49/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 31%|███ | 96/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 27%|██▋ | 85/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 16%|█▌ | 49/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights: 30%|██▉ | 92/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 28%|██▊ | 86/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 27%|██▋ | 85/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 30%|██▉ | 92/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 31%|███ | 97/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 28%|██▊ | 86/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 31%|███ | 97/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 28%|██▊ | 86/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 28%|██▊ | 86/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 28%|██▊ | 88/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 28%|██▊ | 88/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 29%|██▊ | 89/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 29%|██▊ | 89/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 29%|██▉ | 90/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 30%|██▉ | 93/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 16%|█▌ | 50/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights: 29%|██▉ | 90/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 28%|██▊ | 87/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 30%|██▉ | 93/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 29%|██▉ | 91/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 29%|██▉ | 91/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 16%|█▋ | 51/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 32%|███▏ | 98/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 16%|█▋ | 51/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 30%|██▉ | 92/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 30%|██▉ | 92/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 32%|███▏ | 98/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 28%|██▊ | 88/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 17%|█▋ | 52/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 28%|██▊ | 88/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 30%|██▉ | 93/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 31%|███ | 95/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 17%|█▋ | 52/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 30%|██▉ | 93/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 31%|███ | 95/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 27%|██▋ | 85/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 29%|██▊ | 89/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 29%|██▊ | 89/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 27%|██▋ | 85/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 31%|███ | 96/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 53/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 31%|███ | 96/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 32%|███▏ | 100/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 29%|██▉ | 90/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 28%|██▊ | 86/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 17%|█▋ | 53/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights: 32%|███▏ | 100/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 29%|██▉ | 90/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 28%|██▊ | 86/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 31%|███ | 95/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 31%|███ | 97/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 31%|███ | 95/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 31%|███ | 97/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 32%|███▏ | 101/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 29%|██▉ | 91/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 54/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 32%|███▏ | 101/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 29%|██▉ | 91/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 31%|███ | 96/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 32%|███▏ | 98/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 31%|███ | 96/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 17%|█▋ | 54/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights: 32%|███▏ | 98/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 33%|███▎ | 102/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 30%|██▉ | 92/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 28%|██▊ | 88/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 33%|███▎ | 102/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 30%|██▉ | 92/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 31%|███ | 97/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 28%|██▊ | 88/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 18%|█▊ | 55/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 31%|███ | 97/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 33%|███▎ | 103/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 30%|██▉ | 93/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 18%|█▊ | 55/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights: 29%|██▊ | 89/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 33%|███▎ | 103/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 30%|██▉ | 93/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 32%|███▏ | 98/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 32%|███▏ | 100/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 29%|██▊ | 89/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 32%|███▏ | 98/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 32%|███▏ | 100/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 18%|█▊ | 56/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 32%|███▏ | 101/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 29%|██▉ | 90/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 18%|█▊ | 56/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights: 32%|███▏ | 101/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 29%|██▉ | 90/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 34%|███▍ | 105/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 31%|███ | 95/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 34%|███▍ | 105/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 31%|███ | 95/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 32%|███▏ | 100/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 33%|███▎ | 102/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 29%|██▉ | 91/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 32%|███▏ | 100/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 18%|█▊ | 57/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 33%|███▎ | 102/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 29%|██▉ | 91/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 31%|███ | 96/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 34%|███▍ | 106/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 18%|█▊ | 57/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights: 31%|███ | 96/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 32%|███▏ | 101/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 34%|███▍ | 106/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 33%|███▎ | 103/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 30%|██▉ | 92/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 32%|███▏ | 101/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 33%|███▎ | 103/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 30%|██▉ | 92/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 31%|███ | 97/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 34%|███▍ | 107/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 31%|███ | 97/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 19%|█▊ | 58/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 33%|███▎ | 102/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 34%|███▍ | 107/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 30%|██▉ | 93/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 33%|███▎ | 102/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 19%|█▊ | 58/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 30%|██▉ | 93/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 32%|███▏ | 98/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 32%|███▏ | 98/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 34%|███▍ | 105/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 33%|███▎ | 103/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 34%|███▍ | 105/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 33%|███▎ | 103/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 19%|█▉ | 59/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 35%|███▌ | 109/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 34%|███▍ | 106/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 19%|█▉ | 59/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights: 35%|███▌ | 109/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 31%|███ | 95/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 34%|███▍ | 106/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 32%|███▏ | 100/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 31%|███ | 95/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 35%|███▌ | 110/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 32%|███▏ | 100/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 34%|███▍ | 107/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 35%|███▌ | 110/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 34%|███▍ | 105/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 19%|█▉ | 60/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights: 31%|███ | 96/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 34%|███▍ | 107/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 34%|███▍ | 105/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 32%|███▏ | 101/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 31%|███ | 96/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 32%|███▏ | 101/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 20%|█▉ | 61/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 34%|███▍ | 106/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 20%|█▉ | 61/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights: 31%|███ | 97/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 34%|███▍ | 106/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 33%|███▎ | 102/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 31%|███ | 97/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 36%|███▌ | 112/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 33%|███▎ | 102/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 20%|█▉ | 62/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 35%|███▌ | 109/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 36%|███▌ | 112/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 34%|███▍ | 107/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 20%|█▉ | 62/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights: 35%|███▌ | 109/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 32%|███▏ | 98/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 32%|███▏ | 98/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 34%|███▍ | 107/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 32%|███▏ | 100/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 33%|███▎ | 103/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 32%|███▏ | 100/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 36%|███▋ | 113/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 32%|███▏ | 101/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 33%|███▎ | 103/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 32%|███▏ | 101/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 36%|███▋ | 113/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 20%|██ | 63/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 33%|███▎ | 102/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 35%|███▌ | 110/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 33%|███▎ | 102/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 20%|██ | 63/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 35%|███▌ | 110/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 33%|███▎ | 103/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 33%|███▎ | 103/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 34%|███▍ | 105/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 34%|███▍ | 105/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 34%|███▍ | 106/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 34%|███▍ | 105/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 34%|███▍ | 106/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 34%|███▍ | 105/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 21%|██ | 64/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 34%|███▍ | 107/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 34%|███▍ | 106/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 34%|███▍ | 107/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 34%|███▍ | 106/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 21%|██ | 65/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 34%|███▍ | 107/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 21%|██ | 65/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 34%|███▍ | 107/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 21%|██ | 66/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 21%|██ | 66/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 22%|██▏ | 67/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 22%|██▏ | 67/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 22%|██▏ | 68/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 22%|██▏ | 68/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 22%|██▏ | 69/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 40%|████ | 125/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 22%|██▏ | 69/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 40%|████ | 125/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 41%|████ | 126/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 23%|██▎ | 70/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 41%|████ | 126/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 23%|██▎ | 70/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 23%|██▎ | 71/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 41%|████ | 128/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 41%|████ | 128/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 23%|██▎ | 71/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 40%|████ | 125/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 40%|████ | 125/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 23%|██▎ | 72/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 41%|████ | 126/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 23%|██▎ | 72/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 41%|████ | 126/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 40%|████ | 125/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 23%|██▎ | 73/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights: 41%|████ | 128/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 40%|████ | 125/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 41%|████ | 128/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 41%|████ | 126/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 41%|████ | 126/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 24%|██▍ | 74/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 24%|██▍ | 74/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 24%|██▍ | 75/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 41%|████ | 128/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 40%|████ | 125/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 41%|████ | 128/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 40%|████ | 125/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 24%|██▍ | 75/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 41%|████ | 126/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 41%|████ | 126/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 40%|████ | 125/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 40%|████ | 125/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 24%|██▍ | 76/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 41%|████ | 126/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 41%|████ | 128/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 41%|████ | 126/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 25%|██▍ | 77/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 41%|████ | 128/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 25%|██▍ | 77/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 41%|████ | 128/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 25%|██▌ | 78/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 41%|████ | 128/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 25%|██▌ | 78/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 25%|██▌ | 79/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 25%|██▌ | 79/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 26%|██▌ | 80/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 26%|██▌ | 80/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 26%|██▌ | 81/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 26%|██▌ | 81/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 26%|██▋ | 82/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 27%|██▋ | 83/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 27%|██▋ | 83/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 27%|██▋ | 84/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 27%|██▋ | 84/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 27%|██▋ | 85/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 27%|██▋ | 85/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 28%|██▊ | 86/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 28%|██▊ | 86/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 28%|██▊ | 87/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 28%|██▊ | 88/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 28%|██▊ | 88/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 29%|██▊ | 89/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 29%|██▊ | 89/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 29%|██▉ | 90/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 29%|██▉ | 90/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 29%|██▉ | 91/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 29%|██▉ | 91/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 30%|██▉ | 92/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 30%|██▉ | 92/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 81.71it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 81.71it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 30%|██▉ | 93/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 30%|██▉ | 93/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 81.71it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 81.71it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 30%|███ | 94/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 31%|███ | 95/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 31%|███ | 95/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 31%|███ | 96/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 31%|███ | 96/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 31%|███ | 97/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 31%|███ | 97/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 32%|███▏ | 98/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 32%|███▏ | 98/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 32%|███▏ | 99/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 32%|███▏ | 100/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 32%|███▏ | 100/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 32%|███▏ | 101/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 32%|███▏ | 101/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 33%|███▎ | 102/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 33%|███▎ | 102/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 33%|███▎ | 103/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 33%|███▎ | 103/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 34%|███▍ | 105/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 34%|███▍ | 105/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 34%|███▍ | 106/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 34%|███▍ | 106/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 34%|███▍ | 107/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 34%|███▍ | 107/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 192.34it/s, Materializing param=model.norm.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 192.34it/s, Materializing param=model.norm.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 195.96it/s, Materializing param=model.norm.weight]
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 40%|████ | 125/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 40%|████ | 125/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 41%|████ | 126/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 41%|████ | 126/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 41%|████ | 127/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 41%|████ | 128/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 41%|████ | 128/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 41%|████▏ | 129/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 42%|████▏ | 130/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 42%|████▏ | 131/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 42%|████▏ | 132/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 212.69it/s, Materializing param=model.norm.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 212.69it/s, Materializing param=model.norm.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 184.81it/s, Materializing param=model.norm.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.o_proj.weight][WARNING|modeling_utils.py:2496] 2026-04-29 15:44:24,236 >> The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 55%|█████▍ | 171/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 55%|█████▌ | 172/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 261.82it/s, Materializing param=model.norm.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 261.82it/s, Materializing param=model.norm.weight]
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 175.99it/s, Materializing param=model.norm.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 255.68it/s, Materializing param=model.norm.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 255.68it/s, Materializing param=model.norm.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 170.80it/s, Materializing param=model.norm.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 247.65it/s, Materializing param=model.norm.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 247.65it/s, Materializing param=model.norm.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 175.84it/s, Materializing param=model.norm.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 250.91it/s, Materializing param=model.norm.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 250.91it/s, Materializing param=model.norm.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 175.36it/s, Materializing param=model.norm.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 245.34it/s, Materializing param=model.norm.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 245.34it/s, Materializing param=model.norm.weight]
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 173.28it/s, Materializing param=model.norm.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 374.43it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 374.43it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 167.36it/s, Materializing param=model.norm.weight]
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
[INFO|configuration_utils.py:967] 2026-04-29 15:44:24,477 >> loading configuration file generation_config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/generation_config.json
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
[INFO|configuration_utils.py:1014] 2026-04-29 15:44:24,477 >> Generate config GenerationConfig {
"bos_token_id": 151643,
"do_sample": true,
"eos_token_id": [
151645,
151643
],
"pad_token_id": 151643,
"temperature": 0.6,
"top_k": 20,
"top_p": 0.95
}
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
[INFO|dynamic_module_utils.py:406] 2026-04-29 15:44:24,593 >> Could not locate the custom_generate/generate.py inside Qwen/Qwen3-1.7B.
[INFO|2026-04-29 15:44:24] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
[INFO|2026-04-29 15:44:24] llamafactory.model.model_utils.attention:144 >> Using torch SDPA for faster training and inference.
[INFO|2026-04-29 15:44:24] llamafactory.model.adapter:144 >> Upcasting trainable params to float32.
[INFO|2026-04-29 15:44:24] llamafactory.model.adapter:144 >> Fine-tuning method: Full
[INFO|2026-04-29 15:44:24] llamafactory.model.loader:144 >> trainable params: 2,031,739,904 || all params: 2,031,739,904 || trainable%: 100.0000
[WARNING|trainer_utils.py:1234] 2026-04-29 15:44:24,831 >> The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Using network Socket
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ncclCommSplit comm 0x56028de3b5a0 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 parent 0x5602809d0750 splitCount 1 color 1266629538 key 2- Init START
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ncclCommSplit comm 0x558d03437a10 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 parent 0x558cf7486a10 splitCount 1 color 1266629538 key 1- Init START
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ncclCommSplit comm 0x55e3778f3800 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 parent 0x55e36d363080 splitCount 1 color 1266629538 key 7- Init START
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ncclCommSplit comm 0x55c02c235cf0 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 parent 0x55c0201d0a60 splitCount 1 color 1266629538 key 5- Init START
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ncclCommSplit comm 0x5612df7ca6e0 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 parent 0x5612d38dcbf0 splitCount 1 color 1266629538 key 3- Init START
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ncclCommSplit comm 0x5647135d02c0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 parent 0x5647078795e0 splitCount 1 color 1266629538 key 4- Init START
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ncclCommSplit comm 0x560722415b60 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 parent 0x56071a5c6320 splitCount 1 color 1266629538 key 6- Init START
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ncclCommSplit comm 0x55ffc26dd150 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 parent 0x55ffbe0f7c80 splitCount 1 color 1266629538 key 0- Init START
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 6 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NVLS multicast support is not available on dev 6 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 3 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NVLS multicast support is not available on dev 3 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 2 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NVLS multicast support is not available on dev 2 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 1 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NVLS multicast support is not available on dev 1 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 5 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NVLS multicast support is not available on dev 5 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 7 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NVLS multicast support is not available on dev 7 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 4 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NVLS multicast support is not available on dev 4 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NVLS multicast support is not available on dev 0 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO comm 0x558d03437a10 rank 1 nRanks 8 nNodes 1 localRanks 8 localRank 1 MNNVL 0
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO comm 0x55ffc26dd150 rank 0 nRanks 8 nNodes 1 localRanks 8 localRank 0 MNNVL 0
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO comm 0x55e3778f3800 rank 7 nRanks 8 nNodes 1 localRanks 8 localRank 7 MNNVL 0
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO comm 0x560722415b60 rank 6 nRanks 8 nNodes 1 localRanks 8 localRank 6 MNNVL 0
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO comm 0x55c02c235cf0 rank 5 nRanks 8 nNodes 1 localRanks 8 localRank 5 MNNVL 0
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO comm 0x56028de3b5a0 rank 2 nRanks 8 nNodes 1 localRanks 8 localRank 2 MNNVL 0
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO comm 0x5647135d02c0 rank 4 nRanks 8 nNodes 1 localRanks 8 localRank 4 MNNVL 0
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 00/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO comm 0x5612df7ca6e0 rank 3 nRanks 8 nNodes 1 localRanks 8 localRank 3 MNNVL 0
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 01/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 02/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 03/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 04/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 05/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 06/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 07/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 08/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 09/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 10/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 11/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 12/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 13/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 14/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 15/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 16/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 17/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 18/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 19/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 20/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 21/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 22/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 23/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470947:2474553 [0] NCCL INFO [Proxy Service] Device 0 CPU core 60
ywang29-p4d-debug-worker-0:2470947:2474554 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 15
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470948:2474555 [0] NCCL INFO [Proxy Service] Device 1 CPU core 69
ywang29-p4d-debug-worker-0:2470948:2474556 [0] NCCL INFO [Proxy Service UDS] Device 1 CPU core 16
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470952:2474557 [0] NCCL INFO [Proxy Service] Device 4 CPU core 35
ywang29-p4d-debug-worker-0:2470952:2474558 [0] NCCL INFO [Proxy Service UDS] Device 4 CPU core 44
ywang29-p4d-debug-worker-0:2470951:2474559 [0] NCCL INFO [Proxy Service] Device 3 CPU core 58
ywang29-p4d-debug-worker-0:2470951:2474560 [0] NCCL INFO [Proxy Service UDS] Device 3 CPU core 14
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470955:2474561 [0] NCCL INFO [Proxy Service] Device 7 CPU core 77
ywang29-p4d-debug-worker-0:2470955:2474562 [0] NCCL INFO [Proxy Service UDS] Device 7 CPU core 79
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470953:2474563 [0] NCCL INFO [Proxy Service] Device 5 CPU core 32
ywang29-p4d-debug-worker-0:2470953:2474564 [0] NCCL INFO [Proxy Service UDS] Device 5 CPU core 46
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470954:2474565 [0] NCCL INFO [Proxy Service] Device 6 CPU core 93
ywang29-p4d-debug-worker-0:2470954:2474566 [0] NCCL INFO [Proxy Service UDS] Device 6 CPU core 88
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
ywang29-p4d-debug-worker-0:2470949:2474567 [0] NCCL INFO [Proxy Service] Device 2 CPU core 1
ywang29-p4d-debug-worker-0:2470949:2474568 [0] NCCL INFO [Proxy Service UDS] Device 2 CPU core 53
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO CC Off, workFifoBytes 1048576
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ncclCommSplit comm 0x5647135d02c0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 parent 0x5647078795e0 splitCount 1 color 1266629538 key 4 - Init COMPLETE
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ncclCommSplit comm 0x56028de3b5a0 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 parent 0x5602809d0750 splitCount 1 color 1266629538 key 2 - Init COMPLETE
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ncclCommSplit comm 0x55ffc26dd150 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 parent 0x55ffbe0f7c80 splitCount 1 color 1266629538 key 0 - Init COMPLETE
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Init timings - ncclCommSplit: rank 4 nranks 8 total 0.18 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.07, rest 0.06)
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Init timings - ncclCommSplit: rank 0 nranks 8 total 0.24 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.00, topo 0.05, graphs 0.00, connections 0.06, rest 0.13)
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Init timings - ncclCommSplit: rank 2 nranks 8 total 0.15 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.06, rest 0.03)
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ncclCommSplit comm 0x560722415b60 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 parent 0x56071a5c6320 splitCount 1 color 1266629538 key 6 - Init COMPLETE
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Init timings - ncclCommSplit: rank 6 nranks 8 total 0.25 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.02, topo 0.04, graphs 0.00, connections 0.07, rest 0.13)
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ncclCommSplit comm 0x55c02c235cf0 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 parent 0x55c0201d0a60 splitCount 1 color 1266629538 key 5 - Init COMPLETE
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ncclCommSplit comm 0x558d03437a10 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 parent 0x558cf7486a10 splitCount 1 color 1266629538 key 1 - Init COMPLETE
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ncclCommSplit comm 0x5612df7ca6e0 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 parent 0x5612d38dcbf0 splitCount 1 color 1266629538 key 3 - Init COMPLETE
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Init timings - ncclCommSplit: rank 5 nranks 8 total 0.28 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.07, rest 0.16)
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Init timings - ncclCommSplit: rank 1 nranks 8 total 0.26 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.07, rest 0.14)
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ncclCommSplit comm 0x55e3778f3800 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 parent 0x55e36d363080 splitCount 1 color 1266629538 key 7 - Init COMPLETE
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Init timings - ncclCommSplit: rank 3 nranks 8 total 0.16 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.07, rest 0.03)
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Init timings - ncclCommSplit: rank 7 nranks 8 total 0.18 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.07, rest 0.05)
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 00/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 01/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 02/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 03/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 04/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 00/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 05/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 06/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 01/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 07/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 02/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 03/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 08/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 04/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 05/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 09/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 10/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 06/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 07/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 11/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 08/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 09/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 12/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 13/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 14/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 10/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 15/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 11/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 16/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 12/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 16/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 17/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 18/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 17/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 13/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 18/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 14/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 19/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 20/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 15/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 16/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 19/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 16/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 20/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 16/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 17/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 21/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 16/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 17/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 18/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 21/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 22/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 22/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 18/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 17/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 23/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 19/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 23/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 20/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 17/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 18/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 16/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 16/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 19/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 19/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 18/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 21/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 19/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 20/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 20/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 21/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 17/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 20/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 22/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 21/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 17/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 18/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 22/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 18/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 21/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 23/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 22/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 19/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 23/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 19/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 22/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 23/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 20/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 20/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 23/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 21/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 21/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 22/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 22/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 23/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 23/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
Before initializing optimizer states
MA 4.73 GB Max_MA 4.73 GB CA 4.77 GB Max_CA 5 GB
CPU Virtual Memory: used = 70.92 GB, percent = 6.3%
After initializing optimizer states
MA 4.73 GB Max_MA 5.68 GB CA 5.71 GB Max_CA 6 GB
CPU Virtual Memory: used = 70.92 GB, percent = 6.3%
After initializing ZeRO optimizer
MA 4.73 GB Max_MA 4.73 GB CA 5.71 GB Max_CA 6 GB
CPU Virtual Memory: used = 70.92 GB, percent = 6.3%
[INFO|trainer.py:1587] 2026-04-29 15:44:34,254 >> ***** Running training *****
[INFO|trainer.py:1588] 2026-04-29 15:44:34,254 >> Num examples = 60,000
[INFO|trainer.py:1589] 2026-04-29 15:44:34,254 >> Num Epochs = 2
[INFO|trainer.py:1590] 2026-04-29 15:44:34,254 >> Instantaneous batch size per device = 2
[INFO|trainer.py:1593] 2026-04-29 15:44:34,254 >> Total train batch size (w. parallel, distributed & accumulation) = 128
[INFO|trainer.py:1594] 2026-04-29 15:44:34,254 >> Gradient Accumulation steps = 8
[INFO|trainer.py:1595] 2026-04-29 15:44:34,254 >> Total optimization steps = 938
[INFO|trainer.py:1596] 2026-04-29 15:44:34,255 >> Number of trainable parameters = 2,031,739,904
wandb: [wandb.login()] Loaded credentials for https://api.wandb.ai from WANDB_API_KEY.
wandb: Currently logged in as: kkhya (maskmoe) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
wandb: setting up run jl6u15v5
wandb: Tracking run with wandb version 0.25.1
wandb: Run data is saved locally in /nfs/ywang29/lm-factory/wandb/run-20260429_154434-jl6u15v5
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run qwen3_1.7b_math_fft
wandb: ⭐️ View project at https://wandb.ai/maskmoe/MFT-LM
wandb: 🚀 View run at https://wandb.ai/maskmoe/MFT-LM/runs/jl6u15v5
0%| | 0/938 [00:00<?, ?it/s]
0%| | 1/938 [00:05<1:21:11, 5.20s/it]
0%| | 2/938 [00:08<1:08:10, 4.37s/it]
0%| | 3/938 [00:13<1:08:24, 4.39s/it]
0%| | 4/938 [00:17<1:09:08, 4.44s/it]
1%| | 5/938 [00:22<1:07:42, 4.35s/it]
1%| | 6/938 [00:26<1:05:38, 4.23s/it]
1%| | 7/938 [00:30<1:08:49, 4.44s/it]
1%| | 8/938 [00:35<1:08:07, 4.40s/it]
1%| | 9/938 [00:39<1:05:08, 4.21s/it]
1%| | 10/938 [00:43<1:07:18, 4.35s/it]
{'loss': '0.6842', 'grad_norm': '11.14', 'learning_rate': '9.574e-07', 'epoch': '0.02133'}
1%| | 10/938 [00:43<1:07:18, 4.35s/it]
1%| | 11/938 [00:47<1:05:24, 4.23s/it]
1%|▏ | 12/938 [00:51<1:03:37, 4.12s/it]
1%|▏ | 13/938 [00:55<1:01:48, 4.01s/it]
1%|▏ | 14/938 [00:59<1:02:34, 4.06s/it]
2%|▏ | 15/938 [01:03<1:02:42, 4.08s/it]
2%|▏ | 16/938 [01:07<1:03:52, 4.16s/it]
2%|▏ | 17/938 [01:12<1:03:46, 4.15s/it]
2%|▏ | 18/938 [01:16<1:02:46, 4.09s/it]
2%|▏ | 19/938 [01:19<1:01:55, 4.04s/it]
2%|▏ | 20/938 [01:23<1:00:13, 3.94s/it]
{'loss': '0.6097', 'grad_norm': '4.558', 'learning_rate': '2.021e-06', 'epoch': '0.04267'}
2%|▏ | 20/938 [01:23<1:00:13, 3.94s/it]
2%|▏ | 21/938 [01:28<1:02:04, 4.06s/it]
2%|▏ | 22/938 [01:32<1:01:45, 4.05s/it]
2%|▏ | 23/938 [01:35<1:00:57, 4.00s/it]
3%|▎ | 24/938 [01:39<59:52, 3.93s/it]
3%|▎ | 25/938 [01:43<58:15, 3.83s/it]
3%|▎ | 26/938 [01:47<1:00:22, 3.97s/it]
3%|▎ | 27/938 [01:51<1:00:35, 3.99s/it]
3%|▎ | 28/938 [01:55<59:26, 3.92s/it]
3%|▎ | 29/938 [01:59<59:03, 3.90s/it]
3%|▎ | 30/938 [02:03<58:43, 3.88s/it]
{'loss': '0.4652', 'grad_norm': '1.498', 'learning_rate': '3.085e-06', 'epoch': '0.064'}
3%|▎ | 30/938 [02:03<58:43, 3.88s/it]
3%|▎ | 31/938 [02:07<59:38, 3.95s/it]
3%|▎ | 32/938 [02:10<58:23, 3.87s/it]
4%|▎ | 33/938 [02:14<56:48, 3.77s/it]
4%|▎ | 34/938 [02:18<57:58, 3.85s/it]
4%|▎ | 35/938 [02:22<57:50, 3.84s/it]
4%|▍ | 36/938 [02:26<59:02, 3.93s/it]
4%|▍ | 37/938 [02:30<1:01:54, 4.12s/it]
4%|▍ | 38/938 [02:34<1:00:21, 4.02s/it]
4%|▍ | 39/938 [02:38<59:26, 3.97s/it]
4%|▍ | 40/938 [02:43<1:05:36, 4.38s/it]
{'loss': '0.3869', 'grad_norm': '0.7575', 'learning_rate': '4.149e-06', 'epoch': '0.08533'}
4%|▍ | 40/938 [02:43<1:05:36, 4.38s/it]
4%|▍ | 41/938 [02:48<1:06:29, 4.45s/it]
4%|▍ | 42/938 [02:52<1:06:15, 4.44s/it]
5%|▍ | 43/938 [02:58<1:09:33, 4.66s/it]
5%|▍ | 44/938 [03:01<1:05:25, 4.39s/it]
5%|▍ | 45/938 [03:05<1:03:20, 4.26s/it]
5%|▍ | 46/938 [03:09<1:01:24, 4.13s/it]
5%|▌ | 47/938 [03:13<1:01:17, 4.13s/it]
5%|▌ | 48/938 [03:18<1:02:16, 4.20s/it]
5%|▌ | 49/938 [03:22<1:04:53, 4.38s/it]
5%|▌ | 50/938 [03:26<1:02:41, 4.24s/it]
{'loss': '0.3463', 'grad_norm': '0.6512', 'learning_rate': '5.213e-06', 'epoch': '0.1067'}
5%|▌ | 50/938 [03:26<1:02:41, 4.24s/it]
5%|▌ | 51/938 [03:30<1:01:38, 4.17s/it]
6%|▌ | 52/938 [03:34<59:15, 4.01s/it]
6%|▌ | 53/938 [03:38<59:06, 4.01s/it]
6%|▌ | 54/938 [03:43<1:02:10, 4.22s/it]
6%|▌ | 55/938 [03:47<1:03:00, 4.28s/it]
6%|▌ | 56/938 [03:51<1:00:44, 4.13s/it]
6%|▌ | 57/938 [03:55<58:38, 3.99s/it]
6%|▌ | 58/938 [03:59<59:39, 4.07s/it]
6%|▋ | 59/938 [04:03<58:18, 3.98s/it]
6%|▋ | 60/938 [04:07<59:00, 4.03s/it]
{'loss': '0.3305', 'grad_norm': '0.5266', 'learning_rate': '6.277e-06', 'epoch': '0.128'}
6%|▋ | 60/938 [04:07<59:00, 4.03s/it]
7%|▋ | 61/938 [04:11<58:15, 3.99s/it]
7%|▋ | 62/938 [04:15<1:00:41, 4.16s/it]
7%|▋ | 63/938 [04:19<1:00:37, 4.16s/it]
7%|▋ | 64/938 [04:24<1:00:37, 4.16s/it]
7%|▋ | 65/938 [04:28<1:02:16, 4.28s/it]
7%|▋ | 66/938 [04:32<1:01:41, 4.25s/it]
7%|▋ | 67/938 [04:36<1:00:00, 4.13s/it]
7%|▋ | 68/938 [04:40<59:03, 4.07s/it]
7%|▋ | 69/938 [04:44<58:54, 4.07s/it]
7%|▋ | 70/938 [04:49<1:00:20, 4.17s/it]
{'loss': '0.3322', 'grad_norm': '0.5385', 'learning_rate': '7.34e-06', 'epoch': '0.1493'}
7%|▋ | 70/938 [04:49<1:00:20, 4.17s/it]
8%|▊ | 71/938 [04:52<59:03, 4.09s/it]
8%|▊ | 72/938 [04:56<57:40, 4.00s/it]
8%|▊ | 73/938 [05:00<57:24, 3.98s/it]
8%|▊ | 74/938 [05:04<57:14, 3.97s/it]
8%|▊ | 75/938 [05:09<59:06, 4.11s/it]
8%|▊ | 76/938 [05:13<59:03, 4.11s/it]
8%|▊ | 77/938 [05:17<59:24, 4.14s/it]
8%|▊ | 78/938 [05:22<1:04:00, 4.47s/it]
8%|▊ | 79/938 [05:26<1:02:20, 4.35s/it]
9%|▊ | 80/938 [05:31<1:02:35, 4.38s/it]
{'loss': '0.3166', 'grad_norm': '0.5109', 'learning_rate': '8.404e-06', 'epoch': '0.1707'}
9%|▊ | 80/938 [05:31<1:02:35, 4.38s/it]
9%|▊ | 81/938 [05:35<1:01:55, 4.34s/it]
9%|▊ | 82/938 [05:38<58:39, 4.11s/it]
9%|▉ | 83/938 [05:42<57:24, 4.03s/it]
9%|▉ | 84/938 [05:46<55:14, 3.88s/it]
9%|▉ | 85/938 [05:50<55:40, 3.92s/it]
9%|▉ | 86/938 [05:54<55:47, 3.93s/it]
9%|▉ | 87/938 [05:58<55:28, 3.91s/it]
9%|▉ | 88/938 [06:02<56:28, 3.99s/it]
9%|▉ | 89/938 [06:06<58:39, 4.14s/it]
10%|▉ | 90/938 [06:11<1:01:19, 4.34s/it]
{'loss': '0.3102', 'grad_norm': '0.4772', 'learning_rate': '9.468e-06', 'epoch': '0.192'}
10%|▉ | 90/938 [06:11<1:01:19, 4.34s/it]
10%|▉ | 91/938 [06:15<58:30, 4.14s/it]
10%|▉ | 92/938 [06:19<1:00:39, 4.30s/it]
10%|▉ | 93/938 [06:23<59:21, 4.21s/it]
10%|█ | 94/938 [06:28<1:00:36, 4.31s/it]
10%|█ | 95/938 [06:32<59:39, 4.25s/it]
10%|█ | 96/938 [06:36<58:20, 4.16s/it]
10%|█ | 97/938 [06:40<56:49, 4.05s/it]
10%|█ | 98/938 [06:44<56:26, 4.03s/it]
11%|█ | 99/938 [06:48<55:39, 3.98s/it]
11%|█ | 100/938 [06:52<55:53, 4.00s/it]
{'loss': '0.3156', 'grad_norm': '0.4976', 'learning_rate': '9.999e-06', 'epoch': '0.2133'}
11%|█ | 100/938 [06:52<55:53, 4.00s/it]
11%|█ | 101/938 [06:56<56:11, 4.03s/it]
11%|█ | 102/938 [07:01<58:53, 4.23s/it]
11%|█ | 103/938 [07:04<56:49, 4.08s/it]
11%|█ | 104/938 [07:09<59:32, 4.28s/it]
11%|█ | 105/938 [07:13<58:35, 4.22s/it]
11%|█▏ | 106/938 [07:17<55:53, 4.03s/it]
11%|█▏ | 107/938 [07:21<57:11, 4.13s/it]
12%|█▏ | 108/938 [07:25<56:40, 4.10s/it]
12%|█▏ | 109/938 [07:29<56:31, 4.09s/it]
12%|█▏ | 110/938 [07:33<57:18, 4.15s/it]
{'loss': '0.3121', 'grad_norm': '0.4746', 'learning_rate': '9.992e-06', 'epoch': '0.2347'}
12%|█▏ | 110/938 [07:33<57:18, 4.15s/it]
12%|█▏ | 111/938 [07:37<55:17, 4.01s/it]
12%|█▏ | 112/938 [07:42<56:44, 4.12s/it]
12%|█▏ | 113/938 [07:46<56:24, 4.10s/it]
12%|█▏ | 114/938 [07:50<57:57, 4.22s/it]
12%|█▏ | 115/938 [07:54<56:43, 4.14s/it]
12%|█▏ | 116/938 [07:58<56:41, 4.14s/it]
12%|█▏ | 117/938 [08:02<54:10, 3.96s/it]
13%|█▎ | 118/938 [08:06<55:54, 4.09s/it]
13%|█▎ | 119/938 [08:10<53:57, 3.95s/it]
13%|█▎ | 120/938 [08:14<56:14, 4.12s/it]
{'loss': '0.3095', 'grad_norm': '0.4676', 'learning_rate': '9.978e-06', 'epoch': '0.256'}
13%|█▎ | 120/938 [08:14<56:14, 4.12s/it]
13%|█▎ | 121/938 [08:18<53:40, 3.94s/it]
13%|█▎ | 122/938 [08:21<52:42, 3.88s/it]
13%|█▎ | 123/938 [08:25<51:39, 3.80s/it]
13%|█▎ | 124/938 [08:29<51:46, 3.82s/it]
13%|█▎ | 125/938 [08:33<53:41, 3.96s/it]
13%|█▎ | 126/938 [08:37<53:29, 3.95s/it]
14%|█▎ | 127/938 [08:42<54:50, 4.06s/it]
14%|█▎ | 128/938 [08:46<55:15, 4.09s/it]
14%|█▍ | 129/938 [08:50<55:45, 4.14s/it]
14%|█▍ | 130/938 [08:54<54:19, 4.03s/it]
{'loss': '0.3129', 'grad_norm': '0.4763', 'learning_rate': '9.958e-06', 'epoch': '0.2773'}
14%|█▍ | 130/938 [08:54<54:19, 4.03s/it]
14%|█▍ | 131/938 [08:58<54:33, 4.06s/it]
14%|█▍ | 132/938 [09:02<54:39, 4.07s/it]
14%|█▍ | 133/938 [09:07<57:39, 4.30s/it]
14%|█▍ | 134/938 [09:11<57:01, 4.26s/it]
14%|█▍ | 135/938 [09:15<55:31, 4.15s/it]
14%|█▍ | 136/938 [09:18<53:18, 3.99s/it]
15%|█▍ | 137/938 [09:22<53:33, 4.01s/it]
15%|█▍ | 138/938 [09:27<54:20, 4.08s/it]
15%|█▍ | 139/938 [09:31<54:33, 4.10s/it]
15%|█▍ | 140/938 [09:35<54:23, 4.09s/it]
{'loss': '0.3062', 'grad_norm': '0.493', 'learning_rate': '9.93e-06', 'epoch': '0.2987'}
15%|█▍ | 140/938 [09:35<54:23, 4.09s/it]
15%|█▌ | 141/938 [09:39<53:43, 4.05s/it]
15%|█▌ | 142/938 [09:43<52:45, 3.98s/it]
15%|█▌ | 143/938 [09:47<53:34, 4.04s/it]
15%|█▌ | 144/938 [09:52<55:49, 4.22s/it]
15%|█▌ | 145/938 [09:56<56:13, 4.25s/it]
16%|█▌ | 146/938 [09:59<53:01, 4.02s/it]
16%|█▌ | 147/938 [10:03<52:15, 3.96s/it]
16%|█▌ | 148/938 [10:07<49:59, 3.80s/it]
16%|█▌ | 149/938 [10:11<52:10, 3.97s/it]
16%|█▌ | 150/938 [10:15<52:49, 4.02s/it]
{'loss': '0.3069', 'grad_norm': '0.4954', 'learning_rate': '9.896e-06', 'epoch': '0.32'}
16%|█▌ | 150/938 [10:15<52:49, 4.02s/it]
16%|█▌ | 151/938 [10:19<53:40, 4.09s/it]
16%|█▌ | 152/938 [10:23<53:29, 4.08s/it]
16%|█▋ | 153/938 [10:27<52:35, 4.02s/it]
16%|█▋ | 154/938 [10:32<53:39, 4.11s/it]
17%|█▋ | 155/938 [10:35<52:25, 4.02s/it]
17%|█▋ | 156/938 [10:40<55:19, 4.24s/it]
17%|█▋ | 157/938 [10:44<53:47, 4.13s/it]
17%|█▋ | 158/938 [10:47<50:43, 3.90s/it]
17%|█▋ | 159/938 [10:51<50:34, 3.90s/it]
17%|█▋ | 160/938 [10:56<52:48, 4.07s/it]
{'loss': '0.3146', 'grad_norm': '0.485', 'learning_rate': '9.854e-06', 'epoch': '0.3413'}
17%|█▋ | 160/938 [10:56<52:48, 4.07s/it]
17%|█▋ | 161/938 [11:00<52:42, 4.07s/it]
17%|█▋ | 162/938 [11:04<52:27, 4.06s/it]
17%|█▋ | 163/938 [11:08<51:04, 3.95s/it]
17%|█▋ | 164/938 [11:11<50:47, 3.94s/it]
18%|█▊ | 165/938 [11:16<52:37, 4.08s/it]
18%|█▊ | 166/938 [11:20<54:12, 4.21s/it]
18%|█▊ | 167/938 [11:24<51:53, 4.04s/it]
18%|█▊ | 168/938 [11:28<51:12, 3.99s/it]
18%|█▊ | 169/938 [11:32<52:02, 4.06s/it]
18%|█▊ | 170/938 [11:36<52:02, 4.07s/it]
{'loss': '0.3096', 'grad_norm': '0.4601', 'learning_rate': '9.806e-06', 'epoch': '0.3627'}
18%|█▊ | 170/938 [11:36<52:02, 4.07s/it]
18%|█▊ | 171/938 [11:40<50:25, 3.95s/it]
18%|█▊ | 172/938 [11:45<53:44, 4.21s/it]
18%|█▊ | 173/938 [11:49<54:50, 4.30s/it]
19%|█▊ | 174/938 [11:54<55:16, 4.34s/it]
19%|█▊ | 175/938 [11:57<52:26, 4.12s/it]
19%|█▉ | 176/938 [12:02<53:49, 4.24s/it]
19%|█▉ | 177/938 [12:06<52:23, 4.13s/it]
19%|█▉ | 178/938 [12:10<51:44, 4.08s/it]
19%|█▉ | 179/938 [12:15<54:43, 4.33s/it]
19%|█▉ | 180/938 [12:19<55:28, 4.39s/it]
{'loss': '0.307', 'grad_norm': '0.44', 'learning_rate': '9.752e-06', 'epoch': '0.384'}
19%|█▉ | 180/938 [12:19<55:28, 4.39s/it]
19%|█▉ | 181/938 [12:23<55:01, 4.36s/it]
19%|█▉ | 182/938 [12:27<52:34, 4.17s/it]
20%|█▉ | 183/938 [12:32<53:49, 4.28s/it]
20%|█▉ | 184/938 [12:36<52:55, 4.21s/it]
20%|█▉ | 185/938 [12:40<53:29, 4.26s/it]
20%|█▉ | 186/938 [12:44<53:09, 4.24s/it]
20%|█▉ | 187/938 [12:48<50:54, 4.07s/it]
20%|██ | 188/938 [12:51<48:32, 3.88s/it]
20%|██ | 189/938 [12:55<49:06, 3.93s/it]
20%|██ | 190/938 [12:59<48:35, 3.90s/it]
{'loss': '0.3129', 'grad_norm': '0.5087', 'learning_rate': '9.691e-06', 'epoch': '0.4053'}
20%|██ | 190/938 [12:59<48:35, 3.90s/it]
20%|██ | 191/938 [13:03<48:50, 3.92s/it]
20%|██ | 192/938 [13:07<48:24, 3.89s/it]
21%|██ | 193/938 [13:11<48:31, 3.91s/it]
21%|██ | 194/938 [13:15<47:36, 3.84s/it]
21%|██ | 195/938 [13:19<49:52, 4.03s/it]
21%|██ | 196/938 [13:23<50:33, 4.09s/it]
21%|██ | 197/938 [13:27<49:38, 4.02s/it]
21%|██ | 198/938 [13:32<51:07, 4.15s/it]
21%|██ | 199/938 [13:36<50:39, 4.11s/it]
21%|██▏ | 200/938 [13:40<50:11, 4.08s/it]
{'loss': '0.3092', 'grad_norm': '0.4879', 'learning_rate': '9.623e-06', 'epoch': '0.4267'}
21%|██▏ | 200/938 [13:40<50:11, 4.08s/it]
21%|██▏ | 201/938 [13:45<53:11, 4.33s/it]
22%|██▏ | 202/938 [13:49<55:09, 4.50s/it]
22%|██▏ | 203/938 [13:54<53:25, 4.36s/it]
22%|██▏ | 204/938 [13:58<53:57, 4.41s/it]
22%|██▏ | 205/938 [14:02<52:28, 4.30s/it]
22%|██▏ | 206/938 [14:06<51:57, 4.26s/it]
22%|██▏ | 207/938 [14:10<51:35, 4.23s/it]
22%|██▏ | 208/938 [14:14<49:55, 4.10s/it]
22%|██▏ | 209/938 [14:18<49:46, 4.10s/it]
22%|██▏ | 210/938 [14:22<48:41, 4.01s/it]
{'loss': '0.3032', 'grad_norm': '0.4798', 'learning_rate': '9.549e-06', 'epoch': '0.448'}
22%|██▏ | 210/938 [14:22<48:41, 4.01s/it]
22%|██▏ | 211/938 [14:27<50:10, 4.14s/it]
23%|██▎ | 212/938 [14:31<51:53, 4.29s/it]
23%|██▎ | 213/938 [14:36<51:54, 4.30s/it]
23%|██▎ | 214/938 [14:39<49:21, 4.09s/it]
23%|██▎ | 215/938 [14:43<47:48, 3.97s/it]
23%|██▎ | 216/938 [14:47<49:16, 4.10s/it]
23%|██▎ | 217/938 [14:52<51:38, 4.30s/it]
23%|██▎ | 218/938 [14:56<49:56, 4.16s/it]
23%|██▎ | 219/938 [15:00<49:46, 4.15s/it]
23%|██▎ | 220/938 [15:04<50:37, 4.23s/it]
{'loss': '0.3086', 'grad_norm': '0.4887', 'learning_rate': '9.468e-06', 'epoch': '0.4693'}
23%|██▎ | 220/938 [15:04<50:37, 4.23s/it]
24%|██▎ | 221/938 [15:08<49:33, 4.15s/it]
24%|██▎ | 222/938 [15:12<48:25, 4.06s/it]
24%|██▍ | 223/938 [15:16<46:12, 3.88s/it]
24%|██▍ | 224/938 [15:20<46:52, 3.94s/it]
24%|██▍ | 225/938 [15:23<46:07, 3.88s/it]
24%|██▍ | 226/938 [15:28<47:47, 4.03s/it]
24%|██▍ | 227/938 [15:33<50:22, 4.25s/it]
24%|██▍ | 228/938 [15:36<49:02, 4.14s/it]
24%|██▍ | 229/938 [15:40<47:25, 4.01s/it]
25%|██▍ | 230/938 [15:45<51:22, 4.35s/it]
{'loss': '0.3002', 'grad_norm': '0.4873', 'learning_rate': '9.382e-06', 'epoch': '0.4907'}
25%|██▍ | 230/938 [15:45<51:22, 4.35s/it]
25%|██▍ | 231/938 [15:49<50:22, 4.28s/it]
25%|██▍ | 232/938 [15:53<48:35, 4.13s/it]
25%|██▍ | 233/938 [15:57<47:08, 4.01s/it]
25%|██▍ | 234/938 [16:01<46:04, 3.93s/it]
25%|██▌ | 235/938 [16:05<47:08, 4.02s/it]
25%|██▌ | 236/938 [16:09<45:53, 3.92s/it]
25%|██▌ | 237/938 [16:13<47:57, 4.11s/it]
25%|██▌ | 238/938 [16:17<48:27, 4.15s/it]
25%|██▌ | 239/938 [16:21<46:57, 4.03s/it]
26%|██▌ | 240/938 [16:25<45:23, 3.90s/it]
{'loss': '0.299', 'grad_norm': '0.5213', 'learning_rate': '9.289e-06', 'epoch': '0.512'}
26%|██▌ | 240/938 [16:25<45:23, 3.90s/it]
26%|██▌ | 241/938 [16:29<45:32, 3.92s/it]
26%|██▌ | 242/938 [16:33<45:50, 3.95s/it]
26%|██▌ | 243/938 [16:37<48:21, 4.17s/it]
26%|██▌ | 244/938 [16:42<50:19, 4.35s/it]
26%|██▌ | 245/938 [16:47<50:29, 4.37s/it]
26%|██▌ | 246/938 [16:51<49:30, 4.29s/it]
26%|██▋ | 247/938 [16:55<49:13, 4.27s/it]
26%|██▋ | 248/938 [16:59<49:35, 4.31s/it]
27%|██▋ | 249/938 [17:03<48:07, 4.19s/it]
27%|██▋ | 250/938 [17:08<49:09, 4.29s/it]
{'loss': '0.2925', 'grad_norm': '0.4684', 'learning_rate': '9.191e-06', 'epoch': '0.5333'}
27%|██▋ | 250/938 [17:08<49:09, 4.29s/it]
27%|██▋ | 251/938 [17:12<49:57, 4.36s/it]
27%|██▋ | 252/938 [17:16<48:21, 4.23s/it]
27%|██▋ | 253/938 [17:20<45:49, 4.01s/it]
27%|██▋ | 254/938 [17:23<43:20, 3.80s/it]
27%|██▋ | 255/938 [17:27<44:23, 3.90s/it]
27%|██▋ | 256/938 [17:31<44:15, 3.89s/it]
27%|██▋ | 257/938 [17:35<44:32, 3.92s/it]
28%|██▊ | 258/938 [17:39<43:59, 3.88s/it]
28%|██▊ | 259/938 [17:43<45:11, 3.99s/it]
28%|██▊ | 260/938 [17:48<46:31, 4.12s/it]
{'loss': '0.3163', 'grad_norm': '0.4518', 'learning_rate': '9.086e-06', 'epoch': '0.5547'}
28%|██▊ | 260/938 [17:48<46:31, 4.12s/it]
28%|██▊ | 261/938 [17:52<47:32, 4.21s/it]
28%|██▊ | 262/938 [17:56<47:16, 4.20s/it]
28%|██▊ | 263/938 [18:00<45:18, 4.03s/it]
28%|██▊ | 264/938 [18:04<45:03, 4.01s/it]
28%|██▊ | 265/938 [18:07<43:08, 3.85s/it]
28%|██▊ | 266/938 [18:12<45:53, 4.10s/it]
28%|██▊ | 267/938 [18:16<45:51, 4.10s/it]
29%|██▊ | 268/938 [18:20<45:13, 4.05s/it]
29%|██▊ | 269/938 [18:24<44:38, 4.00s/it]
29%|██▉ | 270/938 [18:28<44:36, 4.01s/it]
{'loss': '0.3071', 'grad_norm': '0.4481', 'learning_rate': '8.976e-06', 'epoch': '0.576'}
29%|██▉ | 270/938 [18:28<44:36, 4.01s/it]
29%|██▉ | 271/938 [18:32<44:22, 3.99s/it]
29%|██▉ | 272/938 [18:36<44:47, 4.04s/it]
29%|██▉ | 273/938 [18:39<42:58, 3.88s/it]
29%|██▉ | 274/938 [18:43<42:24, 3.83s/it]
29%|██▉ | 275/938 [18:47<42:16, 3.83s/it]
29%|██▉ | 276/938 [18:51<44:21, 4.02s/it]
30%|██▉ | 277/938 [18:55<44:17, 4.02s/it]
30%|██▉ | 278/938 [19:00<45:15, 4.11s/it]
30%|██▉ | 279/938 [19:04<46:15, 4.21s/it]
30%|██▉ | 280/938 [19:08<46:05, 4.20s/it]
{'loss': '0.3033', 'grad_norm': '0.4673', 'learning_rate': '8.861e-06', 'epoch': '0.5973'}
30%|██▉ | 280/938 [19:08<46:05, 4.20s/it]
30%|██▉ | 281/938 [19:12<45:29, 4.15s/it]
30%|███ | 282/938 [19:16<44:06, 4.03s/it]
30%|███ | 283/938 [19:20<44:11, 4.05s/it]
30%|███ | 284/938 [19:24<43:15, 3.97s/it]
30%|███ | 285/938 [19:28<42:53, 3.94s/it]
30%|███ | 286/938 [19:32<42:39, 3.93s/it]
31%|███ | 287/938 [19:35<41:16, 3.80s/it]
31%|███ | 288/938 [19:39<41:42, 3.85s/it]
31%|███ | 289/938 [19:44<46:00, 4.25s/it]
31%|███ | 290/938 [19:49<48:03, 4.45s/it]
{'loss': '0.2964', 'grad_norm': '0.4601', 'learning_rate': '8.74e-06', 'epoch': '0.6187'}
31%|███ | 290/938 [19:49<48:03, 4.45s/it]
31%|███ | 291/938 [19:53<45:57, 4.26s/it]
31%|███ | 292/938 [19:58<46:53, 4.36s/it]
31%|███ | 293/938 [20:01<44:06, 4.10s/it]
31%|███▏ | 294/938 [20:05<43:25, 4.05s/it]
31%|███▏ | 295/938 [20:10<46:07, 4.30s/it]
32%|███▏ | 296/938 [20:14<44:08, 4.13s/it]
32%|███▏ | 297/938 [20:17<42:13, 3.95s/it]
32%|███▏ | 298/938 [20:23<46:16, 4.34s/it]
32%|███▏ | 299/938 [20:27<46:13, 4.34s/it]
32%|███▏ | 300/938 [20:32<49:07, 4.62s/it]
{'loss': '0.307', 'grad_norm': '0.4463', 'learning_rate': '8.614e-06', 'epoch': '0.64'}
32%|███▏ | 300/938 [20:32<49:07, 4.62s/it]
32%|███▏ | 301/938 [20:36<45:40, 4.30s/it]
32%|███▏ | 302/938 [20:40<44:14, 4.17s/it]
32%|███▏ | 303/938 [20:44<43:21, 4.10s/it]
32%|███▏ | 304/938 [20:48<44:52, 4.25s/it]
33%|███▎ | 305/938 [20:52<44:13, 4.19s/it]
33%|███▎ | 306/938 [20:56<43:14, 4.11s/it]
33%|███▎ | 307/938 [20:59<40:27, 3.85s/it]
33%|███▎ | 308/938 [21:04<41:10, 3.92s/it]
33%|███▎ | 309/938 [21:08<43:16, 4.13s/it]
33%|███▎ | 310/938 [21:12<42:17, 4.04s/it]
{'loss': '0.3074', 'grad_norm': '0.4809', 'learning_rate': '8.483e-06', 'epoch': '0.6613'}
33%|███▎ | 310/938 [21:12<42:17, 4.04s/it]
33%|███▎ | 311/938 [21:16<42:36, 4.08s/it]
33%|███▎ | 312/938 [21:20<42:35, 4.08s/it]
33%|███▎ | 313/938 [21:25<44:24, 4.26s/it]
33%|███▎ | 314/938 [21:29<45:05, 4.34s/it]
34%|███▎ | 315/938 [21:34<44:57, 4.33s/it]
34%|███▎ | 316/938 [21:38<45:15, 4.36s/it]
34%|███▍ | 317/938 [21:42<44:35, 4.31s/it]
34%|███▍ | 318/938 [21:47<44:30, 4.31s/it]
34%|███▍ | 319/938 [21:51<43:30, 4.22s/it]
34%|███▍ | 320/938 [21:55<43:29, 4.22s/it]
{'loss': '0.2807', 'grad_norm': '0.508', 'learning_rate': '8.347e-06', 'epoch': '0.6827'}
34%|███▍ | 320/938 [21:55<43:29, 4.22s/it]
34%|███▍ | 321/938 [21:59<43:57, 4.27s/it]
34%|███▍ | 322/938 [22:04<46:06, 4.49s/it]
34%|███▍ | 323/938 [22:08<45:05, 4.40s/it]
35%|███▍ | 324/938 [22:12<43:51, 4.29s/it]
35%|███▍ | 325/938 [22:17<44:41, 4.37s/it]
35%|███▍ | 326/938 [22:21<44:38, 4.38s/it]
35%|███▍ | 327/938 [22:25<42:34, 4.18s/it]
35%|███▍ | 328/938 [22:29<41:07, 4.04s/it]
35%|███▌ | 329/938 [22:33<41:53, 4.13s/it]
35%|███▌ | 330/938 [22:37<40:43, 4.02s/it]
{'loss': '0.2867', 'grad_norm': '0.4667', 'learning_rate': '8.206e-06', 'epoch': '0.704'}
35%|███▌ | 330/938 [22:37<40:43, 4.02s/it]
35%|███▌ | 331/938 [22:42<43:37, 4.31s/it]
35%|███▌ | 332/938 [22:47<45:11, 4.47s/it]
36%|███▌ | 333/938 [22:52<47:02, 4.67s/it]
36%|███▌ | 334/938 [22:56<45:52, 4.56s/it]
36%|███▌ | 335/938 [23:01<45:25, 4.52s/it]
36%|███▌ | 336/938 [23:05<45:27, 4.53s/it]
36%|███▌ | 337/938 [23:09<44:02, 4.40s/it]
36%|███▌ | 338/938 [23:14<44:46, 4.48s/it]
36%|███▌ | 339/938 [23:19<44:54, 4.50s/it]
36%|███▌ | 340/938 [23:23<44:38, 4.48s/it]
{'loss': '0.3078', 'grad_norm': '0.4525', 'learning_rate': '8.061e-06', 'epoch': '0.7253'}
36%|███▌ | 340/938 [23:23<44:38, 4.48s/it]
36%|███▋ | 341/938 [23:27<44:13, 4.45s/it]
36%|███▋ | 342/938 [23:32<44:05, 4.44s/it]
37%|███▋ | 343/938 [23:37<45:00, 4.54s/it]
37%|███▋ | 344/938 [23:41<44:51, 4.53s/it]
37%|███▋ | 345/938 [23:45<43:38, 4.42s/it]
37%|███▋ | 346/938 [23:50<44:42, 4.53s/it]
37%|███▋ | 347/938 [23:54<43:12, 4.39s/it]
37%|███▋ | 348/938 [23:58<42:40, 4.34s/it]
37%|███▋ | 349/938 [24:02<42:05, 4.29s/it]
37%|███▋ | 350/938 [24:06<40:58, 4.18s/it]
{'loss': '0.2937', 'grad_norm': '0.5167', 'learning_rate': '7.912e-06', 'epoch': '0.7467'}
37%|███▋ | 350/938 [24:06<40:58, 4.18s/it]
37%|███▋ | 351/938 [24:10<39:30, 4.04s/it]
38%|███▊ | 352/938 [24:14<39:22, 4.03s/it]
38%|███▊ | 353/938 [24:19<41:10, 4.22s/it]
38%|███▊ | 354/938 [24:23<41:40, 4.28s/it]
38%|███▊ | 355/938 [24:29<44:54, 4.62s/it]
38%|███▊ | 356/938 [24:33<43:35, 4.49s/it]
38%|███▊ | 357/938 [24:37<43:27, 4.49s/it]
38%|███▊ | 358/938 [24:41<41:30, 4.29s/it]
38%|███▊ | 359/938 [24:45<41:30, 4.30s/it]
38%|███▊ | 360/938 [24:50<41:17, 4.29s/it]
{'loss': '0.2857', 'grad_norm': '0.4678', 'learning_rate': '7.758e-06', 'epoch': '0.768'}
38%|███▊ | 360/938 [24:50<41:17, 4.29s/it]
38%|███▊ | 361/938 [24:55<43:08, 4.49s/it]
39%|███▊ | 362/938 [24:58<40:31, 4.22s/it]
39%|███▊ | 363/938 [25:02<39:23, 4.11s/it]
39%|███▉ | 364/938 [25:06<40:05, 4.19s/it]
39%|███▉ | 365/938 [25:10<39:19, 4.12s/it]
39%|███▉ | 366/938 [25:15<39:12, 4.11s/it]
39%|███▉ | 367/938 [25:19<39:19, 4.13s/it]
39%|███▉ | 368/938 [25:23<39:27, 4.15s/it]
39%|███▉ | 369/938 [25:27<39:03, 4.12s/it]
39%|███▉ | 370/938 [25:32<42:04, 4.44s/it]
{'loss': '0.2912', 'grad_norm': '0.4754', 'learning_rate': '7.601e-06', 'epoch': '0.7893'}
39%|███▉ | 370/938 [25:32<42:04, 4.44s/it]
40%|███▉ | 371/938 [25:37<44:35, 4.72s/it]
40%|███▉ | 372/938 [25:43<47:25, 5.03s/it]
40%|███▉ | 373/938 [25:47<44:42, 4.75s/it]
40%|███▉ | 374/938 [25:52<43:25, 4.62s/it]
40%|███▉ | 375/938 [25:56<43:26, 4.63s/it]
40%|████ | 376/938 [26:01<43:59, 4.70s/it]
40%|████ | 377/938 [26:05<41:49, 4.47s/it]
40%|████ | 378/938 [26:09<40:19, 4.32s/it]
40%|████ | 379/938 [26:13<40:00, 4.29s/it]
41%|████ | 380/938 [26:18<40:10, 4.32s/it]
{'loss': '0.2886', 'grad_norm': '0.4467', 'learning_rate': '7.441e-06', 'epoch': '0.8107'}
41%|████ | 380/938 [26:18<40:10, 4.32s/it]
41%|████ | 381/938 [26:21<38:08, 4.11s/it]
41%|████ | 382/938 [26:25<37:35, 4.06s/it]
41%|████ | 383/938 [26:29<37:40, 4.07s/it]
41%|████ | 384/938 [26:33<36:21, 3.94s/it]
41%|████ | 385/938 [26:37<37:47, 4.10s/it]
41%|████ | 386/938 [26:42<38:30, 4.19s/it]
41%|████▏ | 387/938 [26:46<37:34, 4.09s/it]
41%|████▏ | 388/938 [26:50<39:18, 4.29s/it]
41%|████▏ | 389/938 [26:55<40:00, 4.37s/it]
42%|████▏ | 390/938 [26:59<39:40, 4.34s/it]
{'loss': '0.2949', 'grad_norm': '0.4401', 'learning_rate': '7.277e-06', 'epoch': '0.832'}
42%|████▏ | 390/938 [26:59<39:40, 4.34s/it]
42%|████▏ | 391/938 [27:04<39:24, 4.32s/it]
42%|████▏ | 392/938 [27:08<40:26, 4.44s/it]
42%|████▏ | 393/938 [27:12<39:29, 4.35s/it]
42%|████▏ | 394/938 [27:17<40:22, 4.45s/it]
42%|████▏ | 395/938 [27:22<41:33, 4.59s/it]
42%|████▏ | 396/938 [27:26<38:52, 4.30s/it]
42%|████▏ | 397/938 [27:30<38:00, 4.22s/it]
42%|████▏ | 398/938 [27:34<37:14, 4.14s/it]
43%|████▎ | 399/938 [27:38<38:32, 4.29s/it]
43%|████▎ | 400/938 [27:43<38:37, 4.31s/it]
{'loss': '0.2888', 'grad_norm': '0.463', 'learning_rate': '7.109e-06', 'epoch': '0.8533'}
43%|████▎ | 400/938 [27:43<38:37, 4.31s/it]
43%|████▎ | 401/938 [27:47<37:50, 4.23s/it]
43%|████▎ | 402/938 [27:51<37:16, 4.17s/it]
43%|████▎ | 403/938 [27:55<37:10, 4.17s/it]
43%|████▎ | 404/938 [27:59<36:20, 4.08s/it]
43%|████▎ | 405/938 [28:02<34:51, 3.92s/it]
43%|████▎ | 406/938 [28:07<35:38, 4.02s/it]
43%|████▎ | 407/938 [28:11<36:30, 4.13s/it]
43%|████▎ | 408/938 [28:15<36:48, 4.17s/it]
44%|████▎ | 409/938 [28:20<37:11, 4.22s/it]
44%|████▎ | 410/938 [28:23<36:24, 4.14s/it]
{'loss': '0.3021', 'grad_norm': '0.5064', 'learning_rate': '6.939e-06', 'epoch': '0.8747'}
44%|████▎ | 410/938 [28:23<36:24, 4.14s/it]
44%|████▍ | 411/938 [28:28<36:54, 4.20s/it]
44%|████▍ | 412/938 [28:32<36:28, 4.16s/it]
44%|████▍ | 413/938 [28:36<36:03, 4.12s/it]
44%|████▍ | 414/938 [28:40<36:35, 4.19s/it]
44%|████▍ | 415/938 [28:45<37:15, 4.28s/it]
44%|████▍ | 416/938 [28:49<36:14, 4.17s/it]
44%|████▍ | 417/938 [28:53<36:53, 4.25s/it]
45%|████▍ | 418/938 [28:57<37:03, 4.28s/it]
45%|████▍ | 419/938 [29:02<36:41, 4.24s/it]
45%|████▍ | 420/938 [29:06<36:10, 4.19s/it]
{'loss': '0.2929', 'grad_norm': '0.4812', 'learning_rate': '6.766e-06', 'epoch': '0.896'}
45%|████▍ | 420/938 [29:06<36:10, 4.19s/it]
45%|████▍ | 421/938 [29:10<36:20, 4.22s/it]
45%|████▍ | 422/938 [29:14<35:15, 4.10s/it]
45%|████▌ | 423/938 [29:18<36:08, 4.21s/it]
45%|████▌ | 424/938 [29:22<35:51, 4.19s/it]
45%|████▌ | 425/938 [29:27<35:47, 4.19s/it]
45%|████▌ | 426/938 [29:30<34:01, 3.99s/it]
46%|████▌ | 427/938 [29:34<33:17, 3.91s/it]
46%|████▌ | 428/938 [29:38<33:55, 3.99s/it]
46%|████▌ | 429/938 [29:42<33:34, 3.96s/it]
46%|████▌ | 430/938 [29:46<34:09, 4.04s/it]
{'loss': '0.2888', 'grad_norm': '0.4532', 'learning_rate': '6.591e-06', 'epoch': '0.9173'}
46%|████▌ | 430/938 [29:46<34:09, 4.04s/it]
46%|████▌ | 431/938 [29:50<32:35, 3.86s/it]
46%|████▌ | 432/938 [29:53<32:24, 3.84s/it]
46%|████▌ | 433/938 [29:57<32:11, 3.82s/it]
46%|████▋ | 434/938 [30:02<34:43, 4.13s/it]
46%|████▋ | 435/938 [30:07<38:02, 4.54s/it]
46%|████▋ | 436/938 [30:12<37:47, 4.52s/it]
47%|████▋ | 437/938 [30:16<36:26, 4.36s/it]
47%|████▋ | 438/938 [30:20<35:57, 4.32s/it]
47%|████▋ | 439/938 [30:23<33:12, 3.99s/it]
47%|████▋ | 440/938 [30:28<34:49, 4.20s/it]
{'loss': '0.2961', 'grad_norm': '0.4447', 'learning_rate': '6.414e-06', 'epoch': '0.9387'}
47%|████▋ | 440/938 [30:28<34:49, 4.20s/it]
47%|████▋ | 441/938 [30:32<35:06, 4.24s/it]
47%|████▋ | 442/938 [30:36<33:11, 4.01s/it]
47%|████▋ | 443/938 [30:40<33:27, 4.05s/it]
47%|████▋ | 444/938 [30:44<33:29, 4.07s/it]
47%|████▋ | 445/938 [30:49<34:34, 4.21s/it]
48%|████▊ | 446/938 [30:53<35:37, 4.34s/it]
48%|████▊ | 447/938 [30:57<34:27, 4.21s/it]
48%|████▊ | 448/938 [31:01<33:42, 4.13s/it]
48%|████▊ | 449/938 [31:06<34:21, 4.22s/it]
48%|████▊ | 450/938 [31:09<33:31, 4.12s/it]
{'loss': '0.2955', 'grad_norm': '0.4198', 'learning_rate': '6.234e-06', 'epoch': '0.96'}
48%|████▊ | 450/938 [31:09<33:31, 4.12s/it]
48%|████▊ | 451/938 [31:14<34:52, 4.30s/it]
48%|████▊ | 452/938 [31:20<38:18, 4.73s/it]
48%|████▊ | 453/938 [31:24<37:01, 4.58s/it]
48%|████▊ | 454/938 [31:29<37:15, 4.62s/it]
49%|████▊ | 455/938 [31:34<37:43, 4.69s/it]
49%|████▊ | 456/938 [31:38<37:13, 4.63s/it]
49%|████▊ | 457/938 [31:42<35:28, 4.43s/it]
49%|████▉ | 458/938 [31:47<35:45, 4.47s/it]
49%|████▉ | 459/938 [31:50<33:49, 4.24s/it]
49%|████▉ | 460/938 [31:54<32:37, 4.09s/it]
{'loss': '0.2948', 'grad_norm': '0.4504', 'learning_rate': '6.053e-06', 'epoch': '0.9813'}
49%|████▉ | 460/938 [31:54<32:37, 4.09s/it]
49%|████▉ | 461/938 [31:58<31:45, 3.99s/it]
49%|████▉ | 462/938 [32:02<31:01, 3.91s/it]
49%|████▉ | 463/938 [32:06<31:19, 3.96s/it]
49%|████▉ | 464/938 [32:10<31:39, 4.01s/it]
50%|████▉ | 465/938 [32:14<31:21, 3.98s/it]
50%|████▉ | 466/938 [32:18<32:31, 4.14s/it]
50%|████▉ | 467/938 [32:22<32:13, 4.10s/it]
50%|████▉ | 468/938 [32:26<31:18, 4.00s/it]
50%|█████ | 469/938 [32:29<29:41, 3.80s/it]
50%|█████ | 470/938 [32:34<32:30, 4.17s/it]
{'loss': '0.2972', 'grad_norm': '0.5018', 'learning_rate': '5.87e-06', 'epoch': '1.002'}
50%|█████ | 470/938 [32:34<32:30, 4.17s/it]
50%|█████ | 471/938 [32:39<33:45, 4.34s/it]
50%|█████ | 472/938 [32:43<32:04, 4.13s/it]
50%|█████ | 473/938 [32:46<30:18, 3.91s/it]
51%|█████ | 474/938 [32:50<30:18, 3.92s/it]
51%|█████ | 475/938 [32:54<31:01, 4.02s/it]
51%|█████ | 476/938 [32:59<31:18, 4.07s/it]
51%|█████ | 477/938 [33:03<31:10, 4.06s/it]
51%|█████ | 478/938 [33:07<32:02, 4.18s/it]
51%|█████ | 479/938 [33:11<31:02, 4.06s/it]
51%|█████ | 480/938 [33:15<31:09, 4.08s/it]
{'loss': '0.2648', 'grad_norm': '0.4842', 'learning_rate': '5.686e-06', 'epoch': '1.023'}
51%|█████ | 480/938 [33:15<31:09, 4.08s/it]
51%|█████▏ | 481/938 [33:19<30:30, 4.01s/it]
51%|█████▏ | 482/938 [33:23<30:43, 4.04s/it]
51%|█████▏ | 483/938 [33:26<29:26, 3.88s/it]
52%|█████▏ | 484/938 [33:31<29:48, 3.94s/it]
52%|█████▏ | 485/938 [33:36<32:09, 4.26s/it]
52%|█████▏ | 486/938 [33:40<31:53, 4.23s/it]
52%|█████▏ | 487/938 [33:44<32:24, 4.31s/it]
52%|█████▏ | 488/938 [33:48<31:29, 4.20s/it]
52%|█████▏ | 489/938 [33:52<30:54, 4.13s/it]
52%|█████▏ | 490/938 [33:57<31:57, 4.28s/it]
{'loss': '0.271', 'grad_norm': '0.4402', 'learning_rate': '5.502e-06', 'epoch': '1.045'}
52%|█████▏ | 490/938 [33:57<31:57, 4.28s/it]
52%|█████▏ | 491/938 [34:00<30:38, 4.11s/it]
52%|█████▏ | 492/938 [34:05<31:30, 4.24s/it]
53%|█████▎ | 493/938 [34:09<31:30, 4.25s/it]
53%|█████▎ | 494/938 [34:13<30:53, 4.18s/it]
53%|█████▎ | 495/938 [34:17<30:05, 4.08s/it]
53%|█████▎ | 496/938 [34:22<31:00, 4.21s/it]
53%|█████▎ | 497/938 [34:26<31:54, 4.34s/it]
53%|█████▎ | 498/938 [34:30<30:09, 4.11s/it]
53%|█████▎ | 499/938 [34:34<29:19, 4.01s/it]
53%|█████▎ | 500/938 [34:38<30:08, 4.13s/it]
{'loss': '0.2706', 'grad_norm': '0.4749', 'learning_rate': '5.316e-06', 'epoch': '1.066'}
53%|█████▎ | 500/938 [34:38<30:08, 4.13s/it][INFO|trainer.py:3797] 2026-04-29 16:19:18,046 >> Saving model checkpoint to saves/qwen3_1.7b/math/fft/checkpoint-500
[INFO|configuration_utils.py:432] 2026-04-29 16:19:18,055 >> Configuration saved in saves/qwen3_1.7b/math/fft/checkpoint-500/config.json
[INFO|configuration_utils.py:803] 2026-04-29 16:19:18,058 >> Configuration saved in saves/qwen3_1.7b/math/fft/checkpoint-500/generation_config.json
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s]
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.28s/it]
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.29s/it]
[INFO|modeling_utils.py:3380] 2026-04-29 16:19:24,375 >> Model weights saved in saves/qwen3_1.7b/math/fft/checkpoint-500/model.safetensors
[INFO|tokenization_utils_base.py:3224] 2026-04-29 16:19:24,380 >> chat template saved in saves/qwen3_1.7b/math/fft/checkpoint-500/chat_template.jinja
[INFO|tokenization_utils_base.py:2078] 2026-04-29 16:19:24,384 >> tokenizer config file saved in saves/qwen3_1.7b/math/fft/checkpoint-500/tokenizer_config.json
53%|█████▎ | 501/938 [34:52<51:35, 7.08s/it]
54%|█████▎ | 502/938 [34:56<44:10, 6.08s/it]
54%|█████▎ | 503/938 [34:59<38:54, 5.37s/it]
54%|█████▎ | 504/938 [35:04<36:57, 5.11s/it]
54%|█████▍ | 505/938 [35:08<35:11, 4.88s/it]
54%|█████▍ | 506/938 [35:13<33:51, 4.70s/it]
54%|█████▍ | 507/938 [35:17<32:40, 4.55s/it]
54%|█████▍ | 508/938 [35:22<33:33, 4.68s/it]
54%|█████▍ | 509/938 [35:26<32:07, 4.49s/it]
54%|█████▍ | 510/938 [35:30<31:53, 4.47s/it]
{'loss': '0.2784', 'grad_norm': '0.4592', 'learning_rate': '5.13e-06', 'epoch': '1.087'}
54%|█████▍ | 510/938 [35:30<31:53, 4.47s/it]
54%|█████▍ | 511/938 [35:34<30:50, 4.33s/it]
55%|█████▍ | 512/938 [35:38<30:31, 4.30s/it]
55%|█████▍ | 513/938 [35:44<32:17, 4.56s/it]
55%|█████▍ | 514/938 [35:48<31:03, 4.39s/it]
55%|█████▍ | 515/938 [35:51<29:44, 4.22s/it]
55%|█████▌ | 516/938 [35:56<30:52, 4.39s/it]
55%|█████▌ | 517/938 [36:00<29:49, 4.25s/it]
55%|█████▌ | 518/938 [36:04<29:27, 4.21s/it]
55%|█████▌ | 519/938 [36:09<31:01, 4.44s/it]
55%|█████▌ | 520/938 [36:13<29:14, 4.20s/it]
{'loss': '0.2618', 'grad_norm': '0.4609', 'learning_rate': '4.944e-06', 'epoch': '1.109'}
55%|█████▌ | 520/938 [36:13<29:14, 4.20s/it]
56%|█████▌ | 521/938 [36:17<28:34, 4.11s/it]
56%|█████▌ | 522/938 [36:21<29:05, 4.20s/it]
56%|█████▌ | 523/938 [36:25<29:14, 4.23s/it]
56%|█████▌ | 524/938 [36:30<29:51, 4.33s/it]
56%|█████▌ | 525/938 [36:34<29:14, 4.25s/it]
56%|█████▌ | 526/938 [36:38<27:33, 4.01s/it]
56%|█████▌ | 527/938 [36:41<26:31, 3.87s/it]
56%|█████▋ | 528/938 [36:45<27:25, 4.01s/it]
56%|█████▋ | 529/938 [36:50<28:01, 4.11s/it]
57%|█████▋ | 530/938 [36:54<27:16, 4.01s/it]
{'loss': '0.2712', 'grad_norm': '0.4843', 'learning_rate': '4.758e-06', 'epoch': '1.13'}
57%|█████▋ | 530/938 [36:54<27:16, 4.01s/it]
57%|█████▋ | 531/938 [36:58<27:51, 4.11s/it]
57%|█████▋ | 532/938 [37:03<30:17, 4.48s/it]
57%|█████▋ | 533/938 [37:07<28:37, 4.24s/it]
57%|█████▋ | 534/938 [37:11<27:44, 4.12s/it]
57%|█████▋ | 535/938 [37:15<27:43, 4.13s/it]
57%|█████▋ | 536/938 [37:20<29:20, 4.38s/it]
57%|█████▋ | 537/938 [37:25<30:00, 4.49s/it]
57%|█████▋ | 538/938 [37:29<29:09, 4.37s/it]
57%|█████▋ | 539/938 [37:33<29:10, 4.39s/it]
58%|█████▊ | 540/938 [37:37<27:22, 4.13s/it]
{'loss': '0.2627', 'grad_norm': '0.5324', 'learning_rate': '4.572e-06', 'epoch': '1.151'}
58%|█████▊ | 540/938 [37:37<27:22, 4.13s/it]
58%|█████▊ | 541/938 [37:41<28:24, 4.29s/it]
58%|█████▊ | 542/938 [37:45<27:53, 4.23s/it]
58%|█████▊ | 543/938 [37:49<26:48, 4.07s/it]
58%|█████▊ | 544/938 [37:53<26:41, 4.06s/it]
58%|█████▊ | 545/938 [37:58<27:16, 4.16s/it]
58%|█████▊ | 546/938 [38:01<26:23, 4.04s/it]
58%|█████▊ | 547/938 [38:05<26:15, 4.03s/it]
58%|█████▊ | 548/938 [38:10<27:35, 4.24s/it]
59%|█████▊ | 549/938 [38:14<26:45, 4.13s/it]
59%|█████▊ | 550/938 [38:18<26:52, 4.16s/it]
{'loss': '0.2738', 'grad_norm': '0.4596', 'learning_rate': '4.387e-06', 'epoch': '1.173'}
59%|█████▊ | 550/938 [38:18<26:52, 4.16s/it]
59%|█████▊ | 551/938 [38:22<26:25, 4.10s/it]
59%|█████▉ | 552/938 [38:26<26:24, 4.10s/it]
59%|█████▉ | 553/938 [38:30<26:34, 4.14s/it]
59%|█████▉ | 554/938 [38:34<25:32, 3.99s/it]
59%|█████▉ | 555/938 [38:38<25:14, 3.95s/it]
59%|█████▉ | 556/938 [38:42<25:23, 3.99s/it]
59%|█████▉ | 557/938 [38:46<24:56, 3.93s/it]
59%|█████▉ | 558/938 [38:50<24:26, 3.86s/it]
60%|█████▉ | 559/938 [38:54<25:26, 4.03s/it]
60%|█████▉ | 560/938 [38:58<24:56, 3.96s/it]
{'loss': '0.2654', 'grad_norm': '0.4884', 'learning_rate': '4.203e-06', 'epoch': '1.194'}
60%|█████▉ | 560/938 [38:58<24:56, 3.96s/it]
60%|█████▉ | 561/938 [39:01<24:21, 3.88s/it]
60%|█████▉ | 562/938 [39:06<25:09, 4.01s/it]
60%|██████ | 563/938 [39:10<24:51, 3.98s/it]
60%|██████ | 564/938 [39:13<24:14, 3.89s/it]
60%|██████ | 565/938 [39:18<24:45, 3.98s/it]
60%|██████ | 566/938 [39:21<24:34, 3.96s/it]
60%|██████ | 567/938 [39:26<24:52, 4.02s/it]
61%|██████ | 568/938 [39:30<25:31, 4.14s/it]
61%|██████ | 569/938 [39:35<27:18, 4.44s/it]
61%|██████ | 570/938 [39:40<28:20, 4.62s/it]
{'loss': '0.2673', 'grad_norm': '0.4471', 'learning_rate': '4.02e-06', 'epoch': '1.215'}
61%|██████ | 570/938 [39:40<28:20, 4.62s/it]
61%|██████ | 571/938 [39:44<26:47, 4.38s/it]
61%|██████ | 572/938 [39:48<25:11, 4.13s/it]
61%|██████ | 573/938 [39:52<25:18, 4.16s/it]
61%|██████ | 574/938 [39:55<23:45, 3.92s/it]
61%|██████▏ | 575/938 [40:00<24:30, 4.05s/it]
61%|██████▏ | 576/938 [40:03<24:09, 4.00s/it]
62%|██████▏ | 577/938 [40:08<24:54, 4.14s/it]
62%|██████▏ | 578/938 [40:12<24:08, 4.02s/it]
62%|██████▏ | 579/938 [40:16<23:52, 3.99s/it]
62%|██████▏ | 580/938 [40:20<24:08, 4.05s/it]
{'loss': '0.263', 'grad_norm': '0.4469', 'learning_rate': '3.838e-06', 'epoch': '1.237'}
62%|██████▏ | 580/938 [40:20<24:08, 4.05s/it]
62%|██████▏ | 581/938 [40:24<23:42, 3.99s/it]
62%|██████▏ | 582/938 [40:27<23:25, 3.95s/it]
62%|██████▏ | 583/938 [40:31<22:44, 3.85s/it]
62%|██████▏ | 584/938 [40:37<25:52, 4.38s/it]
62%|██████▏ | 585/938 [40:40<24:17, 4.13s/it]
62%|██████▏ | 586/938 [40:46<27:31, 4.69s/it]
63%|██████▎ | 587/938 [40:51<27:25, 4.69s/it]
63%|██████▎ | 588/938 [40:55<26:50, 4.60s/it]
63%|██████▎ | 589/938 [41:00<26:11, 4.50s/it]
63%|██████▎ | 590/938 [41:04<25:13, 4.35s/it]
{'loss': '0.2718', 'grad_norm': '0.4668', 'learning_rate': '3.658e-06', 'epoch': '1.258'}
63%|██████▎ | 590/938 [41:04<25:13, 4.35s/it]
63%|██████▎ | 591/938 [41:08<25:10, 4.35s/it]
63%|██████▎ | 592/938 [41:11<23:41, 4.11s/it]
63%|██████▎ | 593/938 [41:16<24:42, 4.30s/it]
63%|██████▎ | 594/938 [41:22<26:22, 4.60s/it]
63%|██████▎ | 595/938 [41:26<25:15, 4.42s/it]
64%|██████▎ | 596/938 [41:29<24:13, 4.25s/it]
64%|██████▎ | 597/938 [41:34<25:31, 4.49s/it]
64%|██████▍ | 598/938 [41:38<23:56, 4.22s/it]
64%|██████▍ | 599/938 [41:42<23:23, 4.14s/it]
64%|██████▍ | 600/938 [41:46<23:35, 4.19s/it]
{'loss': '0.2709', 'grad_norm': '0.4404', 'learning_rate': '3.48e-06', 'epoch': '1.279'}
64%|██████▍ | 600/938 [41:46<23:35, 4.19s/it]
64%|██████▍ | 601/938 [41:51<24:02, 4.28s/it]
64%|██████▍ | 602/938 [41:55<24:34, 4.39s/it]
64%|██████▍ | 603/938 [42:00<24:19, 4.36s/it]
64%|██████▍ | 604/938 [42:03<22:36, 4.06s/it]
64%|██████▍ | 605/938 [42:07<22:38, 4.08s/it]
65%|██████▍ | 606/938 [42:12<23:04, 4.17s/it]
65%|██████▍ | 607/938 [42:16<23:26, 4.25s/it]
65%|██████▍ | 608/938 [42:21<23:52, 4.34s/it]
65%|██████▍ | 609/938 [42:25<23:14, 4.24s/it]
65%|██████▌ | 610/938 [42:29<22:58, 4.20s/it]
{'loss': '0.2683', 'grad_norm': '0.5088', 'learning_rate': '3.304e-06', 'epoch': '1.301'}
65%|██████▌ | 610/938 [42:29<22:58, 4.20s/it]
65%|██████▌ | 611/938 [42:33<22:25, 4.11s/it]
65%|██████▌ | 612/938 [42:37<22:43, 4.18s/it]
65%|██████▌ | 613/938 [42:41<22:51, 4.22s/it]
65%|██████▌ | 614/938 [42:45<22:49, 4.23s/it]
66%|██████▌ | 615/938 [42:50<22:34, 4.19s/it]
66%|██████▌ | 616/938 [42:54<22:15, 4.15s/it]
66%|██████▌ | 617/938 [42:57<21:21, 3.99s/it]
66%|██████▌ | 618/938 [43:02<21:59, 4.12s/it]
66%|██████▌ | 619/938 [43:06<21:45, 4.09s/it]
66%|██████▌ | 620/938 [43:10<21:18, 4.02s/it]
{'loss': '0.2715', 'grad_norm': '0.4963', 'learning_rate': '3.13e-06', 'epoch': '1.322'}
66%|██████▌ | 620/938 [43:10<21:18, 4.02s/it]
66%|██████▌ | 621/938 [43:13<21:00, 3.98s/it]
66%|██████▋ | 622/938 [43:18<22:16, 4.23s/it]
66%|██████▋ | 623/938 [43:22<22:11, 4.23s/it]
67%|██████▋ | 624/938 [43:27<22:39, 4.33s/it]
67%|██████▋ | 625/938 [43:32<22:49, 4.37s/it]
67%|██████▋ | 626/938 [43:35<21:50, 4.20s/it]
67%|██████▋ | 627/938 [43:39<21:06, 4.07s/it]
67%|██████▋ | 628/938 [43:43<20:58, 4.06s/it]
67%|██████▋ | 629/938 [43:47<20:05, 3.90s/it]
67%|██████▋ | 630/938 [43:51<20:23, 3.97s/it]
{'loss': '0.2726', 'grad_norm': '0.4921', 'learning_rate': '2.958e-06', 'epoch': '1.343'}
67%|██████▋ | 630/938 [43:51<20:23, 3.97s/it]
67%|██████▋ | 631/938 [43:55<20:03, 3.92s/it]
67%|██████▋ | 632/938 [43:59<20:01, 3.93s/it]
67%|██████▋ | 633/938 [44:02<19:48, 3.90s/it]
68%|██████▊ | 634/938 [44:07<21:27, 4.24s/it]
68%|██████▊ | 635/938 [44:11<20:24, 4.04s/it]
68%|██████▊ | 636/938 [44:15<20:26, 4.06s/it]
68%|██████▊ | 637/938 [44:19<20:42, 4.13s/it]
68%|██████▊ | 638/938 [44:24<20:42, 4.14s/it]
68%|██████▊ | 639/938 [44:28<20:56, 4.20s/it]
68%|██████▊ | 640/938 [44:32<20:11, 4.07s/it]
{'loss': '0.2665', 'grad_norm': '0.4858', 'learning_rate': '2.79e-06', 'epoch': '1.365'}
68%|██████▊ | 640/938 [44:32<20:11, 4.07s/it]
68%|██████▊ | 641/938 [44:36<20:03, 4.05s/it]
68%|██████▊ | 642/938 [44:39<19:42, 3.99s/it]
69%|██████▊ | 643/938 [44:43<19:30, 3.97s/it]
69%|██████▊ | 644/938 [44:48<20:12, 4.12s/it]
69%|██████▉ | 645/938 [44:53<21:14, 4.35s/it]
69%|██████▉ | 646/938 [44:57<20:17, 4.17s/it]
69%|██████▉ | 647/938 [45:00<19:33, 4.03s/it]
69%|██████▉ | 648/938 [45:04<19:18, 4.00s/it]
69%|██████▉ | 649/938 [45:08<18:56, 3.93s/it]
69%|██████▉ | 650/938 [45:12<18:31, 3.86s/it]
{'loss': '0.2647', 'grad_norm': '0.4581', 'learning_rate': '2.625e-06', 'epoch': '1.386'}
69%|██████▉ | 650/938 [45:12<18:31, 3.86s/it]
69%|██████▉ | 651/938 [45:16<18:40, 3.90s/it]
70%|██████▉ | 652/938 [45:20<19:59, 4.19s/it]
70%|██████▉ | 653/938 [45:24<19:13, 4.05s/it]
70%|██████▉ | 654/938 [45:29<19:33, 4.13s/it]
70%|██████▉ | 655/938 [45:32<19:09, 4.06s/it]
70%|██████▉ | 656/938 [45:37<19:19, 4.11s/it]
70%|███████ | 657/938 [45:41<19:32, 4.17s/it]
70%|███████ | 658/938 [45:45<19:41, 4.22s/it]
70%|███████ | 659/938 [45:49<18:56, 4.07s/it]
70%|███████ | 660/938 [45:53<19:03, 4.11s/it]
{'loss': '0.2691', 'grad_norm': '0.4541', 'learning_rate': '2.462e-06', 'epoch': '1.407'}
70%|███████ | 660/938 [45:53<19:03, 4.11s/it]
70%|███████ | 661/938 [45:57<18:50, 4.08s/it]
71%|███████ | 662/938 [46:01<18:39, 4.06s/it]
71%|███████ | 663/938 [46:06<19:08, 4.18s/it]
71%|███████ | 664/938 [46:10<18:35, 4.07s/it]
71%|███████ | 665/938 [46:13<17:45, 3.90s/it]
71%|███████ | 666/938 [46:18<18:33, 4.09s/it]
71%|███████ | 667/938 [46:22<18:26, 4.08s/it]
71%|███████ | 668/938 [46:26<18:29, 4.11s/it]
71%|███████▏ | 669/938 [46:30<18:58, 4.23s/it]
71%|███████▏ | 670/938 [46:34<18:44, 4.20s/it]
{'loss': '0.2636', 'grad_norm': '0.5011', 'learning_rate': '2.304e-06', 'epoch': '1.429'}
71%|███████▏ | 670/938 [46:34<18:44, 4.20s/it]
72%|███████▏ | 671/938 [46:38<17:55, 4.03s/it]
72%|███████▏ | 672/938 [46:42<17:16, 3.90s/it]
72%|███████▏ | 673/938 [46:46<17:26, 3.95s/it]
72%|███████▏ | 674/938 [46:50<17:51, 4.06s/it]
72%|███████▏ | 675/938 [46:55<18:41, 4.27s/it]
72%|███████▏ | 676/938 [46:58<17:53, 4.10s/it]
72%|███████▏ | 677/938 [47:03<17:55, 4.12s/it]
72%|███████▏ | 678/938 [47:07<17:47, 4.11s/it]
72%|███████▏ | 679/938 [47:10<17:06, 3.96s/it]
72%|███████▏ | 680/938 [47:15<17:20, 4.03s/it]
{'loss': '0.2607', 'grad_norm': '0.4419', 'learning_rate': '2.149e-06', 'epoch': '1.45'}
72%|███████▏ | 680/938 [47:15<17:20, 4.03s/it]
73%|███████▎ | 681/938 [47:18<16:31, 3.86s/it]
73%|███████▎ | 682/938 [47:22<16:57, 3.97s/it]
73%|███████▎ | 683/938 [47:27<17:15, 4.06s/it]
73%|███████▎ | 684/938 [47:30<16:50, 3.98s/it]
73%|███████▎ | 685/938 [47:34<17:00, 4.03s/it]
73%|███████▎ | 686/938 [47:39<17:15, 4.11s/it]
73%|███████▎ | 687/938 [47:43<17:36, 4.21s/it]
73%|███████▎ | 688/938 [47:47<16:29, 3.96s/it]
73%|███████▎ | 689/938 [47:51<16:25, 3.96s/it]
74%|███████▎ | 690/938 [47:54<16:02, 3.88s/it]
{'loss': '0.2723', 'grad_norm': '0.4483', 'learning_rate': '1.998e-06', 'epoch': '1.471'}
74%|███████▎ | 690/938 [47:54<16:02, 3.88s/it]
74%|███████▎ | 691/938 [47:58<16:26, 3.99s/it]
74%|███████▍ | 692/938 [48:02<16:10, 3.94s/it]
74%|███████▍ | 693/938 [48:06<16:01, 3.92s/it]
74%|███████▍ | 694/938 [48:10<15:36, 3.84s/it]
74%|███████▍ | 695/938 [48:14<15:35, 3.85s/it]
74%|███████▍ | 696/938 [48:18<15:40, 3.89s/it]
74%|███████▍ | 697/938 [48:22<15:58, 3.98s/it]
74%|███████▍ | 698/938 [48:26<16:08, 4.03s/it]
75%|███████▍ | 699/938 [48:30<16:25, 4.12s/it]
75%|███████▍ | 700/938 [48:35<17:15, 4.35s/it]
{'loss': '0.2778', 'grad_norm': '0.4514', 'learning_rate': '1.851e-06', 'epoch': '1.493'}
75%|███████▍ | 700/938 [48:35<17:15, 4.35s/it]
75%|███████▍ | 701/938 [48:39<16:56, 4.29s/it]
75%|███████▍ | 702/938 [48:43<16:04, 4.09s/it]
75%|███████▍ | 703/938 [48:48<16:35, 4.24s/it]
75%|███████▌ | 704/938 [48:53<17:30, 4.49s/it]
75%|███████▌ | 705/938 [48:56<16:23, 4.22s/it]
75%|███████▌ | 706/938 [49:01<16:30, 4.27s/it]
75%|███████▌ | 707/938 [49:05<16:25, 4.26s/it]
75%|███████▌ | 708/938 [49:09<16:37, 4.34s/it]
76%|███████▌ | 709/938 [49:14<16:37, 4.36s/it]
76%|███████▌ | 710/938 [49:18<16:28, 4.33s/it]
{'loss': '0.2632', 'grad_norm': '0.4736', 'learning_rate': '1.709e-06', 'epoch': '1.514'}
76%|███████▌ | 710/938 [49:18<16:28, 4.33s/it]
76%|███████▌ | 711/938 [49:22<16:19, 4.32s/it]
76%|███████▌ | 712/938 [49:26<15:48, 4.19s/it]
76%|███████▌ | 713/938 [49:30<15:14, 4.06s/it]
76%|███████▌ | 714/938 [49:35<15:45, 4.22s/it]
76%|███████▌ | 715/938 [49:38<15:01, 4.04s/it]
76%|███████▋ | 716/938 [49:42<14:33, 3.93s/it]
76%|███████▋ | 717/938 [49:46<14:20, 3.90s/it]
77%|███████▋ | 718/938 [49:49<14:08, 3.85s/it]
77%|███████▋ | 719/938 [49:53<13:49, 3.79s/it]
77%|███████▋ | 720/938 [49:58<14:40, 4.04s/it]
{'loss': '0.2699', 'grad_norm': '0.4452', 'learning_rate': '1.571e-06', 'epoch': '1.535'}
77%|███████▋ | 720/938 [49:58<14:40, 4.04s/it]
77%|███████▋ | 721/938 [50:03<15:28, 4.28s/it]
77%|███████▋ | 722/938 [50:06<14:41, 4.08s/it]
77%|███████▋ | 723/938 [50:11<15:13, 4.25s/it]
77%|███████▋ | 724/938 [50:15<15:15, 4.28s/it]
77%|███████▋ | 725/938 [50:20<15:49, 4.46s/it]
77%|███████▋ | 726/938 [50:24<15:09, 4.29s/it]
78%|███████▊ | 727/938 [50:28<14:46, 4.20s/it]
78%|███████▊ | 728/938 [50:32<14:17, 4.08s/it]
78%|███████▊ | 729/938 [50:36<14:41, 4.22s/it]
78%|███████▊ | 730/938 [50:41<15:25, 4.45s/it]
{'loss': '0.2556', 'grad_norm': '0.4535', 'learning_rate': '1.438e-06', 'epoch': '1.557'}
78%|███████▊ | 730/938 [50:41<15:25, 4.45s/it]
78%|███████▊ | 731/938 [50:46<15:07, 4.38s/it]
78%|███████▊ | 732/938 [50:50<14:44, 4.29s/it]
78%|███████▊ | 733/938 [50:54<14:54, 4.36s/it]
78%|███████▊ | 734/938 [50:58<14:36, 4.30s/it]
78%|███████▊ | 735/938 [51:02<13:51, 4.10s/it]
78%|███████▊ | 736/938 [51:06<13:56, 4.14s/it]
79%|███████▊ | 737/938 [51:10<13:24, 4.00s/it]
79%|███████▊ | 738/938 [51:14<13:35, 4.08s/it]
79%|███████▉ | 739/938 [51:18<13:02, 3.93s/it]
79%|███████▉ | 740/938 [51:22<13:25, 4.07s/it]
{'loss': '0.2628', 'grad_norm': '0.4523', 'learning_rate': '1.31e-06', 'epoch': '1.578'}
79%|███████▉ | 740/938 [51:22<13:25, 4.07s/it]
79%|███████▉ | 741/938 [51:26<13:34, 4.13s/it]
79%|███████▉ | 742/938 [51:30<13:21, 4.09s/it]
79%|███████▉ | 743/938 [51:34<13:10, 4.06s/it]
79%|███████▉ | 744/938 [51:38<12:53, 3.99s/it]
79%|███████▉ | 745/938 [51:42<12:46, 3.97s/it]
80%|███████▉ | 746/938 [51:46<12:22, 3.87s/it]
80%|███████▉ | 747/938 [51:49<12:06, 3.80s/it]
80%|███████▉ | 748/938 [51:53<12:09, 3.84s/it]
80%|███████▉ | 749/938 [51:57<11:41, 3.71s/it]
80%|███████▉ | 750/938 [52:02<12:46, 4.08s/it]
{'loss': '0.2727', 'grad_norm': '0.4335', 'learning_rate': '1.187e-06', 'epoch': '1.599'}
80%|███████▉ | 750/938 [52:02<12:46, 4.08s/it]
80%|████████ | 751/938 [52:06<13:06, 4.21s/it]
80%|████████ | 752/938 [52:10<12:57, 4.18s/it]
80%|████████ | 753/938 [52:14<12:39, 4.10s/it]
80%|████████ | 754/938 [52:19<13:10, 4.30s/it]
80%|████████ | 755/938 [52:24<13:26, 4.41s/it]
81%|████████ | 756/938 [52:27<12:37, 4.16s/it]
81%|████████ | 757/938 [52:32<13:26, 4.46s/it]
81%|████████ | 758/938 [52:36<13:00, 4.34s/it]
81%|████████ | 759/938 [52:41<12:55, 4.33s/it]
81%|████████ | 760/938 [52:45<12:51, 4.34s/it]
{'loss': '0.2753', 'grad_norm': '0.4579', 'learning_rate': '1.069e-06', 'epoch': '1.621'}
81%|████████ | 760/938 [52:45<12:51, 4.34s/it]
81%|████████ | 761/938 [52:49<12:26, 4.22s/it]
81%|████████ | 762/938 [52:53<12:20, 4.21s/it]
81%|████████▏ | 763/938 [52:57<11:59, 4.11s/it]
81%|████████▏ | 764/938 [53:01<11:59, 4.13s/it]
82%|████████▏ | 765/938 [53:05<11:49, 4.10s/it]
82%|████████▏ | 766/938 [53:09<11:52, 4.14s/it]
82%|████████▏ | 767/938 [53:15<13:16, 4.66s/it]
82%|████████▏ | 768/938 [53:19<12:42, 4.49s/it]
82%|████████▏ | 769/938 [53:23<12:11, 4.33s/it]
82%|████████▏ | 770/938 [53:28<11:56, 4.27s/it]
{'loss': '0.258', 'grad_norm': '0.4879', 'learning_rate': '9.571e-07', 'epoch': '1.642'}
82%|████████▏ | 770/938 [53:28<11:56, 4.27s/it]
82%|████████▏ | 771/938 [53:32<12:10, 4.37s/it]
82%|████████▏ | 772/938 [53:36<11:49, 4.28s/it]
82%|████████▏ | 773/938 [53:40<11:13, 4.08s/it]
83%|████████▎ | 774/938 [53:44<11:29, 4.21s/it]
83%|████████▎ | 775/938 [53:48<11:17, 4.15s/it]
83%|████████▎ | 776/938 [53:52<11:00, 4.08s/it]
83%|████████▎ | 777/938 [53:57<11:13, 4.18s/it]
83%|████████▎ | 778/938 [54:02<11:47, 4.42s/it]
83%|████████▎ | 779/938 [54:06<11:53, 4.49s/it]
83%|████████▎ | 780/938 [54:10<11:26, 4.34s/it]
{'loss': '0.2685', 'grad_norm': '0.4277', 'learning_rate': '8.504e-07', 'epoch': '1.663'}
83%|████████▎ | 780/938 [54:10<11:26, 4.34s/it]
83%|████████▎ | 781/938 [54:15<11:38, 4.45s/it]
83%|████████▎ | 782/938 [54:19<11:13, 4.32s/it]
83%|████████▎ | 783/938 [54:23<11:00, 4.26s/it]
84%|████████▎ | 784/938 [54:27<10:50, 4.23s/it]
84%|████████▎ | 785/938 [54:32<11:28, 4.50s/it]
84%|████████▍ | 786/938 [54:36<10:56, 4.32s/it]
84%|████████▍ | 787/938 [54:40<10:12, 4.05s/it]
84%|████████▍ | 788/938 [54:44<10:23, 4.15s/it]
84%|████████▍ | 789/938 [54:48<10:06, 4.07s/it]
84%|████████▍ | 790/938 [54:52<10:03, 4.08s/it]
{'loss': '0.266', 'grad_norm': '0.4804', 'learning_rate': '7.495e-07', 'epoch': '1.685'}
84%|████████▍ | 790/938 [54:52<10:03, 4.08s/it]
84%|████████▍ | 791/938 [54:56<09:48, 4.01s/it]
84%|████████▍ | 792/938 [55:00<09:55, 4.08s/it]
85%|████████▍ | 793/938 [55:04<09:38, 3.99s/it]
85%|████████▍ | 794/938 [55:09<10:15, 4.27s/it]
85%|████████▍ | 795/938 [55:13<09:50, 4.13s/it]
85%|████████▍ | 796/938 [55:18<10:16, 4.34s/it]
85%|████████▍ | 797/938 [55:21<09:48, 4.17s/it]
85%|████████▌ | 798/938 [55:26<09:52, 4.23s/it]
85%|████████▌ | 799/938 [55:30<09:48, 4.23s/it]
85%|████████▌ | 800/938 [55:34<09:39, 4.20s/it]
{'loss': '0.2572', 'grad_norm': '0.4831', 'learning_rate': '6.544e-07', 'epoch': '1.706'}
85%|████████▌ | 800/938 [55:34<09:39, 4.20s/it]
85%|████████▌ | 801/938 [55:38<09:30, 4.16s/it]
86%|████████▌ | 802/938 [55:42<09:27, 4.18s/it]
86%|████████▌ | 803/938 [55:47<09:27, 4.21s/it]
86%|████████▌ | 804/938 [55:50<09:08, 4.10s/it]
86%|████████▌ | 805/938 [55:54<08:47, 3.96s/it]
86%|████████▌ | 806/938 [55:59<09:01, 4.10s/it]
86%|████████▌ | 807/938 [56:03<09:08, 4.19s/it]
86%|████████▌ | 808/938 [56:08<09:25, 4.35s/it]
86%|████████▌ | 809/938 [56:12<09:12, 4.28s/it]
86%|████████▋ | 810/938 [56:15<08:38, 4.05s/it]
{'loss': '0.2663', 'grad_norm': '0.5086', 'learning_rate': '5.654e-07', 'epoch': '1.727'}
86%|████████▋ | 810/938 [56:15<08:38, 4.05s/it]
86%|████████▋ | 811/938 [56:20<08:46, 4.15s/it]
87%|████████▋ | 812/938 [56:24<08:59, 4.28s/it]
87%|████████▋ | 813/938 [56:28<08:51, 4.25s/it]
87%|████████▋ | 814/938 [56:33<08:52, 4.29s/it]
87%|████████▋ | 815/938 [56:37<08:38, 4.22s/it]
87%|████████▋ | 816/938 [56:41<08:43, 4.29s/it]
87%|████████▋ | 817/938 [56:46<08:42, 4.32s/it]
87%|████████▋ | 818/938 [56:51<09:20, 4.67s/it]
87%|████████▋ | 819/938 [56:56<09:12, 4.64s/it]
87%|████████▋ | 820/938 [57:00<08:35, 4.37s/it]
{'loss': '0.2616', 'grad_norm': '0.478', 'learning_rate': '4.825e-07', 'epoch': '1.749'}
87%|████████▋ | 820/938 [57:00<08:35, 4.37s/it]
88%|████████▊ | 821/938 [57:05<08:53, 4.56s/it]
88%|████████▊ | 822/938 [57:08<08:05, 4.19s/it]
88%|████████▊ | 823/938 [57:12<07:51, 4.10s/it]
88%|████████▊ | 824/938 [57:16<07:42, 4.06s/it]
88%|████████▊ | 825/938 [57:20<07:34, 4.02s/it]
88%|████████▊ | 826/938 [57:24<07:33, 4.05s/it]
88%|████████▊ | 827/938 [57:28<07:27, 4.03s/it]
88%|████████▊ | 828/938 [57:32<07:31, 4.10s/it]
88%|████████▊ | 829/938 [57:36<07:16, 4.00s/it]
88%|████████▊ | 830/938 [57:40<07:15, 4.03s/it]
{'loss': '0.2693', 'grad_norm': '0.4145', 'learning_rate': '4.059e-07', 'epoch': '1.77'}
88%|████████▊ | 830/938 [57:40<07:15, 4.03s/it]
89%|████████▊ | 831/938 [57:44<07:20, 4.11s/it]
89%|████████▊ | 832/938 [57:48<06:56, 3.93s/it]
89%|████████▉ | 833/938 [57:52<07:15, 4.15s/it]
89%|████████▉ | 834/938 [57:56<07:01, 4.05s/it]
89%|████████▉ | 835/938 [58:01<07:07, 4.15s/it]
89%|████████▉ | 836/938 [58:05<07:13, 4.25s/it]
89%|████████▉ | 837/938 [58:09<07:04, 4.20s/it]
89%|████████▉ | 838/938 [58:13<06:45, 4.05s/it]
89%|████████▉ | 839/938 [58:17<06:43, 4.08s/it]
90%|████████▉ | 840/938 [58:21<06:48, 4.17s/it]
{'loss': '0.2727', 'grad_norm': '0.4312', 'learning_rate': '3.357e-07', 'epoch': '1.791'}
90%|████████▉ | 840/938 [58:21<06:48, 4.17s/it]
90%|████████▉ | 841/938 [58:25<06:36, 4.08s/it]
90%|████████▉ | 842/938 [58:30<06:53, 4.31s/it]
90%|████████▉ | 843/938 [58:34<06:36, 4.18s/it]
90%|████████▉ | 844/938 [58:38<06:27, 4.12s/it]
90%|█████████ | 845/938 [58:42<06:24, 4.14s/it]
90%|█████████ | 846/938 [58:46<06:08, 4.00s/it]
90%|█████████ | 847/938 [58:50<06:16, 4.14s/it]
90%|█████████ | 848/938 [58:54<06:14, 4.16s/it]
91%|█████████ | 849/938 [58:58<05:56, 4.00s/it]
91%|█████████ | 850/938 [59:02<05:47, 3.95s/it]
{'loss': '0.276', 'grad_norm': '0.465', 'learning_rate': '2.719e-07', 'epoch': '1.813'}
91%|█████████ | 850/938 [59:02<05:47, 3.95s/it]
91%|█████████ | 851/938 [59:06<05:51, 4.04s/it]
91%|█████████ | 852/938 [59:11<05:56, 4.14s/it]
91%|█████████ | 853/938 [59:15<06:01, 4.26s/it]
91%|█████████ | 854/938 [59:19<05:51, 4.19s/it]
91%|█████████ | 855/938 [59:23<05:42, 4.12s/it]
91%|█████████▏| 856/938 [59:27<05:27, 3.99s/it]
91%|█████████▏| 857/938 [59:31<05:32, 4.10s/it]
91%|█████████▏| 858/938 [59:35<05:23, 4.04s/it]
92%|█████████▏| 859/938 [59:39<05:10, 3.93s/it]
92%|█████████▏| 860/938 [59:43<05:08, 3.96s/it]
{'loss': '0.2737', 'grad_norm': '0.4531', 'learning_rate': '2.146e-07', 'epoch': '1.834'}
92%|█████████▏| 860/938 [59:43<05:08, 3.96s/it]
92%|█████████▏| 861/938 [59:47<05:19, 4.14s/it]
92%|█████████▏| 862/938 [59:51<05:13, 4.13s/it]
92%|█████████▏| 863/938 [59:56<05:27, 4.37s/it]
92%|█████████▏| 864/938 [1:00:00<05:16, 4.28s/it]
92%|█████████▏| 865/938 [1:00:04<05:08, 4.22s/it]
92%|█████████▏| 866/938 [1:00:09<05:02, 4.20s/it]
92%|█████████▏| 867/938 [1:00:13<05:02, 4.26s/it]
93%|█████████▎| 868/938 [1:00:18<05:19, 4.57s/it]
93%|█████████▎| 869/938 [1:00:23<05:21, 4.65s/it]
93%|█████████▎| 870/938 [1:00:28<05:18, 4.69s/it]
{'loss': '0.2506', 'grad_norm': '0.4372', 'learning_rate': '1.64e-07', 'epoch': '1.855'}
93%|█████████▎| 870/938 [1:00:28<05:18, 4.69s/it]
93%|█████████▎| 871/938 [1:00:32<05:06, 4.57s/it]
93%|█████████▎| 872/938 [1:00:36<04:51, 4.41s/it]
93%|█████████▎| 873/938 [1:00:41<04:44, 4.38s/it]
93%|█████████▎| 874/938 [1:00:45<04:34, 4.28s/it]
93%|█████████▎| 875/938 [1:00:49<04:24, 4.19s/it]
93%|█████████▎| 876/938 [1:00:53<04:14, 4.11s/it]
93%|█████████▎| 877/938 [1:00:57<04:17, 4.23s/it]
94%|█████████▎| 878/938 [1:01:01<04:01, 4.03s/it]
94%|█████████▎| 879/938 [1:01:04<03:51, 3.92s/it]
94%|█████████▍| 880/938 [1:01:08<03:46, 3.90s/it]
{'loss': '0.2735', 'grad_norm': '0.464', 'learning_rate': '1.201e-07', 'epoch': '1.877'}
94%|█████████▍| 880/938 [1:01:08<03:46, 3.90s/it]
94%|█████████▍| 881/938 [1:01:12<03:45, 3.95s/it]
94%|█████████▍| 882/938 [1:01:16<03:39, 3.92s/it]
94%|█████████▍| 883/938 [1:01:20<03:40, 4.01s/it]
94%|█████████▍| 884/938 [1:01:25<03:42, 4.12s/it]
94%|█████████▍| 885/938 [1:01:29<03:37, 4.10s/it]
94%|█████████▍| 886/938 [1:01:33<03:43, 4.30s/it]
95%|█████████▍| 887/938 [1:01:38<03:40, 4.32s/it]
95%|█████████▍| 888/938 [1:01:41<03:24, 4.09s/it]
95%|█████████▍| 889/938 [1:01:45<03:18, 4.06s/it]
95%|█████████▍| 890/938 [1:01:49<03:11, 3.99s/it]
{'loss': '0.2667', 'grad_norm': '0.4555', 'learning_rate': '8.294e-08', 'epoch': '1.898'}
95%|█████████▍| 890/938 [1:01:49<03:11, 3.99s/it]
95%|█████████▍| 891/938 [1:01:54<03:19, 4.24s/it]
95%|█████████▌| 892/938 [1:01:58<03:05, 4.02s/it]
95%|█████████▌| 893/938 [1:02:01<02:52, 3.84s/it]
95%|█████████▌| 894/938 [1:02:05<02:55, 3.98s/it]
95%|█████████▌| 895/938 [1:02:09<02:47, 3.89s/it]
96%|█████████▌| 896/938 [1:02:13<02:49, 4.04s/it]
96%|█████████▌| 897/938 [1:02:18<02:51, 4.19s/it]
96%|█████████▌| 898/938 [1:02:22<02:46, 4.16s/it]
96%|█████████▌| 899/938 [1:02:27<02:50, 4.36s/it]
96%|█████████▌| 900/938 [1:02:31<02:41, 4.26s/it]
{'loss': '0.2759', 'grad_norm': '0.4674', 'learning_rate': '5.259e-08', 'epoch': '1.919'}
96%|█████████▌| 900/938 [1:02:31<02:41, 4.26s/it]
96%|█████████▌| 901/938 [1:02:35<02:36, 4.22s/it]
96%|█████████▌| 902/938 [1:02:39<02:27, 4.11s/it]
96%|█████████▋| 903/938 [1:02:43<02:22, 4.07s/it]
96%|█████████▋| 904/938 [1:02:46<02:13, 3.93s/it]
96%|█████████▋| 905/938 [1:02:51<02:13, 4.06s/it]
97%|█████████▋| 906/938 [1:02:55<02:11, 4.12s/it]
97%|█████████▋| 907/938 [1:02:59<02:04, 4.01s/it]
97%|█████████▋| 908/938 [1:03:03<01:59, 3.97s/it]
97%|█████████▋| 909/938 [1:03:07<02:00, 4.16s/it]
97%|█████████▋| 910/938 [1:03:12<02:00, 4.32s/it]
{'loss': '0.2717', 'grad_norm': '0.464', 'learning_rate': '2.91e-08', 'epoch': '1.941'}
97%|█████████▋| 910/938 [1:03:12<02:00, 4.32s/it]
97%|█████████▋| 911/938 [1:03:15<01:50, 4.08s/it]
97%|█████████▋| 912/938 [1:03:19<01:43, 4.00s/it]
97%|█████████▋| 913/938 [1:03:23<01:37, 3.91s/it]
97%|█████████▋| 914/938 [1:03:27<01:31, 3.82s/it]
98%|█████████▊| 915/938 [1:03:30<01:28, 3.85s/it]
98%|█████████▊| 916/938 [1:03:34<01:24, 3.85s/it]
98%|█████████▊| 917/938 [1:03:38<01:17, 3.71s/it]
98%|█████████▊| 918/938 [1:03:42<01:17, 3.87s/it]
98%|█████████▊| 919/938 [1:03:46<01:14, 3.93s/it]
98%|█████████▊| 920/938 [1:03:50<01:13, 4.09s/it]
{'loss': '0.2673', 'grad_norm': '0.4542', 'learning_rate': '1.25e-08', 'epoch': '1.962'}
98%|█████████▊| 920/938 [1:03:50<01:13, 4.09s/it]
98%|█████████▊| 921/938 [1:03:54<01:08, 4.02s/it]
98%|█████████▊| 922/938 [1:03:59<01:07, 4.22s/it]
98%|█████████▊| 923/938 [1:04:03<01:03, 4.21s/it]
99%|█████████▊| 924/938 [1:04:07<00:59, 4.23s/it]
99%|█████████▊| 925/938 [1:04:12<00:55, 4.26s/it]
99%|█████████▊| 926/938 [1:04:16<00:50, 4.24s/it]
99%|█████████▉| 927/938 [1:04:19<00:44, 4.01s/it]
99%|█████████▉| 928/938 [1:04:24<00:40, 4.06s/it]
99%|█████████▉| 929/938 [1:04:27<00:35, 3.99s/it]
99%|█████████▉| 930/938 [1:04:31<00:31, 3.99s/it]
{'loss': '0.2772', 'grad_norm': '0.479', 'learning_rate': '2.805e-09', 'epoch': '1.983'}
99%|█████████▉| 930/938 [1:04:31<00:31, 3.99s/it]
99%|█████████▉| 931/938 [1:04:35<00:27, 3.99s/it]
99%|█████████▉| 932/938 [1:04:40<00:25, 4.22s/it]
99%|█████████▉| 933/938 [1:04:45<00:21, 4.35s/it]
100%|█████████▉| 934/938 [1:04:49<00:17, 4.27s/it]
100%|█████████▉| 935/938 [1:04:53<00:12, 4.15s/it]
100%|█████████▉| 936/938 [1:04:57<00:08, 4.17s/it]
100%|█████████▉| 937/938 [1:05:01<00:04, 4.15s/it]
100%|██████████| 938/938 [1:05:04<00:00, 3.74s/it][INFO|trainer.py:3797] 2026-04-29 16:49:43,034 >> Saving model checkpoint to saves/qwen3_1.7b/math/fft/checkpoint-938
[INFO|configuration_utils.py:432] 2026-04-29 16:49:43,043 >> Configuration saved in saves/qwen3_1.7b/math/fft/checkpoint-938/config.json
[INFO|configuration_utils.py:803] 2026-04-29 16:49:43,046 >> Configuration saved in saves/qwen3_1.7b/math/fft/checkpoint-938/generation_config.json
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s]
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.30s/it]
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.30s/it]
[INFO|modeling_utils.py:3380] 2026-04-29 16:49:49,356 >> Model weights saved in saves/qwen3_1.7b/math/fft/checkpoint-938/model.safetensors
[INFO|tokenization_utils_base.py:3224] 2026-04-29 16:49:49,361 >> chat template saved in saves/qwen3_1.7b/math/fft/checkpoint-938/chat_template.jinja
[INFO|tokenization_utils_base.py:2078] 2026-04-29 16:49:49,365 >> tokenizer config file saved in saves/qwen3_1.7b/math/fft/checkpoint-938/tokenizer_config.json
[INFO|trainer.py:1863] 2026-04-29 16:49:49,592 >>
Training completed. Do not forget to share your model on huggingface.co/models =)
{'train_runtime': '3915', 'train_samples_per_second': '30.65', 'train_steps_per_second': '0.24', 'train_loss': '0.2964', 'epoch': '2'}
100%|██████████| 938/938 [1:05:13<00:00, 3.74s/it]
100%|██████████| 938/938 [1:05:13<00:00, 4.17s/it]
[INFO|trainer.py:3797] 2026-04-29 16:49:51,371 >> Saving model checkpoint to saves/qwen3_1.7b/math/fft
[INFO|configuration_utils.py:432] 2026-04-29 16:49:51,380 >> Configuration saved in saves/qwen3_1.7b/math/fft/config.json
[INFO|configuration_utils.py:803] 2026-04-29 16:49:51,384 >> Configuration saved in saves/qwen3_1.7b/math/fft/generation_config.json
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s]
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.43s/it]
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.43s/it]
[INFO|modeling_utils.py:3380] 2026-04-29 16:49:57,827 >> Model weights saved in saves/qwen3_1.7b/math/fft/model.safetensors
[INFO|tokenization_utils_base.py:3224] 2026-04-29 16:49:57,832 >> chat template saved in saves/qwen3_1.7b/math/fft/chat_template.jinja
[INFO|tokenization_utils_base.py:2078] 2026-04-29 16:49:57,836 >> tokenizer config file saved in saves/qwen3_1.7b/math/fft/tokenizer_config.json
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO comm 0x56028de3b5a0 rank 2 nranks 8 cudaDev 2 busId 201c0 - Destroy COMPLETE
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO comm 0x5612df7ca6e0 rank 3 nranks 8 cudaDev 3 busId 201d0 - Destroy COMPLETE
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO comm 0x55c02c235cf0 rank 5 nranks 8 cudaDev 5 busId 901d0 - Destroy COMPLETE
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO comm 0x5647135d02c0 rank 4 nranks 8 cudaDev 4 busId 901c0 - Destroy COMPLETE
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO comm 0x560722415b60 rank 6 nranks 8 cudaDev 6 busId a01c0 - Destroy COMPLETE
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO comm 0x55c0201d0a60 rank 5 nranks 8 cudaDev 5 busId 901d0 - Destroy COMPLETE
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO comm 0x5647078795e0 rank 4 nranks 8 cudaDev 4 busId 901c0 - Destroy COMPLETE
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO comm 0x5612d38dcbf0 rank 3 nranks 8 cudaDev 3 busId 201d0 - Destroy COMPLETE
[INFO|trainer.py:3797] 2026-04-29 16:50:00,902 >> Saving model checkpoint to saves/qwen3_1.7b/math/fft
[INFO|configuration_utils.py:432] 2026-04-29 16:50:00,911 >> Configuration saved in saves/qwen3_1.7b/math/fft/config.json
[INFO|configuration_utils.py:803] 2026-04-29 16:50:00,913 >> Configuration saved in saves/qwen3_1.7b/math/fft/generation_config.json
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s]ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ENV/Plugin: Closing env plugin ncclEnvDefault
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ENV/Plugin: Closing env plugin ncclEnvDefault
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ENV/Plugin: Closing env plugin ncclEnvDefault
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.57s/it]
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.57s/it]
[INFO|modeling_utils.py:3380] 2026-04-29 16:50:07,495 >> Model weights saved in saves/qwen3_1.7b/math/fft/model.safetensors
[INFO|tokenization_utils_base.py:3224] 2026-04-29 16:50:07,499 >> chat template saved in saves/qwen3_1.7b/math/fft/chat_template.jinja
[INFO|tokenization_utils_base.py:2078] 2026-04-29 16:50:07,503 >> tokenizer config file saved in saves/qwen3_1.7b/math/fft/tokenizer_config.json
[INFO|modelcard.py:266] 2026-04-29 16:50:07,981 >> Dropping the following result as it does not have all the necessary fields:
{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
Processing Files (0 / 0) : | | 0.00B / 0.00B
New Data Upload : | | 0.00B / 0.00B 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
Processing Files (1 / 1) : 0%| | 11.4MB / 4.07GB, 57.0MB/s
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 0%| | 29.6kB / 4.06GB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 0%| | 29.6kB / 4.06GB 
Processing Files (1 / 2) : 0%| | 11.5MB / 4.07GB, 11.5MB/s
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 0%| | 29.6kB / 4.06GB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 0%| | 29.6kB / 4.06GB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 0%| | 636kB / 4.06GB 
Processing Files (1 / 2) : 0%| | 12.1MB / 4.07GB, 7.54MB/s
New Data Upload : 0%| | 607kB / 134MB, 379kB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 0%| | 1.24MB / 4.06GB 
Processing Files (1 / 2) : 0%| | 12.7MB / 4.07GB, 7.04MB/s
New Data Upload : 1%| | 1.21MB / 134MB, 674kB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 0%| | 7.91MB / 4.06GB 
Processing Files (1 / 2) : 0%| | 19.3MB / 4.07GB, 9.67MB/s
New Data Upload : 6%|▌ | 7.88MB / 134MB, 3.94MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 1%| | 21.3MB / 4.06GB 
Processing Files (1 / 2) : 1%| | 32.7MB / 4.07GB, 14.9MB/s
New Data Upload : 16%|█▌ | 21.2MB / 134MB, 9.65MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 1%| | 33.4MB / 4.06GB 
Processing Files (1 / 2) : 1%| | 44.8MB / 4.07GB, 18.7MB/s
New Data Upload : 25%|██▍ | 33.4MB / 134MB, 13.9MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 1%| | 46.1MB / 4.06GB 
Processing Files (1 / 2) : 1%|▏ | 57.5MB / 4.07GB, 22.1MB/s
New Data Upload : 34%|███▍ | 46.1MB / 134MB, 17.7MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 1%|▏ | 58.3MB / 4.06GB 
Processing Files (1 / 2) : 2%|▏ | 69.7MB / 4.07GB, 24.9MB/s
New Data Upload : 43%|████▎ | 58.2MB / 134MB, 20.8MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB 
Processing Files (1 / 2) : 2%|▏ | 78.2MB / 4.07GB, 26.1MB/s
New Data Upload : 50%|████▉ | 66.7MB / 134MB, 22.2MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 2%|▏ | 67.1MB / 4.06GB 
Processing Files (1 / 2) : 2%|▏ | 78.5MB / 4.07GB, 20.7MB/s
New Data Upload : 50%|████▉ | 67.1MB / 134MB, 17.7MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 2%|▏ | 67.7MB / 4.06GB 
Processing Files (1 / 2) : 2%|▏ | 79.1MB / 4.07GB, 19.8MB/s
New Data Upload : 50%|█████ | 67.7MB / 134MB, 16.9MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 2%|▏ | 71.4MB / 4.06GB 
Processing Files (1 / 2) : 2%|▏ | 82.8MB / 4.07GB, 19.7MB/s
New Data Upload : 35%|███▌ | 71.3MB / 201MB, 17.0MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 2%|▏ | 78.0MB / 4.06GB 
Processing Files (1 / 2) : 2%|▏ | 89.4MB / 4.07GB, 20.3MB/s
New Data Upload : 39%|███▉ | 78.0MB / 201MB, 17.7MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 2%|▏ | 98.0MB / 4.06GB 
Processing Files (1 / 2) : 3%|▎ | 109MB / 4.07GB, 23.8MB/s
New Data Upload : 37%|███▋ | 98.0MB / 268MB, 21.3MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 3%|▎ | 122MB / 4.06GB 
Processing Files (1 / 2) : 3%|▎ | 133MB / 4.07GB, 27.7MB/s
New Data Upload : 45%|████▌ | 122MB / 268MB, 25.3MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 4%|▎ | 146MB / 4.06GB 
Processing Files (1 / 2) : 4%|▍ | 157MB / 4.07GB, 31.5MB/s
New Data Upload : 43%|████▎ | 146MB / 335MB, 29.2MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 4%|▍ | 173MB / 4.06GB 
Processing Files (1 / 2) : 5%|▍ | 185MB / 4.07GB, 35.5MB/s
New Data Upload : 52%|█████▏ | 173MB / 335MB, 33.3MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 5%|▍ | 194MB / 4.06GB 
Processing Files (1 / 2) : 5%|▌ | 206MB / 4.07GB, 38.1MB/s
New Data Upload : 58%|█████▊ | 194MB / 335MB, 36.0MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 6%|▌ | 225MB / 4.06GB 
Processing Files (1 / 2) : 6%|▌ | 236MB / 4.07GB, 42.2MB/s
New Data Upload : 56%|█████▌ | 225MB / 403MB, 40.1MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 6%|▌ | 251MB / 4.06GB 
Processing Files (1 / 2) : 6%|▋ | 263MB / 4.07GB, 45.3MB/s
New Data Upload : 62%|██████▏ | 251MB / 403MB, 43.3MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 7%|▋ | 280MB / 4.06GB 
Processing Files (1 / 2) : 7%|▋ | 292MB / 4.07GB, 48.6MB/s
New Data Upload : 70%|██████▉ | 280MB / 403MB, 46.7MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 7%|▋ | 304MB / 4.06GB 
Processing Files (1 / 2) : 8%|▊ | 315MB / 4.07GB, 50.9MB/s
New Data Upload : 65%|██████▍ | 304MB / 470MB, 49.0MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 8%|▊ | 333MB / 4.06GB 
Processing Files (1 / 2) : 8%|▊ | 345MB / 4.07GB, 53.9MB/s
New Data Upload : 71%|███████ | 333MB / 470MB, 52.1MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 9%|▉ | 356MB / 4.06GB 
Processing Files (1 / 2) : 9%|▉ | 368MB / 4.07GB, 55.7MB/s
New Data Upload : 66%|██████▋ | 356MB / 537MB, 54.0MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 9%|▉ | 377MB / 4.06GB 
Processing Files (1 / 2) : 10%|▉ | 389MB / 4.07GB, 57.1MB/s
New Data Upload : 70%|███████ | 377MB / 537MB, 55.5MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 10%|█ | 407MB / 4.06GB 
Processing Files (1 / 2) : 10%|█ | 418MB / 4.07GB, 59.8MB/s
New Data Upload : 67%|██████▋ | 407MB / 604MB, 58.1MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 11%|█ | 447MB / 4.06GB 
Processing Files (1 / 2) : 11%|█ | 458MB / 4.07GB, 63.7MB/s
New Data Upload : 74%|███████▍ | 447MB / 604MB, 62.1MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 12%|█▏ | 480MB / 4.06GB 
Processing Files (1 / 2) : 12%|█▏ | 492MB / 4.07GB, 66.4MB/s
New Data Upload : 72%|███████▏ | 480MB / 671MB, 64.9MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 13%|█▎ | 516MB / 4.06GB 
Processing Files (1 / 2) : 13%|█▎ | 528MB / 4.07GB, 69.5MB/s
New Data Upload : 77%|███████▋ | 516MB / 671MB, 67.9MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 14%|█▍ | 563MB / 4.06GB 
Processing Files (1 / 2) : 14%|█▍ | 575MB / 4.07GB, 73.7MB/s
New Data Upload : 76%|███████▋ | 563MB / 738MB, 72.2MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 15%|█▍ | 594MB / 4.06GB 
Processing Files (1 / 2) : 15%|█▍ | 605MB / 4.07GB, 75.7MB/s
New Data Upload : 81%|████████ | 594MB / 738MB, 74.2MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 15%|█▌ | 621MB / 4.06GB 
Processing Files (1 / 2) : 16%|█▌ | 632MB / 4.07GB, 77.1MB/s
New Data Upload : 77%|███████▋ | 621MB / 805MB, 75.7MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 16%|█▌ | 650MB / 4.06GB 
Processing Files (1 / 2) : 16%|█▌ | 662MB / 4.07GB, 78.8MB/s
New Data Upload : 81%|████████ | 650MB / 805MB, 77.4MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 17%|█▋ | 690MB / 4.06GB 
Processing Files (1 / 2) : 17%|█▋ | 702MB / 4.07GB, 81.6MB/s
New Data Upload : 86%|████████▌ | 690MB / 805MB, 80.3MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 17%|█▋ | 708MB / 4.06GB 
Processing Files (1 / 2) : 18%|█▊ | 720MB / 4.07GB, 81.8MB/s
New Data Upload : 81%|████████ | 708MB / 872MB, 80.5MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 18%|█▊ | 743MB / 4.06GB 
Processing Files (1 / 2) : 19%|█▊ | 754MB / 4.07GB, 83.8MB/s
New Data Upload : 79%|███████▉ | 743MB / 939MB, 82.5MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 19%|█▉ | 774MB / 4.06GB 
Processing Files (1 / 2) : 19%|█▉ | 786MB / 4.07GB, 85.4MB/s
New Data Upload : 82%|████████▏ | 774MB / 939MB, 84.2MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 20%|█▉ | 811MB / 4.06GB 
Processing Files (1 / 2) : 20%|██ | 822MB / 4.07GB, 87.5MB/s
New Data Upload : 81%|████████ | 811MB / 1.01GB, 86.3MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 21%|██ | 846MB / 4.06GB 
Processing Files (1 / 2) : 21%|██ | 857MB / 4.07GB, 89.3MB/s
New Data Upload : 79%|███████▉ | 845MB / 1.07GB, 88.1MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 22%|██▏ | 894MB / 4.06GB 
Processing Files (1 / 2) : 22%|██▏ | 905MB / 4.07GB, 92.4MB/s
New Data Upload : 83%|████████▎ | 894MB / 1.07GB, 91.2MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 23%|██▎ | 939MB / 4.06GB 
Processing Files (1 / 2) : 23%|██▎ | 950MB / 4.07GB, 95.0MB/s
New Data Upload : 82%|████████▏ | 939MB / 1.14GB, 93.9MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 24%|██▍ | 978MB / 4.06GB 
Processing Files (1 / 2) : 24%|██▍ | 990MB / 4.07GB, 97.0MB/s
New Data Upload : 86%|████████▌ | 978MB / 1.14GB, 95.9MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 25%|██▌ | 1.02GB / 4.06GB 
Processing Files (1 / 2) : 25%|██▌ | 1.03GB / 4.07GB, 99.7MB/s
New Data Upload : 89%|████████▉ | 1.02GB / 1.14GB, 99.7MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 26%|██▌ | 1.05GB / 4.06GB 
Processing Files (1 / 2) : 26%|██▌ | 1.06GB / 4.07GB, 103MB/s
New Data Upload : 92%|█████████▏| 1.05GB / 1.14GB, 103MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 26%|██▌ | 1.06GB / 4.06GB 
Processing Files (1 / 2) : 26%|██▋ | 1.07GB / 4.07GB, 104MB/s
New Data Upload : 93%|█████████▎| 1.06GB / 1.14GB, 104MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 27%|██▋ | 1.08GB / 4.06GB 
Processing Files (1 / 2) : 27%|██▋ | 1.09GB / 4.07GB, 106MB/s
New Data Upload : 95%|█████████▍| 1.08GB / 1.14GB, 106MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 27%|██▋ | 1.09GB / 4.06GB 
Processing Files (1 / 2) : 27%|██▋ | 1.10GB / 4.07GB, 107MB/s
New Data Upload : 96%|█████████▌| 1.09GB / 1.14GB, 107MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 27%|██▋ | 1.11GB / 4.06GB 
Processing Files (1 / 2) : 27%|██▋ | 1.12GB / 4.07GB, 108MB/s
New Data Upload : 97%|█████████▋| 1.11GB / 1.14GB, 108MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 28%|██▊ | 1.12GB / 4.06GB 
Processing Files (1 / 2) : 28%|██▊ | 1.13GB / 4.07GB, 110MB/s
New Data Upload : 98%|█████████▊| 1.12GB / 1.14GB, 110MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 29%|██▉ | 1.18GB / 4.06GB 
Processing Files (1 / 2) : 29%|██▉ | 1.19GB / 4.07GB, 116MB/s
New Data Upload : 99%|█████████▉| 1.13GB / 1.14GB, 111MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 29%|██▉ | 1.19GB / 4.06GB 
Processing Files (1 / 2) : 30%|██▉ | 1.21GB / 4.07GB, 117MB/s
New Data Upload : 94%|█████████▍| 1.14GB / 1.21GB, 112MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 29%|██▉ | 1.19GB / 4.06GB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 29%|██▉ | 1.20GB / 4.06GB 
Processing Files (1 / 2) : 30%|██▉ | 1.21GB / 4.07GB, 115MB/s
New Data Upload : 90%|████████▉ | 1.14GB / 1.27GB, 110MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 30%|██▉ | 1.20GB / 4.06GB 
Processing Files (1 / 2) : 30%|██▉ | 1.21GB / 4.07GB, 115MB/s
New Data Upload : 86%|████████▌ | 1.15GB / 1.34GB, 109MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 30%|██▉ | 1.21GB / 4.06GB 
Processing Files (1 / 2) : 30%|███ | 1.22GB / 4.07GB, 114MB/s
New Data Upload : 86%|████████▋ | 1.16GB / 1.34GB, 109MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 31%|███ | 1.25GB / 4.06GB 
Processing Files (1 / 2) : 31%|███ | 1.26GB / 4.07GB, 116MB/s
New Data Upload : 85%|████████▍ | 1.19GB / 1.41GB, 111MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 32%|███▏ | 1.29GB / 4.06GB 
Processing Files (1 / 2) : 32%|███▏ | 1.30GB / 4.07GB, 120MB/s
New Data Upload : 88%|████████▊ | 1.23GB / 1.41GB, 115MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 33%|███▎ | 1.32GB / 4.06GB 
Processing Files (1 / 2) : 33%|███▎ | 1.33GB / 4.07GB, 123MB/s
New Data Upload : 86%|████████▌ | 1.27GB / 1.48GB, 118MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 34%|███▎ | 1.37GB / 4.06GB 
Processing Files (1 / 2) : 34%|███▍ | 1.38GB / 4.07GB, 128MB/s
New Data Upload : 85%|████████▌ | 1.31GB / 1.54GB, 122MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 35%|███▍ | 1.40GB / 4.06GB 
Processing Files (1 / 2) : 35%|███▍ | 1.42GB / 4.07GB, 131MB/s
New Data Upload : 88%|████████▊ | 1.35GB / 1.54GB, 126MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 35%|███▌ | 1.44GB / 4.06GB 
Processing Files (1 / 2) : 36%|███▌ | 1.45GB / 4.07GB, 134MB/s
New Data Upload : 86%|████████▌ | 1.38GB / 1.61GB, 129MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 36%|███▌ | 1.46GB / 4.06GB 
Processing Files (1 / 2) : 36%|███▌ | 1.48GB / 4.07GB, 137MB/s
New Data Upload : 88%|████████▊ | 1.41GB / 1.61GB, 132MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 37%|███▋ | 1.49GB / 4.06GB 
Processing Files (1 / 2) : 37%|███▋ | 1.50GB / 4.07GB, 139MB/s
New Data Upload : 86%|████████▌ | 1.44GB / 1.68GB, 134MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 38%|███▊ | 1.53GB / 4.06GB 
Processing Files (1 / 2) : 38%|███▊ | 1.54GB / 4.07GB, 142MB/s
New Data Upload : 85%|████████▍ | 1.47GB / 1.74GB, 137MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 39%|███▉ | 1.58GB / 4.06GB 
Processing Files (1 / 2) : 39%|███▉ | 1.59GB / 4.07GB, 145MB/s
New Data Upload : 87%|████████▋ | 1.52GB / 1.74GB, 140MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 40%|████ | 1.63GB / 4.06GB 
Processing Files (1 / 2) : 40%|████ | 1.64GB / 4.07GB, 148MB/s
New Data Upload : 87%|████████▋ | 1.57GB / 1.81GB, 142MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 42%|████▏ | 1.69GB / 4.06GB 
Processing Files (1 / 2) : 42%|████▏ | 1.70GB / 4.07GB, 151MB/s
New Data Upload : 87%|████████▋ | 1.63GB / 1.88GB, 146MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 43%|████▎ | 1.75GB / 4.06GB 
Processing Files (1 / 2) : 43%|████▎ | 1.76GB / 4.07GB, 155MB/s
New Data Upload : 90%|█████████ | 1.70GB / 1.88GB, 149MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 44%|████▍ | 1.80GB / 4.06GB 
Processing Files (1 / 2) : 44%|████▍ | 1.81GB / 4.07GB, 157MB/s
New Data Upload : 90%|████████▉ | 1.75GB / 1.95GB, 152MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 45%|████▌ | 1.83GB / 4.06GB 
Processing Files (1 / 2) : 45%|████▌ | 1.84GB / 4.07GB, 158MB/s
New Data Upload : 91%|█████████▏| 1.78GB / 1.95GB, 152MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 46%|████▌ | 1.87GB / 4.06GB 
Processing Files (1 / 2) : 46%|████▌ | 1.88GB / 4.07GB, 159MB/s
New Data Upload : 90%|█████████ | 1.81GB / 2.01GB, 153MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 47%|████▋ | 1.91GB / 4.06GB 
Processing Files (1 / 2) : 47%|████▋ | 1.92GB / 4.07GB, 160MB/s
New Data Upload : 92%|█████████▏| 1.85GB / 2.01GB, 154MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 48%|████▊ | 1.94GB / 4.06GB 
Processing Files (1 / 2) : 48%|████▊ | 1.95GB / 4.07GB, 160MB/s
New Data Upload : 91%|█████████ | 1.88GB / 2.08GB, 155MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 49%|████▊ | 1.97GB / 4.06GB 
Processing Files (1 / 2) : 49%|████▊ | 1.99GB / 4.07GB, 161MB/s
New Data Upload : 92%|█████████▏| 1.92GB / 2.08GB, 156MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 50%|████▉ | 2.02GB / 4.06GB 
Processing Files (1 / 2) : 50%|████▉ | 2.04GB / 4.07GB, 164MB/s
New Data Upload : 92%|█████████▏| 1.97GB / 2.15GB, 158MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 51%|█████ | 2.06GB / 4.06GB 
Processing Files (1 / 2) : 51%|█████ | 2.07GB / 4.07GB, 165MB/s
New Data Upload : 94%|█████████▎| 2.01GB / 2.15GB, 160MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 52%|█████▏ | 2.09GB / 4.06GB 
Processing Files (1 / 2) : 52%|█████▏ | 2.11GB / 4.07GB, 165MB/s
New Data Upload : 92%|█████████▏| 2.04GB / 2.21GB, 160MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 52%|█████▏ | 2.13GB / 4.06GB 
Processing Files (1 / 2) : 53%|█████▎ | 2.14GB / 4.07GB, 165MB/s
New Data Upload : 94%|█████████▍| 2.08GB / 2.21GB, 160MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 53%|█████▎ | 2.16GB / 4.06GB 
Processing Files (1 / 2) : 53%|█████▎ | 2.17GB / 4.07GB, 164MB/s
New Data Upload : 92%|█████████▏| 2.10GB / 2.28GB, 159MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 54%|█████▎ | 2.18GB / 4.06GB 
Processing Files (1 / 2) : 54%|█████▍ | 2.19GB / 4.07GB, 163MB/s
New Data Upload : 91%|█████████ | 2.13GB / 2.35GB, 158MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 54%|█████▍ | 2.21GB / 4.06GB 
Processing Files (1 / 2) : 54%|█████▍ | 2.22GB / 4.07GB, 161MB/s
New Data Upload : 92%|█████████▏| 2.15GB / 2.35GB, 156MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 55%|█████▌ | 2.25GB / 4.06GB 
Processing Files (1 / 2) : 56%|█████▌ | 2.26GB / 4.07GB, 163MB/s
New Data Upload : 91%|█████████ | 2.20GB / 2.41GB, 157MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 56%|█████▌ | 2.28GB / 4.06GB 
Processing Files (1 / 2) : 56%|█████▋ | 2.30GB / 4.07GB, 163MB/s
New Data Upload : 92%|█████████▏| 2.23GB / 2.41GB, 158MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 57%|█████▋ | 2.33GB / 4.06GB 
Processing Files (1 / 2) : 57%|█████▋ | 2.34GB / 4.07GB, 165MB/s
New Data Upload : 92%|█████████▏| 2.27GB / 2.48GB, 159MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 58%|█████▊ | 2.37GB / 4.06GB 
Processing Files (1 / 2) : 58%|█████▊ | 2.38GB / 4.07GB, 164MB/s
New Data Upload : 93%|█████████▎| 2.31GB / 2.48GB, 159MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 59%|█████▉ | 2.41GB / 4.06GB 
Processing Files (1 / 2) : 59%|█████▉ | 2.42GB / 4.07GB, 166MB/s
New Data Upload : 92%|█████████▏| 2.35GB / 2.55GB, 161MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 60%|██████ | 2.44GB / 4.06GB 
Processing Files (1 / 2) : 60%|██████ | 2.46GB / 4.07GB, 167MB/s
New Data Upload : 91%|█████████▏| 2.39GB / 2.62GB, 161MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 61%|██████ | 2.49GB / 4.06GB 
Processing Files (1 / 2) : 61%|██████▏ | 2.50GB / 4.07GB, 168MB/s
New Data Upload : 93%|█████████▎| 2.43GB / 2.62GB, 162MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 62%|██████▏ | 2.52GB / 4.06GB 
Processing Files (1 / 2) : 62%|██████▏ | 2.53GB / 4.07GB, 168MB/s
New Data Upload : 94%|█████████▍| 2.47GB / 2.62GB, 163MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 63%|██████▎ | 2.56GB / 4.06GB 
Processing Files (1 / 2) : 63%|██████▎ | 2.57GB / 4.07GB, 168MB/s
New Data Upload : 93%|█████████▎| 2.51GB / 2.68GB, 163MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 64%|██████▍ | 2.60GB / 4.06GB 
Processing Files (1 / 2) : 64%|██████▍ | 2.61GB / 4.07GB, 167MB/s
New Data Upload : 92%|█████████▏| 2.54GB / 2.75GB, 162MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 65%|██████▍ | 2.64GB / 4.06GB 
Processing Files (1 / 2) : 65%|██████▍ | 2.65GB / 4.07GB, 166MB/s
New Data Upload : 92%|█████████▏| 2.58GB / 2.82GB, 161MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 66%|██████▌ | 2.68GB / 4.06GB 
Processing Files (1 / 2) : 66%|██████▌ | 2.69GB / 4.07GB, 167MB/s
New Data Upload : 93%|█████████▎| 2.62GB / 2.82GB, 161MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 67%|██████▋ | 2.72GB / 4.06GB 
Processing Files (1 / 2) : 67%|██████▋ | 2.73GB / 4.07GB, 167MB/s
New Data Upload : 95%|█████████▍| 2.67GB / 2.82GB, 162MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 68%|██████▊ | 2.76GB / 4.06GB 
Processing Files (1 / 2) : 68%|██████▊ | 2.77GB / 4.07GB, 168MB/s
New Data Upload : 94%|█████████▍| 2.71GB / 2.88GB, 163MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 69%|██████▊ | 2.79GB / 4.06GB 
Processing Files (1 / 2) : 69%|██████▉ | 2.80GB / 4.07GB, 170MB/s
New Data Upload : 95%|█████████▍| 2.74GB / 2.88GB, 165MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 69%|██████▉ | 2.82GB / 4.06GB 
Processing Files (1 / 2) : 70%|██████▉ | 2.83GB / 4.07GB, 171MB/s
New Data Upload : 94%|█████████▍| 2.77GB / 2.95GB, 166MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 70%|███████ | 2.86GB / 4.06GB 
Processing Files (1 / 2) : 70%|███████ | 2.87GB / 4.07GB, 173MB/s
New Data Upload : 95%|█████████▍| 2.80GB / 2.95GB, 168MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 71%|███████ | 2.88GB / 4.06GB 
Processing Files (1 / 2) : 71%|███████ | 2.89GB / 4.07GB, 174MB/s
New Data Upload : 94%|█████████▎| 2.83GB / 3.02GB, 169MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 72%|███████▏ | 2.92GB / 4.06GB 
Processing Files (1 / 2) : 72%|███████▏ | 2.93GB / 4.07GB, 176MB/s
New Data Upload : 93%|█████████▎| 2.86GB / 3.09GB, 171MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 73%|███████▎ | 2.95GB / 4.06GB 
Processing Files (1 / 2) : 73%|███████▎ | 2.96GB / 4.07GB, 173MB/s
New Data Upload : 94%|█████████▍| 2.90GB / 3.09GB, 173MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 73%|███████▎ | 2.99GB / 4.06GB 
Processing Files (1 / 2) : 74%|███████▎ | 3.00GB / 4.07GB, 176MB/s
New Data Upload : 93%|█████████▎| 2.93GB / 3.15GB, 176MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 74%|███████▍ | 3.00GB / 4.06GB 
Processing Files (1 / 2) : 74%|███████▍ | 3.02GB / 4.07GB, 177MB/s
New Data Upload : 94%|█████████▎| 2.95GB / 3.15GB, 177MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 75%|███████▍ | 3.05GB / 4.06GB 
Processing Files (1 / 2) : 75%|███████▌ | 3.06GB / 4.07GB, 181MB/s
New Data Upload : 93%|█████████▎| 2.99GB / 3.22GB, 181MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 76%|███████▌ | 3.08GB / 4.06GB 
Processing Files (1 / 2) : 76%|███████▌ | 3.09GB / 4.07GB, 184MB/s
New Data Upload : 94%|█████████▍| 3.02GB / 3.22GB, 184MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 77%|███████▋ | 3.13GB / 4.06GB 
Processing Files (1 / 2) : 77%|███████▋ | 3.14GB / 4.07GB, 188MB/s
New Data Upload : 93%|█████████▎| 3.07GB / 3.29GB, 188MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 78%|███████▊ | 3.16GB / 4.06GB 
Processing Files (1 / 2) : 78%|███████▊ | 3.18GB / 4.07GB, 188MB/s
New Data Upload : 95%|█████████▍| 3.11GB / 3.29GB, 188MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 79%|███████▉ | 3.20GB / 4.06GB 
Processing Files (1 / 2) : 79%|███████▉ | 3.21GB / 4.07GB, 188MB/s
New Data Upload : 94%|█████████▍| 3.15GB / 3.35GB, 188MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 80%|███████▉ | 3.24GB / 4.06GB 
Processing Files (1 / 2) : 80%|███████▉ | 3.25GB / 4.07GB, 188MB/s
New Data Upload : 95%|█████████▌| 3.19GB / 3.35GB, 188MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 81%|████████ | 3.27GB / 4.06GB 
Processing Files (1 / 2) : 81%|████████ | 3.28GB / 4.07GB, 187MB/s
New Data Upload : 94%|█████████▍| 3.22GB / 3.42GB, 187MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 81%|████████▏ | 3.30GB / 4.06GB 
Processing Files (1 / 2) : 81%|████████▏ | 3.32GB / 4.07GB, 186MB/s
New Data Upload : 95%|█████████▌| 3.25GB / 3.42GB, 186MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 82%|████████▏ | 3.35GB / 4.06GB 
Processing Files (1 / 2) : 82%|████████▏ | 3.36GB / 4.07GB, 187MB/s
New Data Upload : 94%|█████████▍| 3.29GB / 3.49GB, 187MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 84%|████████▎ | 3.40GB / 4.06GB 
Processing Files (1 / 2) : 84%|████████▎ | 3.41GB / 4.07GB, 189MB/s
New Data Upload : 94%|█████████▍| 3.34GB / 3.55GB, 189MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 84%|████████▍ | 3.43GB / 4.06GB 
Processing Files (1 / 2) : 84%|████████▍ | 3.44GB / 4.07GB, 190MB/s
New Data Upload : 95%|█████████▍| 3.37GB / 3.55GB, 190MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 85%|████████▌ | 3.47GB / 4.06GB 
Processing Files (1 / 2) : 85%|████████▌ | 3.48GB / 4.07GB, 190MB/s
New Data Upload : 94%|█████████▍| 3.42GB / 3.62GB, 190MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 86%|████████▋ | 3.51GB / 4.06GB 
Processing Files (1 / 2) : 87%|████████▋ | 3.53GB / 4.07GB, 190MB/s
New Data Upload : 96%|█████████▌| 3.46GB / 3.62GB, 190MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 88%|████████▊ | 3.56GB / 4.06GB 
Processing Files (1 / 2) : 88%|████████▊ | 3.57GB / 4.07GB, 189MB/s
New Data Upload : 95%|█████████▍| 3.50GB / 3.69GB, 189MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 88%|████████▊ | 3.58GB / 4.06GB 
Processing Files (1 / 2) : 88%|████████▊ | 3.59GB / 4.07GB, 185MB/s
New Data Upload : 96%|█████████▌| 3.52GB / 3.69GB, 185MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 89%|████████▉ | 3.61GB / 4.06GB 
Processing Files (1 / 2) : 89%|████████▉ | 3.62GB / 4.07GB, 182MB/s
New Data Upload : 95%|█████████▍| 3.56GB / 3.76GB, 182MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 90%|████████▉ | 3.65GB / 4.06GB 
Processing Files (1 / 2) : 90%|████████▉ | 3.66GB / 4.07GB, 181MB/s
New Data Upload : 96%|█████████▌| 3.59GB / 3.76GB, 181MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 90%|█████████ | 3.68GB / 4.06GB 
Processing Files (1 / 2) : 90%|█████████ | 3.69GB / 4.07GB, 181MB/s
New Data Upload : 95%|█████████▍| 3.62GB / 3.82GB, 181MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 91%|█████████▏| 3.72GB / 4.06GB 
Processing Files (1 / 2) : 92%|█████████▏| 3.73GB / 4.07GB, 181MB/s
New Data Upload : 96%|█████████▌| 3.66GB / 3.82GB, 181MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 92%|█████████▏| 3.75GB / 4.06GB 
Processing Files (1 / 2) : 92%|█████████▏| 3.76GB / 4.07GB, 181MB/s
New Data Upload : 95%|█████████▌| 3.70GB / 3.89GB, 181MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 93%|█████████▎| 3.79GB / 4.06GB 
Processing Files (1 / 2) : 93%|█████████▎| 3.80GB / 4.07GB, 182MB/s
New Data Upload : 94%|█████████▍| 3.74GB / 3.96GB, 182MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 94%|█████████▍| 3.83GB / 4.06GB 
Processing Files (1 / 2) : 94%|█████████▍| 3.84GB / 4.07GB, 182MB/s
New Data Upload : 95%|█████████▌| 3.78GB / 3.96GB, 182MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 95%|█████████▌| 3.88GB / 4.06GB 
Processing Files (1 / 2) : 95%|█████████▌| 3.89GB / 4.07GB, 182MB/s
New Data Upload : 95%|█████████▌| 3.82GB / 4.01GB, 182MB/s 
...ath/fft/training_args.bin: 8%|▊ | 579B / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 96%|█████████▋| 3.92GB / 4.06GB 
...ath/fft/training_args.bin: 8%|▊ | 579B / 7.18kB 
Processing Files (1 / 3) : 96%|█████████▋| 3.93GB / 4.07GB, 182MB/s
New Data Upload : 96%|█████████▋| 3.86GB / 4.01GB, 182MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 97%|█████████▋| 3.96GB / 4.06GB 
...ath/fft/training_args.bin: 13%|█▎ | 909B / 7.18kB 
Processing Files (1 / 3) : 97%|█████████▋| 3.97GB / 4.07GB, 183MB/s
New Data Upload : 97%|█████████▋| 3.90GB / 4.01GB, 183MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 99%|█████████▊| 4.01GB / 4.06GB 
...ath/fft/training_args.bin: 33%|███▎ | 2.40kB / 7.18kB 
Processing Files (1 / 3) : 99%|█████████▊| 4.02GB / 4.07GB, 184MB/s
New Data Upload : 99%|█████████▊| 3.95GB / 4.01GB, 184MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 99%|█████████▉| 4.04GB / 4.06GB 
...ath/fft/training_args.bin: 58%|█████▊ | 4.13kB / 7.18kB 
Processing Files (1 / 3) : 99%|█████████▉| 4.05GB / 4.07GB, 184MB/s
New Data Upload : 99%|█████████▉| 3.98GB / 4.01GB, 184MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|█████████▉| 4.05GB / 4.06GB 
...ath/fft/training_args.bin: 82%|████████▏ | 5.87kB / 7.18kB 
Processing Files (1 / 3) : 100%|█████████▉| 4.06GB / 4.07GB, 184MB/s
New Data Upload : 100%|█████████▉| 4.00GB / 4.01GB, 184MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 99%|█████████▉| 7.11kB / 7.18kB 
Processing Files (1 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 182MB/s
New Data Upload : 100%|█████████▉| 4.01GB / 4.01GB, 182MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 99%|█████████▉| 7.11kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 99%|█████████▉| 7.11kB / 7.18kB 
Processing Files (1 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 174MB/s
New Data Upload : 100%|█████████▉| 4.01GB / 4.01GB, 174MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
Processing Files (3 / 3) : 100%|██████████| 4.07GB / 4.07GB, 170MB/s
New Data Upload : 100%|██████████| 4.01GB / 4.01GB, 170MB/s 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
Processing Files (3 / 3) : 100%|██████████| 4.07GB / 4.07GB, 143MB/s
New Data Upload : 100%|██████████| 4.01GB / 4.01GB, 143MB/s
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB
***** train metrics *****
epoch = 2.0
total_flos = 1136301567GF
train_loss = 0.2964
train_runtime = 1:05:15.33
train_samples_per_second = 30.649
train_steps_per_second = 0.24
Figure saved at: saves/qwen3_1.7b/math/fft/training_loss.png
[WARNING|2026-04-29 16:50:51] llamafactory.extras.ploting:149 >> No metric eval_loss to plot.
[WARNING|2026-04-29 16:50:51] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot.
[INFO|trainer.py:3797] 2026-04-29 16:50:52,761 >> Saving model checkpoint to saves/qwen3_1.7b/math/fft
[INFO|configuration_utils.py:432] 2026-04-29 16:50:52,769 >> Configuration saved in saves/qwen3_1.7b/math/fft/config.json
[INFO|configuration_utils.py:803] 2026-04-29 16:50:52,775 >> Configuration saved in saves/qwen3_1.7b/math/fft/generation_config.json
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s]
Writing model shards: 100%|██████████| 1/1 [00:07<00:00, 7.01s/it]
Writing model shards: 100%|██████████| 1/1 [00:07<00:00, 7.01s/it]
[INFO|modeling_utils.py:3380] 2026-04-29 16:50:59,796 >> Model weights saved in saves/qwen3_1.7b/math/fft/model.safetensors
[INFO|tokenization_utils_base.py:3224] 2026-04-29 16:50:59,801 >> chat template saved in saves/qwen3_1.7b/math/fft/chat_template.jinja
[INFO|tokenization_utils_base.py:2078] 2026-04-29 16:50:59,805 >> tokenizer config file saved in saves/qwen3_1.7b/math/fft/tokenizer_config.json
[INFO|modelcard.py:266] 2026-04-29 16:51:00,080 >> Dropping the following result as it does not have all the necessary fields:
{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
Processing Files (0 / 0) : | | 0.00B / 0.00B
New Data Upload : | | 0.00B / 0.00B 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 2%|▏ | 72.0MB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 2%|▏ | 72.0MB / 4.06GB 
Processing Files (2 / 3) : 2%|▏ | 83.4MB / 4.07GB, ???B/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 5%|▍ | 184MB / 4.06GB 
Processing Files (2 / 3) : 5%|▍ | 195MB / 4.07GB, 561MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 8%|▊ | 320MB / 4.06GB 
Processing Files (2 / 3) : 8%|▊ | 331MB / 4.07GB, 619MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 11%|█ | 448MB / 4.06GB 
Processing Files (2 / 3) : 11%|█▏ | 459MB / 4.07GB, 627MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 14%|█▍ | 576MB / 4.06GB 
Processing Files (2 / 3) : 14%|█▍ | 587MB / 4.07GB, 630MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 18%|█▊ | 712MB / 4.06GB 
Processing Files (2 / 3) : 18%|█▊ | 723MB / 4.07GB, 640MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 21%|██ | 840MB / 4.06GB 
Processing Files (2 / 3) : 21%|██ | 851MB / 4.07GB, 640MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 24%|██▍ | 976MB / 4.06GB 
Processing Files (2 / 3) : 24%|██▍ | 987MB / 4.07GB, 646MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 27%|██▋ | 1.11GB / 4.06GB 
Processing Files (2 / 3) : 28%|██▊ | 1.12GB / 4.07GB, 650MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 30%|███ | 1.24GB / 4.06GB 
Processing Files (2 / 3) : 31%|███ | 1.25GB / 4.07GB, 647MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 34%|███▍ | 1.37GB / 4.06GB 
Processing Files (2 / 3) : 34%|███▍ | 1.38GB / 4.07GB, 650MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 37%|███▋ | 1.51GB / 4.06GB 
Processing Files (2 / 3) : 37%|███▋ | 1.52GB / 4.07GB, 653MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 40%|████ | 1.64GB / 4.06GB 
Processing Files (2 / 3) : 41%|████ | 1.66GB / 4.07GB, 655MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 44%|████▍ | 1.78GB / 4.06GB 
Processing Files (2 / 3) : 44%|████▍ | 1.79GB / 4.07GB, 657MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 47%|████▋ | 1.92GB / 4.06GB 
Processing Files (2 / 3) : 47%|████▋ | 1.93GB / 4.07GB, 659MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 51%|█████ | 2.06GB / 4.06GB 
Processing Files (2 / 3) : 51%|█████ | 2.07GB / 4.07GB, 663MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 55%|█████▍ | 2.23GB / 4.06GB 
Processing Files (2 / 3) : 55%|█████▍ | 2.24GB / 4.07GB, 674MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 59%|█████▉ | 2.40GB / 4.06GB 
Processing Files (2 / 3) : 59%|█████▉ | 2.41GB / 4.07GB, 683MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 63%|██████▎ | 2.56GB / 4.06GB 
Processing Files (2 / 3) : 63%|██████▎ | 2.58GB / 4.07GB, 692MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 67%|██████▋ | 2.73GB / 4.06GB 
Processing Files (2 / 3) : 67%|██████▋ | 2.74GB / 4.07GB, 700MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 71%|███████▏ | 2.90GB / 4.06GB 
Processing Files (2 / 3) : 71%|███████▏ | 2.91GB / 4.07GB, 707MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 75%|███████▌ | 3.07GB / 4.06GB 
Processing Files (2 / 3) : 76%|███████▌ | 3.08GB / 4.07GB, 713MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 79%|███████▉ | 3.23GB / 4.06GB 
Processing Files (2 / 3) : 79%|███████▉ | 3.24GB / 4.07GB, 717MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 84%|████████▎ | 3.40GB / 4.06GB 
Processing Files (2 / 3) : 84%|████████▎ | 3.41GB / 4.07GB, 723MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 88%|████████▊ | 3.56GB / 4.06GB 
Processing Files (2 / 3) : 88%|████████▊ | 3.58GB / 4.07GB, 727MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 92%|█████████▏| 3.73GB / 4.06GB 
Processing Files (2 / 3) : 92%|█████████▏| 3.74GB / 4.07GB, 732MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 96%|█████████▌| 3.91GB / 4.06GB 
Processing Files (2 / 3) : 96%|█████████▌| 3.92GB / 4.07GB, 738MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB 
Processing Files (2 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 738MB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB 
Processing Files (2 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 713MB/s
New Data Upload : 92%|█████████▏| 3.80MB / 4.12MB, 678kB/s 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
Processing Files (3 / 3) : 100%|██████████| 4.07GB / 4.07GB, 644MB/s
New Data Upload : 100%|██████████| 4.12MB / 4.12MB, 664kB/s 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB 
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB 
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB 
Processing Files (3 / 3) : 100%|██████████| 4.07GB / 4.07GB, 554MB/s
New Data Upload : 100%|██████████| 4.12MB / 4.12MB, 572kB/s
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB
Reference in New Issue View Git Blame Copy Permalink