2844 lines
979 KiB
Plaintext
2844 lines
979 KiB
Plaintext
==== STARTING EXPERIMENT: qwen3_1.7b_math_fft ====
|
||
Log File: saves/qwen3_1.7b/math/fft/qwen3_1.7b_math_fft_20260429_154203.log
|
||
HF Hub: https://huggingface.co/KKHYA/qwen3-1.7b-fft-math
|
||
Timestamp: 2026-04-29 15:42:03
|
||
=====================================
|
||
[INFO|2026-04-29 15:42:18] llamafactory.launcher:144 >> Initializing 8 distributed tasks at: 127.0.0.1:49311
|
||
W0429 15:42:19.791000 2470856 site-packages/torch/distributed/run.py:852]
|
||
W0429 15:42:19.791000 2470856 site-packages/torch/distributed/run.py:852] *****************************************
|
||
W0429 15:42:19.791000 2470856 site-packages/torch/distributed/run.py:852] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
|
||
W0429 15:42:19.791000 2470856 site-packages/torch/distributed/run.py:852] *****************************************
|
||
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
|
||
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
|
||
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
|
||
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
|
||
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
|
||
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
|
||
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
|
||
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
|
||
df: /root/.triton/autotune: No such file or directory
|
||
df: df: /root/.triton/autotune/root/.triton/autotunedf: /root/.triton/autotunedf: df: /root/.triton/autotune/root/.triton/autotunedf: /root/.triton/autotune: No such file or directory
|
||
: No such file or directory
|
||
: No such file or directory
|
||
: No such file or directory
|
||
: No such file or directory
|
||
: No such file or directory
|
||
[W429 15:42:34.641839785 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
|
||
[W429 15:42:34.641839403 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
|
||
[W429 15:42:34.648204081 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
|
||
[W429 15:42:34.656834411 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
|
||
[W429 15:42:34.704861810 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
|
||
[W429 15:42:34.738333250 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
|
||
[W429 15:42:34.757560566 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
|
||
[W429 15:42:34.774695972 ProcessGroupNCCL.cpp:929] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO cudaDriverVersion 13000
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NCCL version 2.28.9+cuda13.0
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO cudaDriverVersion 13000
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO cudaDriverVersion 13000
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NCCL version 2.28.9+cuda13.0
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NCCL version 2.28.9+cuda13.0
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO cudaDriverVersion 13000
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NCCL version 2.28.9+cuda13.0
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO cudaDriverVersion 13000
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NCCL version 2.28.9+cuda13.0
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO cudaDriverVersion 13000
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO cudaDriverVersion 13000
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NCCL version 2.28.9+cuda13.0
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NCCL version 2.28.9+cuda13.0
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ENV/Plugin: Could not find: libnccl-env.so
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO cudaDriverVersion 13000
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Bootstrap: Using eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NCCL version 2.28.9+cuda13.0
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Successfully loaded external network plugin /opt/amazon/ofi-nccl/lib/libnccl-net.so
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NET/Socket : Using [0]eth0:10.200.162.206<0>
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ncclCommInitRankConfig comm 0x55e36d363080 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 commId 0xbdc52b61bae6206d - Init START
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ncclCommInitRankConfig comm 0x56071a5c6320 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 commId 0xbdc52b61bae6206d - Init START
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ncclCommInitRankConfig comm 0x558cf7486a10 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 commId 0xbdc52b61bae6206d - Init START
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ncclCommInitRankConfig comm 0x55ffbe0f7c80 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 commId 0xbdc52b61bae6206d - Init START
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO RAS client listening socket at ::1<28028>
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO RAS client listening socket at ::1<28028>
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ncclCommInitRankConfig comm 0x5647078795e0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 commId 0xbdc52b61bae6206d - Init START
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ncclCommInitRankConfig comm 0x5612d38dcbf0 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 commId 0xbdc52b61bae6206d - Init START
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ncclCommInitRankConfig comm 0x5602809d0750 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 commId 0xbdc52b61bae6206d - Init START
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ncclCommInitRankConfig comm 0x55c0201d0a60 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 commId 0xbdc52b61bae6206d - Init START
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO RAS client listening socket at ::1<28028>
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO RAS client listening socket at ::1<28028>
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO RAS client listening socket at ::1<28028>
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO RAS client listening socket at ::1<28028>
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO RAS client listening socket at ::1<28028>
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO RAS client listening socket at ::1<28028>
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Bootstrap timings total 0.008871 (create 0.000085, send 0.000076, recv 0.007974, ring 0.000343, delay 0.000001)
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Bootstrap timings total 0.001031 (create 0.000050, send 0.000072, recv 0.000209, ring 0.000343, delay 0.000001)
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Bootstrap timings total 0.001899 (create 0.000034, send 0.000071, recv 0.000096, ring 0.000324, delay 0.000001)
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Bootstrap timings total 0.005427 (create 0.000034, send 0.000064, recv 0.000224, ring 0.004700, delay 0.000001)
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Bootstrap timings total 0.002049 (create 0.000036, send 0.000068, recv 0.001328, ring 0.000239, delay 0.000001)
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Bootstrap timings total 0.029934 (create 0.000045, send 0.000090, recv 0.024643, ring 0.004705, delay 0.000001)
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Bootstrap timings total 0.000897 (create 0.000036, send 0.000064, recv 0.000252, ring 0.000131, delay 0.000001)
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Bootstrap timings total 0.010645 (create 0.000037, send 0.000077, recv 0.000107, ring 0.000122, delay 0.000001)
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 3 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NVLS multicast support is not available on dev 3 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 4 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NVLS multicast support is not available on dev 4 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 2 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NVLS multicast support is not available on dev 2 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 1 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NVLS multicast support is not available on dev 1 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 5 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NVLS multicast support is not available on dev 5 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 6 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NVLS multicast support is not available on dev 6 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NVLS multicast support is not available on dev 0 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 7 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NVLS multicast support is not available on dev 7 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO comm 0x5612d38dcbf0 rank 3 nRanks 8 nNodes 1 localRanks 8 localRank 3 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO comm 0x55c0201d0a60 rank 5 nRanks 8 nNodes 1 localRanks 8 localRank 5 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO comm 0x5647078795e0 rank 4 nRanks 8 nNodes 1 localRanks 8 localRank 4 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO comm 0x56071a5c6320 rank 6 nRanks 8 nNodes 1 localRanks 8 localRank 6 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO comm 0x558cf7486a10 rank 1 nRanks 8 nNodes 1 localRanks 8 localRank 1 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO comm 0x55e36d363080 rank 7 nRanks 8 nNodes 1 localRanks 8 localRank 7 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO comm 0x5602809d0750 rank 2 nRanks 8 nNodes 1 localRanks 8 localRank 2 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO comm 0x55ffbe0f7c80 rank 0 nRanks 8 nNodes 1 localRanks 8 localRank 0 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 00/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 01/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 02/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 03/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 04/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 05/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 06/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 07/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 08/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 09/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 10/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 11/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 12/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 13/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 14/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 15/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 16/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 17/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 18/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 19/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 20/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 21/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 22/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 23/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470948:2471411 [0] NCCL INFO [Proxy Service UDS] Device 1 CPU core 49
|
||
ywang29-p4d-debug-worker-0:2470948:2471410 [0] NCCL INFO [Proxy Service] Device 1 CPU core 23
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470953:2471412 [0] NCCL INFO [Proxy Service] Device 5 CPU core 80
|
||
ywang29-p4d-debug-worker-0:2470953:2471413 [0] NCCL INFO [Proxy Service UDS] Device 5 CPU core 86
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470947:2471414 [0] NCCL INFO [Proxy Service] Device 0 CPU core 15
|
||
ywang29-p4d-debug-worker-0:2470947:2471415 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 65
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470949:2471416 [0] NCCL INFO [Proxy Service] Device 2 CPU core 9
|
||
ywang29-p4d-debug-worker-0:2470949:2471417 [0] NCCL INFO [Proxy Service UDS] Device 2 CPU core 9
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470952:2471418 [0] NCCL INFO [Proxy Service] Device 4 CPU core 38
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
|
||
ywang29-p4d-debug-worker-0:2470952:2471419 [0] NCCL INFO [Proxy Service UDS] Device 4 CPU core 39
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470955:2471421 [0] NCCL INFO [Proxy Service UDS] Device 7 CPU core 34
|
||
ywang29-p4d-debug-worker-0:2470955:2471420 [0] NCCL INFO [Proxy Service] Device 7 CPU core 82
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470954:2471422 [0] NCCL INFO [Proxy Service] Device 6 CPU core 31
|
||
ywang29-p4d-debug-worker-0:2470954:2471423 [0] NCCL INFO [Proxy Service UDS] Device 6 CPU core 31
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470951:2471424 [0] NCCL INFO [Proxy Service] Device 3 CPU core 4
|
||
ywang29-p4d-debug-worker-0:2470951:2471425 [0] NCCL INFO [Proxy Service UDS] Device 3 CPU core 3
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO CC Off, workFifoBytes 1048576
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO TUNER/Plugin: Using nccl_ofi_tuner (v3)
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ncclCommInitRankConfig comm 0x5602809d0750 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 commId 0xbdc52b61bae6206d - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ncclCommInitRankConfig comm 0x56071a5c6320 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 commId 0xbdc52b61bae6206d - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ncclCommInitRankConfig comm 0x5647078795e0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 commId 0xbdc52b61bae6206d - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ncclCommInitRankConfig comm 0x55ffbe0f7c80 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 commId 0xbdc52b61bae6206d - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Init timings - ncclCommInitRankConfig: rank 2 nranks 8 total 0.52 (kernels 0.17, alloc 0.22, bootstrap 0.00, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.06, rest 0.04)
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Init timings - ncclCommInitRankConfig: rank 4 nranks 8 total 0.53 (kernels 0.18, alloc 0.22, bootstrap 0.00, allgathers 0.00, topo 0.03, graphs 0.00, connections 0.06, rest 0.04)
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Init timings - ncclCommInitRankConfig: rank 6 nranks 8 total 0.53 (kernels 0.18, alloc 0.22, bootstrap 0.01, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.05, rest 0.04)
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Init timings - ncclCommInitRankConfig: rank 0 nranks 8 total 0.53 (kernels 0.18, alloc 0.22, bootstrap 0.01, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.06, rest 0.03)
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ncclCommInitRankConfig comm 0x558cf7486a10 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 commId 0xbdc52b61bae6206d - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ncclCommInitRankConfig comm 0x55e36d363080 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 commId 0xbdc52b61bae6206d - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ncclCommInitRankConfig comm 0x55c0201d0a60 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 commId 0xbdc52b61bae6206d - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Init timings - ncclCommInitRankConfig: rank 1 nranks 8 total 0.53 (kernels 0.18, alloc 0.22, bootstrap 0.01, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.08, rest 0.02)
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Init timings - ncclCommInitRankConfig: rank 7 nranks 8 total 0.53 (kernels 0.18, alloc 0.20, bootstrap 0.03, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.06, rest 0.04)
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Init timings - ncclCommInitRankConfig: rank 5 nranks 8 total 0.53 (kernels 0.18, alloc 0.22, bootstrap 0.00, allgathers 0.00, topo 0.04, graphs 0.00, connections 0.08, rest 0.02)
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ncclCommInitRankConfig comm 0x5612d38dcbf0 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 commId 0xbdc52b61bae6206d - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Init timings - ncclCommInitRankConfig: rank 3 nranks 8 total 0.53 (kernels 0.18, alloc 0.22, bootstrap 0.00, allgathers 0.00, topo 0.03, graphs 0.00, connections 0.06, rest 0.04)
|
||
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 5, world size: 8, device: cuda:5, distributed training: True, compute dtype: torch.bfloat16
|
||
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 4, world size: 8, device: cuda:4, distributed training: True, compute dtype: torch.bfloat16
|
||
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 6, world size: 8, device: cuda:6, distributed training: True, compute dtype: torch.bfloat16
|
||
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 7, world size: 8, device: cuda:7, distributed training: True, compute dtype: torch.bfloat16
|
||
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 1, world size: 8, device: cuda:1, distributed training: True, compute dtype: torch.bfloat16
|
||
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 2, world size: 8, device: cuda:2, distributed training: True, compute dtype: torch.bfloat16
|
||
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 3, world size: 8, device: cuda:3, distributed training: True, compute dtype: torch.bfloat16
|
||
[INFO|2026-04-29 15:42:35] llamafactory.hparams.parser:505 >> Process rank: 0, world size: 8, device: cuda:0, distributed training: True, compute dtype: torch.bfloat16
|
||
[INFO|configuration_utils.py:670] 2026-04-29 15:42:35,891 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/config.json
|
||
[INFO|configuration_utils.py:742] 2026-04-29 15:42:35,894 >> Model config Qwen3Config {
|
||
"architectures": [
|
||
"Qwen3ForCausalLM"
|
||
],
|
||
"attention_bias": false,
|
||
"attention_dropout": 0.0,
|
||
"bos_token_id": 151643,
|
||
"dtype": "bfloat16",
|
||
"eos_token_id": 151645,
|
||
"head_dim": 128,
|
||
"hidden_act": "silu",
|
||
"hidden_size": 2048,
|
||
"initializer_range": 0.02,
|
||
"intermediate_size": 6144,
|
||
"layer_types": [
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention"
|
||
],
|
||
"masked_layers": null,
|
||
"max_position_embeddings": 40960,
|
||
"max_window_layers": 28,
|
||
"model_type": "qwen3",
|
||
"num_attention_heads": 16,
|
||
"num_hidden_layers": 28,
|
||
"num_key_value_heads": 8,
|
||
"pad_token_id": null,
|
||
"rms_norm_eps": 1e-06,
|
||
"rope_parameters": {
|
||
"rope_theta": 1000000,
|
||
"rope_type": "default"
|
||
},
|
||
"sliding_window": null,
|
||
"sparsity_attn": null,
|
||
"sparsity_mlp": null,
|
||
"subnet_mode": null,
|
||
"subnet_type": null,
|
||
"threshold_attn": null,
|
||
"threshold_mlp": null,
|
||
"tie_word_embeddings": true,
|
||
"transformers_version": "5.2.0",
|
||
"use_cache": true,
|
||
"use_sliding_window": false,
|
||
"vocab_size": 151936
|
||
}
|
||
|
||
[INFO|configuration_utils.py:670] 2026-04-29 15:42:39,796 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/config.json
|
||
[INFO|configuration_utils.py:742] 2026-04-29 15:42:39,797 >> Model config Qwen3Config {
|
||
"architectures": [
|
||
"Qwen3ForCausalLM"
|
||
],
|
||
"attention_bias": false,
|
||
"attention_dropout": 0.0,
|
||
"bos_token_id": 151643,
|
||
"dtype": "bfloat16",
|
||
"eos_token_id": 151645,
|
||
"head_dim": 128,
|
||
"hidden_act": "silu",
|
||
"hidden_size": 2048,
|
||
"initializer_range": 0.02,
|
||
"intermediate_size": 6144,
|
||
"layer_types": [
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention"
|
||
],
|
||
"masked_layers": null,
|
||
"max_position_embeddings": 40960,
|
||
"max_window_layers": 28,
|
||
"model_type": "qwen3",
|
||
"num_attention_heads": 16,
|
||
"num_hidden_layers": 28,
|
||
"num_key_value_heads": 8,
|
||
"pad_token_id": null,
|
||
"rms_norm_eps": 1e-06,
|
||
"rope_parameters": {
|
||
"rope_theta": 1000000,
|
||
"rope_type": "default"
|
||
},
|
||
"sliding_window": null,
|
||
"sparsity_attn": null,
|
||
"sparsity_mlp": null,
|
||
"subnet_mode": null,
|
||
"subnet_type": null,
|
||
"threshold_attn": null,
|
||
"threshold_mlp": null,
|
||
"tie_word_embeddings": true,
|
||
"transformers_version": "5.2.0",
|
||
"use_cache": true,
|
||
"use_sliding_window": false,
|
||
"vocab_size": 151936
|
||
}
|
||
|
||
[INFO|configuration_utils.py:670] 2026-04-29 15:42:39,906 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/config.json
|
||
[INFO|configuration_utils.py:742] 2026-04-29 15:42:39,907 >> Model config Qwen3Config {
|
||
"architectures": [
|
||
"Qwen3ForCausalLM"
|
||
],
|
||
"attention_bias": false,
|
||
"attention_dropout": 0.0,
|
||
"bos_token_id": 151643,
|
||
"dtype": "bfloat16",
|
||
"eos_token_id": 151645,
|
||
"head_dim": 128,
|
||
"hidden_act": "silu",
|
||
"hidden_size": 2048,
|
||
"initializer_range": 0.02,
|
||
"intermediate_size": 6144,
|
||
"layer_types": [
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention"
|
||
],
|
||
"masked_layers": null,
|
||
"max_position_embeddings": 40960,
|
||
"max_window_layers": 28,
|
||
"model_type": "qwen3",
|
||
"num_attention_heads": 16,
|
||
"num_hidden_layers": 28,
|
||
"num_key_value_heads": 8,
|
||
"pad_token_id": null,
|
||
"rms_norm_eps": 1e-06,
|
||
"rope_parameters": {
|
||
"rope_theta": 1000000,
|
||
"rope_type": "default"
|
||
},
|
||
"sliding_window": null,
|
||
"sparsity_attn": null,
|
||
"sparsity_mlp": null,
|
||
"subnet_mode": null,
|
||
"subnet_type": null,
|
||
"threshold_attn": null,
|
||
"threshold_mlp": null,
|
||
"tie_word_embeddings": true,
|
||
"transformers_version": "5.2.0",
|
||
"use_cache": true,
|
||
"use_sliding_window": false,
|
||
"vocab_size": 151936
|
||
}
|
||
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 16/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 17/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 18/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 19/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 20/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 21/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 22/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 23/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 00/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 01/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 02/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 03/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 04/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 05/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 06/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 07/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 08/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 09/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 10/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 11/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 12/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 13/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 14/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 15/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 16/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 17/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 18/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 19/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 20/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 21/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 22/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 23/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 16/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 17/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 18/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 19/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 20/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 21/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 22/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 23/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
[INFO|2026-04-29 15:42:41] llamafactory.data.loader:144 >> Loading dataset KKHYA/metamath_converted...
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 16/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 17/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 18/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 19/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 20/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 21/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 22/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 23/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 00/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 01/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 02/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 03/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 04/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 05/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 06/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 07/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 08/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 09/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 10/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 11/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 12/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 13/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 14/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 15/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 16/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 17/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 18/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 19/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 20/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 21/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 22/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 23/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 16/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 17/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 18/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 19/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 20/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 21/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 22/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 23/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 16/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 17/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 18/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 19/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 20/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 21/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 22/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 23/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Setting num_proc from 16 to 2 for the train split as it only contains 2 shards.
|
||
|
||
Generating train split: 0%| | 0/394996 [00:00<?, ? examples/s]
|
||
Generating train split: 1%| | 3000/394996 [00:00<00:47, 8307.48 examples/s]
|
||
Generating train split: 12%|█▏ | 47000/394996 [00:00<00:02, 129958.07 examples/s]
|
||
Generating train split: 24%|██▎ | 93000/394996 [00:00<00:01, 215888.84 examples/s]
|
||
Generating train split: 37%|███▋ | 147000/394996 [00:00<00:00, 305992.13 examples/s]
|
||
Generating train split: 50%|█████ | 199000/394996 [00:00<00:00, 362914.23 examples/s]
|
||
Generating train split: 64%|██████▍ | 252000/394996 [00:00<00:00, 402970.92 examples/s]
|
||
Generating train split: 78%|███████▊ | 309000/394996 [00:00<00:00, 446553.16 examples/s]
|
||
Generating train split: 92%|█████████▏| 365000/394996 [00:01<00:00, 465307.36 examples/s]
|
||
Generating train split: 100%|██████████| 394996/394996 [00:01<00:00, 315529.42 examples/s]
|
||
[INFO|2026-04-29 15:42:50] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset KKHYA/metamath_converted.
|
||
|
||
Converting format of dataset (num_proc=16): 0%| | 0/10000 [00:00<?, ? examples/s]
|
||
Converting format of dataset (num_proc=16): 5%|▌ | 527/10000 [00:00<00:08, 1118.11 examples/s]
|
||
Converting format of dataset (num_proc=16): 100%|██████████| 10000/10000 [00:00<00:00, 14140.08 examples/s]
|
||
[INFO|2026-04-29 15:42:50] llamafactory.data.loader:144 >> Loading dataset KKHYA/numinamath_tir_converted...
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Setting num_proc from 16 back to 1 for the train split to disable multiprocessing as it only contains one shard.
|
||
|
||
Generating train split: 0%| | 0/72441 [00:00<?, ? examples/s]
|
||
Generating train split: 8%|▊ | 6000/72441 [00:00<00:01, 52916.63 examples/s]
|
||
Generating train split: 22%|██▏ | 16000/72441 [00:00<00:00, 75880.62 examples/s]
|
||
Generating train split: 39%|███▊ | 28000/72441 [00:00<00:00, 93879.61 examples/s]
|
||
Generating train split: 54%|█████▍ | 39000/72441 [00:00<00:00, 95367.14 examples/s]
|
||
Generating train split: 68%|██████▊ | 49000/72441 [00:00<00:00, 95909.35 examples/s]
|
||
Generating train split: 84%|████████▍ | 61000/72441 [00:00<00:00, 102761.15 examples/s]
|
||
Generating train split: 100%|██████████| 72441/72441 [00:00<00:00, 103411.87 examples/s]
|
||
Generating train split: 100%|██████████| 72441/72441 [00:00<00:00, 96274.33 examples/s]
|
||
Setting num_proc from 16 back to 1 for the test split to disable multiprocessing as it only contains one shard.
|
||
|
||
Generating test split: 0%| | 0/99 [00:00<?, ? examples/s]
|
||
Generating test split: 100%|██████████| 99/99 [00:00<00:00, 29883.85 examples/s]
|
||
[INFO|2026-04-29 15:43:05] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset KKHYA/numinamath_tir_converted.
|
||
|
||
Converting format of dataset (num_proc=16): 0%| | 0/10000 [00:00<?, ? examples/s]
|
||
Converting format of dataset (num_proc=16): 4%|▍ | 440/10000 [00:00<00:10, 920.19 examples/s]
|
||
Converting format of dataset (num_proc=16): 100%|██████████| 10000/10000 [00:00<00:00, 14552.33 examples/s]
|
||
[INFO|2026-04-29 15:43:05] llamafactory.data.loader:144 >> Loading dataset KKHYA/numinamath_cot_converted...
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Setting num_proc from 16 to 5 for the train split as it only contains 5 shards.
|
||
|
||
Generating train split: 0%| | 0/859493 [00:00<?, ? examples/s]
|
||
Generating train split: 0%| | 1000/859493 [00:00<03:57, 3608.37 examples/s]
|
||
Generating train split: 6%|▌ | 48000/859493 [00:00<00:05, 159542.65 examples/s]
|
||
Generating train split: 14%|█▍ | 124000/859493 [00:00<00:02, 332328.64 examples/s]
|
||
Generating train split: 24%|██▍ | 205000/859493 [00:00<00:01, 464546.22 examples/s]
|
||
Generating train split: 33%|███▎ | 285000/859493 [00:00<00:01, 554177.55 examples/s]
|
||
Generating train split: 42%|████▏ | 359000/859493 [00:00<00:00, 576642.19 examples/s]
|
||
Generating train split: 52%|█████▏ | 443000/859493 [00:00<00:00, 637290.61 examples/s]
|
||
Generating train split: 60%|██████ | 519000/859493 [00:01<00:00, 661365.99 examples/s]
|
||
Generating train split: 69%|██████▉ | 592000/859493 [00:01<00:00, 655939.40 examples/s]
|
||
Generating train split: 79%|███████▊ | 675000/859493 [00:01<00:00, 669802.57 examples/s]
|
||
Generating train split: 87%|████████▋ | 751000/859493 [00:01<00:00, 682799.05 examples/s]
|
||
Generating train split: 96%|█████████▋| 827898/859493 [00:01<00:00, 705069.90 examples/s]
|
||
Generating train split: 100%|██████████| 859493/859493 [00:01<00:00, 532761.42 examples/s]
|
||
Setting num_proc from 16 back to 1 for the test split to disable multiprocessing as it only contains one shard.
|
||
|
||
Generating test split: 0%| | 0/100 [00:00<?, ? examples/s]
|
||
Generating test split: 100%|██████████| 100/100 [00:00<00:00, 11119.28 examples/s]
|
||
[INFO|2026-04-29 15:43:22] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset KKHYA/numinamath_cot_converted.
|
||
|
||
Converting format of dataset (num_proc=16): 0%| | 0/10000 [00:00<?, ? examples/s]
|
||
Converting format of dataset (num_proc=16): 5%|▌ | 520/10000 [00:00<00:10, 869.91 examples/s]
|
||
Converting format of dataset (num_proc=16): 100%|██████████| 10000/10000 [00:00<00:00, 12240.88 examples/s]
|
||
[INFO|2026-04-29 15:43:23] llamafactory.data.loader:144 >> Loading dataset allenai/tulu-3-sft-personas-math...
|
||
Setting num_proc from 16 to 2 for the train split as it only contains 2 shards.
|
||
|
||
Generating train split: 0%| | 0/149960 [00:00<?, ? examples/s]
|
||
Generating train split: 1%| | 1000/149960 [00:00<00:22, 6515.30 examples/s]
|
||
Generating train split: 11%|█▏ | 17000/149960 [00:00<00:01, 76378.12 examples/s]
|
||
Generating train split: 22%|██▏ | 33000/149960 [00:00<00:01, 107818.99 examples/s]
|
||
Generating train split: 34%|███▍ | 51000/149960 [00:00<00:00, 129688.58 examples/s]
|
||
Generating train split: 44%|████▍ | 66000/149960 [00:00<00:00, 133412.83 examples/s]
|
||
Generating train split: 56%|█████▌ | 84000/149960 [00:00<00:00, 139172.97 examples/s]
|
||
Generating train split: 69%|██████▉ | 104000/149960 [00:00<00:00, 154551.66 examples/s]
|
||
Generating train split: 81%|████████ | 121000/149960 [00:00<00:00, 157547.03 examples/s]
|
||
Generating train split: 93%|█████████▎| 140000/149960 [00:01<00:00, 162863.39 examples/s]
|
||
Generating train split: 100%|██████████| 149960/149960 [00:01<00:00, 126921.10 examples/s]
|
||
[INFO|2026-04-29 15:43:31] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset allenai/tulu-3-sft-personas-math.
|
||
|
||
Converting format of dataset (num_proc=16): 0%| | 0/10000 [00:00<?, ? examples/s]
|
||
Converting format of dataset (num_proc=16): 5%|▌ | 529/10000 [00:00<00:10, 911.77 examples/s]
|
||
Converting format of dataset (num_proc=16): 100%|██████████| 10000/10000 [00:00<00:00, 12526.85 examples/s]
|
||
[INFO|2026-04-29 15:43:32] llamafactory.data.loader:144 >> Loading dataset allenai/tulu-3-sft-personas-math-grade...
|
||
Setting num_proc from 16 back to 1 for the train split to disable multiprocessing as it only contains one shard.
|
||
|
||
Generating train split: 0%| | 0/49980 [00:00<?, ? examples/s]
|
||
Generating train split: 30%|███ | 15000/49980 [00:00<00:00, 140747.51 examples/s]
|
||
Generating train split: 72%|███████▏ | 36000/49980 [00:00<00:00, 175603.03 examples/s]
|
||
Generating train split: 100%|██████████| 49980/49980 [00:00<00:00, 182907.76 examples/s]
|
||
[INFO|2026-04-29 15:43:36] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset allenai/tulu-3-sft-personas-math-grade.
|
||
|
||
Converting format of dataset (num_proc=16): 0%| | 0/10000 [00:00<?, ? examples/s]
|
||
Converting format of dataset (num_proc=16): 5%|▍ | 459/10000 [00:00<00:12, 751.02 examples/s]
|
||
Converting format of dataset (num_proc=16): 100%|██████████| 10000/10000 [00:00<00:00, 12057.75 examples/s]
|
||
[INFO|2026-04-29 15:43:37] llamafactory.data.loader:144 >> Loading dataset allenai/tulu-3-sft-personas-algebra...
|
||
Setting num_proc from 16 back to 1 for the train split to disable multiprocessing as it only contains one shard.
|
||
|
||
Generating train split: 0%| | 0/20000 [00:00<?, ? examples/s]
|
||
Generating train split: 40%|████ | 8000/20000 [00:00<00:00, 74061.29 examples/s]
|
||
Generating train split: 100%|██████████| 20000/20000 [00:00<00:00, 100731.63 examples/s]
|
||
[INFO|2026-04-29 15:43:41] llamafactory.data.loader:144 >> Sampled 10000 examples from dataset allenai/tulu-3-sft-personas-algebra.
|
||
|
||
Converting format of dataset (num_proc=16): 0%| | 0/10000 [00:00<?, ? examples/s]
|
||
Converting format of dataset (num_proc=16): 6%|▌ | 567/10000 [00:00<00:09, 973.12 examples/s]
|
||
Converting format of dataset (num_proc=16): 100%|██████████| 10000/10000 [00:00<00:00, 12467.30 examples/s]
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 16/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 17/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 18/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 19/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 20/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 21/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 22/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 23/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
|
||
Running tokenizer on dataset (num_proc=16): 0%| | 0/60000 [00:00<?, ? examples/s]Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
Repo card metadata block was not found. Setting CardData to empty.
|
||
|
||
Running tokenizer on dataset (num_proc=16): 2%|▏ | 1000/60000 [00:03<02:59, 329.09 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 3%|▎ | 2000/60000 [00:03<01:37, 597.68 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 5%|▌ | 3000/60000 [00:04<01:09, 818.81 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 8%|▊ | 4750/60000 [00:04<00:39, 1387.60 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 10%|▉ | 5750/60000 [00:05<00:28, 1874.67 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 11%|█▏ | 6750/60000 [00:05<00:30, 1722.68 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 14%|█▍ | 8500/60000 [00:06<00:22, 2288.73 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 16%|█▌ | 9500/60000 [00:06<00:19, 2590.57 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 18%|█▊ | 10500/60000 [00:07<00:31, 1584.14 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 19%|█▉ | 11500/60000 [00:08<00:26, 1803.92 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 20%|██ | 12250/60000 [00:09<00:36, 1302.72 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 22%|██▏ | 13250/60000 [00:09<00:28, 1660.18 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 24%|██▍ | 14250/60000 [00:09<00:25, 1785.82 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 25%|██▌ | 15250/60000 [00:10<00:27, 1625.85 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 27%|██▋ | 16250/60000 [00:11<00:26, 1626.96 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 29%|██▉ | 17250/60000 [00:11<00:20, 2123.23 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 30%|███ | 18250/60000 [00:11<00:18, 2251.50 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 32%|███▏ | 19250/60000 [00:12<00:16, 2456.03 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 34%|███▍ | 20250/60000 [00:12<00:18, 2172.46 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 37%|███▋ | 22250/60000 [00:13<00:12, 2923.16 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 40%|████ | 24000/60000 [00:13<00:10, 3422.65 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 42%|████▏ | 25000/60000 [00:13<00:12, 2897.39 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 45%|████▍ | 26750/60000 [00:14<00:10, 3200.26 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 46%|████▌ | 27500/60000 [00:14<00:09, 3545.46 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 47%|████▋ | 28250/60000 [00:14<00:10, 2940.95 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 49%|████▉ | 29250/60000 [00:15<00:09, 3161.81 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 50%|█████ | 30250/60000 [00:15<00:12, 2332.51 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 52%|█████▏ | 31000/60000 [00:16<00:12, 2399.25 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 53%|█████▎ | 32000/60000 [00:17<00:16, 1698.81 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 55%|█████▌ | 33000/60000 [00:18<00:18, 1465.71 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 57%|█████▋ | 34000/60000 [00:18<00:15, 1641.74 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 58%|█████▊ | 35000/60000 [00:19<00:15, 1641.56 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 60%|██████ | 36000/60000 [00:19<00:11, 2028.31 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 63%|██████▎ | 38000/60000 [00:20<00:11, 1898.06 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 65%|██████▌ | 39000/60000 [00:20<00:08, 2370.35 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 68%|██████▊ | 41000/60000 [00:20<00:06, 3153.95 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 70%|██████▉ | 41750/60000 [00:21<00:07, 2406.48 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 71%|███████▏ | 42750/60000 [00:21<00:06, 2521.33 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 75%|███████▍ | 44750/60000 [00:22<00:05, 2998.25 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 76%|███████▌ | 45500/60000 [00:22<00:05, 2795.64 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 78%|███████▊ | 46500/60000 [00:23<00:05, 2659.38 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 79%|███████▉ | 47500/60000 [00:23<00:04, 3028.73 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 81%|████████ | 48500/60000 [00:23<00:03, 2909.60 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 82%|████████▎ | 49500/60000 [00:24<00:03, 2941.81 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 84%|████████▍ | 50250/60000 [00:24<00:04, 2075.51 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 85%|████████▌ | 51250/60000 [00:24<00:03, 2676.86 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 87%|████████▋ | 52000/60000 [00:25<00:02, 3023.88 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 88%|████████▊ | 53000/60000 [00:25<00:02, 2461.22 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 90%|█████████ | 54000/60000 [00:25<00:01, 3157.23 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 91%|█████████▏| 54750/60000 [00:26<00:01, 2711.01 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 93%|█████████▎| 55750/60000 [00:27<00:02, 1587.03 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 94%|█████████▍| 56500/60000 [00:27<00:01, 1820.15 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 96%|█████████▌| 57500/60000 [00:27<00:01, 2027.74 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 97%|█████████▋| 58250/60000 [00:29<00:01, 1051.85 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 100%|██████████| 60000/60000 [00:31<00:00, 1037.92 examples/s]
|
||
Running tokenizer on dataset (num_proc=16): 100%|██████████| 60000/60000 [00:31<00:00, 1906.46 examples/s]
|
||
training example:
|
||
input_ids:
|
||
[151644, 872, 198, 67909, 702, 2326, 3039, 438, 1753, 3220, 438, 21475, 13, 2379, 614, 264, 10856, 2790, 315, 400, 22, 21, 13, 2585, 1753, 3220, 1558, 19685, 614, 30, 151645, 198, 151644, 77091, 198, 10061, 594, 9658, 21475, 702, 856, 11192, 624, 12209, 19685, 702, 220, 18, 87, 11192, 624, 785, 2790, 3311, 315, 3220, 807, 614, 374, 856, 488, 220, 18, 87, 284, 220, 19, 87, 198, 1654, 1414, 429, 279, 2790, 3311, 315, 3220, 807, 614, 374, 400, 22, 21, 11, 773, 220, 19, 87, 284, 400, 22, 21, 198, 12509, 6577, 2176, 11067, 315, 279, 23606, 553, 220, 19, 11, 582, 633, 856, 284, 400, 16, 24, 198, 12549, 19685, 702, 2326, 3039, 438, 1753, 3220, 438, 21475, 11, 19685, 702, 220, 18, 353, 400, 16, 24, 284, 400, 20, 22, 198, 54815, 11, 19685, 702, 400, 20, 22, 198, 820, 220, 20, 22, 198, 785, 4226, 374, 25, 220, 20, 22, 151645, 198]
|
||
inputs:
|
||
<|im_start|>user
|
||
Jean has three times as much money as Jane. They have a combined total of $76. How much money does Jean have?<|im_end|>
|
||
<|im_start|>assistant
|
||
Let's assume Jane has x dollars.
|
||
Then Jean has 3x dollars.
|
||
The total amount of money they have is x + 3x = 4x
|
||
We know that the total amount of money they have is $76, so 4x = $76
|
||
Dividing both sides of the equation by 4, we get x = $19
|
||
Since Jean has three times as much money as Jane, Jean has 3 * $19 = $57
|
||
Therefore, Jean has $57
|
||
#### 57
|
||
The answer is: 57<|im_end|>
|
||
|
||
label_ids:
|
||
[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 10061, 594, 9658, 21475, 702, 856, 11192, 624, 12209, 19685, 702, 220, 18, 87, 11192, 624, 785, 2790, 3311, 315, 3220, 807, 614, 374, 856, 488, 220, 18, 87, 284, 220, 19, 87, 198, 1654, 1414, 429, 279, 2790, 3311, 315, 3220, 807, 614, 374, 400, 22, 21, 11, 773, 220, 19, 87, 284, 400, 22, 21, 198, 12509, 6577, 2176, 11067, 315, 279, 23606, 553, 220, 19, 11, 582, 633, 856, 284, 400, 16, 24, 198, 12549, 19685, 702, 2326, 3039, 438, 1753, 3220, 438, 21475, 11, 19685, 702, 220, 18, 353, 400, 16, 24, 284, 400, 20, 22, 198, 54815, 11, 19685, 702, 400, 20, 22, 198, 820, 220, 20, 22, 198, 785, 4226, 374, 25, 220, 20, 22, 151645, 198]
|
||
labels:
|
||
Let's assume Jane has x dollars.
|
||
Then Jean has 3x dollars.
|
||
The total amount of money they have is x + 3x = 4x
|
||
We know that the total amount of money they have is $76, so 4x = $76
|
||
Dividing both sides of the equation by 4, we get x = $19
|
||
Since Jean has three times as much money as Jane, Jean has 3 * $19 = $57
|
||
Therefore, Jean has $57
|
||
#### 57
|
||
The answer is: 57<|im_end|>
|
||
|
||
[INFO|configuration_utils.py:670] 2026-04-29 15:44:15,949 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/config.json
|
||
[INFO|configuration_utils.py:742] 2026-04-29 15:44:15,949 >> Model config Qwen3Config {
|
||
"architectures": [
|
||
"Qwen3ForCausalLM"
|
||
],
|
||
"attention_bias": false,
|
||
"attention_dropout": 0.0,
|
||
"bos_token_id": 151643,
|
||
"dtype": "bfloat16",
|
||
"eos_token_id": 151645,
|
||
"head_dim": 128,
|
||
"hidden_act": "silu",
|
||
"hidden_size": 2048,
|
||
"initializer_range": 0.02,
|
||
"intermediate_size": 6144,
|
||
"layer_types": [
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention",
|
||
"full_attention"
|
||
],
|
||
"masked_layers": null,
|
||
"max_position_embeddings": 40960,
|
||
"max_window_layers": 28,
|
||
"model_type": "qwen3",
|
||
"num_attention_heads": 16,
|
||
"num_hidden_layers": 28,
|
||
"num_key_value_heads": 8,
|
||
"pad_token_id": null,
|
||
"rms_norm_eps": 1e-06,
|
||
"rope_parameters": {
|
||
"rope_theta": 1000000,
|
||
"rope_type": "default"
|
||
},
|
||
"sliding_window": null,
|
||
"sparsity_attn": null,
|
||
"sparsity_mlp": null,
|
||
"subnet_mode": null,
|
||
"subnet_type": null,
|
||
"threshold_attn": null,
|
||
"threshold_mlp": null,
|
||
"tie_word_embeddings": true,
|
||
"transformers_version": "5.2.0",
|
||
"use_cache": true,
|
||
"use_sliding_window": false,
|
||
"vocab_size": 151936
|
||
}
|
||
|
||
[INFO|2026-04-29 15:44:15] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
|
||
[INFO|modeling_utils.py:710] 2026-04-29 15:44:17,465 >> loading weights file model.safetensors from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/model.safetensors.index.json
|
||
|
||
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
|
||
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
|
||
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
|
||
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
|
||
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
|
||
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
|
||
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
|
||
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s]
|
||
Fetching 2 files: 50%|█████ | 1/2 [00:04<00:04, 4.85s/it]
|
||
Fetching 2 files: 100%|██████████| 2/2 [00:04<00:00, 2.43s/it]
|
||
[INFO|modeling_utils.py:779] 2026-04-29 15:44:22,425 >> Will use dtype=torch.bfloat16 as defined in model's config object
|
||
|
||
Fetching 2 files: 50%|█████ | 1/2 [00:04<00:04, 4.02s/it]
|
||
Fetching 2 files: 100%|██████████| 2/2 [00:04<00:00, 2.01s/it]
|
||
[INFO|configuration_utils.py:1014] 2026-04-29 15:44:22,426 >> Generate config GenerationConfig {
|
||
"bos_token_id": 151643,
|
||
"eos_token_id": 151645,
|
||
"output_attentions": false,
|
||
"output_hidden_states": false,
|
||
"use_cache": false
|
||
}
|
||
|
||
|
||
Fetching 2 files: 50%|█████ | 1/2 [00:04<00:04, 4.02s/it]
|
||
Fetching 2 files: 100%|██████████| 2/2 [00:04<00:00, 2.01s/it]
|
||
|
||
Fetching 2 files: 50%|█████ | 1/2 [00:03<00:03, 3.98s/it]
|
||
Fetching 2 files: 100%|██████████| 2/2 [00:03<00:00, 1.99s/it]
|
||
|
||
Fetching 2 files: 50%|█████ | 1/2 [00:03<00:03, 3.98s/it]
|
||
Fetching 2 files: 100%|██████████| 2/2 [00:03<00:00, 1.99s/it]
|
||
|
||
Fetching 2 files: 50%|█████ | 1/2 [00:04<00:04, 4.07s/it]
|
||
Fetching 2 files: 100%|██████████| 2/2 [00:04<00:00, 2.04s/it]
|
||
|
||
Fetching 2 files: 50%|█████ | 1/2 [00:04<00:04, 4.17s/it]
|
||
Fetching 2 files: 100%|██████████| 2/2 [00:04<00:00, 2.08s/it]
|
||
|
||
Fetching 2 files: 50%|█████ | 1/2 [00:04<00:04, 4.12s/it]
|
||
Fetching 2 files: 100%|██████████| 2/2 [00:04<00:00, 2.06s/it]
|
||
|
||
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 13357.66it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 6721.64it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 13025.79it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 6523.02it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 11915.64it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 6105.25it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 12985.46it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 6442.86it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 8867.45it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 4777.11it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 11459.85it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 6159.04it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 12087.33it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 5949.37it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 0/311 [00:00<?, ?it/s]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 10538.45it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%| | 1/311 [00:00<00:00, 5729.92it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:25, 12.28it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:25, 12.28it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:25, 12.28it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:19, 15.56it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:19, 15.56it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:19, 15.56it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:19, 15.74it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:19, 15.74it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:19, 15.74it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:20, 15.33it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:23, 12.95it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:20, 15.33it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:20, 15.33it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:23, 12.95it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:23, 12.95it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:33, 9.20it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:33, 9.20it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:33, 9.20it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:23, 13.41it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:23, 13.41it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:23, 13.41it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:33, 9.20it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:33, 9.34it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:33, 9.34it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/311 [00:00<00:33, 9.34it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:32, 9.20it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:32, 9.34it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:31, 9.20it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:31, 9.34it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:30, 9.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:30, 9.34it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:29, 9.20it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:29, 9.34it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:00<00:28, 9.20it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:00<00:27, 9.20it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:00<00:27, 9.20it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:00<00:28, 9.34it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:00<00:27, 9.34it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:20, 15.33it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:20, 15.33it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:19, 15.74it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:19, 15.56it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:25, 12.28it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:19, 15.74it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:19, 15.56it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:25, 12.28it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.29it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.29it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.29it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:23, 12.95it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:23, 12.95it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:23, 13.29it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:23, 13.29it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:23, 13.20it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:23, 13.11it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:26, 11.41it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:24, 12.43it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:22, 13.20it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:00<00:01, 224.92it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:22, 13.29it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:00<00:01, 215.59it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:22, 13.11it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:25, 11.41it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:21, 13.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 215.59it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:23, 12.43it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:22, 13.41it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%| | 3/311 [00:00<00:22, 13.41it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 1%|▏ | 4/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 2%|▏ | 5/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:21, 13.29it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 2%|▏ | 6/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:00<00:00, 224.92it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:21, 13.11it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:20, 13.20it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:24, 11.41it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 2%|▏ | 7/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 3%|▎ | 8/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.k_norm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 3%|▎ | 9/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 3%|▎ | 10/311 [00:00<00:26, 11.58it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 4%|▎ | 11/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:20, 13.29it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 4%|▍ | 12/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 4%|▍ | 13/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▍ | 14/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:23, 11.41it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:23, 11.41it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 5%|▍ | 15/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:00<00:19, 13.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:00<00:22, 12.43it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:20, 13.11it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 5%|▌ | 16/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 5%|▌ | 17/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 133.40it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 6%|▌ | 18/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 6%|▌ | 19/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 6%|▋ | 20/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 7%|▋ | 21/311 [00:00<00:25, 11.58it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 22/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 7%|▋ | 23/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 8%|▊ | 24/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 8%|▊ | 25/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 8%|▊ | 26/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:00<00:00, 320.70it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:00<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:00<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:00<00:02, 123.52it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:00<00:01, 174.81it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:00<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:00<00:01, 133.40it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:00<00:02, 119.69it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:01<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 9%|▊ | 27/311 [00:01<00:24, 11.58it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:01<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:01<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:01<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:01<00:01, 137.25it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 28/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 29/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:00, 320.70it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 10%|▉ | 30/311 [00:01<00:11, 25.53it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:01<00:02, 119.69it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:01<00:01, 133.40it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:01<00:02, 123.52it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 10%|▉ | 31/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 10%|█ | 32/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 33/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:01<00:01, 174.81it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:00, 320.93it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 11%|█ | 34/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 11%|█▏ | 35/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:01<00:06, 41.63it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 36/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 37/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 38/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 13%|█▎ | 39/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:01<00:04, 57.23it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 40/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 13%|█▎ | 41/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▎ | 42/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 14%|█▍ | 43/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 45.34it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▍ | 44/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 14%|█▍ | 45/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 15%|█▍ | 46/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 15%|█▌ | 47/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 15%|█▌ | 48/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 16%|█▌ | 49/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:01<00:05, 44.30it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 16%|█▌ | 50/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▋ | 51/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 17%|█▋ | 52/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 17%|█▋ | 53/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 17%|█▋ | 54/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 18%|█▊ | 55/311 [00:01<00:10, 25.53it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 18%|█▊ | 56/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 57/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:01<00:05, 42.74it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 19%|█▊ | 58/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 19%|█▉ | 59/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 19%|█▉ | 60/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 61/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 20%|█▉ | 62/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:01, 101.38it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:01<00:05, 41.63it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:01<00:04, 44.30it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:01<00:04, 42.74it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:01<00:03, 57.23it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:01, 101.70it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 20%|██ | 63/311 [00:01<00:09, 25.53it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:01<00:04, 45.34it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 64/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 21%|██ | 65/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:01, 122.32it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 21%|██ | 66/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 22%|██▏ | 67/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 68/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 22%|██▏ | 69/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 23%|██▎ | 70/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 93.46it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 71/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 72/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 23%|██▎ | 73/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:02, 89.49it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██▍ | 74/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 24%|██▍ | 75/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 90.55it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 24%|██▍ | 76/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▍ | 77/311 [00:01<00:04, 58.32it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 25%|██▌ | 78/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 79/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 26%|██▌ | 80/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 26%|██▌ | 81/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 26%|██▋ | 82/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 27%|██▋ | 83/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 27%|██▋ | 84/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:02, 85.35it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 27%|██▋ | 85/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 28%|██▊ | 86/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 28%|██▊ | 87/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 28%|██▊ | 88/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 29%|██▊ | 89/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 29%|██▉ | 90/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 29%|██▉ | 91/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:02, 81.71it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 30%|██▉ | 92/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 81.71it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 81.71it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 85.35it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 138.01it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 93.46it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 30%|██▉ | 93/311 [00:01<00:03, 58.32it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 90.55it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 81.71it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 122.32it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 81.71it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 30%|███ | 94/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 89.49it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 31%|███ | 95/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 31%|███ | 96/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 31%|███ | 97/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 32%|███▏ | 98/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 99/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 143.12it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 32%|███▏ | 100/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 32%|███▏ | 101/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 139.05it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 138.34it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 102/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 33%|███▎ | 103/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 33%|███▎ | 104/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 105/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 34%|███▍ | 106/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:01, 134.31it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 34%|███▍ | 107/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 108/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 35%|███▌ | 109/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 35%|███▌ | 110/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:01, 135.41it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 36%|███▌ | 111/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 36%|███▌ | 112/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 113/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 37%|███▋ | 114/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 115/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 37%|███▋ | 116/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 38%|███▊ | 117/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 38%|███▊ | 118/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 38%|███▊ | 119/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 39%|███▊ | 120/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 39%|███▉ | 121/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 192.34it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 192.34it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 39%|███▉ | 122/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 192.34it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 195.96it/s, Materializing param=model.norm.weight]
|
||
|
||
Loading weights: 40%|███▉ | 123/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 40%|███▉ | 124/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 40%|████ | 125/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 139.05it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 41%|████ | 126/311 [00:01<00:02, 87.65it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 143.12it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 135.41it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 161.52it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 41%|████ | 127/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 138.34it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 41%|████ | 128/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 134.31it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 41%|████▏ | 129/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 42%|████▏ | 130/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 42%|████▏ | 131/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 42%|████▏ | 132/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 43%|████▎ | 133/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 43%|████▎ | 134/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 135/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 44%|████▎ | 136/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 44%|████▍ | 137/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 138/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 45%|████▍ | 139/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 45%|████▌ | 140/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 45%|████▌ | 141/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 142/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 46%|████▌ | 143/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 46%|████▋ | 144/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 47%|████▋ | 145/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 47%|████▋ | 146/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 212.69it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 212.69it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 212.69it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 147/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 184.81it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 48%|████▊ | 148/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 48%|████▊ | 149/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 48%|████▊ | 150/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 49%|████▊ | 151/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 49%|████▉ | 152/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 49%|████▉ | 153/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 50%|████▉ | 154/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 50%|████▉ | 155/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 50%|█████ | 156/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 50%|█████ | 157/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 51%|█████ | 158/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 51%|█████ | 159/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 51%|█████▏ | 160/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 52%|█████▏ | 161/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 52%|█████▏ | 162/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 52%|█████▏ | 163/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 53%|█████▎ | 164/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 165/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 53%|█████▎ | 166/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 54%|█████▎ | 167/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 168/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 54%|█████▍ | 169/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 188.58it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 190.03it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 179.36it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 181.86it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 55%|█████▍ | 170/311 [00:01<00:01, 121.34it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.o_proj.weight][WARNING|modeling_utils.py:2496] 2026-04-29 15:44:24,236 >> The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 184.15it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 55%|█████▍ | 171/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▌ | 172/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 173/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 56%|█████▌ | 174/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 56%|█████▋ | 175/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 57%|█████▋ | 176/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 177/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 57%|█████▋ | 178/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 179/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 180/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 58%|█████▊ | 181/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 59%|█████▊ | 182/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 59%|█████▉ | 183/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 59%|█████▉ | 184/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 59%|█████▉ | 185/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 60%|█████▉ | 186/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 60%|██████ | 187/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 60%|██████ | 188/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 61%|██████ | 189/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 61%|██████ | 190/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 61%|██████▏ | 191/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 192/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 261.82it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 261.82it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 261.82it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 62%|██████▏ | 193/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 175.99it/s, Materializing param=model.norm.weight]
|
||
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 62%|██████▏ | 194/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 195/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 255.68it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 196/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 255.68it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 255.68it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 170.80it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
|
||
Loading weights: 63%|██████▎ | 197/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 64%|██████▎ | 198/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 64%|██████▍ | 199/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 64%|██████▍ | 200/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 247.65it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 247.65it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 247.65it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 65%|██████▍ | 201/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 175.84it/s, Materializing param=model.norm.weight]
|
||
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 65%|██████▍ | 202/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 203/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 250.91it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 66%|██████▌ | 204/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 250.91it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 250.91it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 205/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 175.36it/s, Materializing param=model.norm.weight]
|
||
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 66%|██████▌ | 206/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 67%|██████▋ | 207/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 67%|██████▋ | 208/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 67%|██████▋ | 209/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 68%|██████▊ | 210/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 68%|██████▊ | 211/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 245.34it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 245.34it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 245.34it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 68%|██████▊ | 212/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 173.28it/s, Materializing param=model.norm.weight]
|
||
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 68%|██████▊ | 213/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 69%|██████▉ | 214/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 69%|██████▉ | 215/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 216/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 70%|██████▉ | 217/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 70%|███████ | 218/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 70%|███████ | 219/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 71%|███████ | 220/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 71%|███████ | 221/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 71%|███████▏ | 222/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 223/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 72%|███████▏ | 224/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 72%|███████▏ | 225/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 73%|███████▎ | 226/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 227/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 73%|███████▎ | 228/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 74%|███████▎ | 229/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 74%|███████▍ | 230/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 74%|███████▍ | 231/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 75%|███████▍ | 232/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 75%|███████▍ | 233/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 75%|███████▌ | 234/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 235/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 76%|███████▌ | 236/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 76%|███████▌ | 237/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 238/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 77%|███████▋ | 239/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 77%|███████▋ | 240/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 77%|███████▋ | 241/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 78%|███████▊ | 242/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 78%|███████▊ | 243/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 78%|███████▊ | 244/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 245/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 79%|███████▉ | 246/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 79%|███████▉ | 247/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 80%|███████▉ | 248/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 80%|████████ | 249/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 80%|████████ | 250/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 81%|████████ | 251/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 81%|████████ | 252/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 81%|████████▏ | 253/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 82%|████████▏ | 254/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 82%|████████▏ | 255/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 256/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 83%|████████▎ | 257/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 83%|████████▎ | 258/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 83%|████████▎ | 259/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 84%|████████▎ | 260/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 84%|████████▍ | 261/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 84%|████████▍ | 262/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████▍ | 263/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 85%|████████▍ | 264/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 85%|████████▌ | 265/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 86%|████████▌ | 266/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 267/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 86%|████████▌ | 268/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 86%|████████▋ | 269/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 87%|████████▋ | 270/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 271/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 87%|████████▋ | 272/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 88%|████████▊ | 273/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 88%|████████▊ | 274/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 88%|████████▊ | 275/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 89%|████████▊ | 276/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 89%|████████▉ | 277/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 278/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 90%|████████▉ | 279/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 90%|█████████ | 280/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 90%|█████████ | 281/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 91%|█████████ | 282/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 91%|█████████ | 283/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 91%|█████████▏| 284/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 285/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 92%|█████████▏| 286/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 92%|█████████▏| 287/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 93%|█████████▎| 288/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 289/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 93%|█████████▎| 290/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 94%|█████████▎| 291/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 94%|█████████▍| 292/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 94%|█████████▍| 293/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 95%|█████████▍| 294/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 95%|█████████▍| 295/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 95%|█████████▌| 296/311 [00:01<00:00, 172.13it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 95%|█████████▌| 297/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 96%|█████████▌| 298/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 96%|█████████▌| 299/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 96%|█████████▋| 300/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 301/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 97%|█████████▋| 302/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 97%|█████████▋| 303/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 304/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 98%|█████████▊| 305/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 98%|█████████▊| 306/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▊| 307/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 99%|█████████▉| 308/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 99%|█████████▉| 309/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 100%|█████████▉| 310/311 [00:01<00:00, 374.43it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 374.43it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 374.43it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 311/311 [00:01<00:00, 167.36it/s, Materializing param=model.norm.weight]
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
[INFO|configuration_utils.py:967] 2026-04-29 15:44:24,477 >> loading configuration file generation_config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen3-1.7B/snapshots/70d244cc86ccca08cf5af4e1e306ecf908b1ad5e/generation_config.json
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
[INFO|configuration_utils.py:1014] 2026-04-29 15:44:24,477 >> Generate config GenerationConfig {
|
||
"bos_token_id": 151643,
|
||
"do_sample": true,
|
||
"eos_token_id": [
|
||
151645,
|
||
151643
|
||
],
|
||
"pad_token_id": 151643,
|
||
"temperature": 0.6,
|
||
"top_k": 20,
|
||
"top_p": 0.95
|
||
}
|
||
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
[INFO|dynamic_module_utils.py:406] 2026-04-29 15:44:24,593 >> Could not locate the custom_generate/generate.py inside Qwen/Qwen3-1.7B.
|
||
[INFO|2026-04-29 15:44:24] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
|
||
[INFO|2026-04-29 15:44:24] llamafactory.model.model_utils.attention:144 >> Using torch SDPA for faster training and inference.
|
||
[INFO|2026-04-29 15:44:24] llamafactory.model.adapter:144 >> Upcasting trainable params to float32.
|
||
[INFO|2026-04-29 15:44:24] llamafactory.model.adapter:144 >> Fine-tuning method: Full
|
||
[INFO|2026-04-29 15:44:24] llamafactory.model.loader:144 >> trainable params: 2,031,739,904 || all params: 2,031,739,904 || trainable%: 100.0000
|
||
[WARNING|trainer_utils.py:1234] 2026-04-29 15:44:24,831 >> The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
|
||
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
|
||
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
|
||
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
|
||
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
|
||
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
|
||
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
|
||
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Comm config Blocking set to 1
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Initialized NET plugin Socket
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Assigned NET plugin Socket to comm
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Using network Socket
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ncclCommSplit comm 0x56028de3b5a0 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 parent 0x5602809d0750 splitCount 1 color 1266629538 key 2- Init START
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ncclCommSplit comm 0x558d03437a10 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 parent 0x558cf7486a10 splitCount 1 color 1266629538 key 1- Init START
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ncclCommSplit comm 0x55e3778f3800 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 parent 0x55e36d363080 splitCount 1 color 1266629538 key 7- Init START
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ncclCommSplit comm 0x55c02c235cf0 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 parent 0x55c0201d0a60 splitCount 1 color 1266629538 key 5- Init START
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ncclCommSplit comm 0x5612df7ca6e0 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 parent 0x5612d38dcbf0 splitCount 1 color 1266629538 key 3- Init START
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ncclCommSplit comm 0x5647135d02c0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 parent 0x5647078795e0 splitCount 1 color 1266629538 key 4- Init START
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ncclCommSplit comm 0x560722415b60 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 parent 0x56071a5c6320 splitCount 1 color 1266629538 key 6- Init START
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ncclCommSplit comm 0x55ffc26dd150 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 parent 0x55ffbe0f7c80 splitCount 1 color 1266629538 key 0- Init START
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 6 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NVLS multicast support is not available on dev 6 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 3 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NVLS multicast support is not available on dev 3 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 2 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NVLS multicast support is not available on dev 2 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 1 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NVLS multicast support is not available on dev 1 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 5 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NVLS multicast support is not available on dev 5 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 7 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NVLS multicast support is not available on dev 7 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 4 is 24-47,72-95. (GPU affinity = 24-47,72-95 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NVLS multicast support is not available on dev 4 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ncclTopoGetCpuAffinity: Affinity for GPU 0 is 0-23,48-71. (GPU affinity = 0-23,48-71 ; CPU affinity = 0-95).
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NVLS multicast support is not available on dev 0 (NVLS_NCHANNELS 0)
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO comm 0x558d03437a10 rank 1 nRanks 8 nNodes 1 localRanks 8 localRank 1 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO comm 0x55ffc26dd150 rank 0 nRanks 8 nNodes 1 localRanks 8 localRank 0 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO comm 0x55e3778f3800 rank 7 nRanks 8 nNodes 1 localRanks 8 localRank 7 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO comm 0x560722415b60 rank 6 nRanks 8 nNodes 1 localRanks 8 localRank 6 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO comm 0x55c02c235cf0 rank 5 nRanks 8 nNodes 1 localRanks 8 localRank 5 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO comm 0x56028de3b5a0 rank 2 nRanks 8 nNodes 1 localRanks 8 localRank 2 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO comm 0x5647135d02c0 rank 4 nRanks 8 nNodes 1 localRanks 8 localRank 4 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 00/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO comm 0x5612df7ca6e0 rank 3 nRanks 8 nNodes 1 localRanks 8 localRank 3 MNNVL 0
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 01/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 02/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 03/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 04/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 05/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 06/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 07/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 08/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 09/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 10/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 11/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 12/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 13/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 14/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 15/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 16/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 17/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 18/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 19/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 20/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 21/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 22/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 23/24 : 0 1 2 3 4 5 6 7
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO P2P Chunksize set to 524288
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470947:2474553 [0] NCCL INFO [Proxy Service] Device 0 CPU core 60
|
||
ywang29-p4d-debug-worker-0:2470947:2474554 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 15
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470948:2474555 [0] NCCL INFO [Proxy Service] Device 1 CPU core 69
|
||
ywang29-p4d-debug-worker-0:2470948:2474556 [0] NCCL INFO [Proxy Service UDS] Device 1 CPU core 16
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470952:2474557 [0] NCCL INFO [Proxy Service] Device 4 CPU core 35
|
||
ywang29-p4d-debug-worker-0:2470952:2474558 [0] NCCL INFO [Proxy Service UDS] Device 4 CPU core 44
|
||
ywang29-p4d-debug-worker-0:2470951:2474559 [0] NCCL INFO [Proxy Service] Device 3 CPU core 58
|
||
ywang29-p4d-debug-worker-0:2470951:2474560 [0] NCCL INFO [Proxy Service UDS] Device 3 CPU core 14
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470955:2474561 [0] NCCL INFO [Proxy Service] Device 7 CPU core 77
|
||
ywang29-p4d-debug-worker-0:2470955:2474562 [0] NCCL INFO [Proxy Service UDS] Device 7 CPU core 79
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470953:2474563 [0] NCCL INFO [Proxy Service] Device 5 CPU core 32
|
||
ywang29-p4d-debug-worker-0:2470953:2474564 [0] NCCL INFO [Proxy Service UDS] Device 5 CPU core 46
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470954:2474565 [0] NCCL INFO [Proxy Service] Device 6 CPU core 93
|
||
ywang29-p4d-debug-worker-0:2470954:2474566 [0] NCCL INFO [Proxy Service UDS] Device 6 CPU core 88
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0 isAllCudaP2p 1
|
||
ywang29-p4d-debug-worker-0:2470949:2474567 [0] NCCL INFO [Proxy Service] Device 2 CPU core 1
|
||
ywang29-p4d-debug-worker-0:2470949:2474568 [0] NCCL INFO [Proxy Service UDS] Device 2 CPU core 53
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO CC Off, workFifoBytes 1048576
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ncclCommSplit comm 0x5647135d02c0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 parent 0x5647078795e0 splitCount 1 color 1266629538 key 4 - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO ncclCommSplit comm 0x56028de3b5a0 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 parent 0x5602809d0750 splitCount 1 color 1266629538 key 2 - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO ncclCommSplit comm 0x55ffc26dd150 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 parent 0x55ffbe0f7c80 splitCount 1 color 1266629538 key 0 - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Init timings - ncclCommSplit: rank 4 nranks 8 total 0.18 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.07, rest 0.06)
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Init timings - ncclCommSplit: rank 0 nranks 8 total 0.24 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.00, topo 0.05, graphs 0.00, connections 0.06, rest 0.13)
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Init timings - ncclCommSplit: rank 2 nranks 8 total 0.15 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.06, rest 0.03)
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO ncclCommSplit comm 0x560722415b60 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 parent 0x56071a5c6320 splitCount 1 color 1266629538 key 6 - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Init timings - ncclCommSplit: rank 6 nranks 8 total 0.25 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.02, topo 0.04, graphs 0.00, connections 0.07, rest 0.13)
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ncclCommSplit comm 0x55c02c235cf0 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 parent 0x55c0201d0a60 splitCount 1 color 1266629538 key 5 - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO ncclCommSplit comm 0x558d03437a10 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 parent 0x558cf7486a10 splitCount 1 color 1266629538 key 1 - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ncclCommSplit comm 0x5612df7ca6e0 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 parent 0x5612d38dcbf0 splitCount 1 color 1266629538 key 3 - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Init timings - ncclCommSplit: rank 5 nranks 8 total 0.28 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.07, rest 0.16)
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Init timings - ncclCommSplit: rank 1 nranks 8 total 0.26 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.07, rest 0.14)
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO ncclCommSplit comm 0x55e3778f3800 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 parent 0x55e36d363080 splitCount 1 color 1266629538 key 7 - Init COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Init timings - ncclCommSplit: rank 3 nranks 8 total 0.16 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.07, rest 0.03)
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Init timings - ncclCommSplit: rank 7 nranks 8 total 0.18 (kernels 0.00, alloc 0.00, bootstrap 0.00, allgathers 0.01, topo 0.04, graphs 0.00, connections 0.07, rest 0.05)
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 00/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 01/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 02/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 03/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 04/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 00/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 05/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 06/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 01/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 07/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 02/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 03/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 08/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 04/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 05/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 09/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 10/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 06/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 07/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 11/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 08/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 09/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 12/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 13/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 14/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 10/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 15/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 11/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 16/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 12/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 16/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 17/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 18/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 17/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 13/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 18/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 14/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 19/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 20/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 15/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 16/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 19/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 16/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 20/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 16/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 17/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 21/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 16/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 17/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 18/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 21/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 22/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 22/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 18/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 17/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Channel 23/0 : 5[5] -> 6[6] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 19/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Channel 23/0 : 6[6] -> 7[7] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 20/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 17/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 18/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 16/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 16/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 19/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 19/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 18/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 21/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 19/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 20/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 20/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 21/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 17/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 20/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 22/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 21/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 17/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 18/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 22/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 18/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 21/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Channel 23/0 : 2[2] -> 3[3] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 22/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 19/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Channel 23/0 : 4[4] -> 5[5] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 19/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 22/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Channel 23/0 : 1[1] -> 2[2] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 20/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 20/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Channel 23/0 : 7[7] -> 0[0] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 21/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 21/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 22/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 22/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Channel 23/0 : 3[3] -> 4[4] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Channel 23/0 : 0[0] -> 1[1] via P2P/CUMEM/read
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470948:2470948 [1] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470955:2470955 [7] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
ywang29-p4d-debug-worker-0:2470947:2470947 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
|
||
Before initializing optimizer states
|
||
MA 4.73 GB Max_MA 4.73 GB CA 4.77 GB Max_CA 5 GB
|
||
CPU Virtual Memory: used = 70.92 GB, percent = 6.3%
|
||
After initializing optimizer states
|
||
MA 4.73 GB Max_MA 5.68 GB CA 5.71 GB Max_CA 6 GB
|
||
CPU Virtual Memory: used = 70.92 GB, percent = 6.3%
|
||
After initializing ZeRO optimizer
|
||
MA 4.73 GB Max_MA 4.73 GB CA 5.71 GB Max_CA 6 GB
|
||
CPU Virtual Memory: used = 70.92 GB, percent = 6.3%
|
||
[INFO|trainer.py:1587] 2026-04-29 15:44:34,254 >> ***** Running training *****
|
||
[INFO|trainer.py:1588] 2026-04-29 15:44:34,254 >> Num examples = 60,000
|
||
[INFO|trainer.py:1589] 2026-04-29 15:44:34,254 >> Num Epochs = 2
|
||
[INFO|trainer.py:1590] 2026-04-29 15:44:34,254 >> Instantaneous batch size per device = 2
|
||
[INFO|trainer.py:1593] 2026-04-29 15:44:34,254 >> Total train batch size (w. parallel, distributed & accumulation) = 128
|
||
[INFO|trainer.py:1594] 2026-04-29 15:44:34,254 >> Gradient Accumulation steps = 8
|
||
[INFO|trainer.py:1595] 2026-04-29 15:44:34,254 >> Total optimization steps = 938
|
||
[INFO|trainer.py:1596] 2026-04-29 15:44:34,255 >> Number of trainable parameters = 2,031,739,904
|
||
wandb: [wandb.login()] Loaded credentials for https://api.wandb.ai from WANDB_API_KEY.
|
||
wandb: Currently logged in as: kkhya (maskmoe) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
|
||
wandb: setting up run jl6u15v5
|
||
wandb: Tracking run with wandb version 0.25.1
|
||
wandb: Run data is saved locally in /nfs/ywang29/lm-factory/wandb/run-20260429_154434-jl6u15v5
|
||
wandb: Run `wandb offline` to turn off syncing.
|
||
wandb: Syncing run qwen3_1.7b_math_fft
|
||
wandb: ⭐️ View project at https://wandb.ai/maskmoe/MFT-LM
|
||
wandb: 🚀 View run at https://wandb.ai/maskmoe/MFT-LM/runs/jl6u15v5
|
||
|
||
0%| | 0/938 [00:00<?, ?it/s]
|
||
0%| | 1/938 [00:05<1:21:11, 5.20s/it]
|
||
0%| | 2/938 [00:08<1:08:10, 4.37s/it]
|
||
0%| | 3/938 [00:13<1:08:24, 4.39s/it]
|
||
0%| | 4/938 [00:17<1:09:08, 4.44s/it]
|
||
1%| | 5/938 [00:22<1:07:42, 4.35s/it]
|
||
1%| | 6/938 [00:26<1:05:38, 4.23s/it]
|
||
1%| | 7/938 [00:30<1:08:49, 4.44s/it]
|
||
1%| | 8/938 [00:35<1:08:07, 4.40s/it]
|
||
1%| | 9/938 [00:39<1:05:08, 4.21s/it]
|
||
1%| | 10/938 [00:43<1:07:18, 4.35s/it]
|
||
|
||
{'loss': '0.6842', 'grad_norm': '11.14', 'learning_rate': '9.574e-07', 'epoch': '0.02133'}
|
||
|
||
1%| | 10/938 [00:43<1:07:18, 4.35s/it]
|
||
1%| | 11/938 [00:47<1:05:24, 4.23s/it]
|
||
1%|▏ | 12/938 [00:51<1:03:37, 4.12s/it]
|
||
1%|▏ | 13/938 [00:55<1:01:48, 4.01s/it]
|
||
1%|▏ | 14/938 [00:59<1:02:34, 4.06s/it]
|
||
2%|▏ | 15/938 [01:03<1:02:42, 4.08s/it]
|
||
2%|▏ | 16/938 [01:07<1:03:52, 4.16s/it]
|
||
2%|▏ | 17/938 [01:12<1:03:46, 4.15s/it]
|
||
2%|▏ | 18/938 [01:16<1:02:46, 4.09s/it]
|
||
2%|▏ | 19/938 [01:19<1:01:55, 4.04s/it]
|
||
2%|▏ | 20/938 [01:23<1:00:13, 3.94s/it]
|
||
|
||
{'loss': '0.6097', 'grad_norm': '4.558', 'learning_rate': '2.021e-06', 'epoch': '0.04267'}
|
||
|
||
2%|▏ | 20/938 [01:23<1:00:13, 3.94s/it]
|
||
2%|▏ | 21/938 [01:28<1:02:04, 4.06s/it]
|
||
2%|▏ | 22/938 [01:32<1:01:45, 4.05s/it]
|
||
2%|▏ | 23/938 [01:35<1:00:57, 4.00s/it]
|
||
3%|▎ | 24/938 [01:39<59:52, 3.93s/it]
|
||
3%|▎ | 25/938 [01:43<58:15, 3.83s/it]
|
||
3%|▎ | 26/938 [01:47<1:00:22, 3.97s/it]
|
||
3%|▎ | 27/938 [01:51<1:00:35, 3.99s/it]
|
||
3%|▎ | 28/938 [01:55<59:26, 3.92s/it]
|
||
3%|▎ | 29/938 [01:59<59:03, 3.90s/it]
|
||
3%|▎ | 30/938 [02:03<58:43, 3.88s/it]
|
||
|
||
{'loss': '0.4652', 'grad_norm': '1.498', 'learning_rate': '3.085e-06', 'epoch': '0.064'}
|
||
|
||
3%|▎ | 30/938 [02:03<58:43, 3.88s/it]
|
||
3%|▎ | 31/938 [02:07<59:38, 3.95s/it]
|
||
3%|▎ | 32/938 [02:10<58:23, 3.87s/it]
|
||
4%|▎ | 33/938 [02:14<56:48, 3.77s/it]
|
||
4%|▎ | 34/938 [02:18<57:58, 3.85s/it]
|
||
4%|▎ | 35/938 [02:22<57:50, 3.84s/it]
|
||
4%|▍ | 36/938 [02:26<59:02, 3.93s/it]
|
||
4%|▍ | 37/938 [02:30<1:01:54, 4.12s/it]
|
||
4%|▍ | 38/938 [02:34<1:00:21, 4.02s/it]
|
||
4%|▍ | 39/938 [02:38<59:26, 3.97s/it]
|
||
4%|▍ | 40/938 [02:43<1:05:36, 4.38s/it]
|
||
|
||
{'loss': '0.3869', 'grad_norm': '0.7575', 'learning_rate': '4.149e-06', 'epoch': '0.08533'}
|
||
|
||
4%|▍ | 40/938 [02:43<1:05:36, 4.38s/it]
|
||
4%|▍ | 41/938 [02:48<1:06:29, 4.45s/it]
|
||
4%|▍ | 42/938 [02:52<1:06:15, 4.44s/it]
|
||
5%|▍ | 43/938 [02:58<1:09:33, 4.66s/it]
|
||
5%|▍ | 44/938 [03:01<1:05:25, 4.39s/it]
|
||
5%|▍ | 45/938 [03:05<1:03:20, 4.26s/it]
|
||
5%|▍ | 46/938 [03:09<1:01:24, 4.13s/it]
|
||
5%|▌ | 47/938 [03:13<1:01:17, 4.13s/it]
|
||
5%|▌ | 48/938 [03:18<1:02:16, 4.20s/it]
|
||
5%|▌ | 49/938 [03:22<1:04:53, 4.38s/it]
|
||
5%|▌ | 50/938 [03:26<1:02:41, 4.24s/it]
|
||
|
||
{'loss': '0.3463', 'grad_norm': '0.6512', 'learning_rate': '5.213e-06', 'epoch': '0.1067'}
|
||
|
||
5%|▌ | 50/938 [03:26<1:02:41, 4.24s/it]
|
||
5%|▌ | 51/938 [03:30<1:01:38, 4.17s/it]
|
||
6%|▌ | 52/938 [03:34<59:15, 4.01s/it]
|
||
6%|▌ | 53/938 [03:38<59:06, 4.01s/it]
|
||
6%|▌ | 54/938 [03:43<1:02:10, 4.22s/it]
|
||
6%|▌ | 55/938 [03:47<1:03:00, 4.28s/it]
|
||
6%|▌ | 56/938 [03:51<1:00:44, 4.13s/it]
|
||
6%|▌ | 57/938 [03:55<58:38, 3.99s/it]
|
||
6%|▌ | 58/938 [03:59<59:39, 4.07s/it]
|
||
6%|▋ | 59/938 [04:03<58:18, 3.98s/it]
|
||
6%|▋ | 60/938 [04:07<59:00, 4.03s/it]
|
||
|
||
{'loss': '0.3305', 'grad_norm': '0.5266', 'learning_rate': '6.277e-06', 'epoch': '0.128'}
|
||
|
||
6%|▋ | 60/938 [04:07<59:00, 4.03s/it]
|
||
7%|▋ | 61/938 [04:11<58:15, 3.99s/it]
|
||
7%|▋ | 62/938 [04:15<1:00:41, 4.16s/it]
|
||
7%|▋ | 63/938 [04:19<1:00:37, 4.16s/it]
|
||
7%|▋ | 64/938 [04:24<1:00:37, 4.16s/it]
|
||
7%|▋ | 65/938 [04:28<1:02:16, 4.28s/it]
|
||
7%|▋ | 66/938 [04:32<1:01:41, 4.25s/it]
|
||
7%|▋ | 67/938 [04:36<1:00:00, 4.13s/it]
|
||
7%|▋ | 68/938 [04:40<59:03, 4.07s/it]
|
||
7%|▋ | 69/938 [04:44<58:54, 4.07s/it]
|
||
7%|▋ | 70/938 [04:49<1:00:20, 4.17s/it]
|
||
|
||
{'loss': '0.3322', 'grad_norm': '0.5385', 'learning_rate': '7.34e-06', 'epoch': '0.1493'}
|
||
|
||
7%|▋ | 70/938 [04:49<1:00:20, 4.17s/it]
|
||
8%|▊ | 71/938 [04:52<59:03, 4.09s/it]
|
||
8%|▊ | 72/938 [04:56<57:40, 4.00s/it]
|
||
8%|▊ | 73/938 [05:00<57:24, 3.98s/it]
|
||
8%|▊ | 74/938 [05:04<57:14, 3.97s/it]
|
||
8%|▊ | 75/938 [05:09<59:06, 4.11s/it]
|
||
8%|▊ | 76/938 [05:13<59:03, 4.11s/it]
|
||
8%|▊ | 77/938 [05:17<59:24, 4.14s/it]
|
||
8%|▊ | 78/938 [05:22<1:04:00, 4.47s/it]
|
||
8%|▊ | 79/938 [05:26<1:02:20, 4.35s/it]
|
||
9%|▊ | 80/938 [05:31<1:02:35, 4.38s/it]
|
||
|
||
{'loss': '0.3166', 'grad_norm': '0.5109', 'learning_rate': '8.404e-06', 'epoch': '0.1707'}
|
||
|
||
9%|▊ | 80/938 [05:31<1:02:35, 4.38s/it]
|
||
9%|▊ | 81/938 [05:35<1:01:55, 4.34s/it]
|
||
9%|▊ | 82/938 [05:38<58:39, 4.11s/it]
|
||
9%|▉ | 83/938 [05:42<57:24, 4.03s/it]
|
||
9%|▉ | 84/938 [05:46<55:14, 3.88s/it]
|
||
9%|▉ | 85/938 [05:50<55:40, 3.92s/it]
|
||
9%|▉ | 86/938 [05:54<55:47, 3.93s/it]
|
||
9%|▉ | 87/938 [05:58<55:28, 3.91s/it]
|
||
9%|▉ | 88/938 [06:02<56:28, 3.99s/it]
|
||
9%|▉ | 89/938 [06:06<58:39, 4.14s/it]
|
||
10%|▉ | 90/938 [06:11<1:01:19, 4.34s/it]
|
||
|
||
{'loss': '0.3102', 'grad_norm': '0.4772', 'learning_rate': '9.468e-06', 'epoch': '0.192'}
|
||
|
||
10%|▉ | 90/938 [06:11<1:01:19, 4.34s/it]
|
||
10%|▉ | 91/938 [06:15<58:30, 4.14s/it]
|
||
10%|▉ | 92/938 [06:19<1:00:39, 4.30s/it]
|
||
10%|▉ | 93/938 [06:23<59:21, 4.21s/it]
|
||
10%|█ | 94/938 [06:28<1:00:36, 4.31s/it]
|
||
10%|█ | 95/938 [06:32<59:39, 4.25s/it]
|
||
10%|█ | 96/938 [06:36<58:20, 4.16s/it]
|
||
10%|█ | 97/938 [06:40<56:49, 4.05s/it]
|
||
10%|█ | 98/938 [06:44<56:26, 4.03s/it]
|
||
11%|█ | 99/938 [06:48<55:39, 3.98s/it]
|
||
11%|█ | 100/938 [06:52<55:53, 4.00s/it]
|
||
|
||
{'loss': '0.3156', 'grad_norm': '0.4976', 'learning_rate': '9.999e-06', 'epoch': '0.2133'}
|
||
|
||
11%|█ | 100/938 [06:52<55:53, 4.00s/it]
|
||
11%|█ | 101/938 [06:56<56:11, 4.03s/it]
|
||
11%|█ | 102/938 [07:01<58:53, 4.23s/it]
|
||
11%|█ | 103/938 [07:04<56:49, 4.08s/it]
|
||
11%|█ | 104/938 [07:09<59:32, 4.28s/it]
|
||
11%|█ | 105/938 [07:13<58:35, 4.22s/it]
|
||
11%|█▏ | 106/938 [07:17<55:53, 4.03s/it]
|
||
11%|█▏ | 107/938 [07:21<57:11, 4.13s/it]
|
||
12%|█▏ | 108/938 [07:25<56:40, 4.10s/it]
|
||
12%|█▏ | 109/938 [07:29<56:31, 4.09s/it]
|
||
12%|█▏ | 110/938 [07:33<57:18, 4.15s/it]
|
||
|
||
{'loss': '0.3121', 'grad_norm': '0.4746', 'learning_rate': '9.992e-06', 'epoch': '0.2347'}
|
||
|
||
12%|█▏ | 110/938 [07:33<57:18, 4.15s/it]
|
||
12%|█▏ | 111/938 [07:37<55:17, 4.01s/it]
|
||
12%|█▏ | 112/938 [07:42<56:44, 4.12s/it]
|
||
12%|█▏ | 113/938 [07:46<56:24, 4.10s/it]
|
||
12%|█▏ | 114/938 [07:50<57:57, 4.22s/it]
|
||
12%|█▏ | 115/938 [07:54<56:43, 4.14s/it]
|
||
12%|█▏ | 116/938 [07:58<56:41, 4.14s/it]
|
||
12%|█▏ | 117/938 [08:02<54:10, 3.96s/it]
|
||
13%|█▎ | 118/938 [08:06<55:54, 4.09s/it]
|
||
13%|█▎ | 119/938 [08:10<53:57, 3.95s/it]
|
||
13%|█▎ | 120/938 [08:14<56:14, 4.12s/it]
|
||
|
||
{'loss': '0.3095', 'grad_norm': '0.4676', 'learning_rate': '9.978e-06', 'epoch': '0.256'}
|
||
|
||
13%|█▎ | 120/938 [08:14<56:14, 4.12s/it]
|
||
13%|█▎ | 121/938 [08:18<53:40, 3.94s/it]
|
||
13%|█▎ | 122/938 [08:21<52:42, 3.88s/it]
|
||
13%|█▎ | 123/938 [08:25<51:39, 3.80s/it]
|
||
13%|█▎ | 124/938 [08:29<51:46, 3.82s/it]
|
||
13%|█▎ | 125/938 [08:33<53:41, 3.96s/it]
|
||
13%|█▎ | 126/938 [08:37<53:29, 3.95s/it]
|
||
14%|█▎ | 127/938 [08:42<54:50, 4.06s/it]
|
||
14%|█▎ | 128/938 [08:46<55:15, 4.09s/it]
|
||
14%|█▍ | 129/938 [08:50<55:45, 4.14s/it]
|
||
14%|█▍ | 130/938 [08:54<54:19, 4.03s/it]
|
||
|
||
{'loss': '0.3129', 'grad_norm': '0.4763', 'learning_rate': '9.958e-06', 'epoch': '0.2773'}
|
||
|
||
14%|█▍ | 130/938 [08:54<54:19, 4.03s/it]
|
||
14%|█▍ | 131/938 [08:58<54:33, 4.06s/it]
|
||
14%|█▍ | 132/938 [09:02<54:39, 4.07s/it]
|
||
14%|█▍ | 133/938 [09:07<57:39, 4.30s/it]
|
||
14%|█▍ | 134/938 [09:11<57:01, 4.26s/it]
|
||
14%|█▍ | 135/938 [09:15<55:31, 4.15s/it]
|
||
14%|█▍ | 136/938 [09:18<53:18, 3.99s/it]
|
||
15%|█▍ | 137/938 [09:22<53:33, 4.01s/it]
|
||
15%|█▍ | 138/938 [09:27<54:20, 4.08s/it]
|
||
15%|█▍ | 139/938 [09:31<54:33, 4.10s/it]
|
||
15%|█▍ | 140/938 [09:35<54:23, 4.09s/it]
|
||
|
||
{'loss': '0.3062', 'grad_norm': '0.493', 'learning_rate': '9.93e-06', 'epoch': '0.2987'}
|
||
|
||
15%|█▍ | 140/938 [09:35<54:23, 4.09s/it]
|
||
15%|█▌ | 141/938 [09:39<53:43, 4.05s/it]
|
||
15%|█▌ | 142/938 [09:43<52:45, 3.98s/it]
|
||
15%|█▌ | 143/938 [09:47<53:34, 4.04s/it]
|
||
15%|█▌ | 144/938 [09:52<55:49, 4.22s/it]
|
||
15%|█▌ | 145/938 [09:56<56:13, 4.25s/it]
|
||
16%|█▌ | 146/938 [09:59<53:01, 4.02s/it]
|
||
16%|█▌ | 147/938 [10:03<52:15, 3.96s/it]
|
||
16%|█▌ | 148/938 [10:07<49:59, 3.80s/it]
|
||
16%|█▌ | 149/938 [10:11<52:10, 3.97s/it]
|
||
16%|█▌ | 150/938 [10:15<52:49, 4.02s/it]
|
||
|
||
{'loss': '0.3069', 'grad_norm': '0.4954', 'learning_rate': '9.896e-06', 'epoch': '0.32'}
|
||
|
||
16%|█▌ | 150/938 [10:15<52:49, 4.02s/it]
|
||
16%|█▌ | 151/938 [10:19<53:40, 4.09s/it]
|
||
16%|█▌ | 152/938 [10:23<53:29, 4.08s/it]
|
||
16%|█▋ | 153/938 [10:27<52:35, 4.02s/it]
|
||
16%|█▋ | 154/938 [10:32<53:39, 4.11s/it]
|
||
17%|█▋ | 155/938 [10:35<52:25, 4.02s/it]
|
||
17%|█▋ | 156/938 [10:40<55:19, 4.24s/it]
|
||
17%|█▋ | 157/938 [10:44<53:47, 4.13s/it]
|
||
17%|█▋ | 158/938 [10:47<50:43, 3.90s/it]
|
||
17%|█▋ | 159/938 [10:51<50:34, 3.90s/it]
|
||
17%|█▋ | 160/938 [10:56<52:48, 4.07s/it]
|
||
|
||
{'loss': '0.3146', 'grad_norm': '0.485', 'learning_rate': '9.854e-06', 'epoch': '0.3413'}
|
||
|
||
17%|█▋ | 160/938 [10:56<52:48, 4.07s/it]
|
||
17%|█▋ | 161/938 [11:00<52:42, 4.07s/it]
|
||
17%|█▋ | 162/938 [11:04<52:27, 4.06s/it]
|
||
17%|█▋ | 163/938 [11:08<51:04, 3.95s/it]
|
||
17%|█▋ | 164/938 [11:11<50:47, 3.94s/it]
|
||
18%|█▊ | 165/938 [11:16<52:37, 4.08s/it]
|
||
18%|█▊ | 166/938 [11:20<54:12, 4.21s/it]
|
||
18%|█▊ | 167/938 [11:24<51:53, 4.04s/it]
|
||
18%|█▊ | 168/938 [11:28<51:12, 3.99s/it]
|
||
18%|█▊ | 169/938 [11:32<52:02, 4.06s/it]
|
||
18%|█▊ | 170/938 [11:36<52:02, 4.07s/it]
|
||
|
||
{'loss': '0.3096', 'grad_norm': '0.4601', 'learning_rate': '9.806e-06', 'epoch': '0.3627'}
|
||
|
||
18%|█▊ | 170/938 [11:36<52:02, 4.07s/it]
|
||
18%|█▊ | 171/938 [11:40<50:25, 3.95s/it]
|
||
18%|█▊ | 172/938 [11:45<53:44, 4.21s/it]
|
||
18%|█▊ | 173/938 [11:49<54:50, 4.30s/it]
|
||
19%|█▊ | 174/938 [11:54<55:16, 4.34s/it]
|
||
19%|█▊ | 175/938 [11:57<52:26, 4.12s/it]
|
||
19%|█▉ | 176/938 [12:02<53:49, 4.24s/it]
|
||
19%|█▉ | 177/938 [12:06<52:23, 4.13s/it]
|
||
19%|█▉ | 178/938 [12:10<51:44, 4.08s/it]
|
||
19%|█▉ | 179/938 [12:15<54:43, 4.33s/it]
|
||
19%|█▉ | 180/938 [12:19<55:28, 4.39s/it]
|
||
|
||
{'loss': '0.307', 'grad_norm': '0.44', 'learning_rate': '9.752e-06', 'epoch': '0.384'}
|
||
|
||
19%|█▉ | 180/938 [12:19<55:28, 4.39s/it]
|
||
19%|█▉ | 181/938 [12:23<55:01, 4.36s/it]
|
||
19%|█▉ | 182/938 [12:27<52:34, 4.17s/it]
|
||
20%|█▉ | 183/938 [12:32<53:49, 4.28s/it]
|
||
20%|█▉ | 184/938 [12:36<52:55, 4.21s/it]
|
||
20%|█▉ | 185/938 [12:40<53:29, 4.26s/it]
|
||
20%|█▉ | 186/938 [12:44<53:09, 4.24s/it]
|
||
20%|█▉ | 187/938 [12:48<50:54, 4.07s/it]
|
||
20%|██ | 188/938 [12:51<48:32, 3.88s/it]
|
||
20%|██ | 189/938 [12:55<49:06, 3.93s/it]
|
||
20%|██ | 190/938 [12:59<48:35, 3.90s/it]
|
||
|
||
{'loss': '0.3129', 'grad_norm': '0.5087', 'learning_rate': '9.691e-06', 'epoch': '0.4053'}
|
||
|
||
20%|██ | 190/938 [12:59<48:35, 3.90s/it]
|
||
20%|██ | 191/938 [13:03<48:50, 3.92s/it]
|
||
20%|██ | 192/938 [13:07<48:24, 3.89s/it]
|
||
21%|██ | 193/938 [13:11<48:31, 3.91s/it]
|
||
21%|██ | 194/938 [13:15<47:36, 3.84s/it]
|
||
21%|██ | 195/938 [13:19<49:52, 4.03s/it]
|
||
21%|██ | 196/938 [13:23<50:33, 4.09s/it]
|
||
21%|██ | 197/938 [13:27<49:38, 4.02s/it]
|
||
21%|██ | 198/938 [13:32<51:07, 4.15s/it]
|
||
21%|██ | 199/938 [13:36<50:39, 4.11s/it]
|
||
21%|██▏ | 200/938 [13:40<50:11, 4.08s/it]
|
||
|
||
{'loss': '0.3092', 'grad_norm': '0.4879', 'learning_rate': '9.623e-06', 'epoch': '0.4267'}
|
||
|
||
21%|██▏ | 200/938 [13:40<50:11, 4.08s/it]
|
||
21%|██▏ | 201/938 [13:45<53:11, 4.33s/it]
|
||
22%|██▏ | 202/938 [13:49<55:09, 4.50s/it]
|
||
22%|██▏ | 203/938 [13:54<53:25, 4.36s/it]
|
||
22%|██▏ | 204/938 [13:58<53:57, 4.41s/it]
|
||
22%|██▏ | 205/938 [14:02<52:28, 4.30s/it]
|
||
22%|██▏ | 206/938 [14:06<51:57, 4.26s/it]
|
||
22%|██▏ | 207/938 [14:10<51:35, 4.23s/it]
|
||
22%|██▏ | 208/938 [14:14<49:55, 4.10s/it]
|
||
22%|██▏ | 209/938 [14:18<49:46, 4.10s/it]
|
||
22%|██▏ | 210/938 [14:22<48:41, 4.01s/it]
|
||
|
||
{'loss': '0.3032', 'grad_norm': '0.4798', 'learning_rate': '9.549e-06', 'epoch': '0.448'}
|
||
|
||
22%|██▏ | 210/938 [14:22<48:41, 4.01s/it]
|
||
22%|██▏ | 211/938 [14:27<50:10, 4.14s/it]
|
||
23%|██▎ | 212/938 [14:31<51:53, 4.29s/it]
|
||
23%|██▎ | 213/938 [14:36<51:54, 4.30s/it]
|
||
23%|██▎ | 214/938 [14:39<49:21, 4.09s/it]
|
||
23%|██▎ | 215/938 [14:43<47:48, 3.97s/it]
|
||
23%|██▎ | 216/938 [14:47<49:16, 4.10s/it]
|
||
23%|██▎ | 217/938 [14:52<51:38, 4.30s/it]
|
||
23%|██▎ | 218/938 [14:56<49:56, 4.16s/it]
|
||
23%|██▎ | 219/938 [15:00<49:46, 4.15s/it]
|
||
23%|██▎ | 220/938 [15:04<50:37, 4.23s/it]
|
||
|
||
{'loss': '0.3086', 'grad_norm': '0.4887', 'learning_rate': '9.468e-06', 'epoch': '0.4693'}
|
||
|
||
23%|██▎ | 220/938 [15:04<50:37, 4.23s/it]
|
||
24%|██▎ | 221/938 [15:08<49:33, 4.15s/it]
|
||
24%|██▎ | 222/938 [15:12<48:25, 4.06s/it]
|
||
24%|██▍ | 223/938 [15:16<46:12, 3.88s/it]
|
||
24%|██▍ | 224/938 [15:20<46:52, 3.94s/it]
|
||
24%|██▍ | 225/938 [15:23<46:07, 3.88s/it]
|
||
24%|██▍ | 226/938 [15:28<47:47, 4.03s/it]
|
||
24%|██▍ | 227/938 [15:33<50:22, 4.25s/it]
|
||
24%|██▍ | 228/938 [15:36<49:02, 4.14s/it]
|
||
24%|██▍ | 229/938 [15:40<47:25, 4.01s/it]
|
||
25%|██▍ | 230/938 [15:45<51:22, 4.35s/it]
|
||
|
||
{'loss': '0.3002', 'grad_norm': '0.4873', 'learning_rate': '9.382e-06', 'epoch': '0.4907'}
|
||
|
||
25%|██▍ | 230/938 [15:45<51:22, 4.35s/it]
|
||
25%|██▍ | 231/938 [15:49<50:22, 4.28s/it]
|
||
25%|██▍ | 232/938 [15:53<48:35, 4.13s/it]
|
||
25%|██▍ | 233/938 [15:57<47:08, 4.01s/it]
|
||
25%|██▍ | 234/938 [16:01<46:04, 3.93s/it]
|
||
25%|██▌ | 235/938 [16:05<47:08, 4.02s/it]
|
||
25%|██▌ | 236/938 [16:09<45:53, 3.92s/it]
|
||
25%|██▌ | 237/938 [16:13<47:57, 4.11s/it]
|
||
25%|██▌ | 238/938 [16:17<48:27, 4.15s/it]
|
||
25%|██▌ | 239/938 [16:21<46:57, 4.03s/it]
|
||
26%|██▌ | 240/938 [16:25<45:23, 3.90s/it]
|
||
|
||
{'loss': '0.299', 'grad_norm': '0.5213', 'learning_rate': '9.289e-06', 'epoch': '0.512'}
|
||
|
||
26%|██▌ | 240/938 [16:25<45:23, 3.90s/it]
|
||
26%|██▌ | 241/938 [16:29<45:32, 3.92s/it]
|
||
26%|██▌ | 242/938 [16:33<45:50, 3.95s/it]
|
||
26%|██▌ | 243/938 [16:37<48:21, 4.17s/it]
|
||
26%|██▌ | 244/938 [16:42<50:19, 4.35s/it]
|
||
26%|██▌ | 245/938 [16:47<50:29, 4.37s/it]
|
||
26%|██▌ | 246/938 [16:51<49:30, 4.29s/it]
|
||
26%|██▋ | 247/938 [16:55<49:13, 4.27s/it]
|
||
26%|██▋ | 248/938 [16:59<49:35, 4.31s/it]
|
||
27%|██▋ | 249/938 [17:03<48:07, 4.19s/it]
|
||
27%|██▋ | 250/938 [17:08<49:09, 4.29s/it]
|
||
|
||
{'loss': '0.2925', 'grad_norm': '0.4684', 'learning_rate': '9.191e-06', 'epoch': '0.5333'}
|
||
|
||
27%|██▋ | 250/938 [17:08<49:09, 4.29s/it]
|
||
27%|██▋ | 251/938 [17:12<49:57, 4.36s/it]
|
||
27%|██▋ | 252/938 [17:16<48:21, 4.23s/it]
|
||
27%|██▋ | 253/938 [17:20<45:49, 4.01s/it]
|
||
27%|██▋ | 254/938 [17:23<43:20, 3.80s/it]
|
||
27%|██▋ | 255/938 [17:27<44:23, 3.90s/it]
|
||
27%|██▋ | 256/938 [17:31<44:15, 3.89s/it]
|
||
27%|██▋ | 257/938 [17:35<44:32, 3.92s/it]
|
||
28%|██▊ | 258/938 [17:39<43:59, 3.88s/it]
|
||
28%|██▊ | 259/938 [17:43<45:11, 3.99s/it]
|
||
28%|██▊ | 260/938 [17:48<46:31, 4.12s/it]
|
||
|
||
{'loss': '0.3163', 'grad_norm': '0.4518', 'learning_rate': '9.086e-06', 'epoch': '0.5547'}
|
||
|
||
28%|██▊ | 260/938 [17:48<46:31, 4.12s/it]
|
||
28%|██▊ | 261/938 [17:52<47:32, 4.21s/it]
|
||
28%|██▊ | 262/938 [17:56<47:16, 4.20s/it]
|
||
28%|██▊ | 263/938 [18:00<45:18, 4.03s/it]
|
||
28%|██▊ | 264/938 [18:04<45:03, 4.01s/it]
|
||
28%|██▊ | 265/938 [18:07<43:08, 3.85s/it]
|
||
28%|██▊ | 266/938 [18:12<45:53, 4.10s/it]
|
||
28%|██▊ | 267/938 [18:16<45:51, 4.10s/it]
|
||
29%|██▊ | 268/938 [18:20<45:13, 4.05s/it]
|
||
29%|██▊ | 269/938 [18:24<44:38, 4.00s/it]
|
||
29%|██▉ | 270/938 [18:28<44:36, 4.01s/it]
|
||
|
||
{'loss': '0.3071', 'grad_norm': '0.4481', 'learning_rate': '8.976e-06', 'epoch': '0.576'}
|
||
|
||
29%|██▉ | 270/938 [18:28<44:36, 4.01s/it]
|
||
29%|██▉ | 271/938 [18:32<44:22, 3.99s/it]
|
||
29%|██▉ | 272/938 [18:36<44:47, 4.04s/it]
|
||
29%|██▉ | 273/938 [18:39<42:58, 3.88s/it]
|
||
29%|██▉ | 274/938 [18:43<42:24, 3.83s/it]
|
||
29%|██▉ | 275/938 [18:47<42:16, 3.83s/it]
|
||
29%|██▉ | 276/938 [18:51<44:21, 4.02s/it]
|
||
30%|██▉ | 277/938 [18:55<44:17, 4.02s/it]
|
||
30%|██▉ | 278/938 [19:00<45:15, 4.11s/it]
|
||
30%|██▉ | 279/938 [19:04<46:15, 4.21s/it]
|
||
30%|██▉ | 280/938 [19:08<46:05, 4.20s/it]
|
||
|
||
{'loss': '0.3033', 'grad_norm': '0.4673', 'learning_rate': '8.861e-06', 'epoch': '0.5973'}
|
||
|
||
30%|██▉ | 280/938 [19:08<46:05, 4.20s/it]
|
||
30%|██▉ | 281/938 [19:12<45:29, 4.15s/it]
|
||
30%|███ | 282/938 [19:16<44:06, 4.03s/it]
|
||
30%|███ | 283/938 [19:20<44:11, 4.05s/it]
|
||
30%|███ | 284/938 [19:24<43:15, 3.97s/it]
|
||
30%|███ | 285/938 [19:28<42:53, 3.94s/it]
|
||
30%|███ | 286/938 [19:32<42:39, 3.93s/it]
|
||
31%|███ | 287/938 [19:35<41:16, 3.80s/it]
|
||
31%|███ | 288/938 [19:39<41:42, 3.85s/it]
|
||
31%|███ | 289/938 [19:44<46:00, 4.25s/it]
|
||
31%|███ | 290/938 [19:49<48:03, 4.45s/it]
|
||
|
||
{'loss': '0.2964', 'grad_norm': '0.4601', 'learning_rate': '8.74e-06', 'epoch': '0.6187'}
|
||
|
||
31%|███ | 290/938 [19:49<48:03, 4.45s/it]
|
||
31%|███ | 291/938 [19:53<45:57, 4.26s/it]
|
||
31%|███ | 292/938 [19:58<46:53, 4.36s/it]
|
||
31%|███ | 293/938 [20:01<44:06, 4.10s/it]
|
||
31%|███▏ | 294/938 [20:05<43:25, 4.05s/it]
|
||
31%|███▏ | 295/938 [20:10<46:07, 4.30s/it]
|
||
32%|███▏ | 296/938 [20:14<44:08, 4.13s/it]
|
||
32%|███▏ | 297/938 [20:17<42:13, 3.95s/it]
|
||
32%|███▏ | 298/938 [20:23<46:16, 4.34s/it]
|
||
32%|███▏ | 299/938 [20:27<46:13, 4.34s/it]
|
||
32%|███▏ | 300/938 [20:32<49:07, 4.62s/it]
|
||
|
||
{'loss': '0.307', 'grad_norm': '0.4463', 'learning_rate': '8.614e-06', 'epoch': '0.64'}
|
||
|
||
32%|███▏ | 300/938 [20:32<49:07, 4.62s/it]
|
||
32%|███▏ | 301/938 [20:36<45:40, 4.30s/it]
|
||
32%|███▏ | 302/938 [20:40<44:14, 4.17s/it]
|
||
32%|███▏ | 303/938 [20:44<43:21, 4.10s/it]
|
||
32%|███▏ | 304/938 [20:48<44:52, 4.25s/it]
|
||
33%|███▎ | 305/938 [20:52<44:13, 4.19s/it]
|
||
33%|███▎ | 306/938 [20:56<43:14, 4.11s/it]
|
||
33%|███▎ | 307/938 [20:59<40:27, 3.85s/it]
|
||
33%|███▎ | 308/938 [21:04<41:10, 3.92s/it]
|
||
33%|███▎ | 309/938 [21:08<43:16, 4.13s/it]
|
||
33%|███▎ | 310/938 [21:12<42:17, 4.04s/it]
|
||
|
||
{'loss': '0.3074', 'grad_norm': '0.4809', 'learning_rate': '8.483e-06', 'epoch': '0.6613'}
|
||
|
||
33%|███▎ | 310/938 [21:12<42:17, 4.04s/it]
|
||
33%|███▎ | 311/938 [21:16<42:36, 4.08s/it]
|
||
33%|███▎ | 312/938 [21:20<42:35, 4.08s/it]
|
||
33%|███▎ | 313/938 [21:25<44:24, 4.26s/it]
|
||
33%|███▎ | 314/938 [21:29<45:05, 4.34s/it]
|
||
34%|███▎ | 315/938 [21:34<44:57, 4.33s/it]
|
||
34%|███▎ | 316/938 [21:38<45:15, 4.36s/it]
|
||
34%|███▍ | 317/938 [21:42<44:35, 4.31s/it]
|
||
34%|███▍ | 318/938 [21:47<44:30, 4.31s/it]
|
||
34%|███▍ | 319/938 [21:51<43:30, 4.22s/it]
|
||
34%|███▍ | 320/938 [21:55<43:29, 4.22s/it]
|
||
|
||
{'loss': '0.2807', 'grad_norm': '0.508', 'learning_rate': '8.347e-06', 'epoch': '0.6827'}
|
||
|
||
34%|███▍ | 320/938 [21:55<43:29, 4.22s/it]
|
||
34%|███▍ | 321/938 [21:59<43:57, 4.27s/it]
|
||
34%|███▍ | 322/938 [22:04<46:06, 4.49s/it]
|
||
34%|███▍ | 323/938 [22:08<45:05, 4.40s/it]
|
||
35%|███▍ | 324/938 [22:12<43:51, 4.29s/it]
|
||
35%|███▍ | 325/938 [22:17<44:41, 4.37s/it]
|
||
35%|███▍ | 326/938 [22:21<44:38, 4.38s/it]
|
||
35%|███▍ | 327/938 [22:25<42:34, 4.18s/it]
|
||
35%|███▍ | 328/938 [22:29<41:07, 4.04s/it]
|
||
35%|███▌ | 329/938 [22:33<41:53, 4.13s/it]
|
||
35%|███▌ | 330/938 [22:37<40:43, 4.02s/it]
|
||
|
||
{'loss': '0.2867', 'grad_norm': '0.4667', 'learning_rate': '8.206e-06', 'epoch': '0.704'}
|
||
|
||
35%|███▌ | 330/938 [22:37<40:43, 4.02s/it]
|
||
35%|███▌ | 331/938 [22:42<43:37, 4.31s/it]
|
||
35%|███▌ | 332/938 [22:47<45:11, 4.47s/it]
|
||
36%|███▌ | 333/938 [22:52<47:02, 4.67s/it]
|
||
36%|███▌ | 334/938 [22:56<45:52, 4.56s/it]
|
||
36%|███▌ | 335/938 [23:01<45:25, 4.52s/it]
|
||
36%|███▌ | 336/938 [23:05<45:27, 4.53s/it]
|
||
36%|███▌ | 337/938 [23:09<44:02, 4.40s/it]
|
||
36%|███▌ | 338/938 [23:14<44:46, 4.48s/it]
|
||
36%|███▌ | 339/938 [23:19<44:54, 4.50s/it]
|
||
36%|███▌ | 340/938 [23:23<44:38, 4.48s/it]
|
||
|
||
{'loss': '0.3078', 'grad_norm': '0.4525', 'learning_rate': '8.061e-06', 'epoch': '0.7253'}
|
||
|
||
36%|███▌ | 340/938 [23:23<44:38, 4.48s/it]
|
||
36%|███▋ | 341/938 [23:27<44:13, 4.45s/it]
|
||
36%|███▋ | 342/938 [23:32<44:05, 4.44s/it]
|
||
37%|███▋ | 343/938 [23:37<45:00, 4.54s/it]
|
||
37%|███▋ | 344/938 [23:41<44:51, 4.53s/it]
|
||
37%|███▋ | 345/938 [23:45<43:38, 4.42s/it]
|
||
37%|███▋ | 346/938 [23:50<44:42, 4.53s/it]
|
||
37%|███▋ | 347/938 [23:54<43:12, 4.39s/it]
|
||
37%|███▋ | 348/938 [23:58<42:40, 4.34s/it]
|
||
37%|███▋ | 349/938 [24:02<42:05, 4.29s/it]
|
||
37%|███▋ | 350/938 [24:06<40:58, 4.18s/it]
|
||
|
||
{'loss': '0.2937', 'grad_norm': '0.5167', 'learning_rate': '7.912e-06', 'epoch': '0.7467'}
|
||
|
||
37%|███▋ | 350/938 [24:06<40:58, 4.18s/it]
|
||
37%|███▋ | 351/938 [24:10<39:30, 4.04s/it]
|
||
38%|███▊ | 352/938 [24:14<39:22, 4.03s/it]
|
||
38%|███▊ | 353/938 [24:19<41:10, 4.22s/it]
|
||
38%|███▊ | 354/938 [24:23<41:40, 4.28s/it]
|
||
38%|███▊ | 355/938 [24:29<44:54, 4.62s/it]
|
||
38%|███▊ | 356/938 [24:33<43:35, 4.49s/it]
|
||
38%|███▊ | 357/938 [24:37<43:27, 4.49s/it]
|
||
38%|███▊ | 358/938 [24:41<41:30, 4.29s/it]
|
||
38%|███▊ | 359/938 [24:45<41:30, 4.30s/it]
|
||
38%|███▊ | 360/938 [24:50<41:17, 4.29s/it]
|
||
|
||
{'loss': '0.2857', 'grad_norm': '0.4678', 'learning_rate': '7.758e-06', 'epoch': '0.768'}
|
||
|
||
38%|███▊ | 360/938 [24:50<41:17, 4.29s/it]
|
||
38%|███▊ | 361/938 [24:55<43:08, 4.49s/it]
|
||
39%|███▊ | 362/938 [24:58<40:31, 4.22s/it]
|
||
39%|███▊ | 363/938 [25:02<39:23, 4.11s/it]
|
||
39%|███▉ | 364/938 [25:06<40:05, 4.19s/it]
|
||
39%|███▉ | 365/938 [25:10<39:19, 4.12s/it]
|
||
39%|███▉ | 366/938 [25:15<39:12, 4.11s/it]
|
||
39%|███▉ | 367/938 [25:19<39:19, 4.13s/it]
|
||
39%|███▉ | 368/938 [25:23<39:27, 4.15s/it]
|
||
39%|███▉ | 369/938 [25:27<39:03, 4.12s/it]
|
||
39%|███▉ | 370/938 [25:32<42:04, 4.44s/it]
|
||
|
||
{'loss': '0.2912', 'grad_norm': '0.4754', 'learning_rate': '7.601e-06', 'epoch': '0.7893'}
|
||
|
||
39%|███▉ | 370/938 [25:32<42:04, 4.44s/it]
|
||
40%|███▉ | 371/938 [25:37<44:35, 4.72s/it]
|
||
40%|███▉ | 372/938 [25:43<47:25, 5.03s/it]
|
||
40%|███▉ | 373/938 [25:47<44:42, 4.75s/it]
|
||
40%|███▉ | 374/938 [25:52<43:25, 4.62s/it]
|
||
40%|███▉ | 375/938 [25:56<43:26, 4.63s/it]
|
||
40%|████ | 376/938 [26:01<43:59, 4.70s/it]
|
||
40%|████ | 377/938 [26:05<41:49, 4.47s/it]
|
||
40%|████ | 378/938 [26:09<40:19, 4.32s/it]
|
||
40%|████ | 379/938 [26:13<40:00, 4.29s/it]
|
||
41%|████ | 380/938 [26:18<40:10, 4.32s/it]
|
||
|
||
{'loss': '0.2886', 'grad_norm': '0.4467', 'learning_rate': '7.441e-06', 'epoch': '0.8107'}
|
||
|
||
41%|████ | 380/938 [26:18<40:10, 4.32s/it]
|
||
41%|████ | 381/938 [26:21<38:08, 4.11s/it]
|
||
41%|████ | 382/938 [26:25<37:35, 4.06s/it]
|
||
41%|████ | 383/938 [26:29<37:40, 4.07s/it]
|
||
41%|████ | 384/938 [26:33<36:21, 3.94s/it]
|
||
41%|████ | 385/938 [26:37<37:47, 4.10s/it]
|
||
41%|████ | 386/938 [26:42<38:30, 4.19s/it]
|
||
41%|████▏ | 387/938 [26:46<37:34, 4.09s/it]
|
||
41%|████▏ | 388/938 [26:50<39:18, 4.29s/it]
|
||
41%|████▏ | 389/938 [26:55<40:00, 4.37s/it]
|
||
42%|████▏ | 390/938 [26:59<39:40, 4.34s/it]
|
||
|
||
{'loss': '0.2949', 'grad_norm': '0.4401', 'learning_rate': '7.277e-06', 'epoch': '0.832'}
|
||
|
||
42%|████▏ | 390/938 [26:59<39:40, 4.34s/it]
|
||
42%|████▏ | 391/938 [27:04<39:24, 4.32s/it]
|
||
42%|████▏ | 392/938 [27:08<40:26, 4.44s/it]
|
||
42%|████▏ | 393/938 [27:12<39:29, 4.35s/it]
|
||
42%|████▏ | 394/938 [27:17<40:22, 4.45s/it]
|
||
42%|████▏ | 395/938 [27:22<41:33, 4.59s/it]
|
||
42%|████▏ | 396/938 [27:26<38:52, 4.30s/it]
|
||
42%|████▏ | 397/938 [27:30<38:00, 4.22s/it]
|
||
42%|████▏ | 398/938 [27:34<37:14, 4.14s/it]
|
||
43%|████▎ | 399/938 [27:38<38:32, 4.29s/it]
|
||
43%|████▎ | 400/938 [27:43<38:37, 4.31s/it]
|
||
|
||
{'loss': '0.2888', 'grad_norm': '0.463', 'learning_rate': '7.109e-06', 'epoch': '0.8533'}
|
||
|
||
43%|████▎ | 400/938 [27:43<38:37, 4.31s/it]
|
||
43%|████▎ | 401/938 [27:47<37:50, 4.23s/it]
|
||
43%|████▎ | 402/938 [27:51<37:16, 4.17s/it]
|
||
43%|████▎ | 403/938 [27:55<37:10, 4.17s/it]
|
||
43%|████▎ | 404/938 [27:59<36:20, 4.08s/it]
|
||
43%|████▎ | 405/938 [28:02<34:51, 3.92s/it]
|
||
43%|████▎ | 406/938 [28:07<35:38, 4.02s/it]
|
||
43%|████▎ | 407/938 [28:11<36:30, 4.13s/it]
|
||
43%|████▎ | 408/938 [28:15<36:48, 4.17s/it]
|
||
44%|████▎ | 409/938 [28:20<37:11, 4.22s/it]
|
||
44%|████▎ | 410/938 [28:23<36:24, 4.14s/it]
|
||
|
||
{'loss': '0.3021', 'grad_norm': '0.5064', 'learning_rate': '6.939e-06', 'epoch': '0.8747'}
|
||
|
||
44%|████▎ | 410/938 [28:23<36:24, 4.14s/it]
|
||
44%|████▍ | 411/938 [28:28<36:54, 4.20s/it]
|
||
44%|████▍ | 412/938 [28:32<36:28, 4.16s/it]
|
||
44%|████▍ | 413/938 [28:36<36:03, 4.12s/it]
|
||
44%|████▍ | 414/938 [28:40<36:35, 4.19s/it]
|
||
44%|████▍ | 415/938 [28:45<37:15, 4.28s/it]
|
||
44%|████▍ | 416/938 [28:49<36:14, 4.17s/it]
|
||
44%|████▍ | 417/938 [28:53<36:53, 4.25s/it]
|
||
45%|████▍ | 418/938 [28:57<37:03, 4.28s/it]
|
||
45%|████▍ | 419/938 [29:02<36:41, 4.24s/it]
|
||
45%|████▍ | 420/938 [29:06<36:10, 4.19s/it]
|
||
|
||
{'loss': '0.2929', 'grad_norm': '0.4812', 'learning_rate': '6.766e-06', 'epoch': '0.896'}
|
||
|
||
45%|████▍ | 420/938 [29:06<36:10, 4.19s/it]
|
||
45%|████▍ | 421/938 [29:10<36:20, 4.22s/it]
|
||
45%|████▍ | 422/938 [29:14<35:15, 4.10s/it]
|
||
45%|████▌ | 423/938 [29:18<36:08, 4.21s/it]
|
||
45%|████▌ | 424/938 [29:22<35:51, 4.19s/it]
|
||
45%|████▌ | 425/938 [29:27<35:47, 4.19s/it]
|
||
45%|████▌ | 426/938 [29:30<34:01, 3.99s/it]
|
||
46%|████▌ | 427/938 [29:34<33:17, 3.91s/it]
|
||
46%|████▌ | 428/938 [29:38<33:55, 3.99s/it]
|
||
46%|████▌ | 429/938 [29:42<33:34, 3.96s/it]
|
||
46%|████▌ | 430/938 [29:46<34:09, 4.04s/it]
|
||
|
||
{'loss': '0.2888', 'grad_norm': '0.4532', 'learning_rate': '6.591e-06', 'epoch': '0.9173'}
|
||
|
||
46%|████▌ | 430/938 [29:46<34:09, 4.04s/it]
|
||
46%|████▌ | 431/938 [29:50<32:35, 3.86s/it]
|
||
46%|████▌ | 432/938 [29:53<32:24, 3.84s/it]
|
||
46%|████▌ | 433/938 [29:57<32:11, 3.82s/it]
|
||
46%|████▋ | 434/938 [30:02<34:43, 4.13s/it]
|
||
46%|████▋ | 435/938 [30:07<38:02, 4.54s/it]
|
||
46%|████▋ | 436/938 [30:12<37:47, 4.52s/it]
|
||
47%|████▋ | 437/938 [30:16<36:26, 4.36s/it]
|
||
47%|████▋ | 438/938 [30:20<35:57, 4.32s/it]
|
||
47%|████▋ | 439/938 [30:23<33:12, 3.99s/it]
|
||
47%|████▋ | 440/938 [30:28<34:49, 4.20s/it]
|
||
|
||
{'loss': '0.2961', 'grad_norm': '0.4447', 'learning_rate': '6.414e-06', 'epoch': '0.9387'}
|
||
|
||
47%|████▋ | 440/938 [30:28<34:49, 4.20s/it]
|
||
47%|████▋ | 441/938 [30:32<35:06, 4.24s/it]
|
||
47%|████▋ | 442/938 [30:36<33:11, 4.01s/it]
|
||
47%|████▋ | 443/938 [30:40<33:27, 4.05s/it]
|
||
47%|████▋ | 444/938 [30:44<33:29, 4.07s/it]
|
||
47%|████▋ | 445/938 [30:49<34:34, 4.21s/it]
|
||
48%|████▊ | 446/938 [30:53<35:37, 4.34s/it]
|
||
48%|████▊ | 447/938 [30:57<34:27, 4.21s/it]
|
||
48%|████▊ | 448/938 [31:01<33:42, 4.13s/it]
|
||
48%|████▊ | 449/938 [31:06<34:21, 4.22s/it]
|
||
48%|████▊ | 450/938 [31:09<33:31, 4.12s/it]
|
||
|
||
{'loss': '0.2955', 'grad_norm': '0.4198', 'learning_rate': '6.234e-06', 'epoch': '0.96'}
|
||
|
||
48%|████▊ | 450/938 [31:09<33:31, 4.12s/it]
|
||
48%|████▊ | 451/938 [31:14<34:52, 4.30s/it]
|
||
48%|████▊ | 452/938 [31:20<38:18, 4.73s/it]
|
||
48%|████▊ | 453/938 [31:24<37:01, 4.58s/it]
|
||
48%|████▊ | 454/938 [31:29<37:15, 4.62s/it]
|
||
49%|████▊ | 455/938 [31:34<37:43, 4.69s/it]
|
||
49%|████▊ | 456/938 [31:38<37:13, 4.63s/it]
|
||
49%|████▊ | 457/938 [31:42<35:28, 4.43s/it]
|
||
49%|████▉ | 458/938 [31:47<35:45, 4.47s/it]
|
||
49%|████▉ | 459/938 [31:50<33:49, 4.24s/it]
|
||
49%|████▉ | 460/938 [31:54<32:37, 4.09s/it]
|
||
|
||
{'loss': '0.2948', 'grad_norm': '0.4504', 'learning_rate': '6.053e-06', 'epoch': '0.9813'}
|
||
|
||
49%|████▉ | 460/938 [31:54<32:37, 4.09s/it]
|
||
49%|████▉ | 461/938 [31:58<31:45, 3.99s/it]
|
||
49%|████▉ | 462/938 [32:02<31:01, 3.91s/it]
|
||
49%|████▉ | 463/938 [32:06<31:19, 3.96s/it]
|
||
49%|████▉ | 464/938 [32:10<31:39, 4.01s/it]
|
||
50%|████▉ | 465/938 [32:14<31:21, 3.98s/it]
|
||
50%|████▉ | 466/938 [32:18<32:31, 4.14s/it]
|
||
50%|████▉ | 467/938 [32:22<32:13, 4.10s/it]
|
||
50%|████▉ | 468/938 [32:26<31:18, 4.00s/it]
|
||
50%|█████ | 469/938 [32:29<29:41, 3.80s/it]
|
||
50%|█████ | 470/938 [32:34<32:30, 4.17s/it]
|
||
|
||
{'loss': '0.2972', 'grad_norm': '0.5018', 'learning_rate': '5.87e-06', 'epoch': '1.002'}
|
||
|
||
50%|█████ | 470/938 [32:34<32:30, 4.17s/it]
|
||
50%|█████ | 471/938 [32:39<33:45, 4.34s/it]
|
||
50%|█████ | 472/938 [32:43<32:04, 4.13s/it]
|
||
50%|█████ | 473/938 [32:46<30:18, 3.91s/it]
|
||
51%|█████ | 474/938 [32:50<30:18, 3.92s/it]
|
||
51%|█████ | 475/938 [32:54<31:01, 4.02s/it]
|
||
51%|█████ | 476/938 [32:59<31:18, 4.07s/it]
|
||
51%|█████ | 477/938 [33:03<31:10, 4.06s/it]
|
||
51%|█████ | 478/938 [33:07<32:02, 4.18s/it]
|
||
51%|█████ | 479/938 [33:11<31:02, 4.06s/it]
|
||
51%|█████ | 480/938 [33:15<31:09, 4.08s/it]
|
||
|
||
{'loss': '0.2648', 'grad_norm': '0.4842', 'learning_rate': '5.686e-06', 'epoch': '1.023'}
|
||
|
||
51%|█████ | 480/938 [33:15<31:09, 4.08s/it]
|
||
51%|█████▏ | 481/938 [33:19<30:30, 4.01s/it]
|
||
51%|█████▏ | 482/938 [33:23<30:43, 4.04s/it]
|
||
51%|█████▏ | 483/938 [33:26<29:26, 3.88s/it]
|
||
52%|█████▏ | 484/938 [33:31<29:48, 3.94s/it]
|
||
52%|█████▏ | 485/938 [33:36<32:09, 4.26s/it]
|
||
52%|█████▏ | 486/938 [33:40<31:53, 4.23s/it]
|
||
52%|█████▏ | 487/938 [33:44<32:24, 4.31s/it]
|
||
52%|█████▏ | 488/938 [33:48<31:29, 4.20s/it]
|
||
52%|█████▏ | 489/938 [33:52<30:54, 4.13s/it]
|
||
52%|█████▏ | 490/938 [33:57<31:57, 4.28s/it]
|
||
|
||
{'loss': '0.271', 'grad_norm': '0.4402', 'learning_rate': '5.502e-06', 'epoch': '1.045'}
|
||
|
||
52%|█████▏ | 490/938 [33:57<31:57, 4.28s/it]
|
||
52%|█████▏ | 491/938 [34:00<30:38, 4.11s/it]
|
||
52%|█████▏ | 492/938 [34:05<31:30, 4.24s/it]
|
||
53%|█████▎ | 493/938 [34:09<31:30, 4.25s/it]
|
||
53%|█████▎ | 494/938 [34:13<30:53, 4.18s/it]
|
||
53%|█████▎ | 495/938 [34:17<30:05, 4.08s/it]
|
||
53%|█████▎ | 496/938 [34:22<31:00, 4.21s/it]
|
||
53%|█████▎ | 497/938 [34:26<31:54, 4.34s/it]
|
||
53%|█████▎ | 498/938 [34:30<30:09, 4.11s/it]
|
||
53%|█████▎ | 499/938 [34:34<29:19, 4.01s/it]
|
||
53%|█████▎ | 500/938 [34:38<30:08, 4.13s/it]
|
||
|
||
{'loss': '0.2706', 'grad_norm': '0.4749', 'learning_rate': '5.316e-06', 'epoch': '1.066'}
|
||
|
||
53%|█████▎ | 500/938 [34:38<30:08, 4.13s/it][INFO|trainer.py:3797] 2026-04-29 16:19:18,046 >> Saving model checkpoint to saves/qwen3_1.7b/math/fft/checkpoint-500
|
||
[INFO|configuration_utils.py:432] 2026-04-29 16:19:18,055 >> Configuration saved in saves/qwen3_1.7b/math/fft/checkpoint-500/config.json
|
||
[INFO|configuration_utils.py:803] 2026-04-29 16:19:18,058 >> Configuration saved in saves/qwen3_1.7b/math/fft/checkpoint-500/generation_config.json
|
||
|
||
|
||
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s][A
|
||
|
||
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.28s/it][A
|
||
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.29s/it]
|
||
[INFO|modeling_utils.py:3380] 2026-04-29 16:19:24,375 >> Model weights saved in saves/qwen3_1.7b/math/fft/checkpoint-500/model.safetensors
|
||
[INFO|tokenization_utils_base.py:3224] 2026-04-29 16:19:24,380 >> chat template saved in saves/qwen3_1.7b/math/fft/checkpoint-500/chat_template.jinja
|
||
[INFO|tokenization_utils_base.py:2078] 2026-04-29 16:19:24,384 >> tokenizer config file saved in saves/qwen3_1.7b/math/fft/checkpoint-500/tokenizer_config.json
|
||
|
||
53%|█████▎ | 501/938 [34:52<51:35, 7.08s/it]
|
||
54%|█████▎ | 502/938 [34:56<44:10, 6.08s/it]
|
||
54%|█████▎ | 503/938 [34:59<38:54, 5.37s/it]
|
||
54%|█████▎ | 504/938 [35:04<36:57, 5.11s/it]
|
||
54%|█████▍ | 505/938 [35:08<35:11, 4.88s/it]
|
||
54%|█████▍ | 506/938 [35:13<33:51, 4.70s/it]
|
||
54%|█████▍ | 507/938 [35:17<32:40, 4.55s/it]
|
||
54%|█████▍ | 508/938 [35:22<33:33, 4.68s/it]
|
||
54%|█████▍ | 509/938 [35:26<32:07, 4.49s/it]
|
||
54%|█████▍ | 510/938 [35:30<31:53, 4.47s/it]
|
||
|
||
{'loss': '0.2784', 'grad_norm': '0.4592', 'learning_rate': '5.13e-06', 'epoch': '1.087'}
|
||
|
||
54%|█████▍ | 510/938 [35:30<31:53, 4.47s/it]
|
||
54%|█████▍ | 511/938 [35:34<30:50, 4.33s/it]
|
||
55%|█████▍ | 512/938 [35:38<30:31, 4.30s/it]
|
||
55%|█████▍ | 513/938 [35:44<32:17, 4.56s/it]
|
||
55%|█████▍ | 514/938 [35:48<31:03, 4.39s/it]
|
||
55%|█████▍ | 515/938 [35:51<29:44, 4.22s/it]
|
||
55%|█████▌ | 516/938 [35:56<30:52, 4.39s/it]
|
||
55%|█████▌ | 517/938 [36:00<29:49, 4.25s/it]
|
||
55%|█████▌ | 518/938 [36:04<29:27, 4.21s/it]
|
||
55%|█████▌ | 519/938 [36:09<31:01, 4.44s/it]
|
||
55%|█████▌ | 520/938 [36:13<29:14, 4.20s/it]
|
||
|
||
{'loss': '0.2618', 'grad_norm': '0.4609', 'learning_rate': '4.944e-06', 'epoch': '1.109'}
|
||
|
||
55%|█████▌ | 520/938 [36:13<29:14, 4.20s/it]
|
||
56%|█████▌ | 521/938 [36:17<28:34, 4.11s/it]
|
||
56%|█████▌ | 522/938 [36:21<29:05, 4.20s/it]
|
||
56%|█████▌ | 523/938 [36:25<29:14, 4.23s/it]
|
||
56%|█████▌ | 524/938 [36:30<29:51, 4.33s/it]
|
||
56%|█████▌ | 525/938 [36:34<29:14, 4.25s/it]
|
||
56%|█████▌ | 526/938 [36:38<27:33, 4.01s/it]
|
||
56%|█████▌ | 527/938 [36:41<26:31, 3.87s/it]
|
||
56%|█████▋ | 528/938 [36:45<27:25, 4.01s/it]
|
||
56%|█████▋ | 529/938 [36:50<28:01, 4.11s/it]
|
||
57%|█████▋ | 530/938 [36:54<27:16, 4.01s/it]
|
||
|
||
{'loss': '0.2712', 'grad_norm': '0.4843', 'learning_rate': '4.758e-06', 'epoch': '1.13'}
|
||
|
||
57%|█████▋ | 530/938 [36:54<27:16, 4.01s/it]
|
||
57%|█████▋ | 531/938 [36:58<27:51, 4.11s/it]
|
||
57%|█████▋ | 532/938 [37:03<30:17, 4.48s/it]
|
||
57%|█████▋ | 533/938 [37:07<28:37, 4.24s/it]
|
||
57%|█████▋ | 534/938 [37:11<27:44, 4.12s/it]
|
||
57%|█████▋ | 535/938 [37:15<27:43, 4.13s/it]
|
||
57%|█████▋ | 536/938 [37:20<29:20, 4.38s/it]
|
||
57%|█████▋ | 537/938 [37:25<30:00, 4.49s/it]
|
||
57%|█████▋ | 538/938 [37:29<29:09, 4.37s/it]
|
||
57%|█████▋ | 539/938 [37:33<29:10, 4.39s/it]
|
||
58%|█████▊ | 540/938 [37:37<27:22, 4.13s/it]
|
||
|
||
{'loss': '0.2627', 'grad_norm': '0.5324', 'learning_rate': '4.572e-06', 'epoch': '1.151'}
|
||
|
||
58%|█████▊ | 540/938 [37:37<27:22, 4.13s/it]
|
||
58%|█████▊ | 541/938 [37:41<28:24, 4.29s/it]
|
||
58%|█████▊ | 542/938 [37:45<27:53, 4.23s/it]
|
||
58%|█████▊ | 543/938 [37:49<26:48, 4.07s/it]
|
||
58%|█████▊ | 544/938 [37:53<26:41, 4.06s/it]
|
||
58%|█████▊ | 545/938 [37:58<27:16, 4.16s/it]
|
||
58%|█████▊ | 546/938 [38:01<26:23, 4.04s/it]
|
||
58%|█████▊ | 547/938 [38:05<26:15, 4.03s/it]
|
||
58%|█████▊ | 548/938 [38:10<27:35, 4.24s/it]
|
||
59%|█████▊ | 549/938 [38:14<26:45, 4.13s/it]
|
||
59%|█████▊ | 550/938 [38:18<26:52, 4.16s/it]
|
||
|
||
{'loss': '0.2738', 'grad_norm': '0.4596', 'learning_rate': '4.387e-06', 'epoch': '1.173'}
|
||
|
||
59%|█████▊ | 550/938 [38:18<26:52, 4.16s/it]
|
||
59%|█████▊ | 551/938 [38:22<26:25, 4.10s/it]
|
||
59%|█████▉ | 552/938 [38:26<26:24, 4.10s/it]
|
||
59%|█████▉ | 553/938 [38:30<26:34, 4.14s/it]
|
||
59%|█████▉ | 554/938 [38:34<25:32, 3.99s/it]
|
||
59%|█████▉ | 555/938 [38:38<25:14, 3.95s/it]
|
||
59%|█████▉ | 556/938 [38:42<25:23, 3.99s/it]
|
||
59%|█████▉ | 557/938 [38:46<24:56, 3.93s/it]
|
||
59%|█████▉ | 558/938 [38:50<24:26, 3.86s/it]
|
||
60%|█████▉ | 559/938 [38:54<25:26, 4.03s/it]
|
||
60%|█████▉ | 560/938 [38:58<24:56, 3.96s/it]
|
||
|
||
{'loss': '0.2654', 'grad_norm': '0.4884', 'learning_rate': '4.203e-06', 'epoch': '1.194'}
|
||
|
||
60%|█████▉ | 560/938 [38:58<24:56, 3.96s/it]
|
||
60%|█████▉ | 561/938 [39:01<24:21, 3.88s/it]
|
||
60%|█████▉ | 562/938 [39:06<25:09, 4.01s/it]
|
||
60%|██████ | 563/938 [39:10<24:51, 3.98s/it]
|
||
60%|██████ | 564/938 [39:13<24:14, 3.89s/it]
|
||
60%|██████ | 565/938 [39:18<24:45, 3.98s/it]
|
||
60%|██████ | 566/938 [39:21<24:34, 3.96s/it]
|
||
60%|██████ | 567/938 [39:26<24:52, 4.02s/it]
|
||
61%|██████ | 568/938 [39:30<25:31, 4.14s/it]
|
||
61%|██████ | 569/938 [39:35<27:18, 4.44s/it]
|
||
61%|██████ | 570/938 [39:40<28:20, 4.62s/it]
|
||
|
||
{'loss': '0.2673', 'grad_norm': '0.4471', 'learning_rate': '4.02e-06', 'epoch': '1.215'}
|
||
|
||
61%|██████ | 570/938 [39:40<28:20, 4.62s/it]
|
||
61%|██████ | 571/938 [39:44<26:47, 4.38s/it]
|
||
61%|██████ | 572/938 [39:48<25:11, 4.13s/it]
|
||
61%|██████ | 573/938 [39:52<25:18, 4.16s/it]
|
||
61%|██████ | 574/938 [39:55<23:45, 3.92s/it]
|
||
61%|██████▏ | 575/938 [40:00<24:30, 4.05s/it]
|
||
61%|██████▏ | 576/938 [40:03<24:09, 4.00s/it]
|
||
62%|██████▏ | 577/938 [40:08<24:54, 4.14s/it]
|
||
62%|██████▏ | 578/938 [40:12<24:08, 4.02s/it]
|
||
62%|██████▏ | 579/938 [40:16<23:52, 3.99s/it]
|
||
62%|██████▏ | 580/938 [40:20<24:08, 4.05s/it]
|
||
|
||
{'loss': '0.263', 'grad_norm': '0.4469', 'learning_rate': '3.838e-06', 'epoch': '1.237'}
|
||
|
||
62%|██████▏ | 580/938 [40:20<24:08, 4.05s/it]
|
||
62%|██████▏ | 581/938 [40:24<23:42, 3.99s/it]
|
||
62%|██████▏ | 582/938 [40:27<23:25, 3.95s/it]
|
||
62%|██████▏ | 583/938 [40:31<22:44, 3.85s/it]
|
||
62%|██████▏ | 584/938 [40:37<25:52, 4.38s/it]
|
||
62%|██████▏ | 585/938 [40:40<24:17, 4.13s/it]
|
||
62%|██████▏ | 586/938 [40:46<27:31, 4.69s/it]
|
||
63%|██████▎ | 587/938 [40:51<27:25, 4.69s/it]
|
||
63%|██████▎ | 588/938 [40:55<26:50, 4.60s/it]
|
||
63%|██████▎ | 589/938 [41:00<26:11, 4.50s/it]
|
||
63%|██████▎ | 590/938 [41:04<25:13, 4.35s/it]
|
||
|
||
{'loss': '0.2718', 'grad_norm': '0.4668', 'learning_rate': '3.658e-06', 'epoch': '1.258'}
|
||
|
||
63%|██████▎ | 590/938 [41:04<25:13, 4.35s/it]
|
||
63%|██████▎ | 591/938 [41:08<25:10, 4.35s/it]
|
||
63%|██████▎ | 592/938 [41:11<23:41, 4.11s/it]
|
||
63%|██████▎ | 593/938 [41:16<24:42, 4.30s/it]
|
||
63%|██████▎ | 594/938 [41:22<26:22, 4.60s/it]
|
||
63%|██████▎ | 595/938 [41:26<25:15, 4.42s/it]
|
||
64%|██████▎ | 596/938 [41:29<24:13, 4.25s/it]
|
||
64%|██████▎ | 597/938 [41:34<25:31, 4.49s/it]
|
||
64%|██████▍ | 598/938 [41:38<23:56, 4.22s/it]
|
||
64%|██████▍ | 599/938 [41:42<23:23, 4.14s/it]
|
||
64%|██████▍ | 600/938 [41:46<23:35, 4.19s/it]
|
||
|
||
{'loss': '0.2709', 'grad_norm': '0.4404', 'learning_rate': '3.48e-06', 'epoch': '1.279'}
|
||
|
||
64%|██████▍ | 600/938 [41:46<23:35, 4.19s/it]
|
||
64%|██████▍ | 601/938 [41:51<24:02, 4.28s/it]
|
||
64%|██████▍ | 602/938 [41:55<24:34, 4.39s/it]
|
||
64%|██████▍ | 603/938 [42:00<24:19, 4.36s/it]
|
||
64%|██████▍ | 604/938 [42:03<22:36, 4.06s/it]
|
||
64%|██████▍ | 605/938 [42:07<22:38, 4.08s/it]
|
||
65%|██████▍ | 606/938 [42:12<23:04, 4.17s/it]
|
||
65%|██████▍ | 607/938 [42:16<23:26, 4.25s/it]
|
||
65%|██████▍ | 608/938 [42:21<23:52, 4.34s/it]
|
||
65%|██████▍ | 609/938 [42:25<23:14, 4.24s/it]
|
||
65%|██████▌ | 610/938 [42:29<22:58, 4.20s/it]
|
||
|
||
{'loss': '0.2683', 'grad_norm': '0.5088', 'learning_rate': '3.304e-06', 'epoch': '1.301'}
|
||
|
||
65%|██████▌ | 610/938 [42:29<22:58, 4.20s/it]
|
||
65%|██████▌ | 611/938 [42:33<22:25, 4.11s/it]
|
||
65%|██████▌ | 612/938 [42:37<22:43, 4.18s/it]
|
||
65%|██████▌ | 613/938 [42:41<22:51, 4.22s/it]
|
||
65%|██████▌ | 614/938 [42:45<22:49, 4.23s/it]
|
||
66%|██████▌ | 615/938 [42:50<22:34, 4.19s/it]
|
||
66%|██████▌ | 616/938 [42:54<22:15, 4.15s/it]
|
||
66%|██████▌ | 617/938 [42:57<21:21, 3.99s/it]
|
||
66%|██████▌ | 618/938 [43:02<21:59, 4.12s/it]
|
||
66%|██████▌ | 619/938 [43:06<21:45, 4.09s/it]
|
||
66%|██████▌ | 620/938 [43:10<21:18, 4.02s/it]
|
||
|
||
{'loss': '0.2715', 'grad_norm': '0.4963', 'learning_rate': '3.13e-06', 'epoch': '1.322'}
|
||
|
||
66%|██████▌ | 620/938 [43:10<21:18, 4.02s/it]
|
||
66%|██████▌ | 621/938 [43:13<21:00, 3.98s/it]
|
||
66%|██████▋ | 622/938 [43:18<22:16, 4.23s/it]
|
||
66%|██████▋ | 623/938 [43:22<22:11, 4.23s/it]
|
||
67%|██████▋ | 624/938 [43:27<22:39, 4.33s/it]
|
||
67%|██████▋ | 625/938 [43:32<22:49, 4.37s/it]
|
||
67%|██████▋ | 626/938 [43:35<21:50, 4.20s/it]
|
||
67%|██████▋ | 627/938 [43:39<21:06, 4.07s/it]
|
||
67%|██████▋ | 628/938 [43:43<20:58, 4.06s/it]
|
||
67%|██████▋ | 629/938 [43:47<20:05, 3.90s/it]
|
||
67%|██████▋ | 630/938 [43:51<20:23, 3.97s/it]
|
||
|
||
{'loss': '0.2726', 'grad_norm': '0.4921', 'learning_rate': '2.958e-06', 'epoch': '1.343'}
|
||
|
||
67%|██████▋ | 630/938 [43:51<20:23, 3.97s/it]
|
||
67%|██████▋ | 631/938 [43:55<20:03, 3.92s/it]
|
||
67%|██████▋ | 632/938 [43:59<20:01, 3.93s/it]
|
||
67%|██████▋ | 633/938 [44:02<19:48, 3.90s/it]
|
||
68%|██████▊ | 634/938 [44:07<21:27, 4.24s/it]
|
||
68%|██████▊ | 635/938 [44:11<20:24, 4.04s/it]
|
||
68%|██████▊ | 636/938 [44:15<20:26, 4.06s/it]
|
||
68%|██████▊ | 637/938 [44:19<20:42, 4.13s/it]
|
||
68%|██████▊ | 638/938 [44:24<20:42, 4.14s/it]
|
||
68%|██████▊ | 639/938 [44:28<20:56, 4.20s/it]
|
||
68%|██████▊ | 640/938 [44:32<20:11, 4.07s/it]
|
||
|
||
{'loss': '0.2665', 'grad_norm': '0.4858', 'learning_rate': '2.79e-06', 'epoch': '1.365'}
|
||
|
||
68%|██████▊ | 640/938 [44:32<20:11, 4.07s/it]
|
||
68%|██████▊ | 641/938 [44:36<20:03, 4.05s/it]
|
||
68%|██████▊ | 642/938 [44:39<19:42, 3.99s/it]
|
||
69%|██████▊ | 643/938 [44:43<19:30, 3.97s/it]
|
||
69%|██████▊ | 644/938 [44:48<20:12, 4.12s/it]
|
||
69%|██████▉ | 645/938 [44:53<21:14, 4.35s/it]
|
||
69%|██████▉ | 646/938 [44:57<20:17, 4.17s/it]
|
||
69%|██████▉ | 647/938 [45:00<19:33, 4.03s/it]
|
||
69%|██████▉ | 648/938 [45:04<19:18, 4.00s/it]
|
||
69%|██████▉ | 649/938 [45:08<18:56, 3.93s/it]
|
||
69%|██████▉ | 650/938 [45:12<18:31, 3.86s/it]
|
||
|
||
{'loss': '0.2647', 'grad_norm': '0.4581', 'learning_rate': '2.625e-06', 'epoch': '1.386'}
|
||
|
||
69%|██████▉ | 650/938 [45:12<18:31, 3.86s/it]
|
||
69%|██████▉ | 651/938 [45:16<18:40, 3.90s/it]
|
||
70%|██████▉ | 652/938 [45:20<19:59, 4.19s/it]
|
||
70%|██████▉ | 653/938 [45:24<19:13, 4.05s/it]
|
||
70%|██████▉ | 654/938 [45:29<19:33, 4.13s/it]
|
||
70%|██████▉ | 655/938 [45:32<19:09, 4.06s/it]
|
||
70%|██████▉ | 656/938 [45:37<19:19, 4.11s/it]
|
||
70%|███████ | 657/938 [45:41<19:32, 4.17s/it]
|
||
70%|███████ | 658/938 [45:45<19:41, 4.22s/it]
|
||
70%|███████ | 659/938 [45:49<18:56, 4.07s/it]
|
||
70%|███████ | 660/938 [45:53<19:03, 4.11s/it]
|
||
|
||
{'loss': '0.2691', 'grad_norm': '0.4541', 'learning_rate': '2.462e-06', 'epoch': '1.407'}
|
||
|
||
70%|███████ | 660/938 [45:53<19:03, 4.11s/it]
|
||
70%|███████ | 661/938 [45:57<18:50, 4.08s/it]
|
||
71%|███████ | 662/938 [46:01<18:39, 4.06s/it]
|
||
71%|███████ | 663/938 [46:06<19:08, 4.18s/it]
|
||
71%|███████ | 664/938 [46:10<18:35, 4.07s/it]
|
||
71%|███████ | 665/938 [46:13<17:45, 3.90s/it]
|
||
71%|███████ | 666/938 [46:18<18:33, 4.09s/it]
|
||
71%|███████ | 667/938 [46:22<18:26, 4.08s/it]
|
||
71%|███████ | 668/938 [46:26<18:29, 4.11s/it]
|
||
71%|███████▏ | 669/938 [46:30<18:58, 4.23s/it]
|
||
71%|███████▏ | 670/938 [46:34<18:44, 4.20s/it]
|
||
|
||
{'loss': '0.2636', 'grad_norm': '0.5011', 'learning_rate': '2.304e-06', 'epoch': '1.429'}
|
||
|
||
71%|███████▏ | 670/938 [46:34<18:44, 4.20s/it]
|
||
72%|███████▏ | 671/938 [46:38<17:55, 4.03s/it]
|
||
72%|███████▏ | 672/938 [46:42<17:16, 3.90s/it]
|
||
72%|███████▏ | 673/938 [46:46<17:26, 3.95s/it]
|
||
72%|███████▏ | 674/938 [46:50<17:51, 4.06s/it]
|
||
72%|███████▏ | 675/938 [46:55<18:41, 4.27s/it]
|
||
72%|███████▏ | 676/938 [46:58<17:53, 4.10s/it]
|
||
72%|███████▏ | 677/938 [47:03<17:55, 4.12s/it]
|
||
72%|███████▏ | 678/938 [47:07<17:47, 4.11s/it]
|
||
72%|███████▏ | 679/938 [47:10<17:06, 3.96s/it]
|
||
72%|███████▏ | 680/938 [47:15<17:20, 4.03s/it]
|
||
|
||
{'loss': '0.2607', 'grad_norm': '0.4419', 'learning_rate': '2.149e-06', 'epoch': '1.45'}
|
||
|
||
72%|███████▏ | 680/938 [47:15<17:20, 4.03s/it]
|
||
73%|███████▎ | 681/938 [47:18<16:31, 3.86s/it]
|
||
73%|███████▎ | 682/938 [47:22<16:57, 3.97s/it]
|
||
73%|███████▎ | 683/938 [47:27<17:15, 4.06s/it]
|
||
73%|███████▎ | 684/938 [47:30<16:50, 3.98s/it]
|
||
73%|███████▎ | 685/938 [47:34<17:00, 4.03s/it]
|
||
73%|███████▎ | 686/938 [47:39<17:15, 4.11s/it]
|
||
73%|███████▎ | 687/938 [47:43<17:36, 4.21s/it]
|
||
73%|███████▎ | 688/938 [47:47<16:29, 3.96s/it]
|
||
73%|███████▎ | 689/938 [47:51<16:25, 3.96s/it]
|
||
74%|███████▎ | 690/938 [47:54<16:02, 3.88s/it]
|
||
|
||
{'loss': '0.2723', 'grad_norm': '0.4483', 'learning_rate': '1.998e-06', 'epoch': '1.471'}
|
||
|
||
74%|███████▎ | 690/938 [47:54<16:02, 3.88s/it]
|
||
74%|███████▎ | 691/938 [47:58<16:26, 3.99s/it]
|
||
74%|███████▍ | 692/938 [48:02<16:10, 3.94s/it]
|
||
74%|███████▍ | 693/938 [48:06<16:01, 3.92s/it]
|
||
74%|███████▍ | 694/938 [48:10<15:36, 3.84s/it]
|
||
74%|███████▍ | 695/938 [48:14<15:35, 3.85s/it]
|
||
74%|███████▍ | 696/938 [48:18<15:40, 3.89s/it]
|
||
74%|███████▍ | 697/938 [48:22<15:58, 3.98s/it]
|
||
74%|███████▍ | 698/938 [48:26<16:08, 4.03s/it]
|
||
75%|███████▍ | 699/938 [48:30<16:25, 4.12s/it]
|
||
75%|███████▍ | 700/938 [48:35<17:15, 4.35s/it]
|
||
|
||
{'loss': '0.2778', 'grad_norm': '0.4514', 'learning_rate': '1.851e-06', 'epoch': '1.493'}
|
||
|
||
75%|███████▍ | 700/938 [48:35<17:15, 4.35s/it]
|
||
75%|███████▍ | 701/938 [48:39<16:56, 4.29s/it]
|
||
75%|███████▍ | 702/938 [48:43<16:04, 4.09s/it]
|
||
75%|███████▍ | 703/938 [48:48<16:35, 4.24s/it]
|
||
75%|███████▌ | 704/938 [48:53<17:30, 4.49s/it]
|
||
75%|███████▌ | 705/938 [48:56<16:23, 4.22s/it]
|
||
75%|███████▌ | 706/938 [49:01<16:30, 4.27s/it]
|
||
75%|███████▌ | 707/938 [49:05<16:25, 4.26s/it]
|
||
75%|███████▌ | 708/938 [49:09<16:37, 4.34s/it]
|
||
76%|███████▌ | 709/938 [49:14<16:37, 4.36s/it]
|
||
76%|███████▌ | 710/938 [49:18<16:28, 4.33s/it]
|
||
|
||
{'loss': '0.2632', 'grad_norm': '0.4736', 'learning_rate': '1.709e-06', 'epoch': '1.514'}
|
||
|
||
76%|███████▌ | 710/938 [49:18<16:28, 4.33s/it]
|
||
76%|███████▌ | 711/938 [49:22<16:19, 4.32s/it]
|
||
76%|███████▌ | 712/938 [49:26<15:48, 4.19s/it]
|
||
76%|███████▌ | 713/938 [49:30<15:14, 4.06s/it]
|
||
76%|███████▌ | 714/938 [49:35<15:45, 4.22s/it]
|
||
76%|███████▌ | 715/938 [49:38<15:01, 4.04s/it]
|
||
76%|███████▋ | 716/938 [49:42<14:33, 3.93s/it]
|
||
76%|███████▋ | 717/938 [49:46<14:20, 3.90s/it]
|
||
77%|███████▋ | 718/938 [49:49<14:08, 3.85s/it]
|
||
77%|███████▋ | 719/938 [49:53<13:49, 3.79s/it]
|
||
77%|███████▋ | 720/938 [49:58<14:40, 4.04s/it]
|
||
|
||
{'loss': '0.2699', 'grad_norm': '0.4452', 'learning_rate': '1.571e-06', 'epoch': '1.535'}
|
||
|
||
77%|███████▋ | 720/938 [49:58<14:40, 4.04s/it]
|
||
77%|███████▋ | 721/938 [50:03<15:28, 4.28s/it]
|
||
77%|███████▋ | 722/938 [50:06<14:41, 4.08s/it]
|
||
77%|███████▋ | 723/938 [50:11<15:13, 4.25s/it]
|
||
77%|███████▋ | 724/938 [50:15<15:15, 4.28s/it]
|
||
77%|███████▋ | 725/938 [50:20<15:49, 4.46s/it]
|
||
77%|███████▋ | 726/938 [50:24<15:09, 4.29s/it]
|
||
78%|███████▊ | 727/938 [50:28<14:46, 4.20s/it]
|
||
78%|███████▊ | 728/938 [50:32<14:17, 4.08s/it]
|
||
78%|███████▊ | 729/938 [50:36<14:41, 4.22s/it]
|
||
78%|███████▊ | 730/938 [50:41<15:25, 4.45s/it]
|
||
|
||
{'loss': '0.2556', 'grad_norm': '0.4535', 'learning_rate': '1.438e-06', 'epoch': '1.557'}
|
||
|
||
78%|███████▊ | 730/938 [50:41<15:25, 4.45s/it]
|
||
78%|███████▊ | 731/938 [50:46<15:07, 4.38s/it]
|
||
78%|███████▊ | 732/938 [50:50<14:44, 4.29s/it]
|
||
78%|███████▊ | 733/938 [50:54<14:54, 4.36s/it]
|
||
78%|███████▊ | 734/938 [50:58<14:36, 4.30s/it]
|
||
78%|███████▊ | 735/938 [51:02<13:51, 4.10s/it]
|
||
78%|███████▊ | 736/938 [51:06<13:56, 4.14s/it]
|
||
79%|███████▊ | 737/938 [51:10<13:24, 4.00s/it]
|
||
79%|███████▊ | 738/938 [51:14<13:35, 4.08s/it]
|
||
79%|███████▉ | 739/938 [51:18<13:02, 3.93s/it]
|
||
79%|███████▉ | 740/938 [51:22<13:25, 4.07s/it]
|
||
|
||
{'loss': '0.2628', 'grad_norm': '0.4523', 'learning_rate': '1.31e-06', 'epoch': '1.578'}
|
||
|
||
79%|███████▉ | 740/938 [51:22<13:25, 4.07s/it]
|
||
79%|███████▉ | 741/938 [51:26<13:34, 4.13s/it]
|
||
79%|███████▉ | 742/938 [51:30<13:21, 4.09s/it]
|
||
79%|███████▉ | 743/938 [51:34<13:10, 4.06s/it]
|
||
79%|███████▉ | 744/938 [51:38<12:53, 3.99s/it]
|
||
79%|███████▉ | 745/938 [51:42<12:46, 3.97s/it]
|
||
80%|███████▉ | 746/938 [51:46<12:22, 3.87s/it]
|
||
80%|███████▉ | 747/938 [51:49<12:06, 3.80s/it]
|
||
80%|███████▉ | 748/938 [51:53<12:09, 3.84s/it]
|
||
80%|███████▉ | 749/938 [51:57<11:41, 3.71s/it]
|
||
80%|███████▉ | 750/938 [52:02<12:46, 4.08s/it]
|
||
|
||
{'loss': '0.2727', 'grad_norm': '0.4335', 'learning_rate': '1.187e-06', 'epoch': '1.599'}
|
||
|
||
80%|███████▉ | 750/938 [52:02<12:46, 4.08s/it]
|
||
80%|████████ | 751/938 [52:06<13:06, 4.21s/it]
|
||
80%|████████ | 752/938 [52:10<12:57, 4.18s/it]
|
||
80%|████████ | 753/938 [52:14<12:39, 4.10s/it]
|
||
80%|████████ | 754/938 [52:19<13:10, 4.30s/it]
|
||
80%|████████ | 755/938 [52:24<13:26, 4.41s/it]
|
||
81%|████████ | 756/938 [52:27<12:37, 4.16s/it]
|
||
81%|████████ | 757/938 [52:32<13:26, 4.46s/it]
|
||
81%|████████ | 758/938 [52:36<13:00, 4.34s/it]
|
||
81%|████████ | 759/938 [52:41<12:55, 4.33s/it]
|
||
81%|████████ | 760/938 [52:45<12:51, 4.34s/it]
|
||
|
||
{'loss': '0.2753', 'grad_norm': '0.4579', 'learning_rate': '1.069e-06', 'epoch': '1.621'}
|
||
|
||
81%|████████ | 760/938 [52:45<12:51, 4.34s/it]
|
||
81%|████████ | 761/938 [52:49<12:26, 4.22s/it]
|
||
81%|████████ | 762/938 [52:53<12:20, 4.21s/it]
|
||
81%|████████▏ | 763/938 [52:57<11:59, 4.11s/it]
|
||
81%|████████▏ | 764/938 [53:01<11:59, 4.13s/it]
|
||
82%|████████▏ | 765/938 [53:05<11:49, 4.10s/it]
|
||
82%|████████▏ | 766/938 [53:09<11:52, 4.14s/it]
|
||
82%|████████▏ | 767/938 [53:15<13:16, 4.66s/it]
|
||
82%|████████▏ | 768/938 [53:19<12:42, 4.49s/it]
|
||
82%|████████▏ | 769/938 [53:23<12:11, 4.33s/it]
|
||
82%|████████▏ | 770/938 [53:28<11:56, 4.27s/it]
|
||
|
||
{'loss': '0.258', 'grad_norm': '0.4879', 'learning_rate': '9.571e-07', 'epoch': '1.642'}
|
||
|
||
82%|████████▏ | 770/938 [53:28<11:56, 4.27s/it]
|
||
82%|████████▏ | 771/938 [53:32<12:10, 4.37s/it]
|
||
82%|████████▏ | 772/938 [53:36<11:49, 4.28s/it]
|
||
82%|████████▏ | 773/938 [53:40<11:13, 4.08s/it]
|
||
83%|████████▎ | 774/938 [53:44<11:29, 4.21s/it]
|
||
83%|████████▎ | 775/938 [53:48<11:17, 4.15s/it]
|
||
83%|████████▎ | 776/938 [53:52<11:00, 4.08s/it]
|
||
83%|████████▎ | 777/938 [53:57<11:13, 4.18s/it]
|
||
83%|████████▎ | 778/938 [54:02<11:47, 4.42s/it]
|
||
83%|████████▎ | 779/938 [54:06<11:53, 4.49s/it]
|
||
83%|████████▎ | 780/938 [54:10<11:26, 4.34s/it]
|
||
|
||
{'loss': '0.2685', 'grad_norm': '0.4277', 'learning_rate': '8.504e-07', 'epoch': '1.663'}
|
||
|
||
83%|████████▎ | 780/938 [54:10<11:26, 4.34s/it]
|
||
83%|████████▎ | 781/938 [54:15<11:38, 4.45s/it]
|
||
83%|████████▎ | 782/938 [54:19<11:13, 4.32s/it]
|
||
83%|████████▎ | 783/938 [54:23<11:00, 4.26s/it]
|
||
84%|████████▎ | 784/938 [54:27<10:50, 4.23s/it]
|
||
84%|████████▎ | 785/938 [54:32<11:28, 4.50s/it]
|
||
84%|████████▍ | 786/938 [54:36<10:56, 4.32s/it]
|
||
84%|████████▍ | 787/938 [54:40<10:12, 4.05s/it]
|
||
84%|████████▍ | 788/938 [54:44<10:23, 4.15s/it]
|
||
84%|████████▍ | 789/938 [54:48<10:06, 4.07s/it]
|
||
84%|████████▍ | 790/938 [54:52<10:03, 4.08s/it]
|
||
|
||
{'loss': '0.266', 'grad_norm': '0.4804', 'learning_rate': '7.495e-07', 'epoch': '1.685'}
|
||
|
||
84%|████████▍ | 790/938 [54:52<10:03, 4.08s/it]
|
||
84%|████████▍ | 791/938 [54:56<09:48, 4.01s/it]
|
||
84%|████████▍ | 792/938 [55:00<09:55, 4.08s/it]
|
||
85%|████████▍ | 793/938 [55:04<09:38, 3.99s/it]
|
||
85%|████████▍ | 794/938 [55:09<10:15, 4.27s/it]
|
||
85%|████████▍ | 795/938 [55:13<09:50, 4.13s/it]
|
||
85%|████████▍ | 796/938 [55:18<10:16, 4.34s/it]
|
||
85%|████████▍ | 797/938 [55:21<09:48, 4.17s/it]
|
||
85%|████████▌ | 798/938 [55:26<09:52, 4.23s/it]
|
||
85%|████████▌ | 799/938 [55:30<09:48, 4.23s/it]
|
||
85%|████████▌ | 800/938 [55:34<09:39, 4.20s/it]
|
||
|
||
{'loss': '0.2572', 'grad_norm': '0.4831', 'learning_rate': '6.544e-07', 'epoch': '1.706'}
|
||
|
||
85%|████████▌ | 800/938 [55:34<09:39, 4.20s/it]
|
||
85%|████████▌ | 801/938 [55:38<09:30, 4.16s/it]
|
||
86%|████████▌ | 802/938 [55:42<09:27, 4.18s/it]
|
||
86%|████████▌ | 803/938 [55:47<09:27, 4.21s/it]
|
||
86%|████████▌ | 804/938 [55:50<09:08, 4.10s/it]
|
||
86%|████████▌ | 805/938 [55:54<08:47, 3.96s/it]
|
||
86%|████████▌ | 806/938 [55:59<09:01, 4.10s/it]
|
||
86%|████████▌ | 807/938 [56:03<09:08, 4.19s/it]
|
||
86%|████████▌ | 808/938 [56:08<09:25, 4.35s/it]
|
||
86%|████████▌ | 809/938 [56:12<09:12, 4.28s/it]
|
||
86%|████████▋ | 810/938 [56:15<08:38, 4.05s/it]
|
||
|
||
{'loss': '0.2663', 'grad_norm': '0.5086', 'learning_rate': '5.654e-07', 'epoch': '1.727'}
|
||
|
||
86%|████████▋ | 810/938 [56:15<08:38, 4.05s/it]
|
||
86%|████████▋ | 811/938 [56:20<08:46, 4.15s/it]
|
||
87%|████████▋ | 812/938 [56:24<08:59, 4.28s/it]
|
||
87%|████████▋ | 813/938 [56:28<08:51, 4.25s/it]
|
||
87%|████████▋ | 814/938 [56:33<08:52, 4.29s/it]
|
||
87%|████████▋ | 815/938 [56:37<08:38, 4.22s/it]
|
||
87%|████████▋ | 816/938 [56:41<08:43, 4.29s/it]
|
||
87%|████████▋ | 817/938 [56:46<08:42, 4.32s/it]
|
||
87%|████████▋ | 818/938 [56:51<09:20, 4.67s/it]
|
||
87%|████████▋ | 819/938 [56:56<09:12, 4.64s/it]
|
||
87%|████████▋ | 820/938 [57:00<08:35, 4.37s/it]
|
||
|
||
{'loss': '0.2616', 'grad_norm': '0.478', 'learning_rate': '4.825e-07', 'epoch': '1.749'}
|
||
|
||
87%|████████▋ | 820/938 [57:00<08:35, 4.37s/it]
|
||
88%|████████▊ | 821/938 [57:05<08:53, 4.56s/it]
|
||
88%|████████▊ | 822/938 [57:08<08:05, 4.19s/it]
|
||
88%|████████▊ | 823/938 [57:12<07:51, 4.10s/it]
|
||
88%|████████▊ | 824/938 [57:16<07:42, 4.06s/it]
|
||
88%|████████▊ | 825/938 [57:20<07:34, 4.02s/it]
|
||
88%|████████▊ | 826/938 [57:24<07:33, 4.05s/it]
|
||
88%|████████▊ | 827/938 [57:28<07:27, 4.03s/it]
|
||
88%|████████▊ | 828/938 [57:32<07:31, 4.10s/it]
|
||
88%|████████▊ | 829/938 [57:36<07:16, 4.00s/it]
|
||
88%|████████▊ | 830/938 [57:40<07:15, 4.03s/it]
|
||
|
||
{'loss': '0.2693', 'grad_norm': '0.4145', 'learning_rate': '4.059e-07', 'epoch': '1.77'}
|
||
|
||
88%|████████▊ | 830/938 [57:40<07:15, 4.03s/it]
|
||
89%|████████▊ | 831/938 [57:44<07:20, 4.11s/it]
|
||
89%|████████▊ | 832/938 [57:48<06:56, 3.93s/it]
|
||
89%|████████▉ | 833/938 [57:52<07:15, 4.15s/it]
|
||
89%|████████▉ | 834/938 [57:56<07:01, 4.05s/it]
|
||
89%|████████▉ | 835/938 [58:01<07:07, 4.15s/it]
|
||
89%|████████▉ | 836/938 [58:05<07:13, 4.25s/it]
|
||
89%|████████▉ | 837/938 [58:09<07:04, 4.20s/it]
|
||
89%|████████▉ | 838/938 [58:13<06:45, 4.05s/it]
|
||
89%|████████▉ | 839/938 [58:17<06:43, 4.08s/it]
|
||
90%|████████▉ | 840/938 [58:21<06:48, 4.17s/it]
|
||
|
||
{'loss': '0.2727', 'grad_norm': '0.4312', 'learning_rate': '3.357e-07', 'epoch': '1.791'}
|
||
|
||
90%|████████▉ | 840/938 [58:21<06:48, 4.17s/it]
|
||
90%|████████▉ | 841/938 [58:25<06:36, 4.08s/it]
|
||
90%|████████▉ | 842/938 [58:30<06:53, 4.31s/it]
|
||
90%|████████▉ | 843/938 [58:34<06:36, 4.18s/it]
|
||
90%|████████▉ | 844/938 [58:38<06:27, 4.12s/it]
|
||
90%|█████████ | 845/938 [58:42<06:24, 4.14s/it]
|
||
90%|█████████ | 846/938 [58:46<06:08, 4.00s/it]
|
||
90%|█████████ | 847/938 [58:50<06:16, 4.14s/it]
|
||
90%|█████████ | 848/938 [58:54<06:14, 4.16s/it]
|
||
91%|█████████ | 849/938 [58:58<05:56, 4.00s/it]
|
||
91%|█████████ | 850/938 [59:02<05:47, 3.95s/it]
|
||
|
||
{'loss': '0.276', 'grad_norm': '0.465', 'learning_rate': '2.719e-07', 'epoch': '1.813'}
|
||
|
||
91%|█████████ | 850/938 [59:02<05:47, 3.95s/it]
|
||
91%|█████████ | 851/938 [59:06<05:51, 4.04s/it]
|
||
91%|█████████ | 852/938 [59:11<05:56, 4.14s/it]
|
||
91%|█████████ | 853/938 [59:15<06:01, 4.26s/it]
|
||
91%|█████████ | 854/938 [59:19<05:51, 4.19s/it]
|
||
91%|█████████ | 855/938 [59:23<05:42, 4.12s/it]
|
||
91%|█████████▏| 856/938 [59:27<05:27, 3.99s/it]
|
||
91%|█████████▏| 857/938 [59:31<05:32, 4.10s/it]
|
||
91%|█████████▏| 858/938 [59:35<05:23, 4.04s/it]
|
||
92%|█████████▏| 859/938 [59:39<05:10, 3.93s/it]
|
||
92%|█████████▏| 860/938 [59:43<05:08, 3.96s/it]
|
||
|
||
{'loss': '0.2737', 'grad_norm': '0.4531', 'learning_rate': '2.146e-07', 'epoch': '1.834'}
|
||
|
||
92%|█████████▏| 860/938 [59:43<05:08, 3.96s/it]
|
||
92%|█████████▏| 861/938 [59:47<05:19, 4.14s/it]
|
||
92%|█████████▏| 862/938 [59:51<05:13, 4.13s/it]
|
||
92%|█████████▏| 863/938 [59:56<05:27, 4.37s/it]
|
||
92%|█████████▏| 864/938 [1:00:00<05:16, 4.28s/it]
|
||
92%|█████████▏| 865/938 [1:00:04<05:08, 4.22s/it]
|
||
92%|█████████▏| 866/938 [1:00:09<05:02, 4.20s/it]
|
||
92%|█████████▏| 867/938 [1:00:13<05:02, 4.26s/it]
|
||
93%|█████████▎| 868/938 [1:00:18<05:19, 4.57s/it]
|
||
93%|█████████▎| 869/938 [1:00:23<05:21, 4.65s/it]
|
||
93%|█████████▎| 870/938 [1:00:28<05:18, 4.69s/it]
|
||
|
||
{'loss': '0.2506', 'grad_norm': '0.4372', 'learning_rate': '1.64e-07', 'epoch': '1.855'}
|
||
|
||
93%|█████████▎| 870/938 [1:00:28<05:18, 4.69s/it]
|
||
93%|█████████▎| 871/938 [1:00:32<05:06, 4.57s/it]
|
||
93%|█████████▎| 872/938 [1:00:36<04:51, 4.41s/it]
|
||
93%|█████████▎| 873/938 [1:00:41<04:44, 4.38s/it]
|
||
93%|█████████▎| 874/938 [1:00:45<04:34, 4.28s/it]
|
||
93%|█████████▎| 875/938 [1:00:49<04:24, 4.19s/it]
|
||
93%|█████████▎| 876/938 [1:00:53<04:14, 4.11s/it]
|
||
93%|█████████▎| 877/938 [1:00:57<04:17, 4.23s/it]
|
||
94%|█████████▎| 878/938 [1:01:01<04:01, 4.03s/it]
|
||
94%|█████████▎| 879/938 [1:01:04<03:51, 3.92s/it]
|
||
94%|█████████▍| 880/938 [1:01:08<03:46, 3.90s/it]
|
||
|
||
{'loss': '0.2735', 'grad_norm': '0.464', 'learning_rate': '1.201e-07', 'epoch': '1.877'}
|
||
|
||
94%|█████████▍| 880/938 [1:01:08<03:46, 3.90s/it]
|
||
94%|█████████▍| 881/938 [1:01:12<03:45, 3.95s/it]
|
||
94%|█████████▍| 882/938 [1:01:16<03:39, 3.92s/it]
|
||
94%|█████████▍| 883/938 [1:01:20<03:40, 4.01s/it]
|
||
94%|█████████▍| 884/938 [1:01:25<03:42, 4.12s/it]
|
||
94%|█████████▍| 885/938 [1:01:29<03:37, 4.10s/it]
|
||
94%|█████████▍| 886/938 [1:01:33<03:43, 4.30s/it]
|
||
95%|█████████▍| 887/938 [1:01:38<03:40, 4.32s/it]
|
||
95%|█████████▍| 888/938 [1:01:41<03:24, 4.09s/it]
|
||
95%|█████████▍| 889/938 [1:01:45<03:18, 4.06s/it]
|
||
95%|█████████▍| 890/938 [1:01:49<03:11, 3.99s/it]
|
||
|
||
{'loss': '0.2667', 'grad_norm': '0.4555', 'learning_rate': '8.294e-08', 'epoch': '1.898'}
|
||
|
||
95%|█████████▍| 890/938 [1:01:49<03:11, 3.99s/it]
|
||
95%|█████████▍| 891/938 [1:01:54<03:19, 4.24s/it]
|
||
95%|█████████▌| 892/938 [1:01:58<03:05, 4.02s/it]
|
||
95%|█████████▌| 893/938 [1:02:01<02:52, 3.84s/it]
|
||
95%|█████████▌| 894/938 [1:02:05<02:55, 3.98s/it]
|
||
95%|█████████▌| 895/938 [1:02:09<02:47, 3.89s/it]
|
||
96%|█████████▌| 896/938 [1:02:13<02:49, 4.04s/it]
|
||
96%|█████████▌| 897/938 [1:02:18<02:51, 4.19s/it]
|
||
96%|█████████▌| 898/938 [1:02:22<02:46, 4.16s/it]
|
||
96%|█████████▌| 899/938 [1:02:27<02:50, 4.36s/it]
|
||
96%|█████████▌| 900/938 [1:02:31<02:41, 4.26s/it]
|
||
|
||
{'loss': '0.2759', 'grad_norm': '0.4674', 'learning_rate': '5.259e-08', 'epoch': '1.919'}
|
||
|
||
96%|█████████▌| 900/938 [1:02:31<02:41, 4.26s/it]
|
||
96%|█████████▌| 901/938 [1:02:35<02:36, 4.22s/it]
|
||
96%|█████████▌| 902/938 [1:02:39<02:27, 4.11s/it]
|
||
96%|█████████▋| 903/938 [1:02:43<02:22, 4.07s/it]
|
||
96%|█████████▋| 904/938 [1:02:46<02:13, 3.93s/it]
|
||
96%|█████████▋| 905/938 [1:02:51<02:13, 4.06s/it]
|
||
97%|█████████▋| 906/938 [1:02:55<02:11, 4.12s/it]
|
||
97%|█████████▋| 907/938 [1:02:59<02:04, 4.01s/it]
|
||
97%|█████████▋| 908/938 [1:03:03<01:59, 3.97s/it]
|
||
97%|█████████▋| 909/938 [1:03:07<02:00, 4.16s/it]
|
||
97%|█████████▋| 910/938 [1:03:12<02:00, 4.32s/it]
|
||
|
||
{'loss': '0.2717', 'grad_norm': '0.464', 'learning_rate': '2.91e-08', 'epoch': '1.941'}
|
||
|
||
97%|█████████▋| 910/938 [1:03:12<02:00, 4.32s/it]
|
||
97%|█████████▋| 911/938 [1:03:15<01:50, 4.08s/it]
|
||
97%|█████████▋| 912/938 [1:03:19<01:43, 4.00s/it]
|
||
97%|█████████▋| 913/938 [1:03:23<01:37, 3.91s/it]
|
||
97%|█████████▋| 914/938 [1:03:27<01:31, 3.82s/it]
|
||
98%|█████████▊| 915/938 [1:03:30<01:28, 3.85s/it]
|
||
98%|█████████▊| 916/938 [1:03:34<01:24, 3.85s/it]
|
||
98%|█████████▊| 917/938 [1:03:38<01:17, 3.71s/it]
|
||
98%|█████████▊| 918/938 [1:03:42<01:17, 3.87s/it]
|
||
98%|█████████▊| 919/938 [1:03:46<01:14, 3.93s/it]
|
||
98%|█████████▊| 920/938 [1:03:50<01:13, 4.09s/it]
|
||
|
||
{'loss': '0.2673', 'grad_norm': '0.4542', 'learning_rate': '1.25e-08', 'epoch': '1.962'}
|
||
|
||
98%|█████████▊| 920/938 [1:03:50<01:13, 4.09s/it]
|
||
98%|█████████▊| 921/938 [1:03:54<01:08, 4.02s/it]
|
||
98%|█████████▊| 922/938 [1:03:59<01:07, 4.22s/it]
|
||
98%|█████████▊| 923/938 [1:04:03<01:03, 4.21s/it]
|
||
99%|█████████▊| 924/938 [1:04:07<00:59, 4.23s/it]
|
||
99%|█████████▊| 925/938 [1:04:12<00:55, 4.26s/it]
|
||
99%|█████████▊| 926/938 [1:04:16<00:50, 4.24s/it]
|
||
99%|█████████▉| 927/938 [1:04:19<00:44, 4.01s/it]
|
||
99%|█████████▉| 928/938 [1:04:24<00:40, 4.06s/it]
|
||
99%|█████████▉| 929/938 [1:04:27<00:35, 3.99s/it]
|
||
99%|█████████▉| 930/938 [1:04:31<00:31, 3.99s/it]
|
||
|
||
{'loss': '0.2772', 'grad_norm': '0.479', 'learning_rate': '2.805e-09', 'epoch': '1.983'}
|
||
|
||
99%|█████████▉| 930/938 [1:04:31<00:31, 3.99s/it]
|
||
99%|█████████▉| 931/938 [1:04:35<00:27, 3.99s/it]
|
||
99%|█████████▉| 932/938 [1:04:40<00:25, 4.22s/it]
|
||
99%|█████████▉| 933/938 [1:04:45<00:21, 4.35s/it]
|
||
100%|█████████▉| 934/938 [1:04:49<00:17, 4.27s/it]
|
||
100%|█████████▉| 935/938 [1:04:53<00:12, 4.15s/it]
|
||
100%|█████████▉| 936/938 [1:04:57<00:08, 4.17s/it]
|
||
100%|█████████▉| 937/938 [1:05:01<00:04, 4.15s/it]
|
||
100%|██████████| 938/938 [1:05:04<00:00, 3.74s/it][INFO|trainer.py:3797] 2026-04-29 16:49:43,034 >> Saving model checkpoint to saves/qwen3_1.7b/math/fft/checkpoint-938
|
||
[INFO|configuration_utils.py:432] 2026-04-29 16:49:43,043 >> Configuration saved in saves/qwen3_1.7b/math/fft/checkpoint-938/config.json
|
||
[INFO|configuration_utils.py:803] 2026-04-29 16:49:43,046 >> Configuration saved in saves/qwen3_1.7b/math/fft/checkpoint-938/generation_config.json
|
||
|
||
|
||
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s][A
|
||
|
||
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.30s/it][A
|
||
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.30s/it]
|
||
[INFO|modeling_utils.py:3380] 2026-04-29 16:49:49,356 >> Model weights saved in saves/qwen3_1.7b/math/fft/checkpoint-938/model.safetensors
|
||
[INFO|tokenization_utils_base.py:3224] 2026-04-29 16:49:49,361 >> chat template saved in saves/qwen3_1.7b/math/fft/checkpoint-938/chat_template.jinja
|
||
[INFO|tokenization_utils_base.py:2078] 2026-04-29 16:49:49,365 >> tokenizer config file saved in saves/qwen3_1.7b/math/fft/checkpoint-938/tokenizer_config.json
|
||
[INFO|trainer.py:1863] 2026-04-29 16:49:49,592 >>
|
||
|
||
Training completed. Do not forget to share your model on huggingface.co/models =)
|
||
|
||
|
||
|
||
|
||
{'train_runtime': '3915', 'train_samples_per_second': '30.65', 'train_steps_per_second': '0.24', 'train_loss': '0.2964', 'epoch': '2'}
|
||
|
||
100%|██████████| 938/938 [1:05:13<00:00, 3.74s/it]
|
||
100%|██████████| 938/938 [1:05:13<00:00, 4.17s/it]
|
||
[INFO|trainer.py:3797] 2026-04-29 16:49:51,371 >> Saving model checkpoint to saves/qwen3_1.7b/math/fft
|
||
[INFO|configuration_utils.py:432] 2026-04-29 16:49:51,380 >> Configuration saved in saves/qwen3_1.7b/math/fft/config.json
|
||
[INFO|configuration_utils.py:803] 2026-04-29 16:49:51,384 >> Configuration saved in saves/qwen3_1.7b/math/fft/generation_config.json
|
||
|
||
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s]
|
||
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.43s/it]
|
||
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.43s/it]
|
||
[INFO|modeling_utils.py:3380] 2026-04-29 16:49:57,827 >> Model weights saved in saves/qwen3_1.7b/math/fft/model.safetensors
|
||
[INFO|tokenization_utils_base.py:3224] 2026-04-29 16:49:57,832 >> chat template saved in saves/qwen3_1.7b/math/fft/chat_template.jinja
|
||
[INFO|tokenization_utils_base.py:2078] 2026-04-29 16:49:57,836 >> tokenizer config file saved in saves/qwen3_1.7b/math/fft/tokenizer_config.json
|
||
ywang29-p4d-debug-worker-0:2470949:2470949 [2] NCCL INFO comm 0x56028de3b5a0 rank 2 nranks 8 cudaDev 2 busId 201c0 - Destroy COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO comm 0x5612df7ca6e0 rank 3 nranks 8 cudaDev 3 busId 201d0 - Destroy COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO comm 0x55c02c235cf0 rank 5 nranks 8 cudaDev 5 busId 901d0 - Destroy COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO comm 0x5647135d02c0 rank 4 nranks 8 cudaDev 4 busId 901c0 - Destroy COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470954:2470954 [6] NCCL INFO comm 0x560722415b60 rank 6 nranks 8 cudaDev 6 busId a01c0 - Destroy COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO comm 0x55c0201d0a60 rank 5 nranks 8 cudaDev 5 busId 901d0 - Destroy COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO comm 0x5647078795e0 rank 4 nranks 8 cudaDev 4 busId 901c0 - Destroy COMPLETE
|
||
ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO comm 0x5612d38dcbf0 rank 3 nranks 8 cudaDev 3 busId 201d0 - Destroy COMPLETE
|
||
[INFO|trainer.py:3797] 2026-04-29 16:50:00,902 >> Saving model checkpoint to saves/qwen3_1.7b/math/fft
|
||
[INFO|configuration_utils.py:432] 2026-04-29 16:50:00,911 >> Configuration saved in saves/qwen3_1.7b/math/fft/config.json
|
||
[INFO|configuration_utils.py:803] 2026-04-29 16:50:00,913 >> Configuration saved in saves/qwen3_1.7b/math/fft/generation_config.json
|
||
|
||
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s]ywang29-p4d-debug-worker-0:2470951:2470951 [3] NCCL INFO ENV/Plugin: Closing env plugin ncclEnvDefault
|
||
ywang29-p4d-debug-worker-0:2470952:2470952 [4] NCCL INFO ENV/Plugin: Closing env plugin ncclEnvDefault
|
||
ywang29-p4d-debug-worker-0:2470953:2470953 [5] NCCL INFO ENV/Plugin: Closing env plugin ncclEnvDefault
|
||
|
||
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.57s/it]
|
||
Writing model shards: 100%|██████████| 1/1 [00:06<00:00, 6.57s/it]
|
||
[INFO|modeling_utils.py:3380] 2026-04-29 16:50:07,495 >> Model weights saved in saves/qwen3_1.7b/math/fft/model.safetensors
|
||
[INFO|tokenization_utils_base.py:3224] 2026-04-29 16:50:07,499 >> chat template saved in saves/qwen3_1.7b/math/fft/chat_template.jinja
|
||
[INFO|tokenization_utils_base.py:2078] 2026-04-29 16:50:07,503 >> tokenizer config file saved in saves/qwen3_1.7b/math/fft/tokenizer_config.json
|
||
[INFO|modelcard.py:266] 2026-04-29 16:50:07,981 >> Dropping the following result as it does not have all the necessary fields:
|
||
{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
|
||
|
||
Processing Files (0 / 0) : | | 0.00B / 0.00B
|
||
|
||
New Data Upload : | | 0.00B / 0.00B [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
Processing Files (1 / 1) : 0%| | 11.4MB / 4.07GB, 57.0MB/s
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 0%| | 29.6kB / 4.06GB [A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 0%| | 29.6kB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 0%| | 11.5MB / 4.07GB, 11.5MB/s
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 0%| | 29.6kB / 4.06GB [A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 0%| | 29.6kB / 4.06GB [A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 0%| | 636kB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 0%| | 12.1MB / 4.07GB, 7.54MB/s
|
||
|
||
New Data Upload : 0%| | 607kB / 134MB, 379kB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 0%| | 1.24MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 0%| | 12.7MB / 4.07GB, 7.04MB/s
|
||
|
||
New Data Upload : 1%| | 1.21MB / 134MB, 674kB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 0%| | 7.91MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 0%| | 19.3MB / 4.07GB, 9.67MB/s
|
||
|
||
New Data Upload : 6%|▌ | 7.88MB / 134MB, 3.94MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 1%| | 21.3MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 1%| | 32.7MB / 4.07GB, 14.9MB/s
|
||
|
||
New Data Upload : 16%|█▌ | 21.2MB / 134MB, 9.65MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 1%| | 33.4MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 1%| | 44.8MB / 4.07GB, 18.7MB/s
|
||
|
||
New Data Upload : 25%|██▍ | 33.4MB / 134MB, 13.9MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 1%| | 46.1MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 1%|▏ | 57.5MB / 4.07GB, 22.1MB/s
|
||
|
||
New Data Upload : 34%|███▍ | 46.1MB / 134MB, 17.7MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 1%|▏ | 58.3MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 2%|▏ | 69.7MB / 4.07GB, 24.9MB/s
|
||
|
||
New Data Upload : 43%|████▎ | 58.2MB / 134MB, 20.8MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 2%|▏ | 78.2MB / 4.07GB, 26.1MB/s
|
||
|
||
New Data Upload : 50%|████▉ | 66.7MB / 134MB, 22.2MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB [A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB [A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 2%|▏ | 66.7MB / 4.06GB [A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 2%|▏ | 67.1MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 2%|▏ | 78.5MB / 4.07GB, 20.7MB/s
|
||
|
||
New Data Upload : 50%|████▉ | 67.1MB / 134MB, 17.7MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 2%|▏ | 67.7MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 2%|▏ | 79.1MB / 4.07GB, 19.8MB/s
|
||
|
||
New Data Upload : 50%|█████ | 67.7MB / 134MB, 16.9MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 2%|▏ | 71.4MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 2%|▏ | 82.8MB / 4.07GB, 19.7MB/s
|
||
|
||
New Data Upload : 35%|███▌ | 71.3MB / 201MB, 17.0MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 2%|▏ | 78.0MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 2%|▏ | 89.4MB / 4.07GB, 20.3MB/s
|
||
|
||
New Data Upload : 39%|███▉ | 78.0MB / 201MB, 17.7MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 2%|▏ | 98.0MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 3%|▎ | 109MB / 4.07GB, 23.8MB/s
|
||
|
||
New Data Upload : 37%|███▋ | 98.0MB / 268MB, 21.3MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 3%|▎ | 122MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 3%|▎ | 133MB / 4.07GB, 27.7MB/s
|
||
|
||
New Data Upload : 45%|████▌ | 122MB / 268MB, 25.3MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 4%|▎ | 146MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 4%|▍ | 157MB / 4.07GB, 31.5MB/s
|
||
|
||
New Data Upload : 43%|████▎ | 146MB / 335MB, 29.2MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 4%|▍ | 173MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 5%|▍ | 185MB / 4.07GB, 35.5MB/s
|
||
|
||
New Data Upload : 52%|█████▏ | 173MB / 335MB, 33.3MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 5%|▍ | 194MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 5%|▌ | 206MB / 4.07GB, 38.1MB/s
|
||
|
||
New Data Upload : 58%|█████▊ | 194MB / 335MB, 36.0MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 6%|▌ | 225MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 6%|▌ | 236MB / 4.07GB, 42.2MB/s
|
||
|
||
New Data Upload : 56%|█████▌ | 225MB / 403MB, 40.1MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 6%|▌ | 251MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 6%|▋ | 263MB / 4.07GB, 45.3MB/s
|
||
|
||
New Data Upload : 62%|██████▏ | 251MB / 403MB, 43.3MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 7%|▋ | 280MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 7%|▋ | 292MB / 4.07GB, 48.6MB/s
|
||
|
||
New Data Upload : 70%|██████▉ | 280MB / 403MB, 46.7MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 7%|▋ | 304MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 8%|▊ | 315MB / 4.07GB, 50.9MB/s
|
||
|
||
New Data Upload : 65%|██████▍ | 304MB / 470MB, 49.0MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 8%|▊ | 333MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 8%|▊ | 345MB / 4.07GB, 53.9MB/s
|
||
|
||
New Data Upload : 71%|███████ | 333MB / 470MB, 52.1MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 9%|▉ | 356MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 9%|▉ | 368MB / 4.07GB, 55.7MB/s
|
||
|
||
New Data Upload : 66%|██████▋ | 356MB / 537MB, 54.0MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 9%|▉ | 377MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 10%|▉ | 389MB / 4.07GB, 57.1MB/s
|
||
|
||
New Data Upload : 70%|███████ | 377MB / 537MB, 55.5MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 10%|█ | 407MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 10%|█ | 418MB / 4.07GB, 59.8MB/s
|
||
|
||
New Data Upload : 67%|██████▋ | 407MB / 604MB, 58.1MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 11%|█ | 447MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 11%|█ | 458MB / 4.07GB, 63.7MB/s
|
||
|
||
New Data Upload : 74%|███████▍ | 447MB / 604MB, 62.1MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 12%|█▏ | 480MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 12%|█▏ | 492MB / 4.07GB, 66.4MB/s
|
||
|
||
New Data Upload : 72%|███████▏ | 480MB / 671MB, 64.9MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 13%|█▎ | 516MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 13%|█▎ | 528MB / 4.07GB, 69.5MB/s
|
||
|
||
New Data Upload : 77%|███████▋ | 516MB / 671MB, 67.9MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 14%|█▍ | 563MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 14%|█▍ | 575MB / 4.07GB, 73.7MB/s
|
||
|
||
New Data Upload : 76%|███████▋ | 563MB / 738MB, 72.2MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 15%|█▍ | 594MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 15%|█▍ | 605MB / 4.07GB, 75.7MB/s
|
||
|
||
New Data Upload : 81%|████████ | 594MB / 738MB, 74.2MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 15%|█▌ | 621MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 16%|█▌ | 632MB / 4.07GB, 77.1MB/s
|
||
|
||
New Data Upload : 77%|███████▋ | 621MB / 805MB, 75.7MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 16%|█▌ | 650MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 16%|█▌ | 662MB / 4.07GB, 78.8MB/s
|
||
|
||
New Data Upload : 81%|████████ | 650MB / 805MB, 77.4MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 17%|█▋ | 690MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 17%|█▋ | 702MB / 4.07GB, 81.6MB/s
|
||
|
||
New Data Upload : 86%|████████▌ | 690MB / 805MB, 80.3MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 17%|█▋ | 708MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 18%|█▊ | 720MB / 4.07GB, 81.8MB/s
|
||
|
||
New Data Upload : 81%|████████ | 708MB / 872MB, 80.5MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 18%|█▊ | 743MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 19%|█▊ | 754MB / 4.07GB, 83.8MB/s
|
||
|
||
New Data Upload : 79%|███████▉ | 743MB / 939MB, 82.5MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 19%|█▉ | 774MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 19%|█▉ | 786MB / 4.07GB, 85.4MB/s
|
||
|
||
New Data Upload : 82%|████████▏ | 774MB / 939MB, 84.2MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 20%|█▉ | 811MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 20%|██ | 822MB / 4.07GB, 87.5MB/s
|
||
|
||
New Data Upload : 81%|████████ | 811MB / 1.01GB, 86.3MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 21%|██ | 846MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 21%|██ | 857MB / 4.07GB, 89.3MB/s
|
||
|
||
New Data Upload : 79%|███████▉ | 845MB / 1.07GB, 88.1MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 22%|██▏ | 894MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 22%|██▏ | 905MB / 4.07GB, 92.4MB/s
|
||
|
||
New Data Upload : 83%|████████▎ | 894MB / 1.07GB, 91.2MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 23%|██▎ | 939MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 23%|██▎ | 950MB / 4.07GB, 95.0MB/s
|
||
|
||
New Data Upload : 82%|████████▏ | 939MB / 1.14GB, 93.9MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 24%|██▍ | 978MB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 24%|██▍ | 990MB / 4.07GB, 97.0MB/s
|
||
|
||
New Data Upload : 86%|████████▌ | 978MB / 1.14GB, 95.9MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 25%|██▌ | 1.02GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 25%|██▌ | 1.03GB / 4.07GB, 99.7MB/s
|
||
|
||
New Data Upload : 89%|████████▉ | 1.02GB / 1.14GB, 99.7MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 26%|██▌ | 1.05GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 26%|██▌ | 1.06GB / 4.07GB, 103MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 1.05GB / 1.14GB, 103MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 26%|██▌ | 1.06GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 26%|██▋ | 1.07GB / 4.07GB, 104MB/s
|
||
|
||
New Data Upload : 93%|█████████▎| 1.06GB / 1.14GB, 104MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 27%|██▋ | 1.08GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 27%|██▋ | 1.09GB / 4.07GB, 106MB/s
|
||
|
||
New Data Upload : 95%|█████████▍| 1.08GB / 1.14GB, 106MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 27%|██▋ | 1.09GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 27%|██▋ | 1.10GB / 4.07GB, 107MB/s
|
||
|
||
New Data Upload : 96%|█████████▌| 1.09GB / 1.14GB, 107MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 27%|██▋ | 1.11GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 27%|██▋ | 1.12GB / 4.07GB, 108MB/s
|
||
|
||
New Data Upload : 97%|█████████▋| 1.11GB / 1.14GB, 108MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 28%|██▊ | 1.12GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 28%|██▊ | 1.13GB / 4.07GB, 110MB/s
|
||
|
||
New Data Upload : 98%|█████████▊| 1.12GB / 1.14GB, 110MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 29%|██▉ | 1.18GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 29%|██▉ | 1.19GB / 4.07GB, 116MB/s
|
||
|
||
New Data Upload : 99%|█████████▉| 1.13GB / 1.14GB, 111MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 29%|██▉ | 1.19GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 30%|██▉ | 1.21GB / 4.07GB, 117MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 1.14GB / 1.21GB, 112MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 29%|██▉ | 1.19GB / 4.06GB [A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 29%|██▉ | 1.20GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 30%|██▉ | 1.21GB / 4.07GB, 115MB/s
|
||
|
||
New Data Upload : 90%|████████▉ | 1.14GB / 1.27GB, 110MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 30%|██▉ | 1.20GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 30%|██▉ | 1.21GB / 4.07GB, 115MB/s
|
||
|
||
New Data Upload : 86%|████████▌ | 1.15GB / 1.34GB, 109MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 30%|██▉ | 1.21GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 30%|███ | 1.22GB / 4.07GB, 114MB/s
|
||
|
||
New Data Upload : 86%|████████▋ | 1.16GB / 1.34GB, 109MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 31%|███ | 1.25GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 31%|███ | 1.26GB / 4.07GB, 116MB/s
|
||
|
||
New Data Upload : 85%|████████▍ | 1.19GB / 1.41GB, 111MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 32%|███▏ | 1.29GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 32%|███▏ | 1.30GB / 4.07GB, 120MB/s
|
||
|
||
New Data Upload : 88%|████████▊ | 1.23GB / 1.41GB, 115MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 33%|███▎ | 1.32GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 33%|███▎ | 1.33GB / 4.07GB, 123MB/s
|
||
|
||
New Data Upload : 86%|████████▌ | 1.27GB / 1.48GB, 118MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 34%|███▎ | 1.37GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 34%|███▍ | 1.38GB / 4.07GB, 128MB/s
|
||
|
||
New Data Upload : 85%|████████▌ | 1.31GB / 1.54GB, 122MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 35%|███▍ | 1.40GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 35%|███▍ | 1.42GB / 4.07GB, 131MB/s
|
||
|
||
New Data Upload : 88%|████████▊ | 1.35GB / 1.54GB, 126MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 35%|███▌ | 1.44GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 36%|███▌ | 1.45GB / 4.07GB, 134MB/s
|
||
|
||
New Data Upload : 86%|████████▌ | 1.38GB / 1.61GB, 129MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 36%|███▌ | 1.46GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 36%|███▌ | 1.48GB / 4.07GB, 137MB/s
|
||
|
||
New Data Upload : 88%|████████▊ | 1.41GB / 1.61GB, 132MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 37%|███▋ | 1.49GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 37%|███▋ | 1.50GB / 4.07GB, 139MB/s
|
||
|
||
New Data Upload : 86%|████████▌ | 1.44GB / 1.68GB, 134MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 38%|███▊ | 1.53GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 38%|███▊ | 1.54GB / 4.07GB, 142MB/s
|
||
|
||
New Data Upload : 85%|████████▍ | 1.47GB / 1.74GB, 137MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 39%|███▉ | 1.58GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 39%|███▉ | 1.59GB / 4.07GB, 145MB/s
|
||
|
||
New Data Upload : 87%|████████▋ | 1.52GB / 1.74GB, 140MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 40%|████ | 1.63GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 40%|████ | 1.64GB / 4.07GB, 148MB/s
|
||
|
||
New Data Upload : 87%|████████▋ | 1.57GB / 1.81GB, 142MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 42%|████▏ | 1.69GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 42%|████▏ | 1.70GB / 4.07GB, 151MB/s
|
||
|
||
New Data Upload : 87%|████████▋ | 1.63GB / 1.88GB, 146MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 43%|████▎ | 1.75GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 43%|████▎ | 1.76GB / 4.07GB, 155MB/s
|
||
|
||
New Data Upload : 90%|█████████ | 1.70GB / 1.88GB, 149MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 44%|████▍ | 1.80GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 44%|████▍ | 1.81GB / 4.07GB, 157MB/s
|
||
|
||
New Data Upload : 90%|████████▉ | 1.75GB / 1.95GB, 152MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 45%|████▌ | 1.83GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 45%|████▌ | 1.84GB / 4.07GB, 158MB/s
|
||
|
||
New Data Upload : 91%|█████████▏| 1.78GB / 1.95GB, 152MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 46%|████▌ | 1.87GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 46%|████▌ | 1.88GB / 4.07GB, 159MB/s
|
||
|
||
New Data Upload : 90%|█████████ | 1.81GB / 2.01GB, 153MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 47%|████▋ | 1.91GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 47%|████▋ | 1.92GB / 4.07GB, 160MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 1.85GB / 2.01GB, 154MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 48%|████▊ | 1.94GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 48%|████▊ | 1.95GB / 4.07GB, 160MB/s
|
||
|
||
New Data Upload : 91%|█████████ | 1.88GB / 2.08GB, 155MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 49%|████▊ | 1.97GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 49%|████▊ | 1.99GB / 4.07GB, 161MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 1.92GB / 2.08GB, 156MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 50%|████▉ | 2.02GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 50%|████▉ | 2.04GB / 4.07GB, 164MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 1.97GB / 2.15GB, 158MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 51%|█████ | 2.06GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 51%|█████ | 2.07GB / 4.07GB, 165MB/s
|
||
|
||
New Data Upload : 94%|█████████▎| 2.01GB / 2.15GB, 160MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 52%|█████▏ | 2.09GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 52%|█████▏ | 2.11GB / 4.07GB, 165MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 2.04GB / 2.21GB, 160MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 52%|█████▏ | 2.13GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 53%|█████▎ | 2.14GB / 4.07GB, 165MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 2.08GB / 2.21GB, 160MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 53%|█████▎ | 2.16GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 53%|█████▎ | 2.17GB / 4.07GB, 164MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 2.10GB / 2.28GB, 159MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 54%|█████▎ | 2.18GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 54%|█████▍ | 2.19GB / 4.07GB, 163MB/s
|
||
|
||
New Data Upload : 91%|█████████ | 2.13GB / 2.35GB, 158MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 54%|█████▍ | 2.21GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 54%|█████▍ | 2.22GB / 4.07GB, 161MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 2.15GB / 2.35GB, 156MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 55%|█████▌ | 2.25GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 56%|█████▌ | 2.26GB / 4.07GB, 163MB/s
|
||
|
||
New Data Upload : 91%|█████████ | 2.20GB / 2.41GB, 157MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 56%|█████▌ | 2.28GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 56%|█████▋ | 2.30GB / 4.07GB, 163MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 2.23GB / 2.41GB, 158MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 57%|█████▋ | 2.33GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 57%|█████▋ | 2.34GB / 4.07GB, 165MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 2.27GB / 2.48GB, 159MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 58%|█████▊ | 2.37GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 58%|█████▊ | 2.38GB / 4.07GB, 164MB/s
|
||
|
||
New Data Upload : 93%|█████████▎| 2.31GB / 2.48GB, 159MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 59%|█████▉ | 2.41GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 59%|█████▉ | 2.42GB / 4.07GB, 166MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 2.35GB / 2.55GB, 161MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 60%|██████ | 2.44GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 60%|██████ | 2.46GB / 4.07GB, 167MB/s
|
||
|
||
New Data Upload : 91%|█████████▏| 2.39GB / 2.62GB, 161MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 61%|██████ | 2.49GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 61%|██████▏ | 2.50GB / 4.07GB, 168MB/s
|
||
|
||
New Data Upload : 93%|█████████▎| 2.43GB / 2.62GB, 162MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 62%|██████▏ | 2.52GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 62%|██████▏ | 2.53GB / 4.07GB, 168MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 2.47GB / 2.62GB, 163MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 63%|██████▎ | 2.56GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 63%|██████▎ | 2.57GB / 4.07GB, 168MB/s
|
||
|
||
New Data Upload : 93%|█████████▎| 2.51GB / 2.68GB, 163MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 64%|██████▍ | 2.60GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 64%|██████▍ | 2.61GB / 4.07GB, 167MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 2.54GB / 2.75GB, 162MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 65%|██████▍ | 2.64GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 65%|██████▍ | 2.65GB / 4.07GB, 166MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 2.58GB / 2.82GB, 161MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 66%|██████▌ | 2.68GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 66%|██████▌ | 2.69GB / 4.07GB, 167MB/s
|
||
|
||
New Data Upload : 93%|█████████▎| 2.62GB / 2.82GB, 161MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 67%|██████▋ | 2.72GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 67%|██████▋ | 2.73GB / 4.07GB, 167MB/s
|
||
|
||
New Data Upload : 95%|█████████▍| 2.67GB / 2.82GB, 162MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 68%|██████▊ | 2.76GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 68%|██████▊ | 2.77GB / 4.07GB, 168MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 2.71GB / 2.88GB, 163MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 69%|██████▊ | 2.79GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 69%|██████▉ | 2.80GB / 4.07GB, 170MB/s
|
||
|
||
New Data Upload : 95%|█████████▍| 2.74GB / 2.88GB, 165MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 69%|██████▉ | 2.82GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 70%|██████▉ | 2.83GB / 4.07GB, 171MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 2.77GB / 2.95GB, 166MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 70%|███████ | 2.86GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 70%|███████ | 2.87GB / 4.07GB, 173MB/s
|
||
|
||
New Data Upload : 95%|█████████▍| 2.80GB / 2.95GB, 168MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 71%|███████ | 2.88GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 71%|███████ | 2.89GB / 4.07GB, 174MB/s
|
||
|
||
New Data Upload : 94%|█████████▎| 2.83GB / 3.02GB, 169MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 72%|███████▏ | 2.92GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 72%|███████▏ | 2.93GB / 4.07GB, 176MB/s
|
||
|
||
New Data Upload : 93%|█████████▎| 2.86GB / 3.09GB, 171MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 73%|███████▎ | 2.95GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 73%|███████▎ | 2.96GB / 4.07GB, 173MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 2.90GB / 3.09GB, 173MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 73%|███████▎ | 2.99GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 74%|███████▎ | 3.00GB / 4.07GB, 176MB/s
|
||
|
||
New Data Upload : 93%|█████████▎| 2.93GB / 3.15GB, 176MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 74%|███████▍ | 3.00GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 74%|███████▍ | 3.02GB / 4.07GB, 177MB/s
|
||
|
||
New Data Upload : 94%|█████████▎| 2.95GB / 3.15GB, 177MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 75%|███████▍ | 3.05GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 75%|███████▌ | 3.06GB / 4.07GB, 181MB/s
|
||
|
||
New Data Upload : 93%|█████████▎| 2.99GB / 3.22GB, 181MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 76%|███████▌ | 3.08GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 76%|███████▌ | 3.09GB / 4.07GB, 184MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 3.02GB / 3.22GB, 184MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 77%|███████▋ | 3.13GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 77%|███████▋ | 3.14GB / 4.07GB, 188MB/s
|
||
|
||
New Data Upload : 93%|█████████▎| 3.07GB / 3.29GB, 188MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 78%|███████▊ | 3.16GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 78%|███████▊ | 3.18GB / 4.07GB, 188MB/s
|
||
|
||
New Data Upload : 95%|█████████▍| 3.11GB / 3.29GB, 188MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 79%|███████▉ | 3.20GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 79%|███████▉ | 3.21GB / 4.07GB, 188MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 3.15GB / 3.35GB, 188MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 80%|███████▉ | 3.24GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 80%|███████▉ | 3.25GB / 4.07GB, 188MB/s
|
||
|
||
New Data Upload : 95%|█████████▌| 3.19GB / 3.35GB, 188MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 81%|████████ | 3.27GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 81%|████████ | 3.28GB / 4.07GB, 187MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 3.22GB / 3.42GB, 187MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 81%|████████▏ | 3.30GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 81%|████████▏ | 3.32GB / 4.07GB, 186MB/s
|
||
|
||
New Data Upload : 95%|█████████▌| 3.25GB / 3.42GB, 186MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 82%|████████▏ | 3.35GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 82%|████████▏ | 3.36GB / 4.07GB, 187MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 3.29GB / 3.49GB, 187MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 84%|████████▎ | 3.40GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 84%|████████▎ | 3.41GB / 4.07GB, 189MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 3.34GB / 3.55GB, 189MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 84%|████████▍ | 3.43GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 84%|████████▍ | 3.44GB / 4.07GB, 190MB/s
|
||
|
||
New Data Upload : 95%|█████████▍| 3.37GB / 3.55GB, 190MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 85%|████████▌ | 3.47GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 85%|████████▌ | 3.48GB / 4.07GB, 190MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 3.42GB / 3.62GB, 190MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 86%|████████▋ | 3.51GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 87%|████████▋ | 3.53GB / 4.07GB, 190MB/s
|
||
|
||
New Data Upload : 96%|█████████▌| 3.46GB / 3.62GB, 190MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 88%|████████▊ | 3.56GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 88%|████████▊ | 3.57GB / 4.07GB, 189MB/s
|
||
|
||
New Data Upload : 95%|█████████▍| 3.50GB / 3.69GB, 189MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 88%|████████▊ | 3.58GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 88%|████████▊ | 3.59GB / 4.07GB, 185MB/s
|
||
|
||
New Data Upload : 96%|█████████▌| 3.52GB / 3.69GB, 185MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 89%|████████▉ | 3.61GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 89%|████████▉ | 3.62GB / 4.07GB, 182MB/s
|
||
|
||
New Data Upload : 95%|█████████▍| 3.56GB / 3.76GB, 182MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 90%|████████▉ | 3.65GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 90%|████████▉ | 3.66GB / 4.07GB, 181MB/s
|
||
|
||
New Data Upload : 96%|█████████▌| 3.59GB / 3.76GB, 181MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 90%|█████████ | 3.68GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 90%|█████████ | 3.69GB / 4.07GB, 181MB/s
|
||
|
||
New Data Upload : 95%|█████████▍| 3.62GB / 3.82GB, 181MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 91%|█████████▏| 3.72GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 92%|█████████▏| 3.73GB / 4.07GB, 181MB/s
|
||
|
||
New Data Upload : 96%|█████████▌| 3.66GB / 3.82GB, 181MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 92%|█████████▏| 3.75GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 92%|█████████▏| 3.76GB / 4.07GB, 181MB/s
|
||
|
||
New Data Upload : 95%|█████████▌| 3.70GB / 3.89GB, 181MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 93%|█████████▎| 3.79GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 93%|█████████▎| 3.80GB / 4.07GB, 182MB/s
|
||
|
||
New Data Upload : 94%|█████████▍| 3.74GB / 3.96GB, 182MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 94%|█████████▍| 3.83GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 94%|█████████▍| 3.84GB / 4.07GB, 182MB/s
|
||
|
||
New Data Upload : 95%|█████████▌| 3.78GB / 3.96GB, 182MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 95%|█████████▌| 3.88GB / 4.06GB [A[A[A
|
||
Processing Files (1 / 2) : 95%|█████████▌| 3.89GB / 4.07GB, 182MB/s
|
||
|
||
New Data Upload : 95%|█████████▌| 3.82GB / 4.01GB, 182MB/s [A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 8%|▊ | 579B / 7.18kB [A[A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 96%|█████████▋| 3.92GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 8%|▊ | 579B / 7.18kB [A[A[A[A
|
||
Processing Files (1 / 3) : 96%|█████████▋| 3.93GB / 4.07GB, 182MB/s
|
||
|
||
New Data Upload : 96%|█████████▋| 3.86GB / 4.01GB, 182MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 97%|█████████▋| 3.96GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 13%|█▎ | 909B / 7.18kB [A[A[A[A
|
||
Processing Files (1 / 3) : 97%|█████████▋| 3.97GB / 4.07GB, 183MB/s
|
||
|
||
New Data Upload : 97%|█████████▋| 3.90GB / 4.01GB, 183MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 99%|█████████▊| 4.01GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 33%|███▎ | 2.40kB / 7.18kB [A[A[A[A
|
||
Processing Files (1 / 3) : 99%|█████████▊| 4.02GB / 4.07GB, 184MB/s
|
||
|
||
New Data Upload : 99%|█████████▊| 3.95GB / 4.01GB, 184MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 99%|█████████▉| 4.04GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 58%|█████▊ | 4.13kB / 7.18kB [A[A[A[A
|
||
Processing Files (1 / 3) : 99%|█████████▉| 4.05GB / 4.07GB, 184MB/s
|
||
|
||
New Data Upload : 99%|█████████▉| 3.98GB / 4.01GB, 184MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|█████████▉| 4.05GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 82%|████████▏ | 5.87kB / 7.18kB [A[A[A[A
|
||
Processing Files (1 / 3) : 100%|█████████▉| 4.06GB / 4.07GB, 184MB/s
|
||
|
||
New Data Upload : 100%|█████████▉| 4.00GB / 4.01GB, 184MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 99%|█████████▉| 7.11kB / 7.18kB [A[A[A[A
|
||
Processing Files (1 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 182MB/s
|
||
|
||
New Data Upload : 100%|█████████▉| 4.01GB / 4.01GB, 182MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 99%|█████████▉| 7.11kB / 7.18kB [A[A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 99%|█████████▉| 7.11kB / 7.18kB [A[A[A[A
|
||
Processing Files (1 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 174MB/s
|
||
|
||
New Data Upload : 100%|█████████▉| 4.01GB / 4.01GB, 174MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A[A[A
|
||
Processing Files (3 / 3) : 100%|██████████| 4.07GB / 4.07GB, 170MB/s
|
||
|
||
New Data Upload : 100%|██████████| 4.01GB / 4.01GB, 170MB/s [A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A[A[A
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A[A[A
|
||
Processing Files (3 / 3) : 100%|██████████| 4.07GB / 4.07GB, 143MB/s
|
||
|
||
New Data Upload : 100%|██████████| 4.01GB / 4.01GB, 143MB/s
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB
|
||
***** train metrics *****
|
||
epoch = 2.0
|
||
total_flos = 1136301567GF
|
||
train_loss = 0.2964
|
||
train_runtime = 1:05:15.33
|
||
train_samples_per_second = 30.649
|
||
train_steps_per_second = 0.24
|
||
Figure saved at: saves/qwen3_1.7b/math/fft/training_loss.png
|
||
[WARNING|2026-04-29 16:50:51] llamafactory.extras.ploting:149 >> No metric eval_loss to plot.
|
||
[WARNING|2026-04-29 16:50:51] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot.
|
||
[INFO|trainer.py:3797] 2026-04-29 16:50:52,761 >> Saving model checkpoint to saves/qwen3_1.7b/math/fft
|
||
[INFO|configuration_utils.py:432] 2026-04-29 16:50:52,769 >> Configuration saved in saves/qwen3_1.7b/math/fft/config.json
|
||
[INFO|configuration_utils.py:803] 2026-04-29 16:50:52,775 >> Configuration saved in saves/qwen3_1.7b/math/fft/generation_config.json
|
||
|
||
Writing model shards: 0%| | 0/1 [00:00<?, ?it/s]
|
||
Writing model shards: 100%|██████████| 1/1 [00:07<00:00, 7.01s/it]
|
||
Writing model shards: 100%|██████████| 1/1 [00:07<00:00, 7.01s/it]
|
||
[INFO|modeling_utils.py:3380] 2026-04-29 16:50:59,796 >> Model weights saved in saves/qwen3_1.7b/math/fft/model.safetensors
|
||
[INFO|tokenization_utils_base.py:3224] 2026-04-29 16:50:59,801 >> chat template saved in saves/qwen3_1.7b/math/fft/chat_template.jinja
|
||
[INFO|tokenization_utils_base.py:2078] 2026-04-29 16:50:59,805 >> tokenizer config file saved in saves/qwen3_1.7b/math/fft/tokenizer_config.json
|
||
[INFO|modelcard.py:266] 2026-04-29 16:51:00,080 >> Dropping the following result as it does not have all the necessary fields:
|
||
{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
|
||
|
||
Processing Files (0 / 0) : | | 0.00B / 0.00B
|
||
|
||
New Data Upload : | | 0.00B / 0.00B [A
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 2%|▏ | 72.0MB / 4.06GB [A[A[A[A
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 2%|▏ | 72.0MB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 2%|▏ | 83.4MB / 4.07GB, ???B/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 5%|▍ | 184MB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 5%|▍ | 195MB / 4.07GB, 561MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 8%|▊ | 320MB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 8%|▊ | 331MB / 4.07GB, 619MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 11%|█ | 448MB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 11%|█▏ | 459MB / 4.07GB, 627MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 14%|█▍ | 576MB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 14%|█▍ | 587MB / 4.07GB, 630MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 18%|█▊ | 712MB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 18%|█▊ | 723MB / 4.07GB, 640MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 21%|██ | 840MB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 21%|██ | 851MB / 4.07GB, 640MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 24%|██▍ | 976MB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 24%|██▍ | 987MB / 4.07GB, 646MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 27%|██▋ | 1.11GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 28%|██▊ | 1.12GB / 4.07GB, 650MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 30%|███ | 1.24GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 31%|███ | 1.25GB / 4.07GB, 647MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 34%|███▍ | 1.37GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 34%|███▍ | 1.38GB / 4.07GB, 650MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 37%|███▋ | 1.51GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 37%|███▋ | 1.52GB / 4.07GB, 653MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 40%|████ | 1.64GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 41%|████ | 1.66GB / 4.07GB, 655MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 44%|████▍ | 1.78GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 44%|████▍ | 1.79GB / 4.07GB, 657MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 47%|████▋ | 1.92GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 47%|████▋ | 1.93GB / 4.07GB, 659MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 51%|█████ | 2.06GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 51%|█████ | 2.07GB / 4.07GB, 663MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 55%|█████▍ | 2.23GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 55%|█████▍ | 2.24GB / 4.07GB, 674MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 59%|█████▉ | 2.40GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 59%|█████▉ | 2.41GB / 4.07GB, 683MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 63%|██████▎ | 2.56GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 63%|██████▎ | 2.58GB / 4.07GB, 692MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 67%|██████▋ | 2.73GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 67%|██████▋ | 2.74GB / 4.07GB, 700MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 71%|███████▏ | 2.90GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 71%|███████▏ | 2.91GB / 4.07GB, 707MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 75%|███████▌ | 3.07GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 76%|███████▌ | 3.08GB / 4.07GB, 713MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 79%|███████▉ | 3.23GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 79%|███████▉ | 3.24GB / 4.07GB, 717MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 84%|████████▎ | 3.40GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 84%|████████▎ | 3.41GB / 4.07GB, 723MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 88%|████████▊ | 3.56GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 88%|████████▊ | 3.58GB / 4.07GB, 727MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 92%|█████████▏| 3.73GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 92%|█████████▏| 3.74GB / 4.07GB, 732MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 96%|█████████▌| 3.91GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 96%|█████████▌| 3.92GB / 4.07GB, 738MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 738MB/s
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB [A[A[A[A
|
||
Processing Files (2 / 3) : 100%|█████████▉| 4.07GB / 4.07GB, 713MB/s
|
||
|
||
New Data Upload : 92%|█████████▏| 3.80MB / 4.12MB, 678kB/s [A
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB [A[A[A[A
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|█████████▉| 4.06GB / 4.06GB [A[A[A[A
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A[A
|
||
Processing Files (3 / 3) : 100%|██████████| 4.07GB / 4.07GB, 644MB/s
|
||
|
||
New Data Upload : 100%|██████████| 4.12MB / 4.12MB, 664kB/s [A
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A[A
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A[A
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A[A
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A[A
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A[A
|
||
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB [A[A
|
||
|
||
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB [A[A[A
|
||
|
||
|
||
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB [A[A[A[A
|
||
Processing Files (3 / 3) : 100%|██████████| 4.07GB / 4.07GB, 554MB/s
|
||
|
||
New Data Upload : 100%|██████████| 4.12MB / 4.12MB, 572kB/s
|
||
|
||
...ath/fft/training_args.bin: 100%|██████████| 7.18kB / 7.18kB
|
||
|
||
...b/math/fft/tokenizer.json: 100%|██████████| 11.4MB / 11.4MB
|
||
|
||
...ath/fft/model.safetensors: 100%|██████████| 4.06GB / 4.06GB
|