KKHYA/qwen3-14b-fft-coding
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
main
qwen3-14b-fft-coding/eval_coding/fft_20260430_105324.log
ModelHub XC 896b02ea7c 初始化项目，由ModelHub XC社区提供模型 
Model: KKHYA/qwen3-14b-fft-coding
Source: Original Platform
2026-07-17 11:25:11 +08:00

1188 lines
1.3 MiB
Raw Permalink Blame History

The following values were not passed to `accelerate launch` and had defaults used instead:
More than one GPU was found, enabling multi-GPU training.
If this was unintended please pass in `--num_processes=1`.
`--dynamo_backend` was set to a value of `'no'`
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
2026-04-30:10:53:43 INFO [_cli.run:358] [rank=4] non-main process detected, suppressing wandb logging.
2026-04-30:10:53:43 INFO [_cli.run:358] [rank=6] non-main process detected, suppressing wandb logging.
2026-04-30:10:53:43 INFO [_cli.run:358] [rank=1] non-main process detected, suppressing wandb logging.
2026-04-30:10:53:43 INFO [_cli.run:358] [rank=3] non-main process detected, suppressing wandb logging.
2026-04-30:10:53:43 INFO [_cli.run:358] [rank=5] non-main process detected, suppressing wandb logging.
2026-04-30:10:53:43 INFO [_cli.run:358] [rank=7] non-main process detected, suppressing wandb logging.
2026-04-30:10:53:43 INFO [_cli.run:358] [rank=2] non-main process detected, suppressing wandb logging.
wandb: [wandb.login()] Loaded credentials for https://api.wandb.ai from WANDB_API_KEY.
wandb: Currently logged in as: kkhya (maskmoe) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.26.1
wandb: Run data is saved locally in /nfs/ywang29/lm-factory/wandb/run-20260430_105344-jm4o6xvg
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run fft_coding
wandb: ⭐️ View project at https://wandb.ai/maskmoe/MFT-LM-eval
wandb: 🚀 View run at https://wandb.ai/maskmoe/MFT-LM-eval/runs/jm4o6xvg
2026-04-30:10:54:46 INFO [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:47 INFO [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:55:00 INFO [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:01 INFO [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:2'}
2026-04-30:10:55:02 INFO [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:3'}
2026-04-30:10:55:02 INFO [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:7'}
2026-04-30:10:55:02 INFO [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:4'}
2026-04-30:10:55:02 INFO [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:1'}
2026-04-30:10:55:02 INFO [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:6'}
2026-04-30:10:55:02 INFO [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:5'}
2026-04-30:10:55:02 INFO [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:0'}
Loading weights: 0%| | 0/443 [00:00<?, ?it/s] Loading weights: 0%| | 0/443 [00:00<?, ?it/s] Loading weights: 0%| | 0/443 [00:00<?, ?it/s] Loading weights: 0%| | 0/443 [00:00<?, ?it/s] Loading weights: 0%| | 1/443 [00:00<00:00, 8289.14it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 1/443 [00:00<00:00, 5282.50it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 1/443 [00:00<00:00, 8473.34it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 1/443 [00:00<00:00, 6159.04it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 1/443 [00:00<00:00, 5391.14it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 1/443 [00:00<00:00, 4275.54it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 0/443 [00:00<?, ?it/s] Loading weights: 0%| | 1/443 [00:00<00:00, 13443.28it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 1/443 [00:00<00:00, 8886.24it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 1/443 [00:00<00:00, 5562.74it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 0/443 [00:00<?, ?it/s] Loading weights: 0%| | 1/443 [00:00<00:00, 3134.76it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 1/443 [00:00<00:00, 9383.23it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 1/443 [00:00<00:00, 5874.38it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 0/443 [00:00<?, ?it/s] Loading weights: 0%| | 1/443 [00:00<00:00, 8035.07it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 1/443 [00:00<00:00, 5184.55it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 0/443 [00:00<?, ?it/s] Loading weights: 0%| | 1/443 [00:00<00:00, 8065.97it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 1/443 [00:00<00:00, 4718.00it/s, Materializing param=lm_head.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=lm_head.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=lm_head.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=lm_head.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=lm_head.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=lm_head.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=lm_head.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=lm_head.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:14, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:15, 9.56s/it, Materializing param=lm_head.weight] Loading weights: 0%| | 2/443 [00:19<1:10:15, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 0%| | 2/443 [00:19<1:10:15, 9.56s/it, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 3/443 [00:19<1:10:05, 9.56s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/443 [00:19<1:10:05, 9.56s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 4/443 [00:19<1:09:56, 9.56s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 4/443 [00:19<1:09:56, 9.56s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 5/443 [00:20<24:15, 3.32s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 5/443 [00:20<24:15, 3.32s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%| | 5/443 [00:20<24:15, 3.32s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%|▏ | 6/443 [00:21<19:55, 2.74s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%|▏ | 6/443 [00:21<19:55, 2.74s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 1%|▏ | 6/443 [00:21<19:55, 2.74s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 7/443 [00:22<16:22, 2.25s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 7/443 [00:22<16:22, 2.25s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/443 [00:22<16:22, 2.25s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 8/443 [00:22<16:20, 2.25s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 8/443 [00:22<16:20, 2.25s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 9/443 [00:22<16:18, 2.25s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 9/443 [00:22<16:18, 2.25s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 10/443 [00:22<16:15, 2.25s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 10/443 [00:22<16:15, 2.25s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 11/443 [00:22<06:59, 1.03it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 11/443 [00:22<06:59, 1.03it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 2%|▏ | 11/443 [00:22<06:59, 1.03it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 3%|▎ | 12/443 [00:22<06:58, 1.03it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 12/443 [00:22<06:58, 1.03it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 13/443 [00:22<05:07, 1.40it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 13/443 [00:22<05:07, 1.40it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 13/443 [00:22<05:07, 1.40it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 1%| | 3/443 [00:22<52:15, 7.13s/it, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 3/443 [00:22<52:15, 7.13s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/443 [00:22<52:15, 7.13s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 4/443 [00:22<52:08, 7.13s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 4/443 [00:22<52:08, 7.13s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 3%|▎ | 14/443 [00:22<05:06, 1.40it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 14/443 [00:22<05:06, 1.40it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 15/443 [00:22<05:06, 1.40it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 3%|▎ | 15/443 [00:22<05:06, 1.40it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 1%| | 3/443 [00:23<52:53, 7.21s/it, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 3/443 [00:23<52:53, 7.21s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/443 [00:23<52:53, 7.21s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 4/443 [00:23<52:46, 7.21s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 4/443 [00:23<52:46, 7.21s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 5/443 [00:23<24:41, 3.38s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 5/443 [00:23<24:41, 3.38s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%| | 5/443 [00:23<24:41, 3.38s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%| | 5/443 [00:23<24:55, 3.41s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 5/443 [00:23<24:55, 3.41s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%| | 5/443 [00:23<24:55, 3.41s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%|▏ | 6/443 [00:23<18:21, 2.52s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%|▏ | 6/443 [00:23<18:21, 2.52s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 1%|▏ | 6/443 [00:23<18:21, 2.52s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 1%|▏ | 6/443 [00:23<18:36, 2.55s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%|▏ | 6/443 [00:23<18:36, 2.55s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 1%|▏ | 6/443 [00:23<18:36, 2.55s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 7/443 [00:24<13:51, 1.91s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 7/443 [00:24<13:51, 1.91s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/443 [00:24<13:51, 1.91s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 8/443 [00:24<13:49, 1.91s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 8/443 [00:24<13:49, 1.91s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 9/443 [00:24<13:47, 1.91s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 9/443 [00:24<13:47, 1.91s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 4%|▎ | 16/443 [00:24<04:15, 1.67it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 4%|▎ | 16/443 [00:24<04:15, 1.67it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▎ | 16/443 [00:24<04:15, 1.67it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 2%|▏ | 10/443 [00:24<13:45, 1.91s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 10/443 [00:24<13:45, 1.91s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 7/443 [00:24<14:01, 1.93s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 7/443 [00:24<14:01, 1.93s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/443 [00:24<14:01, 1.93s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 8/443 [00:24<13:59, 1.93s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 8/443 [00:24<13:59, 1.93s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 9/443 [00:24<13:57, 1.93s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 9/443 [00:24<13:57, 1.93s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 10/443 [00:24<13:55, 1.93s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 10/443 [00:24<13:55, 1.93s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 11/443 [00:24<05:26, 1.32it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 11/443 [00:24<05:26, 1.32it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 2%|▏ | 11/443 [00:24<05:26, 1.32it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 3%|▎ | 12/443 [00:24<05:25, 1.32it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 12/443 [00:24<05:25, 1.32it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 2%|▏ | 11/443 [00:24<05:36, 1.28it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 11/443 [00:24<05:36, 1.28it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 2%|▏ | 11/443 [00:24<05:36, 1.28it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 3%|▎ | 12/443 [00:24<05:36, 1.28it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 12/443 [00:24<05:36, 1.28it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 13/443 [00:24<05:35, 1.28it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 13/443 [00:24<05:35, 1.28it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 14/443 [00:24<05:34, 1.28it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 14/443 [00:24<05:34, 1.28it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 15/443 [00:24<05:33, 1.28it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 3%|▎ | 15/443 [00:24<05:33, 1.28it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 3%|▎ | 13/443 [00:24<03:59, 1.80it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 13/443 [00:24<03:59, 1.80it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 13/443 [00:24<03:59, 1.80it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 14/443 [00:24<03:58, 1.80it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 14/443 [00:24<03:58, 1.80it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 4%|▍ | 17/443 [00:24<04:27, 1.59it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▍ | 17/443 [00:24<04:27, 1.59it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 17/443 [00:24<04:27, 1.59it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 3%|▎ | 15/443 [00:25<03:21, 2.13it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 15/443 [00:25<03:21, 2.13it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 3%|▎ | 15/443 [00:25<03:21, 2.13it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 4%|▎ | 16/443 [00:25<03:20, 2.13it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▎ | 16/443 [00:25<03:20, 2.13it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 1%| | 3/443 [00:25<59:26, 8.11s/it, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 3/443 [00:25<59:26, 8.11s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/443 [00:25<59:26, 8.11s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 4/443 [00:25<59:18, 8.11s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 4/443 [00:25<59:18, 8.11s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 4%|▎ | 16/443 [00:25<03:14, 2.19it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 4%|▎ | 16/443 [00:25<03:14, 2.19it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▎ | 16/443 [00:25<03:14, 2.19it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▍ | 18/443 [00:25<04:29, 1.58it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 18/443 [00:25<04:29, 1.58it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 18/443 [00:25<04:29, 1.58it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 19/443 [00:25<04:28, 1.58it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 4%|▍ | 19/443 [00:25<04:28, 1.58it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 5%|▍ | 20/443 [00:25<04:28, 1.58it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▍ | 20/443 [00:25<04:28, 1.58it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▍ | 21/443 [00:25<04:27, 1.58it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 21/443 [00:25<04:27, 1.58it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 1%| | 3/443 [00:25<1:00:42, 8.28s/it, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 3/443 [00:25<1:00:42, 8.28s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/443 [00:25<1:00:42, 8.28s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 4/443 [00:25<1:00:34, 8.28s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 4/443 [00:25<1:00:34, 8.28s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 4%|▍ | 17/443 [00:25<03:11, 2.23it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▍ | 17/443 [00:25<03:11, 2.23it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 17/443 [00:25<03:11, 2.23it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 1%| | 3/443 [00:25<1:00:48, 8.29s/it, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 3/443 [00:25<1:00:48, 8.29s/it, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 3/443 [00:25<1:00:48, 8.29s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/443 [00:25<1:00:48, 8.29s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/443 [00:25<1:00:48, 8.29s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/443 [00:25<1:00:48, 8.29s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 4/443 [00:25<1:00:40, 8.29s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 4/443 [00:25<1:00:40, 8.29s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 4/443 [00:25<1:00:40, 8.29s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 4/443 [00:25<1:00:40, 8.29s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 4%|▍ | 17/443 [00:25<03:07, 2.28it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▍ | 17/443 [00:25<03:07, 2.28it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 17/443 [00:25<03:07, 2.28it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 5%|▍ | 22/443 [00:25<02:28, 2.83it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 22/443 [00:25<02:28, 2.83it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▍ | 22/443 [00:25<02:28, 2.83it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▌ | 23/443 [00:25<02:28, 2.83it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 23/443 [00:25<02:28, 2.83it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 1%| | 5/443 [00:25<28:35, 3.92s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 5/443 [00:25<28:35, 3.92s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%| | 5/443 [00:25<28:35, 3.92s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 5%|▌ | 24/443 [00:26<02:06, 3.30it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 24/443 [00:26<02:06, 3.30it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 5%|▌ | 24/443 [00:26<02:06, 3.30it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 6%|▌ | 25/443 [00:26<02:06, 3.30it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 25/443 [00:26<02:06, 3.30it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 1%| | 5/443 [00:26<29:02, 3.98s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 5/443 [00:26<29:02, 3.98s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%| | 5/443 [00:26<29:02, 3.98s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 4%|▍ | 18/443 [00:26<03:29, 2.03it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 18/443 [00:26<03:29, 2.03it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 18/443 [00:26<03:29, 2.03it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 19/443 [00:26<03:28, 2.03it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 4%|▍ | 19/443 [00:26<03:28, 2.03it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 5%|▍ | 20/443 [00:26<03:28, 2.03it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▍ | 20/443 [00:26<03:28, 2.03it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 1%| | 5/443 [00:26<29:04, 3.98s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 5/443 [00:26<29:04, 3.98s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%| | 5/443 [00:26<29:04, 3.98s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 5%|▍ | 21/443 [00:26<03:27, 2.03it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 21/443 [00:26<03:27, 2.03it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 1%| | 5/443 [00:26<29:06, 3.99s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 5/443 [00:26<29:06, 3.99s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%| | 5/443 [00:26<29:06, 3.99s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%| | 3/443 [00:26<1:03:18, 8.63s/it, Materializing param=model.embed_tokens.weight] Loading weights: 1%| | 3/443 [00:26<1:03:18, 8.63s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 3/443 [00:26<1:03:18, 8.63s/it, Materializing param=model.layers.0.input_layernorm.weight] Loading weights: 1%| | 4/443 [00:26<1:03:09, 8.63s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 4/443 [00:26<1:03:09, 8.63s/it, Materializing param=model.layers.0.mlp.down_proj.weight] Loading weights: 1%| | 5/443 [00:26<1:03:01, 8.63s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%| | 5/443 [00:26<1:03:01, 8.63s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 4%|▍ | 18/443 [00:26<03:25, 2.07it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 18/443 [00:26<03:25, 2.07it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 18/443 [00:26<03:25, 2.07it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 19/443 [00:26<03:25, 2.07it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 4%|▍ | 19/443 [00:26<03:25, 2.07it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 5%|▍ | 20/443 [00:26<03:24, 2.07it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▍ | 20/443 [00:26<03:24, 2.07it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▍ | 22/443 [00:26<01:56, 3.61it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 22/443 [00:26<01:56, 3.61it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▍ | 22/443 [00:26<01:56, 3.61it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▌ | 23/443 [00:26<01:56, 3.61it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 23/443 [00:26<01:56, 3.61it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▍ | 21/443 [00:26<03:24, 2.07it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 21/443 [00:26<03:24, 2.07it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▌ | 24/443 [00:26<01:36, 4.36it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 24/443 [00:26<01:36, 4.36it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 5%|▌ | 24/443 [00:26<01:36, 4.36it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 1%|▏ | 6/443 [00:26<21:58, 3.02s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%|▏ | 6/443 [00:26<21:58, 3.02s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 1%|▏ | 6/443 [00:26<21:58, 3.02s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 5%|▍ | 22/443 [00:26<01:53, 3.72it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 22/443 [00:26<01:53, 3.72it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▍ | 22/443 [00:26<01:53, 3.72it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▌ | 23/443 [00:26<01:52, 3.72it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 23/443 [00:26<01:52, 3.72it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 6%|▌ | 25/443 [00:26<01:35, 4.36it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 25/443 [00:26<01:35, 4.36it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 1%|▏ | 6/443 [00:26<21:37, 2.97s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%|▏ | 6/443 [00:26<21:37, 2.97s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 1%|▏ | 6/443 [00:26<21:37, 2.97s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 5%|▌ | 24/443 [00:26<01:29, 4.67it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 24/443 [00:26<01:29, 4.67it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 5%|▌ | 24/443 [00:26<01:29, 4.67it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 6%|▌ | 25/443 [00:26<01:29, 4.67it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 25/443 [00:26<01:29, 4.67it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 26/443 [00:26<01:29, 4.67it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 26/443 [00:26<01:29, 4.67it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 2%|▏ | 7/443 [00:27<16:33, 2.28s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 7/443 [00:27<16:33, 2.28s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/443 [00:27<16:33, 2.28s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 8/443 [00:27<16:30, 2.28s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 8/443 [00:27<16:30, 2.28s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 9/443 [00:27<16:28, 2.28s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 9/443 [00:27<16:28, 2.28s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 1%|▏ | 6/443 [00:27<22:26, 3.08s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%|▏ | 6/443 [00:27<22:26, 3.08s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 1%|▏ | 6/443 [00:27<22:26, 3.08s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 1%|▏ | 6/443 [00:27<22:26, 3.08s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%|▏ | 6/443 [00:27<22:26, 3.08s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 1%|▏ | 6/443 [00:27<22:26, 3.08s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 10/443 [00:27<16:26, 2.28s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 10/443 [00:27<16:26, 2.28s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 1%|▏ | 6/443 [00:27<23:43, 3.26s/it, Materializing param=model.layers.0.mlp.gate_proj.weight] Loading weights: 1%|▏ | 6/443 [00:27<23:43, 3.26s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 1%|▏ | 6/443 [00:27<23:43, 3.26s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 11/443 [00:27<06:27, 1.12it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 11/443 [00:27<06:27, 1.12it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 2%|▏ | 11/443 [00:27<06:27, 1.12it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 3%|▎ | 12/443 [00:27<06:26, 1.12it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 12/443 [00:27<06:26, 1.12it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 6%|▌ | 27/443 [00:27<01:22, 5.02it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 27/443 [00:27<01:22, 5.02it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▌ | 27/443 [00:27<01:22, 5.02it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▌ | 26/443 [00:27<02:41, 2.58it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 26/443 [00:27<02:41, 2.58it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 26/443 [00:27<02:41, 2.58it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 27/443 [00:27<02:41, 2.58it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▌ | 27/443 [00:27<02:41, 2.58it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 3%|▎ | 13/443 [00:27<04:38, 1.54it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 13/443 [00:27<04:38, 1.54it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 13/443 [00:27<04:38, 1.54it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 6%|▌ | 26/443 [00:27<01:56, 3.57it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 26/443 [00:27<01:56, 3.57it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 26/443 [00:27<01:56, 3.57it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 27/443 [00:27<01:56, 3.57it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▌ | 27/443 [00:27<01:56, 3.57it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 3%|▎ | 14/443 [00:27<04:37, 1.54it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 14/443 [00:27<04:37, 1.54it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 15/443 [00:27<04:37, 1.54it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 3%|▎ | 15/443 [00:27<04:37, 1.54it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 2%|▏ | 7/443 [00:27<17:07, 2.36s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 7/443 [00:27<17:07, 2.36s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/443 [00:27<17:07, 2.36s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 8/443 [00:27<17:05, 2.36s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 8/443 [00:27<17:05, 2.36s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 9/443 [00:27<17:03, 2.36s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 9/443 [00:27<17:03, 2.36s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 10/443 [00:27<17:00, 2.36s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 10/443 [00:27<17:00, 2.36s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 11/443 [00:27<06:40, 1.08it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 11/443 [00:27<06:40, 1.08it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 2%|▏ | 11/443 [00:27<06:40, 1.08it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 3%|▎ | 12/443 [00:27<06:39, 1.08it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 12/443 [00:27<06:39, 1.08it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 2%|▏ | 7/443 [00:27<19:06, 2.63s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 7/443 [00:27<19:06, 2.63s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/443 [00:27<19:06, 2.63s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 8/443 [00:27<19:04, 2.63s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 8/443 [00:27<19:04, 2.63s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 9/443 [00:27<19:01, 2.63s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 9/443 [00:27<19:01, 2.63s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 10/443 [00:27<18:59, 2.63s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 10/443 [00:27<18:59, 2.63s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 7/443 [00:27<17:25, 2.40s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 7/443 [00:27<17:25, 2.40s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/443 [00:27<17:25, 2.40s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 8/443 [00:27<17:23, 2.40s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 8/443 [00:27<17:23, 2.40s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 9/443 [00:27<17:20, 2.40s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 9/443 [00:27<17:20, 2.40s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 7/443 [00:27<17:25, 2.40s/it, Materializing param=model.layers.0.mlp.up_proj.weight] Loading weights: 2%|▏ | 7/443 [00:27<17:25, 2.40s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 7/443 [00:27<17:25, 2.40s/it, Materializing param=model.layers.0.post_attention_layernorm.weight] Loading weights: 2%|▏ | 8/443 [00:27<17:23, 2.40s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 8/443 [00:27<17:23, 2.40s/it, Materializing param=model.layers.0.self_attn.k_norm.weight] Loading weights: 2%|▏ | 9/443 [00:27<17:21, 2.40s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 9/443 [00:27<17:21, 2.40s/it, Materializing param=model.layers.0.self_attn.k_proj.weight] Loading weights: 2%|▏ | 10/443 [00:27<17:18, 2.40s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 10/443 [00:27<17:18, 2.40s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 10/443 [00:27<17:18, 2.40s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 10/443 [00:27<17:18, 2.40s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 3%|▎ | 13/443 [00:27<04:44, 1.51it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 13/443 [00:27<04:44, 1.51it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 13/443 [00:27<04:44, 1.51it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 14/443 [00:27<04:43, 1.51it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 14/443 [00:27<04:43, 1.51it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 15/443 [00:27<04:43, 1.51it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 3%|▎ | 15/443 [00:27<04:43, 1.51it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 2%|▏ | 11/443 [00:27<08:15, 1.15s/it, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 11/443 [00:27<08:15, 1.15s/it, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 2%|▏ | 11/443 [00:27<08:15, 1.15s/it, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 3%|▎ | 12/443 [00:27<08:14, 1.15s/it, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 12/443 [00:27<08:14, 1.15s/it, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 2%|▏ | 11/443 [00:27<06:43, 1.07it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 11/443 [00:27<06:43, 1.07it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 2%|▏ | 11/443 [00:27<06:43, 1.07it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 3%|▎ | 12/443 [00:27<06:42, 1.07it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 12/443 [00:27<06:42, 1.07it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<03:16, 2.17it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<03:16, 2.17it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<03:16, 2.17it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 3%|▎ | 13/443 [00:28<05:57, 1.20it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 13/443 [00:28<05:57, 1.20it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 13/443 [00:28<05:57, 1.20it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 14/443 [00:28<05:56, 1.20it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 14/443 [00:28<05:56, 1.20it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 15/443 [00:28<05:55, 1.20it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 3%|▎ | 15/443 [00:28<05:55, 1.20it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 2%|▏ | 11/443 [00:28<06:51, 1.05it/s, Materializing param=model.layers.0.self_attn.o_proj.weight] Loading weights: 2%|▏ | 11/443 [00:28<06:51, 1.05it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 2%|▏ | 11/443 [00:28<06:51, 1.05it/s, Materializing param=model.layers.0.self_attn.q_norm.weight] Loading weights: 3%|▎ | 12/443 [00:28<06:50, 1.05it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 12/443 [00:28<06:50, 1.05it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 13/443 [00:28<06:49, 1.05it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 13/443 [00:28<06:49, 1.05it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 13/443 [00:28<04:46, 1.50it/s, Materializing param=model.layers.0.self_attn.q_proj.weight] Loading weights: 3%|▎ | 13/443 [00:28<04:46, 1.50it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 13/443 [00:28<04:46, 1.50it/s, Materializing param=model.layers.0.self_attn.v_proj.weight] Loading weights: 3%|▎ | 14/443 [00:28<06:48, 1.05it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 14/443 [00:28<06:48, 1.05it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 15/443 [00:28<06:47, 1.05it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 3%|▎ | 15/443 [00:28<06:47, 1.05it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 3%|▎ | 14/443 [00:28<04:45, 1.50it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 14/443 [00:28<04:45, 1.50it/s, Materializing param=model.layers.1.input_layernorm.weight] Loading weights: 3%|▎ | 15/443 [00:28<04:44, 1.50it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 3%|▎ | 15/443 [00:28<04:44, 1.50it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<03:16, 2.17it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<03:16, 2.17it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<03:16, 2.17it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<03:11, 2.22it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<03:11, 2.22it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<03:11, 2.22it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<04:01, 1.77it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<04:01, 1.77it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<04:01, 1.77it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<03:33, 2.00it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<03:33, 2.00it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<03:33, 2.00it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<03:17, 2.16it/s, Materializing param=model.layers.1.mlp.down_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<03:17, 2.16it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▎ | 16/443 [00:28<03:17, 2.16it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<03:09, 2.25it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<03:09, 2.25it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<03:09, 2.25it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 18/443 [00:28<03:08, 2.25it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 18/443 [00:28<03:08, 2.25it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 18/443 [00:28<03:08, 2.25it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 19/443 [00:28<03:08, 2.25it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 4%|▍ | 19/443 [00:28<03:08, 2.25it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 5%|▍ | 20/443 [00:28<03:07, 2.25it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▍ | 20/443 [00:28<03:07, 2.25it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 6%|▋ | 28/443 [00:28<02:39, 2.60it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▋ | 28/443 [00:28<02:41, 2.57it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▋ | 28/443 [00:28<03:19, 2.08it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▋ | 28/443 [00:28<02:39, 2.60it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:28<02:41, 2.57it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:28<02:39, 2.60it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:28<03:19, 2.08it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:28<02:41, 2.57it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:28<03:19, 2.08it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 5%|▍ | 21/443 [00:28<03:07, 2.25it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 21/443 [00:28<03:07, 2.25it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<04:01, 1.77it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<04:01, 1.77it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<03:26, 2.07it/s, Materializing param=model.layers.1.mlp.gate_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<03:26, 2.07it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<03:26, 2.07it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<03:17, 2.16it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 17/443 [00:28<03:17, 2.16it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 5%|▍ | 22/443 [00:28<01:36, 4.34it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 22/443 [00:28<01:36, 4.34it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▍ | 22/443 [00:28<01:36, 4.34it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▌ | 23/443 [00:28<01:36, 4.34it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 23/443 [00:28<01:36, 4.34it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:16, 5.49it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:16, 5.49it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:16, 5.49it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 4%|▍ | 18/443 [00:29<03:02, 2.33it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 18/443 [00:29<03:02, 2.33it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 18/443 [00:29<03:02, 2.33it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 19/443 [00:29<03:01, 2.33it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 4%|▍ | 19/443 [00:29<03:01, 2.33it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 5%|▍ | 20/443 [00:29<03:01, 2.33it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▍ | 20/443 [00:29<03:01, 2.33it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 6%|▌ | 25/443 [00:29<01:16, 5.49it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 25/443 [00:29<01:16, 5.49it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 26/443 [00:29<01:15, 5.49it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 26/443 [00:29<01:15, 5.49it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 5%|▍ | 21/443 [00:29<03:00, 2.33it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 21/443 [00:29<03:00, 2.33it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 22/443 [00:29<01:35, 4.40it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 22/443 [00:29<01:35, 4.40it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▍ | 22/443 [00:29<01:35, 4.40it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▌ | 23/443 [00:29<01:35, 4.40it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 23/443 [00:29<01:35, 4.40it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 4%|▍ | 18/443 [00:29<03:39, 1.94it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 18/443 [00:29<03:39, 1.94it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 18/443 [00:29<03:39, 1.94it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 19/443 [00:29<03:38, 1.94it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 4%|▍ | 19/443 [00:29<03:38, 1.94it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 5%|▍ | 20/443 [00:29<03:38, 1.94it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▍ | 20/443 [00:29<03:38, 1.94it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▍ | 21/443 [00:29<03:37, 1.94it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 21/443 [00:29<03:37, 1.94it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 4%|▍ | 18/443 [00:29<03:18, 2.14it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 18/443 [00:29<03:18, 2.14it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 18/443 [00:29<03:18, 2.14it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 19/443 [00:29<03:18, 2.14it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 4%|▍ | 19/443 [00:29<03:18, 2.14it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 5%|▍ | 20/443 [00:29<03:17, 2.14it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▍ | 20/443 [00:29<03:17, 2.14it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 4%|▍ | 18/443 [00:29<03:08, 2.26it/s, Materializing param=model.layers.1.mlp.up_proj.weight] Loading weights: 4%|▍ | 18/443 [00:29<03:08, 2.26it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 18/443 [00:29<03:08, 2.26it/s, Materializing param=model.layers.1.post_attention_layernorm.weight] Loading weights: 4%|▍ | 19/443 [00:29<03:07, 2.26it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 4%|▍ | 19/443 [00:29<03:07, 2.26it/s, Materializing param=model.layers.1.self_attn.k_norm.weight] Loading weights: 5%|▍ | 20/443 [00:29<03:07, 2.26it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▍ | 20/443 [00:29<03:07, 2.26it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:17, 5.40it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:17, 5.40it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:17, 5.40it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 5%|▍ | 21/443 [00:29<03:06, 2.26it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 21/443 [00:29<03:06, 2.26it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 22/443 [00:29<02:09, 3.25it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 22/443 [00:29<02:09, 3.25it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▍ | 22/443 [00:29<02:09, 3.25it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▌ | 23/443 [00:29<02:09, 3.25it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 23/443 [00:29<02:09, 3.25it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 6%|▌ | 25/443 [00:29<01:17, 5.40it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 25/443 [00:29<01:17, 5.40it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 26/443 [00:29<01:17, 5.40it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 26/443 [00:29<01:17, 5.40it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 5%|▍ | 21/443 [00:29<02:05, 3.35it/s, Materializing param=model.layers.1.self_attn.k_proj.weight] Loading weights: 5%|▍ | 21/443 [00:29<02:05, 3.35it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 21/443 [00:29<02:05, 3.35it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 22/443 [00:29<02:05, 3.35it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▍ | 22/443 [00:29<02:05, 3.35it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▌ | 23/443 [00:29<02:05, 3.35it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 23/443 [00:29<02:05, 3.35it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▍ | 22/443 [00:29<01:51, 3.78it/s, Materializing param=model.layers.1.self_attn.o_proj.weight] Loading weights: 5%|▍ | 22/443 [00:29<01:51, 3.78it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▍ | 22/443 [00:29<01:51, 3.78it/s, Materializing param=model.layers.1.self_attn.q_norm.weight] Loading weights: 5%|▌ | 23/443 [00:29<01:51, 3.78it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 23/443 [00:29<01:51, 3.78it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:44, 4.01it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:44, 4.01it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:44, 4.01it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 6%|▌ | 27/443 [00:29<01:11, 5.85it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 27/443 [00:29<01:11, 5.85it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▌ | 27/443 [00:29<01:11, 5.85it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▌ | 25/443 [00:29<01:44, 4.01it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 25/443 [00:29<01:44, 4.01it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 26/443 [00:29<01:44, 4.01it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 26/443 [00:29<01:44, 4.01it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:25, 4.92it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:25, 4.92it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:25, 4.92it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 6%|▌ | 25/443 [00:29<01:24, 4.92it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 25/443 [00:29<01:24, 4.92it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 26/443 [00:29<01:24, 4.92it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 26/443 [00:29<01:24, 4.92it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:29, 4.67it/s, Materializing param=model.layers.1.self_attn.q_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:29, 4.67it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 5%|▌ | 24/443 [00:29<01:29, 4.67it/s, Materializing param=model.layers.1.self_attn.v_proj.weight] Loading weights: 6%|▌ | 25/443 [00:29<01:29, 4.67it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 25/443 [00:29<01:29, 4.67it/s, Materializing param=model.layers.2.input_layernorm.weight] Loading weights: 6%|▌ | 26/443 [00:29<01:29, 4.67it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 26/443 [00:29<01:29, 4.67it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 27/443 [00:29<01:09, 5.96it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 27/443 [00:29<01:09, 5.96it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▌ | 27/443 [00:29<01:09, 5.96it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▋ | 28/443 [00:29<01:10, 5.85it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:29<01:10, 5.85it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▌ | 27/443 [00:29<01:28, 4.69it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 27/443 [00:29<01:28, 4.69it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▌ | 27/443 [00:29<01:28, 4.69it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▌ | 27/443 [00:30<01:18, 5.33it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 27/443 [00:30<01:18, 5.33it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▌ | 27/443 [00:30<01:18, 5.33it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▌ | 27/443 [00:30<01:21, 5.11it/s, Materializing param=model.layers.2.mlp.down_proj.weight] Loading weights: 6%|▌ | 27/443 [00:30<01:21, 5.11it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▌ | 27/443 [00:30<01:21, 5.11it/s, Materializing param=model.layers.2.mlp.gate_proj.weight] Loading weights: 6%|▋ | 28/443 [00:30<01:09, 5.96it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:30<01:09, 5.96it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:30<01:28, 4.69it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:30<01:28, 4.69it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:30<01:17, 5.33it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:30<01:17, 5.33it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:30<01:21, 5.11it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 6%|▋ | 28/443 [00:30<01:21, 5.11it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 7%|▋ | 29/443 [00:30<04:25, 1.56it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 7%|▋ | 29/443 [00:30<04:25, 1.56it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:32, 4.47it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:55, 3.57it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 7%|▋ | 29/443 [00:30<04:25, 1.56it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:32, 4.48it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:40, 4.11it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:45, 3.91it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:55, 3.57it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 29/443 [00:30<04:53, 1.41it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 7%|▋ | 29/443 [00:30<04:18, 1.60it/s, Materializing param=model.layers.2.mlp.up_proj.weight] Loading weights: 7%|▋ | 30/443 [00:30<04:24, 1.56it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:40, 4.11it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:55, 3.57it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:45, 3.91it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 29/443 [00:30<04:53, 1.41it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 30/443 [00:30<04:24, 1.56it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:40, 4.11it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:45, 3.91it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:32, 4.47it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 29/443 [00:30<04:53, 1.41it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 30/443 [00:30<01:55, 3.57it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:32, 4.47it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 29/443 [00:30<04:18, 1.60it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 30/443 [00:30<01:40, 4.11it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 30/443 [00:30<01:45, 3.91it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 30/443 [00:30<01:55, 3.57it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 30/443 [00:30<01:45, 3.91it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 30/443 [00:30<01:40, 4.11it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 29/443 [00:30<04:18, 1.60it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 31/443 [00:30<04:24, 1.56it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 31/443 [00:30<04:24, 1.56it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 31/443 [00:30<01:55, 3.57it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:32, 4.48it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 30/443 [00:30<01:32, 4.47it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 31/443 [00:30<01:45, 3.91it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 31/443 [00:30<01:55, 3.57it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 31/443 [00:30<01:45, 3.91it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 31/443 [00:30<01:40, 4.11it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 30/443 [00:30<04:53, 1.41it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 30/443 [00:30<01:32, 4.47it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 31/443 [00:30<01:40, 4.11it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 30/443 [00:30<04:53, 1.41it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 30/443 [00:30<04:17, 1.60it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 30/443 [00:30<04:17, 1.60it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 31/443 [00:30<04:52, 1.41it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 31/443 [00:30<01:32, 4.47it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 31/443 [00:30<04:52, 1.41it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 31/443 [00:30<01:32, 4.47it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 31/443 [00:30<04:16, 1.60it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 31/443 [00:30<04:16, 1.60it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 29/443 [00:30<01:32, 4.48it/s, Materializing param=model.layers.2.post_attention_layernorm.weight] Loading weights: 7%|▋ | 30/443 [00:30<01:32, 4.48it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 30/443 [00:30<01:32, 4.48it/s, Materializing param=model.layers.2.self_attn.k_norm.weight] Loading weights: 7%|▋ | 31/443 [00:30<01:31, 4.48it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 31/443 [00:30<01:31, 4.48it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:15, 5.47it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:06, 6.21it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:15, 5.47it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:06, 6.21it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:15, 5.47it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:06, 6.21it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<02:56, 2.32it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<02:38, 2.59it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:20, 5.12it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<02:38, 2.59it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<02:38, 2.59it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:20, 5.12it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:20, 5.12it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<02:56, 2.32it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:06, 6.20it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<02:56, 2.32it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:10, 5.81it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<02:37, 2.60it/s, Materializing param=model.layers.2.self_attn.k_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:10, 5.81it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:10, 5.81it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<02:37, 2.60it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<02:37, 2.60it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:06, 6.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 32/443 [00:30<01:06, 6.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 33/443 [00:31<01:05, 6.21it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<01:05, 6.21it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<03:06, 2.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight] Loading weights: 7%|▋ | 33/443 [00:31<01:14, 5.47it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<01:20, 5.12it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<02:38, 2.59it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<01:10, 5.81it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<01:14, 5.47it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<01:06, 6.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<03:06, 2.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<02:37, 2.60it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<01:20, 5.12it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<03:06, 2.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<01:10, 5.81it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<02:37, 2.60it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 7%|▋ | 33/443 [00:31<02:38, 2.59it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:25, 4.80it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:29, 4.57it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:25, 4.80it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<03:05, 2.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<02:25, 2.81it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:25, 4.80it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:29, 4.57it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:22, 4.97it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 8%|▊ | 34/443 [00:31<03:05, 2.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:29, 4.57it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<02:25, 2.81it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<02:25, 2.81it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:22, 4.97it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:22, 4.97it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 7%|▋ | 33/443 [00:31<01:06, 6.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:18, 5.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:18, 5.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:18, 5.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<02:26, 2.79it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 8%|▊ | 34/443 [00:31<02:26, 2.79it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<02:26, 2.79it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<02:26, 2.79it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<02:26, 2.79it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:50, 3.69it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:50, 3.69it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:50, 3.69it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 37/443 [00:31<01:50, 3.69it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 37/443 [00:31<01:50, 3.69it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<01:22, 4.97it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<01:22, 4.97it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:28, 4.64it/s, Materializing param=model.layers.2.self_attn.q_norm.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:28, 4.64it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 34/443 [00:31<01:28, 4.64it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<01:27, 4.64it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<01:27, 4.64it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<02:14, 3.04it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<02:18, 2.95it/s, Materializing param=model.layers.2.self_attn.q_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<02:14, 3.04it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<02:14, 3.04it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<02:18, 2.95it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<02:18, 2.95it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<01:29, 4.57it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<01:29, 4.57it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<01:25, 4.80it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<01:25, 4.80it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<01:18, 5.20it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 35/443 [00:31<01:18, 5.20it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:12, 5.65it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:12, 5.65it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:12, 5.65it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 37/443 [00:31<01:11, 5.65it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 37/443 [00:31<01:11, 5.65it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 36/443 [00:31<02:13, 3.04it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 36/443 [00:31<02:13, 3.04it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 37/443 [00:31<02:13, 3.04it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 37/443 [00:31<02:13, 3.04it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:09, 5.84it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:09, 5.84it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:09, 5.84it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 37/443 [00:31<01:09, 5.84it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 37/443 [00:31<01:09, 5.84it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:27, 4.64it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:27, 4.64it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 37/443 [00:31<01:27, 4.64it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 37/443 [00:31<01:27, 4.64it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:14, 5.44it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:14, 5.44it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:14, 5.44it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 36/443 [00:31<02:17, 2.95it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 36/443 [00:31<02:17, 2.95it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 37/443 [00:31<02:17, 2.95it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 37/443 [00:31<02:17, 2.95it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:17, 5.23it/s, Materializing param=model.layers.2.self_attn.v_proj.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:17, 5.23it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 8%|▊ | 36/443 [00:31<01:17, 5.23it/s, Materializing param=model.layers.3.input_layernorm.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:17, 2.94it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 8%|▊ | 37/443 [00:33<01:17, 5.23it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<03:24, 1.98it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:45, 2.45it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:41, 2.50it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<03:14, 2.08it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<03:13, 2.09it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:17, 2.94it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 8%|▊ | 37/443 [00:33<01:17, 5.23it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:45, 2.45it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<03:24, 1.98it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:17, 2.94it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<03:13, 2.09it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<03:14, 2.08it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:45, 2.45it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<03:24, 1.98it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<03:13, 2.09it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<03:14, 2.08it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:48, 2.40it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:48, 2.40it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 8%|▊ | 37/443 [00:33<01:14, 5.44it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:48, 2.40it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 8%|▊ | 37/443 [00:33<01:14, 5.44it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:41, 2.50it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:45, 2.45it/s, Materializing param=model.layers.3.mlp.down_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:41, 2.50it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:45, 2.45it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▊ | 38/443 [00:33<02:45, 2.45it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:21, 1.55it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:17, 1.57it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:18, 1.56it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<03:34, 1.88it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:42, 1.43it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:43, 1.43it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:14, 1.59it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:17, 1.57it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:42, 1.43it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:21, 1.55it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<03:34, 1.88it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:18, 1.56it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:17, 1.57it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:42, 1.43it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:21, 1.55it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:43, 1.43it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:18, 1.56it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<03:34, 1.88it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:43, 1.43it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:56, 1.36it/s, Materializing param=model.layers.3.mlp.gate_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:56, 1.36it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:56, 1.36it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:14, 1.59it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 39/443 [00:35<04:14, 1.59it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 40/443 [00:37<05:50, 1.15it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 40/443 [00:37<06:28, 1.04it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 40/443 [00:37<06:28, 1.04it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 40/443 [00:37<05:56, 1.13it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 40/443 [00:37<06:28, 1.04it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 40/443 [00:37<06:15, 1.07it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 40/443 [00:37<06:13, 1.08it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 40/443 [00:37<05:56, 1.13it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 40/443 [00:37<05:52, 1.14it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 40/443 [00:37<04:59, 1.34it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 40/443 [00:37<05:56, 1.13it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 41/443 [00:37<06:27, 1.04it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 40/443 [00:37<06:13, 1.08it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 40/443 [00:37<06:15, 1.07it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 41/443 [00:37<06:27, 1.04it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 40/443 [00:37<06:13, 1.08it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 40/443 [00:37<06:15, 1.07it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 40/443 [00:37<04:59, 1.34it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 40/443 [00:37<05:52, 1.14it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 41/443 [00:37<05:55, 1.13it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 42/443 [00:37<06:27, 1.04it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 40/443 [00:37<05:52, 1.14it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 40/443 [00:37<04:59, 1.34it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 40/443 [00:37<05:54, 1.14it/s, Materializing param=model.layers.3.mlp.up_proj.weight] Loading weights: 9%|▉ | 41/443 [00:37<06:12, 1.08it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 42/443 [00:37<06:27, 1.04it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 40/443 [00:37<05:50, 1.15it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 41/443 [00:37<06:12, 1.08it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 41/443 [00:37<05:55, 1.13it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 40/443 [00:37<05:54, 1.14it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 41/443 [00:37<04:58, 1.34it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 42/443 [00:37<06:11, 1.08it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 41/443 [00:37<05:51, 1.14it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 41/443 [00:37<04:58, 1.34it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 41/443 [00:37<06:14, 1.07it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 42/443 [00:37<06:11, 1.08it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 42/443 [00:37<05:55, 1.13it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 41/443 [00:37<05:51, 1.14it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 40/443 [00:37<05:54, 1.14it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 41/443 [00:37<06:14, 1.07it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 42/443 [00:37<05:55, 1.13it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 42/443 [00:37<04:58, 1.34it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 42/443 [00:37<05:50, 1.14it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 42/443 [00:37<06:13, 1.07it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 42/443 [00:37<05:50, 1.14it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 42/443 [00:37<04:58, 1.34it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 41/443 [00:37<05:53, 1.14it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 42/443 [00:37<06:13, 1.07it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 40/443 [00:37<05:50, 1.15it/s, Materializing param=model.layers.3.post_attention_layernorm.weight] Loading weights: 9%|▉ | 41/443 [00:37<05:53, 1.14it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 42/443 [00:37<05:52, 1.14it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 42/443 [00:37<05:52, 1.14it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 41/443 [00:37<05:49, 1.15it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 41/443 [00:37<05:49, 1.15it/s, Materializing param=model.layers.3.self_attn.k_norm.weight] Loading weights: 9%|▉ | 42/443 [00:37<05:48, 1.15it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 9%|▉ | 42/443 [00:37<05:48, 1.15it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:23, 1.97it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:23, 1.97it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:05, 2.15it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:23, 1.97it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:22, 1.98it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:20, 1.99it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:32, 1.88it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:22, 1.98it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:32, 1.88it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:32, 1.88it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:32, 1.88it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:22, 1.98it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:32, 1.88it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:32, 1.88it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:20, 1.99it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:20, 1.99it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:05, 2.15it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:05, 2.15it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:21, 1.98it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:21, 1.98it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:21, 1.98it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:36, 1.84it/s, Materializing param=model.layers.3.self_attn.k_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:36, 1.84it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 43/443 [00:37<03:36, 1.84it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<03:36, 1.84it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|▉ | 44/443 [00:37<03:36, 1.84it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|█ | 45/443 [00:37<03:35, 1.84it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 45/443 [00:37<03:35, 1.84it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<02:53, 2.29it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<02:53, 2.29it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|▉ | 44/443 [00:37<02:53, 2.29it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|█ | 45/443 [00:37<02:53, 2.29it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 45/443 [00:37<02:53, 2.29it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<03:04, 2.17it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<03:04, 2.17it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|▉ | 44/443 [00:37<03:04, 2.17it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|█ | 45/443 [00:37<03:03, 2.17it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 45/443 [00:37<03:03, 2.17it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<02:47, 2.39it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<02:59, 2.22it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<02:47, 2.39it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|▉ | 44/443 [00:37<02:47, 2.39it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|▉ | 44/443 [00:37<02:59, 2.22it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|▉ | 44/443 [00:37<02:59, 2.22it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|█ | 45/443 [00:37<02:46, 2.39it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 45/443 [00:37<02:46, 2.39it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 45/443 [00:37<02:59, 2.22it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 45/443 [00:37<02:59, 2.22it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<03:07, 2.13it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<03:07, 2.13it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|▉ | 44/443 [00:37<03:07, 2.13it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|█ | 45/443 [00:37<03:07, 2.13it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 45/443 [00:37<03:07, 2.13it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<02:59, 2.22it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<02:59, 2.22it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|▉ | 44/443 [00:37<02:59, 2.22it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|█ | 46/443 [00:37<02:05, 3.15it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:16, 2.90it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:05, 3.15it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:16, 2.90it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:11, 3.02it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:29, 2.66it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:11, 3.02it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:05, 3.15it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:16, 2.90it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:17, 2.89it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:29, 2.66it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:11, 3.02it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:29, 2.66it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:17, 2.89it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:17, 2.89it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:11, 3.02it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:11, 3.02it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 45/443 [00:37<02:37, 2.53it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|█ | 45/443 [00:37<02:37, 2.53it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 45/443 [00:37<02:37, 2.53it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:36, 2.53it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:11, 3.02it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<02:36, 2.53it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<03:14, 2.05it/s, Materializing param=model.layers.3.self_attn.o_proj.weight] Loading weights: 10%|▉ | 44/443 [00:37<03:14, 2.05it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|▉ | 44/443 [00:37<03:14, 2.05it/s, Materializing param=model.layers.3.self_attn.q_norm.weight] Loading weights: 10%|█ | 45/443 [00:37<03:14, 2.05it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 45/443 [00:37<03:14, 2.05it/s, Materializing param=model.layers.3.self_attn.q_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<03:13, 2.05it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 10%|█ | 46/443 [00:37<03:13, 2.05it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<01:54, 3.45it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<01:58, 3.33it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<01:54, 3.45it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 47/443 [00:37<02:12, 2.99it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<01:54, 3.45it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 47/443 [00:37<02:12, 2.99it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 47/443 [00:37<01:58, 3.33it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 47/443 [00:37<02:12, 2.99it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 47/443 [00:37<01:58, 3.33it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 47/443 [00:37<03:13, 2.05it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 47/443 [00:37<01:51, 3.56it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<03:13, 2.05it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 47/443 [00:37<01:47, 3.67it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<01:51, 3.56it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 47/443 [00:37<01:54, 3.45it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<01:51, 3.56it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 48/443 [00:37<02:12, 2.99it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<01:47, 3.67it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 47/443 [00:37<01:59, 3.31it/s, Materializing param=model.layers.3.self_attn.v_proj.weight] Loading weights: 11%|█ | 48/443 [00:37<03:12, 2.05it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<01:47, 3.67it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 48/443 [00:37<03:12, 2.05it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 48/443 [00:37<02:12, 2.99it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<01:59, 3.31it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 48/443 [00:37<01:50, 3.56it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<01:59, 3.31it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 48/443 [00:37<01:50, 3.56it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 48/443 [00:37<01:47, 3.67it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 48/443 [00:37<01:47, 3.67it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<01:54, 3.45it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 48/443 [00:37<01:59, 3.31it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 48/443 [00:37<01:59, 3.31it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 47/443 [00:37<01:54, 3.45it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 48/443 [00:37<01:54, 3.45it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 48/443 [00:37<01:54, 3.45it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 48/443 [00:39<04:20, 1.51it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 48/443 [00:39<04:20, 1.51it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 48/443 [00:39<04:20, 1.51it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<02:54, 2.25it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:25, 1.92it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:30, 1.87it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:23, 1.94it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 48/443 [00:39<04:17, 1.53it/s, Materializing param=model.layers.4.input_layernorm.weight] Loading weights: 11%|█ | 49/443 [00:39<02:54, 2.25it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:25, 1.92it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:34, 1.84it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:23, 1.94it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<04:20, 1.51it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<02:54, 2.25it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:25, 1.92it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:31, 1.86it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:23, 1.94it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<04:20, 1.51it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:34, 1.84it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 48/443 [00:39<04:17, 1.53it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 48/443 [00:39<04:17, 1.53it/s, Materializing param=model.layers.4.mlp.down_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:31, 1.86it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:34, 1.84it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:31, 1.86it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:30, 1.87it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<04:16, 1.53it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<03:30, 1.87it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█ | 49/443 [00:39<04:16, 1.53it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:25, 1.21it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:37, 1.16it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:45, 1.14it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:27, 1.20it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:48, 1.13it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:25, 1.21it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:27, 1.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:37, 1.16it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:48, 1.13it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:27, 1.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:25, 1.21it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:37, 1.16it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:32, 1.18it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:48, 1.13it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:35, 1.17it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:32, 1.18it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:32, 1.18it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:35, 1.17it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:35, 1.17it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<04:30, 1.45it/s, Materializing param=model.layers.4.mlp.gate_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:45, 1.14it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<04:30, 1.45it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<04:30, 1.45it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 11%|█▏ | 50/443 [00:42<05:45, 1.14it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<04:30, 1.45it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 51/443 [00:42<04:30, 1.45it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<04:29, 1.45it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<04:29, 1.45it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 53/443 [00:42<04:28, 1.45it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 53/443 [00:42<04:28, 1.45it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<04:28, 1.45it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<04:28, 1.45it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:42<02:29, 2.59it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:42<02:29, 2.59it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 55/443 [00:42<02:29, 2.59it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:18, 1.23it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:18, 1.23it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:18, 1.23it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:17, 1.23it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:17, 1.23it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:17, 1.23it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:17, 1.23it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:28, 1.19it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:28, 1.19it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:28, 1.19it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:27, 1.19it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:27, 1.19it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:26, 1.19it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:26, 1.19it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:14, 1.25it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:14, 1.25it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:14, 1.25it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:13, 1.25it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:13, 1.25it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:12, 1.25it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:12, 1.25it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:32, 1.18it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:32, 1.18it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:32, 1.18it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:31, 1.18it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:31, 1.18it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:30, 1.18it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:30, 1.18it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<05:16, 1.23it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<05:16, 1.23it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 13%|█▎ | 56/443 [00:42<02:27, 2.62it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:18, 1.23it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:18, 1.23it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:18, 1.23it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 13%|█▎ | 56/443 [00:42<02:27, 2.62it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 56/443 [00:42<02:27, 2.62it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:42<02:27, 2.62it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:42<02:27, 2.62it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:17, 1.23it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:17, 1.23it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:17, 1.23it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:17, 1.23it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<05:11, 1.25it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<05:11, 1.25it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<05:25, 1.19it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<05:25, 1.19it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:25, 1.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:25, 1.20it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:25, 1.20it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:24, 1.20it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:24, 1.20it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:24, 1.20it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:24, 1.20it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<05:29, 1.18it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<05:29, 1.18it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:27, 1.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:27, 1.20it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 51/443 [00:42<05:27, 1.20it/s, Materializing param=model.layers.4.post_attention_layernorm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:26, 1.20it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 52/443 [00:42<05:26, 1.20it/s, Materializing param=model.layers.4.self_attn.k_norm.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:26, 1.20it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 53/443 [00:42<05:26, 1.20it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<05:16, 1.23it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<05:16, 1.23it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 13%|█▎ | 58/443 [00:42<02:26, 2.62it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 58/443 [00:42<02:26, 2.62it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 59/443 [00:42<02:26, 2.62it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 59/443 [00:42<02:26, 2.62it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<05:25, 1.20it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:42<05:25, 1.20it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:33, 2.53it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:33, 2.53it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:33, 2.53it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 54/443 [00:43<02:55, 2.22it/s, Materializing param=model.layers.4.self_attn.k_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:43<02:55, 2.22it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 54/443 [00:43<02:55, 2.22it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:54, 2.22it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:54, 2.22it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:54, 2.22it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:54, 2.22it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:35, 2.49it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:35, 2.49it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:35, 2.49it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:36, 2.48it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:36, 2.48it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:36, 2.48it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:36, 2.48it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:36, 2.48it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:32, 2.53it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:32, 2.53it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:55, 3.35it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:55, 3.35it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:55, 3.35it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:33, 2.53it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:33, 2.53it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:33, 2.53it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:32, 2.53it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:32, 2.53it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 58/443 [00:43<01:54, 3.35it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 58/443 [00:43<01:54, 3.35it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 59/443 [00:43<01:54, 3.35it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 59/443 [00:43<01:54, 3.35it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:34, 2.51it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:34, 2.51it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:34, 2.51it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:33, 2.51it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:33, 2.51it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:37, 2.47it/s, Materializing param=model.layers.4.self_attn.o_proj.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:37, 2.47it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 12%|█▏ | 55/443 [00:43<02:37, 2.47it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:36, 2.47it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:36, 2.47it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:56, 3.33it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:56, 3.33it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:56, 3.33it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:53, 3.40it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:53, 3.40it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:53, 3.40it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 58/443 [00:43<01:55, 3.33it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 58/443 [00:43<01:55, 3.33it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 59/443 [00:43<01:55, 3.33it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 59/443 [00:43<01:55, 3.33it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:57, 3.27it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:57, 3.27it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:57, 3.27it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 58/443 [00:43<01:53, 3.40it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 58/443 [00:43<01:53, 3.40it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 59/443 [00:43<01:52, 3.40it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 59/443 [00:43<01:52, 3.40it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:59, 3.22it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:59, 3.22it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<01:59, 3.22it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 58/443 [00:43<01:57, 3.27it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 58/443 [00:43<01:57, 3.27it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 59/443 [00:43<01:57, 3.27it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 59/443 [00:43<01:57, 3.27it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:23, 2.70it/s, Materializing param=model.layers.4.self_attn.q_norm.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:23, 2.70it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 56/443 [00:43<02:23, 2.70it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<02:23, 2.70it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<02:23, 2.70it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<02:03, 3.13it/s, Materializing param=model.layers.4.self_attn.q_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<02:03, 3.13it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 57/443 [00:43<02:03, 3.13it/s, Materializing param=model.layers.4.self_attn.v_proj.weight] Loading weights: 13%|█▎ | 58/443 [00:43<01:59, 3.22it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 58/443 [00:43<01:59, 3.22it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 59/443 [00:43<01:59, 3.22it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 59/443 [00:43<01:59, 3.22it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 58/443 [00:43<02:22, 2.70it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 58/443 [00:43<02:22, 2.70it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 59/443 [00:43<02:22, 2.70it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 59/443 [00:43<02:22, 2.70it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 58/443 [00:43<02:02, 3.13it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 58/443 [00:43<02:02, 3.13it/s, Materializing param=model.layers.5.input_layernorm.weight] Loading weights: 13%|█▎ | 59/443 [00:43<02:02, 3.13it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 13%|█▎ | 59/443 [00:43<02:02, 3.13it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:12, 2.89it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:14, 2.84it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:12, 2.89it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:14, 2.84it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:31, 2.53it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:12, 2.89it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:14, 2.84it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:10, 2.93it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:14, 2.85it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:31, 2.53it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:15, 2.83it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:14, 2.85it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:31, 2.53it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:20, 2.72it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:14, 2.85it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:15, 2.83it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:15, 2.82it/s, Materializing param=model.layers.5.mlp.down_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:20, 2.72it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:15, 2.83it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:44<02:31, 2.53it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:20, 2.72it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:10, 2.93it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:15, 2.82it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:44<02:31, 2.53it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:15, 2.82it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:44<02:15, 2.83it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:44<02:15, 2.83it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▎ | 60/443 [00:44<02:10, 2.93it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:24, 2.65it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:24, 2.65it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:24, 2.65it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:29, 2.55it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:29, 2.55it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:29, 2.55it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:26, 2.61it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:26, 2.61it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:26, 2.61it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:27, 2.58it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:27, 2.58it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:27, 2.58it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:24, 2.65it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:24, 2.65it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:24, 2.65it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:26, 2.61it/s, Materializing param=model.layers.5.mlp.gate_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:26, 2.61it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 61/443 [00:45<02:26, 2.61it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:40, 1.73it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:24, 1.86it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:24, 1.86it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:24, 1.86it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:40, 1.73it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:40, 1.73it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:21, 1.89it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:35, 1.77it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:21, 1.89it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:40, 1.73it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:40, 1.73it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:40, 1.73it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:21, 1.89it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:41, 1.72it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:44, 1.70it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:36, 1.76it/s, Materializing param=model.layers.5.mlp.up_proj.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:39, 1.73it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:40, 1.73it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:39, 1.73it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:41, 1.72it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:36, 1.76it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:44, 1.70it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:41, 1.72it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:40, 1.73it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:36, 1.76it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:20, 1.89it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:44, 1.70it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:20, 1.89it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:35, 1.76it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:39, 1.73it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:35, 1.77it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:20, 1.89it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:35, 1.76it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:39, 1.73it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:40, 1.72it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:43, 1.70it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:20, 1.89it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:43, 1.70it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:40, 1.72it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 62/443 [00:46<03:35, 1.77it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:35, 1.76it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:39, 1.73it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:35, 1.76it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:39, 1.73it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:43, 1.70it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:39, 1.72it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:43, 1.70it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:39, 1.72it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:34, 1.77it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:34, 1.77it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:33, 1.77it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:33, 1.77it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:15, 2.80it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:15, 2.80it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:14, 2.80it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:15, 2.80it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:14, 2.80it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:12, 2.85it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:12, 2.85it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:14, 2.80it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:12, 2.85it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:04, 2.06it/s, Materializing param=model.layers.5.post_attention_layernorm.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:12, 2.85it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:04, 2.06it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:12, 2.85it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 63/443 [00:46<03:04, 2.06it/s, Materializing param=model.layers.5.self_attn.k_norm.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:12, 2.85it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:03, 2.06it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 14%|█▍ | 64/443 [00:46<03:03, 2.06it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<03:03, 2.06it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<03:03, 2.06it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:12, 2.84it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:11, 2.88it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:12, 2.84it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:10, 2.89it/s, Materializing param=model.layers.5.self_attn.k_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:12, 2.84it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:10, 2.89it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:11, 2.88it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:11, 2.88it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 65/443 [00:46<02:10, 2.89it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:10, 2.88it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:10, 2.88it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▌ | 67/443 [00:46<02:10, 2.88it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 67/443 [00:46<02:10, 2.88it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:03, 3.05it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:03, 3.05it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:03, 3.05it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:05, 3.02it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:05, 3.02it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:05, 3.02it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▌ | 67/443 [00:46<02:04, 3.02it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 67/443 [00:46<02:04, 3.02it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:03, 3.05it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:03, 3.05it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:03, 3.05it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:04, 3.03it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:04, 3.03it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:04, 3.03it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:04, 3.03it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:04, 3.03it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:04, 3.03it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▌ | 67/443 [00:46<02:04, 3.03it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 67/443 [00:46<02:04, 3.03it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:04, 3.02it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:04, 3.02it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▍ | 66/443 [00:46<02:04, 3.02it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▌ | 67/443 [00:46<02:04, 3.02it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 67/443 [00:46<02:04, 3.02it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:35, 3.94it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:35, 3.94it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:35, 3.94it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 67/443 [00:47<01:52, 3.34it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▌ | 67/443 [00:47<01:52, 3.34it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 67/443 [00:47<01:52, 3.34it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:52, 3.34it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:52, 3.34it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:38, 3.82it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:38, 3.82it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:38, 3.82it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 67/443 [00:47<01:54, 3.28it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▌ | 67/443 [00:47<01:54, 3.28it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 67/443 [00:47<01:54, 3.28it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:54, 3.28it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:54, 3.28it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:33, 4.00it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:33, 4.00it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:33, 4.00it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 67/443 [00:47<01:52, 3.33it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▌ | 67/443 [00:47<01:52, 3.33it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 67/443 [00:47<01:52, 3.33it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:52, 3.33it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:52, 3.33it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:34, 3.98it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:34, 3.98it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<01:34, 3.98it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:54, 3.28it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:54, 3.28it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:37, 3.82it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:27, 4.29it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:37, 3.82it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:52, 3.34it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:27, 4.29it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:52, 3.34it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:27, 4.29it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 70/443 [00:47<01:37, 3.82it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 70/443 [00:47<01:53, 3.28it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 70/443 [00:47<01:37, 3.82it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 70/443 [00:47<01:53, 3.28it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 70/443 [00:47<01:26, 4.29it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:33, 4.00it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 70/443 [00:47<01:51, 3.34it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 70/443 [00:47<01:26, 4.29it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:52, 3.33it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:33, 4.00it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:52, 3.33it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 70/443 [00:47<01:51, 3.34it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 70/443 [00:47<01:33, 4.00it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 70/443 [00:47<01:33, 4.00it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:33, 3.98it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 69/443 [00:47<01:33, 3.98it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 70/443 [00:47<01:33, 3.98it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 70/443 [00:47<01:33, 3.98it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 15%|█▍ | 66/443 [00:47<02:15, 2.77it/s, Materializing param=model.layers.5.self_attn.o_proj.weight] Loading weights: 15%|█▍ | 66/443 [00:47<02:15, 2.77it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▍ | 66/443 [00:47<02:15, 2.77it/s, Materializing param=model.layers.5.self_attn.q_norm.weight] Loading weights: 15%|█▌ | 67/443 [00:47<02:15, 2.77it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 67/443 [00:47<02:15, 2.77it/s, Materializing param=model.layers.5.self_attn.q_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<02:15, 2.77it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 15%|█▌ | 68/443 [00:47<02:15, 2.77it/s, Materializing param=model.layers.5.self_attn.v_proj.weight] Loading weights: 16%|█▌ | 69/443 [00:47<02:14, 2.77it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 69/443 [00:47<02:14, 2.77it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 70/443 [00:47<02:14, 2.77it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 70/443 [00:47<02:14, 2.77it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:28, 2.50it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:15, 2.74it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 70/443 [00:48<02:30, 2.48it/s, Materializing param=model.layers.6.input_layernorm.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:03, 3.00it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:15, 2.74it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:17, 2.71it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:12, 2.81it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:15, 2.74it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 70/443 [00:48<02:30, 2.48it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:28, 2.50it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:15, 2.74it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:15, 2.74it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:28, 2.50it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:17, 2.71it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 70/443 [00:48<02:30, 2.48it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:15, 2.74it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:12, 2.81it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:12, 2.81it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:17, 2.71it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:48<02:11, 2.81it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:03, 3.00it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:48<02:11, 2.81it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:30, 2.48it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:30, 2.48it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:48<02:03, 3.00it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:49<02:19, 2.66it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:49<02:29, 2.49it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:49<02:19, 2.66it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:49<02:29, 2.49it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:49<02:19, 2.66it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:49<02:18, 2.68it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:49<02:29, 2.49it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:49<02:18, 2.68it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:49<02:18, 2.68it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:49<02:18, 2.68it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:49<02:18, 2.68it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:49<02:18, 2.68it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 73/443 [00:50<03:03, 2.02it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 73/443 [00:50<02:52, 2.15it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 73/443 [00:50<02:52, 2.15it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 73/443 [00:50<02:52, 2.15it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 16%|█▋ | 73/443 [00:50<03:03, 2.02it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 73/443 [00:50<03:03, 2.02it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 73/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 73/443 [00:50<02:40, 2.31it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 73/443 [00:50<02:52, 2.15it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 17%|█▋ | 75/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 75/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 16%|█▋ | 73/443 [00:50<02:52, 2.15it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<03:02, 2.02it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 16%|█▋ | 73/443 [00:50<02:40, 2.31it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 73/443 [00:50<02:52, 2.15it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<03:02, 2.02it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 16%|█▋ | 73/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 73/443 [00:50<02:40, 2.31it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 73/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 17%|█▋ | 75/443 [00:50<03:02, 2.02it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 74/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 75/443 [00:50<03:02, 2.02it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 74/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<02:39, 2.31it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 75/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 75/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 74/443 [00:50<02:39, 2.31it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<02:51, 2.15it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 75/443 [00:50<02:50, 2.15it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 75/443 [00:50<02:39, 2.31it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 75/443 [00:50<02:50, 2.15it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 75/443 [00:50<02:39, 2.31it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:45, 3.47it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:41, 3.63it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:45, 3.47it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:41, 3.63it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:45, 3.47it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:41, 3.63it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:42, 3.58it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:44, 3.51it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:44, 3.51it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:42, 3.58it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:44, 3.51it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:42, 3.58it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:45, 3.47it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:45, 3.47it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 16%|█▌ | 71/443 [00:50<03:20, 1.86it/s, Materializing param=model.layers.6.mlp.down_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:50<03:20, 1.86it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▌ | 71/443 [00:50<03:20, 1.86it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:50<03:19, 1.86it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:50<03:19, 1.86it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 73/443 [00:50<03:19, 1.86it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 73/443 [00:50<03:19, 1.86it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<03:18, 1.86it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<03:18, 1.86it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 75/443 [00:50<03:18, 1.86it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 75/443 [00:50<03:18, 1.86it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:50<03:16, 1.89it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:53, 3.22it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:50<03:16, 1.89it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:50<03:16, 1.89it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 73/443 [00:50<03:15, 1.89it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 73/443 [00:50<03:15, 1.89it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<03:15, 1.89it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<03:15, 1.89it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 75/443 [00:50<03:14, 1.89it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 75/443 [00:50<03:14, 1.89it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:53, 3.22it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<01:53, 3.22it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:53, 3.22it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:53, 3.22it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:53, 3.22it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:53, 3.22it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:39, 3.69it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:39, 3.69it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:39, 3.69it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:38, 3.69it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:42, 3.58it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:38, 3.69it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:42, 3.58it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:25, 4.28it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:25, 4.28it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:25, 4.28it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:41, 3.59it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:41, 3.59it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:41, 3.59it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:41, 3.59it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:41, 3.59it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<03:17, 1.86it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<03:17, 1.86it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<03:14, 1.89it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<03:14, 1.89it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:50<03:15, 1.90it/s, Materializing param=model.layers.6.mlp.gate_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:50<03:15, 1.90it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 72/443 [00:50<03:15, 1.90it/s, Materializing param=model.layers.6.mlp.up_proj.weight] Loading weights: 16%|█▋ | 73/443 [00:50<03:14, 1.90it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 16%|█▋ | 73/443 [00:50<03:14, 1.90it/s, Materializing param=model.layers.6.post_attention_layernorm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<03:14, 1.90it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 74/443 [00:50<03:14, 1.90it/s, Materializing param=model.layers.6.self_attn.k_norm.weight] Loading weights: 17%|█▋ | 75/443 [00:50<03:13, 1.90it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 75/443 [00:50<03:13, 1.90it/s, Materializing param=model.layers.6.self_attn.k_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<03:13, 1.90it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 76/443 [00:50<03:13, 1.90it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:43, 3.52it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:43, 3.52it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:43, 3.52it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:43, 3.52it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:43, 3.52it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:38, 3.70it/s, Materializing param=model.layers.6.self_attn.o_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:38, 3.70it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 17%|█▋ | 77/443 [00:50<01:38, 3.70it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:38, 3.70it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:38, 3.70it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 17%|█▋ | 77/443 [00:50<03:12, 1.90it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 17%|█▋ | 77/443 [00:50<03:12, 1.90it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:46, 3.41it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:46, 3.41it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:46, 3.41it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:31, 3.98it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:31, 3.98it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:27, 4.14it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:31, 3.98it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:37, 3.75it/s, Materializing param=model.layers.6.self_attn.q_norm.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:25, 4.27it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:46, 3.41it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:26, 4.19it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:27, 4.14it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:46, 3.41it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:27, 4.14it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:25, 4.27it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:26, 4.19it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:25, 4.27it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:26, 4.19it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:33, 3.90it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:33, 3.90it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:37, 3.75it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:33, 3.90it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 78/443 [00:50<01:37, 3.75it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:36, 3.75it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:36, 3.75it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:29, 4.07it/s, Materializing param=model.layers.6.self_attn.q_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:29, 4.07it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 79/443 [00:50<01:29, 4.07it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:18, 4.65it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:31, 3.98it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:19, 4.59it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:24, 4.27it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:31, 3.98it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:19, 4.59it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:18, 4.65it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:19, 4.59it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:18, 4.65it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:17, 4.65it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:17, 4.65it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:22, 4.41it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:22, 4.38it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:24, 4.27it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:19, 4.58it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:22, 4.41it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:22, 4.38it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:22, 4.41it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:20, 4.48it/s, Materializing param=model.layers.6.self_attn.v_proj.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:19, 4.58it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:22, 4.38it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:19, 4.58it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:08, 5.27it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:22, 4.41it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:08, 5.27it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:22, 4.38it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:22, 4.41it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:08, 5.27it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:19, 4.58it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:22, 4.38it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:19, 4.58it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:20, 4.48it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 80/443 [00:50<01:20, 4.48it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:20, 4.48it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:50<01:20, 4.48it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:52<02:37, 2.30it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 81/443 [00:52<03:24, 1.77it/s, Materializing param=model.layers.7.input_layernorm.weight] Loading weights: 18%|█▊ | 81/443 [00:52<02:37, 2.30it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:52<02:37, 2.30it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:52<03:24, 1.77it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 18%|█▊ | 81/443 [00:52<03:24, 1.77it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<02:28, 2.44it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<03:07, 1.93it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<02:45, 2.18it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<01:08, 5.27it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<03:07, 1.93it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<03:10, 1.90it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<03:05, 1.95it/s, Materializing param=model.layers.7.mlp.down_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<03:07, 1.93it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<03:23, 1.77it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<01:08, 5.27it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<02:45, 2.18it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<02:37, 2.30it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<02:45, 2.18it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<02:37, 2.30it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<03:10, 1.90it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<03:05, 1.95it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<03:23, 1.77it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<03:10, 1.90it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<02:28, 2.44it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<03:05, 1.95it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 82/443 [00:52<02:28, 2.44it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:00, 1.50it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:01, 1.49it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<03:50, 1.56it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:00, 1.50it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:01, 1.49it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:34, 1.31it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:36, 1.30it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<03:33, 1.68it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:39, 1.29it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:00, 1.50it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:01, 1.49it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:36, 1.30it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<03:50, 1.56it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:36, 1.30it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:34, 1.31it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:39, 1.29it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<03:50, 1.56it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:34, 1.31it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<04:39, 1.29it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<03:33, 1.68it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<03:33, 1.68it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<03:13, 1.86it/s, Materializing param=model.layers.7.mlp.gate_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<03:13, 1.86it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▊ | 83/443 [00:54<03:13, 1.86it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▉ | 84/443 [00:56<04:45, 1.26it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▉ | 84/443 [00:56<04:18, 1.39it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▉ | 84/443 [00:56<04:55, 1.21it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▉ | 84/443 [00:56<04:18, 1.39it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<04:55, 1.21it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<04:18, 1.39it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<04:55, 1.21it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<05:19, 1.13it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▉ | 85/443 [00:56<04:17, 1.39it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 85/443 [00:56<04:54, 1.21it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<04:45, 1.26it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 85/443 [00:56<04:17, 1.39it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<05:19, 1.13it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<05:59, 1.00s/it, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▉ | 85/443 [00:56<04:54, 1.21it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<05:19, 1.13it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<05:20, 1.12it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▉ | 84/443 [00:56<06:02, 1.01s/it, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▉ | 86/443 [00:56<04:16, 1.39it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 84/443 [00:56<05:59, 1.00s/it, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<05:57, 1.00it/s, Materializing param=model.layers.7.mlp.up_proj.weight] Loading weights: 19%|█▉ | 84/443 [00:56<04:45, 1.26it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<05:20, 1.12it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 86/443 [00:56<04:16, 1.39it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 84/443 [00:56<05:59, 1.00s/it, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<05:20, 1.12it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<05:57, 1.00it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<06:02, 1.01s/it, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 85/443 [00:56<05:58, 1.00s/it, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<05:57, 1.00it/s, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 84/443 [00:56<06:02, 1.01s/it, Materializing param=model.layers.7.post_attention_layernorm.weight] Loading weights: 19%|█▉ | 85/443 [00:56<05:58, 1.00s/it, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 85/443 [00:56<05:19, 1.12it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 85/443 [00:56<04:44, 1.26it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 85/443 [00:56<05:19, 1.12it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 85/443 [00:56<05:18, 1.13it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 86/443 [00:56<05:57, 1.00s/it, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 85/443 [00:56<05:56, 1.00it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 85/443 [00:56<05:18, 1.13it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 86/443 [00:56<05:57, 1.00s/it, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 85/443 [00:56<06:01, 1.01s/it, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 86/443 [00:56<05:18, 1.12it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 85/443 [00:56<04:44, 1.26it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 85/443 [00:56<05:56, 1.00it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 86/443 [00:56<05:18, 1.12it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 85/443 [00:56<06:01, 1.01s/it, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 19%|█▉ | 86/443 [00:56<05:17, 1.13it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 86/443 [00:56<05:17, 1.13it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 86/443 [00:56<05:55, 1.00it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 86/443 [00:56<05:55, 1.00it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 86/443 [00:56<04:53, 1.21it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 86/443 [00:56<06:00, 1.01s/it, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 86/443 [00:56<04:53, 1.21it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 86/443 [00:56<06:00, 1.01s/it, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:07, 1.90it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<02:57, 2.00it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<02:59, 1.99it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:00, 1.97it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:08, 1.89it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:06, 1.91it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<02:59, 1.99it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:07, 1.90it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 19%|█▉ | 86/443 [00:56<03:18, 1.80it/s, Materializing param=model.layers.7.self_attn.k_norm.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:06, 1.91it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:00, 1.97it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<02:57, 2.00it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:07, 1.90it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<02:59, 1.99it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:08, 1.89it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:06, 1.91it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:00, 1.97it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<02:57, 2.00it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:08, 1.89it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 19%|█▉ | 86/443 [00:56<03:18, 1.80it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 19%|█▉ | 86/443 [00:56<03:18, 1.80it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:18, 1.80it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<03:18, 1.80it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<02:43, 2.18it/s, Materializing param=model.layers.7.self_attn.k_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<02:43, 2.18it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 87/443 [00:56<02:43, 2.18it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 88/443 [00:57<02:50, 2.08it/s, Materializing param=model.layers.7.self_attn.o_proj.weight] Loading weights: 20%|█▉ | 88/443 [00:57<02:59, 1.97it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<02:59, 1.97it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<02:58, 1.99it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<03:06, 1.90it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<02:58, 1.99it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<03:06, 1.90it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<03:06, 1.91it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<02:50, 2.08it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<03:06, 1.91it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|██ | 89/443 [00:57<02:40, 2.20it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|██ | 89/443 [00:57<02:40, 2.20it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|█▉ | 88/443 [00:57<02:42, 2.18it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<02:50, 2.08it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|██ | 89/443 [00:57<02:40, 2.20it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|█▉ | 88/443 [00:57<02:57, 2.00it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<02:42, 2.18it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<02:57, 2.00it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<03:07, 1.89it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|██ | 89/443 [00:57<02:29, 2.37it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|█▉ | 88/443 [00:57<03:07, 1.89it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|██ | 89/443 [00:57<02:29, 2.37it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<02:37, 2.25it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|██ | 89/443 [00:57<02:29, 2.37it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<02:37, 2.25it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<02:43, 2.17it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|██ | 89/443 [00:57<02:37, 2.25it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<02:43, 2.17it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<02:43, 2.17it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<02:42, 2.18it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|██ | 89/443 [00:57<02:42, 2.18it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<02:42, 2.18it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<02:44, 2.15it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<02:44, 2.15it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<02:44, 2.15it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<02:32, 2.31it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<02:32, 2.31it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<03:05, 1.91it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|██ | 90/443 [00:57<02:45, 2.14it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<02:32, 2.31it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<02:40, 2.20it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<02:45, 2.14it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<03:10, 1.85it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|██ | 90/443 [00:57<02:40, 2.20it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<02:45, 2.14it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<02:40, 2.20it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<03:05, 1.91it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<03:10, 1.85it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<03:05, 1.91it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<03:10, 1.85it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<03:10, 1.85it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<03:04, 1.91it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<03:10, 1.85it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<03:04, 1.91it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<02:51, 2.06it/s, Materializing param=model.layers.7.self_attn.q_norm.weight] Loading weights: 20%|██ | 89/443 [00:57<02:51, 2.06it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 89/443 [00:57<02:51, 2.06it/s, Materializing param=model.layers.7.self_attn.q_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<02:51, 2.06it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 90/443 [00:57<02:51, 2.06it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 21%|██ | 91/443 [00:57<02:50, 2.06it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 91/443 [00:57<02:50, 2.06it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 92/443 [00:57<02:50, 2.06it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 92/443 [00:57<02:50, 2.06it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 91/443 [00:57<02:44, 2.15it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 91/443 [00:57<02:44, 2.15it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 91/443 [00:57<02:32, 2.31it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 92/443 [00:57<02:43, 2.15it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 92/443 [00:57<02:43, 2.15it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 91/443 [00:57<02:32, 2.31it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 92/443 [00:57<02:32, 2.31it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 92/443 [00:57<02:32, 2.31it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 91/443 [00:57<02:39, 2.20it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 91/443 [00:57<02:39, 2.20it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 92/443 [00:57<02:39, 2.20it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 92/443 [00:57<02:39, 2.20it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 91/443 [00:57<03:09, 1.85it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 91/443 [00:57<03:09, 1.85it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 92/443 [00:57<03:09, 1.85it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 91/443 [00:57<02:44, 2.14it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 92/443 [00:57<03:09, 1.85it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 91/443 [00:57<02:44, 2.14it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 91/443 [00:57<03:03, 1.91it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 91/443 [00:57<03:03, 1.91it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 92/443 [00:57<03:03, 1.91it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 92/443 [00:57<03:03, 1.91it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:07, 2.75it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:11, 2.66it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<01:58, 2.96it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:11, 2.66it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:10, 2.69it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:09, 2.69it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:07, 2.75it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:11, 2.66it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:11, 2.66it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:10, 2.69it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:09, 2.69it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:07, 2.75it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:10, 2.69it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:09, 2.69it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:11, 2.66it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<02:11, 2.66it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<01:58, 2.96it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<01:58, 2.96it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 20%|██ | 90/443 [00:58<02:40, 2.20it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 20%|██ | 90/443 [00:58<02:40, 2.20it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 21%|██ | 91/443 [00:58<03:01, 1.94it/s, Materializing param=model.layers.7.self_attn.v_proj.weight] Loading weights: 21%|██ | 91/443 [00:58<03:01, 1.94it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 91/443 [00:58<03:01, 1.94it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 92/443 [00:58<03:00, 1.94it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 92/443 [00:58<03:00, 1.94it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<03:00, 1.94it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [00:58<03:00, 1.94it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:29, 1.67it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:34, 1.63it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:13, 1.80it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:44, 1.55it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:26, 1.69it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:41, 1.58it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:34, 1.63it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:13, 1.80it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:29, 1.67it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:44, 1.55it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:22, 1.72it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:34, 1.63it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:26, 1.69it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:13, 1.80it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:29, 1.67it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:41, 1.58it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:44, 1.55it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:26, 1.69it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██ | 94/443 [01:00<03:41, 1.58it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:00<03:28, 1.67it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 21%|██▏ | 95/443 [01:00<03:28, 1.67it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 21%|██ | 94/443 [01:00<03:22, 1.72it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 22%|██▏ | 96/443 [01:00<03:28, 1.67it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 96/443 [01:00<03:28, 1.67it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 21%|██ | 94/443 [01:00<03:22, 1.72it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 22%|██▏ | 97/443 [01:00<03:27, 1.67it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 97/443 [01:00<03:27, 1.67it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:00<03:26, 1.67it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:00<03:26, 1.67it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:02, 2.82it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:02, 2.82it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:02, 2.82it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 23%|██▎ | 100/443 [01:01<02:01, 2.82it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 100/443 [01:01<02:01, 2.82it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:39, 1.58it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:39, 1.58it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:39, 1.58it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<03:39, 1.58it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<03:39, 1.58it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 97/443 [01:01<03:38, 1.58it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 97/443 [01:01<03:38, 1.58it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:16, 1.77it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:16, 1.77it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:16, 1.77it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<03:16, 1.77it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<03:16, 1.77it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:27, 1.68it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:27, 1.68it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:27, 1.68it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<03:26, 1.68it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<03:26, 1.68it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 97/443 [01:01<03:26, 1.68it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 97/443 [01:01<03:26, 1.68it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:42, 1.56it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:42, 1.56it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:42, 1.56it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<03:42, 1.56it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<03:42, 1.56it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 97/443 [01:01<03:41, 1.56it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 97/443 [01:01<03:41, 1.56it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 21%|██ | 92/443 [01:01<05:10, 1.13it/s, Materializing param=model.layers.8.input_layernorm.weight] Loading weights: 21%|██ | 92/443 [01:01<05:10, 1.13it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 92/443 [01:01<05:10, 1.13it/s, Materializing param=model.layers.8.mlp.down_proj.weight] Loading weights: 21%|██ | 93/443 [01:01<05:09, 1.13it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 93/443 [01:01<05:09, 1.13it/s, Materializing param=model.layers.8.mlp.gate_proj.weight] Loading weights: 21%|██ | 94/443 [01:01<05:08, 1.13it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██ | 94/443 [01:01<05:08, 1.13it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:01<05:07, 1.13it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 21%|██▏ | 95/443 [01:01<05:07, 1.13it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<05:06, 1.13it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<05:06, 1.13it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 97/443 [01:01<05:05, 1.13it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 97/443 [01:01<05:05, 1.13it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:25, 1.69it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:25, 1.69it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:25, 1.69it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<03:24, 1.69it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<03:24, 1.69it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 97/443 [01:01<03:24, 1.69it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 97/443 [01:01<03:24, 1.69it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:35, 1.61it/s, Materializing param=model.layers.8.mlp.up_proj.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:35, 1.61it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 21%|██▏ | 95/443 [01:01<03:35, 1.61it/s, Materializing param=model.layers.8.post_attention_layernorm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<03:35, 1.61it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 96/443 [01:01<03:35, 1.61it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 97/443 [01:01<03:34, 1.61it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 97/443 [01:01<03:34, 1.61it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 97/443 [01:01<02:27, 2.34it/s, Materializing param=model.layers.8.self_attn.k_norm.weight] Loading weights: 22%|██▏ | 97/443 [01:01<02:27, 2.34it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 97/443 [01:01<02:27, 2.34it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:27, 2.34it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:27, 2.34it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:10, 2.65it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:10, 2.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:10, 2.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:09, 2.67it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:10, 2.64it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:09, 2.67it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:05, 2.74it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:09, 2.67it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:10, 2.64it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:10, 2.64it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:10, 2.65it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:10, 2.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:10, 2.64it/s, Materializing param=model.layers.8.self_attn.k_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:10, 2.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:10, 2.64it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:05, 2.74it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:10, 2.64it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 98/443 [01:01<02:05, 2.74it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:45, 3.25it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:45, 3.25it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:45, 3.25it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:26, 2.34it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:26, 2.34it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 23%|██▎ | 100/443 [01:01<02:26, 2.34it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 100/443 [01:01<02:26, 2.34it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:41, 3.35it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:41, 3.35it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:41, 3.35it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 103/443 [01:01<01:41, 3.35it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:01<01:41, 3.35it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:09, 2.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:09, 2.65it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:09, 2.65it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 23%|██▎ | 100/443 [01:01<02:09, 2.65it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 100/443 [01:01<02:09, 2.65it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:11, 2.62it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:11, 2.62it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:11, 2.62it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 23%|██▎ | 100/443 [01:01<02:10, 2.62it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 100/443 [01:01<02:10, 2.62it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<02:10, 2.62it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<02:10, 2.62it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 102/443 [01:01<02:10, 2.62it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 102/443 [01:01<02:10, 2.62it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 103/443 [01:01<02:09, 2.62it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:01<02:09, 2.62it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:35, 3.57it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:35, 3.57it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:35, 3.57it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:35, 3.57it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:35, 3.57it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:14, 2.56it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:14, 2.56it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:14, 2.56it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 23%|██▎ | 100/443 [01:01<02:14, 2.56it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 100/443 [01:01<02:14, 2.56it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:11, 2.61it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:11, 2.61it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:11, 2.61it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:09, 2.65it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:09, 2.65it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 23%|██▎ | 100/443 [01:01<01:58, 2.89it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 23%|██▎ | 100/443 [01:01<01:58, 2.89it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 100/443 [01:01<01:58, 2.89it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:16, 2.52it/s, Materializing param=model.layers.8.self_attn.o_proj.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:16, 2.52it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 22%|██▏ | 99/443 [01:01<02:16, 2.52it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 23%|██▎ | 100/443 [01:01<02:15, 2.52it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 100/443 [01:01<02:15, 2.52it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:36, 3.54it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:36, 3.54it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:36, 3.54it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:36, 3.54it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:36, 3.54it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:37, 3.49it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:37, 3.49it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:37, 3.49it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:38, 3.49it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:38, 3.49it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:38, 3.49it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:49, 3.13it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:49, 3.13it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:49, 3.13it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 100/443 [01:01<01:56, 2.96it/s, Materializing param=model.layers.8.self_attn.q_norm.weight] Loading weights: 23%|██▎ | 100/443 [01:01<01:56, 2.96it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 100/443 [01:01<01:56, 2.96it/s, Materializing param=model.layers.8.self_attn.q_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:55, 2.96it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 101/443 [01:01<01:55, 2.96it/s, Materializing param=model.layers.8.self_attn.v_proj.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:37, 3.49it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:37, 3.49it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:37, 3.49it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:48, 3.13it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:48, 3.13it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:37, 3.49it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 103/443 [01:01<01:37, 3.49it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:01<01:37, 3.49it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:55, 2.96it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 102/443 [01:01<01:55, 2.96it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 103/443 [01:01<01:55, 2.96it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:01<01:55, 2.96it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:46, 2.04it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:05, 2.70it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:06, 2.67it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:46, 2.04it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:12, 2.56it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:46, 2.04it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:06, 2.67it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:03<02:33, 2.21it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:06, 2.67it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:12, 2.56it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:03<02:34, 2.20it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:12, 2.56it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:03<02:33, 2.21it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:05, 2.70it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:03<02:34, 2.20it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:03<02:33, 2.21it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:03<02:34, 2.20it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:34, 2.20it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:33, 2.21it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:05, 2.70it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:03<02:27, 2.30it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:34, 2.20it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:33, 2.21it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:03<02:36, 2.18it/s, Materializing param=model.layers.9.input_layernorm.weight] Loading weights: 23%|██▎ | 103/443 [01:03<02:27, 2.30it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:03<02:27, 2.30it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:03<02:36, 2.18it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:03<02:05, 2.70it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 23%|██▎ | 103/443 [01:03<02:36, 2.18it/s, Materializing param=model.layers.9.mlp.down_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:35, 2.18it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:35, 2.18it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:27, 2.30it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:03<02:05, 2.70it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 23%|██▎ | 104/443 [01:03<02:27, 2.30it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<01:57, 2.86it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<01:57, 2.86it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 106/443 [01:04<01:57, 2.86it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<01:57, 2.86it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<01:57, 2.86it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 108/443 [01:04<01:57, 2.86it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 108/443 [01:04<01:57, 2.86it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:18, 2.45it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:18, 2.45it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:18, 2.45it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:24, 2.34it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:15, 2.50it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:19, 2.43it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:24, 2.34it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:15, 2.50it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:24, 2.34it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:15, 2.50it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:19, 2.43it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:19, 2.43it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:17, 2.46it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:17, 2.46it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:17, 2.46it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:52, 1.96it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:52, 1.96it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:52, 1.96it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:17, 2.46it/s, Materializing param=model.layers.9.mlp.gate_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:17, 2.46it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▎ | 105/443 [01:04<02:17, 2.46it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<01:18, 4.26it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<01:18, 4.26it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<01:18, 4.26it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:50, 1.98it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:50, 1.98it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:50, 1.98it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:49, 1.98it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:49, 1.98it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:49, 1.98it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:49, 1.98it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:20, 2.40it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:20, 2.40it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:20, 2.40it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:19, 2.40it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:19, 2.40it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:19, 2.40it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:19, 2.40it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:48, 1.98it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:48, 1.98it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:23, 2.35it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:23, 2.35it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:23, 2.35it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:22, 2.35it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:22, 2.35it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:22, 2.35it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:22, 2.35it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:19, 2.40it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:19, 2.40it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:22, 2.35it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:22, 2.35it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:31, 2.23it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:32, 2.21it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:31, 2.23it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:31, 2.23it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:32, 2.21it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:32, 2.21it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:30, 2.23it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:30, 2.23it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:32, 2.21it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:32, 2.21it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:30, 2.23it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:30, 2.23it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:31, 2.21it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:31, 2.21it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:32, 2.22it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:32, 2.22it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:32, 2.22it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:31, 2.22it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:31, 2.22it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:31, 2.22it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:31, 2.22it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:38, 2.12it/s, Materializing param=model.layers.9.mlp.up_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:29, 2.23it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:38, 2.12it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:29, 2.23it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 24%|██▍ | 106/443 [01:04<02:38, 2.12it/s, Materializing param=model.layers.9.post_attention_layernorm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:38, 2.12it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 107/443 [01:04<02:38, 2.12it/s, Materializing param=model.layers.9.self_attn.k_norm.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:37, 2.12it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 24%|██▍ | 108/443 [01:04<02:37, 2.12it/s, Materializing param=model.layers.9.self_attn.k_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:31, 2.21it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:31, 2.21it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:30, 2.22it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:30, 2.22it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:37, 2.12it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 109/443 [01:04<02:37, 2.12it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:28, 3.76it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:28, 3.76it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:28, 3.76it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:19, 4.18it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:19, 4.18it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:18, 4.26it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:28, 3.76it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:19, 4.18it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:28, 3.76it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:18, 4.26it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:23, 4.00it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:23, 4.00it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:23, 4.00it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:17, 4.28it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:17, 4.28it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:17, 4.28it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:17, 4.28it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:17, 4.28it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:20, 4.15it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:20, 4.15it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:20, 4.15it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:20, 4.15it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:20, 4.15it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:22, 4.05it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:22, 4.05it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:22, 4.05it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:21, 4.05it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:21, 4.05it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:20, 4.12it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:20, 4.12it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 110/443 [01:04<01:20, 4.12it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:20, 4.12it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 111/443 [01:04<01:20, 4.12it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:06, 4.96it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:16, 4.30it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:06, 5.00it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:24, 3.91it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:06, 4.98it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 111/443 [01:05<01:21, 4.07it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:06, 4.96it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:16, 4.30it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:06, 5.00it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:06, 4.98it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:06, 4.96it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:16, 4.30it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:06, 5.00it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:06, 4.98it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 111/443 [01:05<01:21, 4.07it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 111/443 [01:05<01:21, 4.07it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:24, 3.91it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:21, 4.07it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:24, 3.91it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:21, 4.07it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:08, 4.84it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:08, 4.84it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:08, 4.84it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:05<01:32, 3.58it/s, Materializing param=model.layers.9.self_attn.o_proj.weight] Loading weights: 25%|██▍ | 110/443 [01:05<01:32, 3.58it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▍ | 110/443 [01:05<01:32, 3.58it/s, Materializing param=model.layers.9.self_attn.q_norm.weight] Loading weights: 25%|██▌ | 111/443 [01:05<01:32, 3.58it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 111/443 [01:05<01:32, 3.58it/s, Materializing param=model.layers.9.self_attn.q_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:32, 3.58it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 25%|██▌ | 112/443 [01:05<01:32, 3.58it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:06, 4.98it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:06, 4.98it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:05, 5.00it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:10, 4.70it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:05, 5.00it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:15, 4.36it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:02, 5.28it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:10, 4.70it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:10, 4.70it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:15, 4.36it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:15, 4.36it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 114/443 [01:05<00:52, 6.24it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:02, 5.28it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:02, 5.28it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 114/443 [01:05<00:52, 6.24it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 114/443 [01:05<00:52, 6.24it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 114/443 [01:05<01:15, 4.36it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 114/443 [01:05<01:02, 5.28it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:06, 4.96it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 114/443 [01:05<01:15, 4.36it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:06, 4.96it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 114/443 [01:05<01:02, 5.28it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:08, 4.84it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:08, 4.84it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 114/443 [01:05<00:53, 6.19it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 114/443 [01:05<00:53, 6.19it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 114/443 [01:05<00:53, 6.19it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 114/443 [01:05<00:54, 6.03it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 114/443 [01:05<00:54, 6.03it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 114/443 [01:05<00:54, 6.03it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 114/443 [01:05<01:43, 3.17it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 114/443 [01:05<01:27, 3.77it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 114/443 [01:05<01:43, 3.17it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 114/443 [01:05<01:43, 3.17it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 114/443 [01:05<01:27, 3.77it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<01:43, 3.17it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<01:43, 3.17it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 114/443 [01:05<01:27, 3.77it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<00:54, 6.03it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<01:20, 4.06it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<00:52, 6.24it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<00:53, 6.19it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<00:54, 6.03it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<01:20, 4.06it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<00:52, 6.24it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<01:32, 3.56it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<01:20, 4.06it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<00:53, 6.19it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<01:26, 3.77it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<01:26, 3.77it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<01:32, 3.56it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<01:32, 3.56it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:32, 3.58it/s, Materializing param=model.layers.9.self_attn.v_proj.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:32, 3.58it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 113/443 [01:05<01:32, 3.58it/s, Materializing param=model.layers.10.input_layernorm.weight] Loading weights: 26%|██▌ | 114/443 [01:05<01:31, 3.58it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 114/443 [01:05<01:31, 3.58it/s, Materializing param=model.layers.10.mlp.down_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<01:31, 3.58it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 115/443 [01:05<01:31, 3.58it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:42, 2.01it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:29, 2.18it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:19, 2.35it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:00, 2.72it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:30, 2.17it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:41, 2.02it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:42, 2.01it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:29, 2.18it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:00, 2.72it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:30, 2.17it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:19, 2.35it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:42, 2.01it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:41, 2.02it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:29, 2.18it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:00, 2.72it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:30, 2.17it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:41, 2.02it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:19, 2.35it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<03:00, 1.81it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<03:00, 1.81it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<03:00, 1.81it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:29, 2.19it/s, Materializing param=model.layers.10.mlp.gate_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:29, 2.19it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▌ | 116/443 [01:07<02:29, 2.19it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:29, 2.19it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:29, 2.19it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:28, 2.19it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:28, 2.19it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 119/443 [01:07<01:36, 3.34it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 119/443 [01:07<01:36, 3.34it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 119/443 [01:07<01:36, 3.34it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<01:36, 3.34it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<01:36, 3.34it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:11, 2.47it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:28, 2.20it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▋ | 117/443 [01:07<01:56, 2.79it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:20, 2.32it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:11, 2.47it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:28, 2.20it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:11, 2.47it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<01:56, 2.79it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:28, 2.20it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:42, 2.01it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:20, 2.32it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:28, 2.19it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:20, 2.32it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<01:56, 2.79it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:28, 2.19it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:28, 2.19it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:28, 2.19it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:11, 2.47it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:28, 2.19it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:11, 2.47it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:42, 2.01it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:42, 2.01it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:27, 2.20it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<01:56, 2.79it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:20, 2.32it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:27, 2.20it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<01:56, 2.79it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:20, 2.32it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:21, 2.31it/s, Materializing param=model.layers.10.mlp.up_proj.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:41, 2.01it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:21, 2.31it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 26%|██▋ | 117/443 [01:07<02:21, 2.31it/s, Materializing param=model.layers.10.post_attention_layernorm.weight] Loading weights: 27%|██▋ | 119/443 [01:07<01:56, 2.79it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 119/443 [01:07<01:56, 2.79it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 119/443 [01:07<02:27, 2.20it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 119/443 [01:07<02:19, 2.32it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:20, 2.31it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 119/443 [01:07<02:27, 2.20it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:41, 2.01it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 118/443 [01:07<02:20, 2.31it/s, Materializing param=model.layers.10.self_attn.k_norm.weight] Loading weights: 27%|██▋ | 119/443 [01:07<02:19, 2.32it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 119/443 [01:07<02:20, 2.31it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 119/443 [01:07<02:20, 2.31it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 119/443 [01:07<02:41, 2.01it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 119/443 [01:07<02:41, 2.01it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 119/443 [01:07<02:10, 2.47it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 119/443 [01:07<02:28, 2.19it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 119/443 [01:07<02:10, 2.47it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 119/443 [01:07<02:28, 2.19it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<02:19, 2.32it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<02:19, 2.32it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<02:26, 2.20it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<02:26, 2.20it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<01:55, 2.79it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<01:55, 2.79it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<02:10, 2.47it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<02:10, 2.47it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<02:27, 2.19it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<02:27, 2.19it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<02:19, 2.31it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:07<02:19, 2.31it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:18, 4.09it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:20, 4.01it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:18, 4.08it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:29, 3.60it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:15, 4.25it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:20, 4.01it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:18, 4.08it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:15, 4.25it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:29, 3.60it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:18, 4.08it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:20, 4.01it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:18, 4.09it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:15, 4.25it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:29, 3.60it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:18, 4.09it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:18, 4.08it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:15, 4.25it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:18, 4.08it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:29, 3.60it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:15, 4.25it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:29, 3.60it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:18, 4.09it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:18, 4.09it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:20, 4.01it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:20, 4.01it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:08<01:36, 3.34it/s, Materializing param=model.layers.10.self_attn.k_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:08<01:36, 3.34it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 120/443 [01:08<01:36, 3.34it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:36, 3.34it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:36, 3.34it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:36, 3.34it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:36, 3.34it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:10, 4.57it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:08, 4.69it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:09, 4.58it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:08, 4.67it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:11, 4.47it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:08, 4.69it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:10, 4.57it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:09, 4.58it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:08, 4.69it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:10, 4.57it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:09, 4.58it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:11, 4.47it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:11, 4.47it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:08, 4.67it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:08, 4.67it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:17, 4.11it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:17, 4.11it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:17, 4.11it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:17, 4.11it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:17, 4.11it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:09, 4.58it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:09, 4.58it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:08, 4.67it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:08, 4.67it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 125/443 [01:08<01:08, 4.67it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:08<01:08, 4.67it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:08, 4.69it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:08, 4.69it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 125/443 [01:08<01:07, 4.69it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:08<01:07, 4.69it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:11, 4.47it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:11, 4.47it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:09, 4.57it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:09, 4.57it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:28, 3.65it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:36, 3.33it/s, Materializing param=model.layers.10.self_attn.o_proj.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:28, 3.65it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:28, 3.65it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:36, 3.33it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 27%|██▋ | 121/443 [01:08<01:36, 3.33it/s, Materializing param=model.layers.10.self_attn.q_norm.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:27, 3.65it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:27, 3.65it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:27, 3.65it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:27, 3.65it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:36, 3.33it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 122/443 [01:08<01:36, 3.33it/s, Materializing param=model.layers.10.self_attn.q_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:36, 3.33it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 123/443 [01:08<01:36, 3.33it/s, Materializing param=model.layers.10.self_attn.v_proj.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:35, 3.33it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:35, 3.33it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:27, 3.65it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 124/443 [01:08<01:27, 3.65it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 125/443 [01:08<01:35, 3.33it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:08<01:35, 3.33it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:08<01:27, 3.65it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:08<01:27, 3.65it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:10<02:20, 2.27it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 126/443 [01:10<01:42, 3.09it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<02:02, 2.58it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<02:01, 2.62it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:10<02:22, 2.24it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 126/443 [01:10<01:48, 2.93it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:10<02:20, 2.27it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<01:42, 3.09it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<02:01, 2.62it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:10<02:27, 2.16it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 28%|██▊ | 125/443 [01:10<02:20, 2.27it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<01:48, 2.93it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<02:01, 2.62it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<01:42, 3.09it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<01:48, 2.93it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:10<02:27, 2.16it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:10<02:22, 2.24it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:10<02:27, 2.16it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:10<02:22, 2.24it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<02:02, 2.58it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<02:02, 2.58it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<02:26, 2.16it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<02:21, 2.24it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<02:26, 2.16it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<02:19, 2.27it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<02:21, 2.24it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:10<02:19, 2.27it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<02:34, 2.05it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:09, 1.67it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:05, 1.70it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:02, 1.73it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:05, 1.70it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<02:59, 1.76it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<02:34, 2.05it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:05, 1.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<02:42, 1.95it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<02:34, 2.05it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:02, 1.73it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<02:59, 1.76it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:05, 1.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:05, 1.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:12<03:43, 1.42it/s, Materializing param=model.layers.11.input_layernorm.weight] Loading weights: 29%|██▊ | 127/443 [01:12<02:42, 1.95it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<02:59, 1.76it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:02, 1.73it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<02:42, 1.95it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:05, 1.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:12<03:43, 1.42it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:09, 1.67it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 28%|██▊ | 125/443 [01:12<03:43, 1.42it/s, Materializing param=model.layers.11.mlp.down_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:09, 1.67it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:12<03:42, 1.42it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 28%|██▊ | 126/443 [01:12<03:42, 1.42it/s, Materializing param=model.layers.11.mlp.gate_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:41, 1.42it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▊ | 127/443 [01:12<03:41, 1.42it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:08, 1.27it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▉ | 128/443 [01:14<03:41, 1.42it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:08, 1.27it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:08, 1.27it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<03:41, 1.42it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<03:41, 1.42it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:12, 1.25it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:16, 1.23it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:12, 1.25it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<03:40, 1.42it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<03:30, 1.50it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:12, 1.25it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:11, 1.25it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▉ | 129/443 [01:14<03:40, 1.42it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:04, 1.29it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▉ | 128/443 [01:14<03:30, 1.50it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:11, 1.25it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<03:31, 1.49it/s, Materializing param=model.layers.11.mlp.up_proj.weight] Loading weights: 29%|██▉ | 129/443 [01:14<04:11, 1.25it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<03:30, 1.50it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:11, 1.25it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:04, 1.29it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<03:31, 1.49it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 130/443 [01:14<03:39, 1.42it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:16, 1.23it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<04:11, 1.25it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<03:31, 1.49it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 130/443 [01:14<03:39, 1.42it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:04, 1.29it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<03:29, 1.50it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<04:10, 1.25it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<03:29, 1.50it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<04:10, 1.25it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<03:30, 1.49it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<04:04, 1.29it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<03:30, 1.49it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<04:04, 1.29it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 128/443 [01:14<04:16, 1.23it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 130/443 [01:14<04:09, 1.25it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 130/443 [01:14<04:10, 1.25it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 130/443 [01:14<04:09, 1.25it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 130/443 [01:14<03:29, 1.49it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 130/443 [01:14<04:03, 1.29it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 130/443 [01:14<04:10, 1.25it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 130/443 [01:14<03:29, 1.49it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 130/443 [01:14<04:03, 1.29it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 129/443 [01:14<04:15, 1.23it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<04:15, 1.23it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 130/443 [01:14<04:14, 1.23it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 130/443 [01:14<04:14, 1.23it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 29%|██▉ | 129/443 [01:14<03:26, 1.52it/s, Materializing param=model.layers.11.post_attention_layernorm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<03:26, 1.52it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 29%|██▉ | 129/443 [01:14<03:26, 1.52it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:20, 2.22it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:30, 2.08it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:20, 2.22it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:26, 2.12it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:20, 2.22it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:26, 2.12it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:26, 2.12it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:29, 2.09it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:30, 2.08it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 29%|██▉ | 130/443 [01:14<02:38, 1.97it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:29, 2.09it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 29%|██▉ | 130/443 [01:14<02:38, 1.97it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:29, 2.09it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:30, 2.08it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 29%|██▉ | 130/443 [01:14<02:38, 1.97it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:37, 1.97it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:37, 1.97it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:29, 2.09it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:16, 2.29it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:29, 2.09it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:16, 2.29it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:29, 2.09it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 131/443 [01:14<02:16, 2.29it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:33, 2.03it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:26, 2.12it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:25, 2.14it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:33, 2.03it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:25, 2.14it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:33, 2.03it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:26, 2.12it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|███ | 133/443 [01:14<02:09, 2.39it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|███ | 133/443 [01:14<02:32, 2.03it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:25, 2.14it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|███ | 133/443 [01:14<02:09, 2.39it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:28, 2.09it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:22, 2.19it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:18, 2.25it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<02:09, 2.39it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:34, 2.02it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<02:32, 2.03it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:28, 2.09it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:22, 2.19it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 29%|██▉ | 130/443 [01:14<03:20, 1.56it/s, Materializing param=model.layers.11.self_attn.k_norm.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:18, 2.25it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:22, 2.19it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 29%|██▉ | 130/443 [01:14<03:20, 1.56it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:18, 2.25it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|███ | 133/443 [01:14<02:24, 2.14it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<02:11, 2.36it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 29%|██▉ | 130/443 [01:14<03:20, 1.56it/s, Materializing param=model.layers.11.self_attn.k_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<02:24, 2.14it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<02:21, 2.19it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<02:11, 2.36it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<02:21, 2.19it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<02:11, 2.36it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<02:18, 2.25it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<02:18, 2.25it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:34, 2.02it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|██▉ | 131/443 [01:14<03:19, 1.56it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 132/443 [01:14<02:34, 2.02it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|██▉ | 131/443 [01:14<03:19, 1.56it/s, Materializing param=model.layers.11.self_attn.o_proj.weight] Loading weights: 30%|██▉ | 132/443 [01:14<03:18, 1.56it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|██▉ | 132/443 [01:14<03:18, 1.56it/s, Materializing param=model.layers.11.self_attn.q_norm.weight] Loading weights: 30%|███ | 133/443 [01:14<02:33, 2.02it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<03:18, 1.56it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<03:18, 1.56it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 133/443 [01:14<02:33, 2.02it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:18, 2.24it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:19, 2.22it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:13, 2.32it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:06, 2.45it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:12, 2.33it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<01:53, 2.73it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:08, 2.41it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:06, 2.45it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:18, 2.24it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:12, 2.33it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:19, 2.22it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<01:53, 2.73it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:08, 2.41it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:18, 2.24it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:12, 2.33it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:19, 2.22it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<01:53, 2.73it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:08, 2.41it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:06, 2.45it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:13, 2.32it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:13, 2.32it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:04, 2.48it/s, Materializing param=model.layers.11.self_attn.q_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:04, 2.48it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 134/443 [01:15<02:04, 2.48it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 135/443 [01:15<02:04, 2.48it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 30%|███ | 135/443 [01:15<02:04, 2.48it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 31%|███ | 136/443 [01:15<02:03, 2.48it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 136/443 [01:15<02:03, 2.48it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 30%|███ | 135/443 [01:15<02:07, 2.41it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 30%|███ | 135/443 [01:15<02:07, 2.41it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 30%|███ | 135/443 [01:15<02:17, 2.24it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 30%|███ | 135/443 [01:15<02:17, 2.24it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 31%|███ | 136/443 [01:15<02:07, 2.41it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 136/443 [01:15<02:07, 2.41it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 136/443 [01:15<02:17, 2.24it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 136/443 [01:15<02:17, 2.24it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 30%|███ | 135/443 [01:15<02:12, 2.33it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 30%|███ | 135/443 [01:15<02:12, 2.33it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 30%|███ | 135/443 [01:15<02:12, 2.32it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 30%|███ | 135/443 [01:15<02:18, 2.22it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 30%|███ | 135/443 [01:15<02:18, 2.22it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 31%|███ | 136/443 [01:15<02:18, 2.22it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 136/443 [01:15<02:18, 2.22it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 30%|███ | 135/443 [01:15<02:12, 2.32it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 31%|███ | 136/443 [01:15<02:12, 2.32it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 30%|███ | 135/443 [01:15<02:05, 2.45it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 30%|███ | 135/443 [01:15<02:05, 2.45it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 31%|███ | 136/443 [01:15<02:12, 2.32it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 136/443 [01:15<02:05, 2.45it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 136/443 [01:15<02:05, 2.45it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 136/443 [01:17<03:09, 1.62it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 31%|███ | 137/443 [01:17<02:43, 1.87it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:47, 1.82it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:35, 1.97it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:46, 1.83it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:37, 1.94it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:47, 1.82it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:39, 1.92it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:35, 1.97it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 136/443 [01:17<03:09, 1.62it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:46, 1.83it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:37, 1.94it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:35, 1.97it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:47, 1.82it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 136/443 [01:17<03:09, 1.62it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:46, 1.83it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:39, 1.92it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:37, 1.94it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:39, 1.92it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:43, 1.87it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<03:08, 1.62it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<03:08, 1.62it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<02:43, 1.87it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 30%|███ | 135/443 [01:17<03:18, 1.55it/s, Materializing param=model.layers.11.self_attn.v_proj.weight] Loading weights: 30%|███ | 135/443 [01:17<03:18, 1.55it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 30%|███ | 135/443 [01:17<03:18, 1.55it/s, Materializing param=model.layers.12.input_layernorm.weight] Loading weights: 31%|███ | 136/443 [01:17<03:18, 1.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 136/443 [01:17<03:18, 1.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<03:17, 1.55it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 137/443 [01:17<03:17, 1.55it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:26, 1.48it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:58, 1.28it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:51, 1.32it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:26, 1.48it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:56, 1.29it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<04:09, 1.22it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:52, 1.31it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:58, 1.28it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:26, 1.48it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:51, 1.32it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<04:04, 1.25it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<04:08, 1.23it/s, Materializing param=model.layers.12.mlp.gate_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:58, 1.28it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:51, 1.32it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:56, 1.29it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:52, 1.31it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:56, 1.29it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<03:52, 1.31it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<04:09, 1.22it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<04:08, 1.23it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<04:09, 1.22it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<04:08, 1.23it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███ | 138/443 [01:19<04:04, 1.25it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███▏ | 139/443 [01:19<03:55, 1.29it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 31%|███▏ | 139/443 [01:19<03:55, 1.29it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 31%|███ | 138/443 [01:19<04:04, 1.25it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 32%|███▏ | 140/443 [01:19<03:54, 1.29it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 140/443 [01:19<03:54, 1.29it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 141/443 [01:19<03:53, 1.29it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 141/443 [01:19<03:53, 1.29it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:19<03:53, 1.29it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:19<03:53, 1.29it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:19<02:00, 2.50it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:19<02:00, 2.50it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 143/443 [01:19<02:00, 2.50it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 33%|███▎ | 144/443 [01:19<01:59, 2.50it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 144/443 [01:19<01:59, 2.50it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:52, 1.31it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:52, 1.31it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:52, 1.31it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 32%|███▏ | 140/443 [01:20<03:51, 1.31it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 140/443 [01:20<03:51, 1.31it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 141/443 [01:20<03:50, 1.31it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 141/443 [01:20<03:50, 1.31it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:41, 2.93it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:41, 2.93it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:41, 2.93it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:20<03:50, 1.31it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:20<03:50, 1.31it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:45, 1.35it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:45, 1.35it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:45, 1.35it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 32%|███▏ | 140/443 [01:20<03:45, 1.35it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 140/443 [01:20<03:45, 1.35it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 141/443 [01:20<03:44, 1.35it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 141/443 [01:20<03:44, 1.35it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:41, 2.93it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:41, 2.93it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:48, 1.33it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:48, 1.33it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:48, 1.33it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 32%|███▏ | 140/443 [01:20<03:47, 1.33it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 140/443 [01:20<03:47, 1.33it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 141/443 [01:20<03:46, 1.33it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 141/443 [01:20<03:46, 1.33it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:20<03:43, 1.35it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:20<03:43, 1.35it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:20<03:45, 1.33it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:20<03:45, 1.33it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:31, 1.44it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:31, 1.44it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:31, 1.44it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 31%|███▏ | 139/443 [01:20<04:06, 1.23it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 32%|███▏ | 140/443 [01:20<03:31, 1.44it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 140/443 [01:20<03:31, 1.44it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 31%|███▏ | 139/443 [01:20<04:06, 1.23it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 31%|███▏ | 139/443 [01:20<04:06, 1.23it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 32%|███▏ | 141/443 [01:20<03:30, 1.44it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 141/443 [01:20<03:30, 1.44it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 140/443 [01:20<04:06, 1.23it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 140/443 [01:20<04:06, 1.23it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 141/443 [01:20<04:05, 1.23it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 141/443 [01:20<04:05, 1.23it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:58, 1.27it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:58, 1.27it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 31%|███▏ | 139/443 [01:20<03:58, 1.27it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 32%|███▏ | 140/443 [01:20<03:58, 1.27it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 140/443 [01:20<03:58, 1.27it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 141/443 [01:20<03:57, 1.27it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 141/443 [01:20<03:57, 1.27it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 31%|███▏ | 139/443 [01:20<04:08, 1.22it/s, Materializing param=model.layers.12.mlp.up_proj.weight] Loading weights: 31%|███▏ | 139/443 [01:20<04:08, 1.22it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 31%|███▏ | 139/443 [01:20<04:08, 1.22it/s, Materializing param=model.layers.12.post_attention_layernorm.weight] Loading weights: 32%|███▏ | 140/443 [01:20<04:07, 1.22it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 140/443 [01:20<04:07, 1.22it/s, Materializing param=model.layers.12.self_attn.k_norm.weight] Loading weights: 32%|███▏ | 141/443 [01:20<04:06, 1.22it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 141/443 [01:20<04:06, 1.22it/s, Materializing param=model.layers.12.self_attn.k_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<01:58, 2.52it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<01:58, 2.52it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 142/443 [01:20<03:29, 1.44it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:20<03:29, 1.44it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<01:58, 2.52it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 142/443 [01:20<03:56, 1.27it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:20<03:56, 1.27it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:20<04:04, 1.23it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:20<04:04, 1.23it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:20<04:05, 1.22it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 142/443 [01:20<04:05, 1.22it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<01:59, 2.51it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<01:59, 2.51it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 143/443 [01:20<01:59, 2.51it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 143/443 [01:20<01:59, 2.50it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<01:59, 2.50it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 143/443 [01:20<01:59, 2.50it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:59, 2.50it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:59, 2.50it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<02:04, 2.41it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<02:04, 2.41it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 143/443 [01:20<02:04, 2.41it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 143/443 [01:20<02:04, 2.41it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<02:04, 2.41it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 143/443 [01:20<02:04, 2.41it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 33%|███▎ | 144/443 [01:20<02:04, 2.41it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 144/443 [01:20<02:04, 2.41it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<01:55, 2.59it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<01:55, 2.59it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 143/443 [01:20<01:55, 2.59it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:55, 2.59it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:55, 2.59it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:49, 2.74it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:49, 2.74it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:49, 2.74it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:48, 2.74it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:48, 2.74it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<02:03, 2.42it/s, Materializing param=model.layers.12.self_attn.o_proj.weight] Loading weights: 32%|███▏ | 143/443 [01:20<02:03, 2.42it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 32%|███▏ | 143/443 [01:20<02:03, 2.42it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 33%|███▎ | 144/443 [01:20<02:03, 2.42it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 144/443 [01:20<02:03, 2.42it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:48, 2.74it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:48, 2.74it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:48, 2.74it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:48, 2.74it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:50, 2.71it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:50, 2.71it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:50, 2.71it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:50, 2.71it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:50, 2.71it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:50, 2.72it/s, Materializing param=model.layers.12.self_attn.q_norm.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:50, 2.72it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 144/443 [01:20<01:50, 2.72it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:49, 2.72it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:49, 2.72it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:49, 2.71it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:49, 2.71it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:49, 2.71it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:49, 2.71it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:35, 3.12it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:35, 3.12it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:35, 3.12it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:49, 2.72it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:49, 2.72it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:48, 2.72it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:48, 2.72it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:35, 3.11it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:35, 3.11it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:35, 3.11it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:35, 3.12it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:35, 3.12it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:34, 3.12it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:34, 3.12it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:38, 3.02it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:38, 3.02it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:38, 3.02it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:35, 3.11it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:35, 3.11it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:35, 3.11it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:35, 3.11it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:33, 3.18it/s, Materializing param=model.layers.12.self_attn.q_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:33, 3.18it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 145/443 [01:20<01:33, 3.18it/s, Materializing param=model.layers.12.self_attn.v_proj.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:38, 3.02it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:38, 3.02it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:37, 3.02it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:37, 3.02it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:33, 3.18it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 146/443 [01:20<01:33, 3.18it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:33, 3.18it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 147/443 [01:20<01:33, 3.18it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:20, 3.68it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 147/443 [01:21<01:58, 2.50it/s, Materializing param=model.layers.13.input_layernorm.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:21, 3.63it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 147/443 [01:21<01:58, 2.50it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:20, 3.68it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:21, 3.63it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 147/443 [01:21<01:58, 2.50it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:20, 3.68it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:21, 3.63it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:58, 2.50it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:58, 2.50it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:24, 3.49it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:24, 3.49it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:24, 3.49it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:24, 3.49it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:24, 3.49it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:24, 3.49it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:28, 3.34it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:28, 3.34it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:28, 3.34it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:29, 3.31it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:29, 3.31it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:29, 3.31it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:25, 3.44it/s, Materializing param=model.layers.13.mlp.down_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:25, 3.44it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 33%|███▎ | 148/443 [01:21<01:25, 3.44it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:27, 1.99it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:15, 2.17it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:18, 2.13it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:15, 2.17it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:27, 1.99it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:15, 2.17it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:15, 2.17it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:27, 1.99it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:18, 2.13it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:17, 2.14it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:15, 2.17it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:18, 2.13it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:15, 2.17it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:15, 2.16it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:15, 2.17it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:11, 2.24it/s, Materializing param=model.layers.13.mlp.gate_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:15, 2.17it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:17, 2.14it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:15, 2.17it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:11, 2.24it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:15, 2.16it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:11, 2.24it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:17, 2.14it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▎ | 149/443 [01:23<02:15, 2.16it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:11, 2.24it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:11, 2.24it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:10, 2.24it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:10, 2.24it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:10, 2.24it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:10, 2.24it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:09, 2.24it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:09, 2.24it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:17, 3.71it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:17, 3.71it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:17, 3.71it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 155/443 [01:23<01:17, 3.71it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▍ | 155/443 [01:23<01:17, 3.71it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:23<01:17, 3.71it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:23<01:17, 3.71it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:16, 2.14it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:16, 2.14it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:16, 2.14it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:16, 2.14it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:16, 2.14it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:15, 2.14it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:15, 2.14it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 35%|███▌ | 157/443 [01:23<01:17, 3.71it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 35%|███▌ | 157/443 [01:23<01:17, 3.71it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 36%|███▌ | 158/443 [01:23<01:16, 3.71it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 158/443 [01:23<01:16, 3.71it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:15, 2.14it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:15, 2.14it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:14, 3.89it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:14, 3.89it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:14, 3.89it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 155/443 [01:23<01:13, 3.89it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▍ | 155/443 [01:23<01:13, 3.89it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:28, 1.97it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:30, 1.95it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:28, 1.97it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:28, 1.97it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:28, 1.97it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:28, 1.97it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:30, 1.95it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:30, 1.95it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:29, 1.95it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:29, 1.95it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:29, 1.95it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:29, 1.95it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:31, 1.93it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:31, 1.93it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:31, 1.93it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:30, 1.93it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:30, 1.93it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:30, 1.93it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:30, 1.93it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:29, 1.96it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:29, 1.96it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:29, 1.96it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:28, 1.96it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:28, 1.96it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:30, 1.95it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:30, 1.95it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:30, 1.95it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:29, 1.95it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:29, 1.95it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:38, 1.85it/s, Materializing param=model.layers.13.mlp.up_proj.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:38, 1.85it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 150/443 [01:23<02:38, 1.85it/s, Materializing param=model.layers.13.post_attention_layernorm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:38, 1.85it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 151/443 [01:23<02:38, 1.85it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:37, 1.85it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:37, 1.85it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:37, 1.85it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:37, 1.85it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:28, 1.95it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:29, 1.93it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:29, 1.93it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:28, 1.95it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:27, 1.97it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:27, 1.97it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:27, 1.97it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:27, 1.97it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:29, 1.95it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 152/443 [01:23<02:29, 1.95it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:28, 1.95it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<02:28, 1.95it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:23<00:59, 4.79it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:23<00:59, 4.79it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:23<00:59, 4.79it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 157/443 [01:23<00:59, 4.79it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 35%|███▌ | 157/443 [01:23<00:59, 4.79it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 36%|███▌ | 158/443 [01:23<00:59, 4.79it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 158/443 [01:23<00:59, 4.79it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 34%|███▍ | 152/443 [01:23<01:46, 2.73it/s, Materializing param=model.layers.13.self_attn.k_norm.weight] Loading weights: 34%|███▍ | 152/443 [01:23<01:46, 2.73it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 34%|███▍ | 152/443 [01:23<01:46, 2.73it/s, Materializing param=model.layers.13.self_attn.k_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<01:46, 2.73it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 153/443 [01:23<01:46, 2.73it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:46, 2.73it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:46, 2.73it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 155/443 [01:23<01:45, 2.73it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▍ | 155/443 [01:23<01:45, 2.73it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:22, 3.51it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:28, 3.25it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:22, 3.52it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:22, 3.51it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:28, 3.25it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:22, 3.52it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:28, 3.25it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:22, 3.52it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 155/443 [01:23<01:28, 3.25it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:22, 3.51it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 155/443 [01:23<01:28, 3.25it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:22, 3.51it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:22, 3.51it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:22, 3.51it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 155/443 [01:23<01:21, 3.51it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▍ | 155/443 [01:23<01:21, 3.51it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:21, 3.54it/s, Materializing param=model.layers.13.self_attn.o_proj.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:21, 3.54it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 154/443 [01:23<01:21, 3.54it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 155/443 [01:23<01:21, 3.54it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▍ | 155/443 [01:23<01:21, 3.54it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:01, 4.67it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:01, 4.67it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:01, 4.67it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:06, 4.29it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:06, 4.29it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:06, 4.29it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▍ | 155/443 [01:24<01:17, 3.71it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 155/443 [01:24<01:17, 3.71it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▍ | 155/443 [01:24<01:17, 3.71it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:17, 3.71it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:17, 3.71it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:12, 3.94it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:12, 3.94it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:12, 3.94it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 157/443 [01:24<01:06, 4.29it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 35%|███▌ | 157/443 [01:24<01:06, 4.29it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:06, 4.29it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:06, 4.29it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 35%|███▌ | 157/443 [01:24<01:01, 4.67it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 35%|███▌ | 157/443 [01:24<01:01, 4.67it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:01, 4.67it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:01, 4.67it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 35%|███▌ | 157/443 [01:24<01:12, 3.94it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 35%|███▌ | 157/443 [01:24<01:12, 3.94it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 35%|███▍ | 155/443 [01:24<01:19, 3.63it/s, Materializing param=model.layers.13.self_attn.q_norm.weight] Loading weights: 35%|███▍ | 155/443 [01:24<01:19, 3.63it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▍ | 155/443 [01:24<01:19, 3.63it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:19, 3.63it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:19, 3.63it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 157/443 [01:24<01:17, 3.71it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 35%|███▌ | 157/443 [01:24<01:17, 3.71it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:16, 3.71it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:16, 3.71it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:08, 4.19it/s, Materializing param=model.layers.13.self_attn.q_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:08, 4.19it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 156/443 [01:24<01:08, 4.19it/s, Materializing param=model.layers.13.self_attn.v_proj.weight] Loading weights: 35%|███▌ | 157/443 [01:24<01:18, 3.63it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 35%|███▌ | 157/443 [01:24<01:18, 3.63it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 35%|███▌ | 157/443 [01:24<01:08, 4.19it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 35%|███▌ | 157/443 [01:24<01:08, 4.19it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:08, 4.19it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:08, 4.19it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:18, 3.61it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:03, 4.47it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:11, 3.97it/s, Materializing param=model.layers.14.input_layernorm.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:05, 4.31it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:16, 3.73it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:11, 3.97it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:03, 4.51it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:04, 4.40it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:05, 4.31it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:16, 3.73it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:11, 3.97it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:05, 4.31it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:03, 4.47it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:18, 3.61it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:03, 4.51it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:11, 3.97it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:16, 3.73it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:04, 4.40it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:18, 3.61it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:03, 4.47it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:03, 4.51it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:04, 4.40it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:11, 3.97it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:18, 3.61it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:18, 3.61it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 158/443 [01:24<01:11, 3.97it/s, Materializing param=model.layers.14.mlp.down_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:11, 3.97it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 159/443 [01:24<01:11, 3.97it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:10, 2.16it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:14, 2.10it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:10, 2.16it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:05, 2.26it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:05, 2.26it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:19, 2.02it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:13, 2.11it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:10, 2.16it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:10, 2.16it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:05, 2.26it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:14, 2.10it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:13, 2.11it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:10, 2.16it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:10, 2.16it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:05, 2.26it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:19, 2.02it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:14, 2.10it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:13, 2.11it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:19, 2.02it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:05, 2.26it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:05, 2.26it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:09, 2.19it/s, Materializing param=model.layers.14.mlp.gate_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:09, 2.19it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▌ | 160/443 [01:26<02:09, 2.19it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:11, 1.47it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:23, 1.39it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:23, 1.39it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:23, 1.39it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:23, 1.39it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:11, 1.47it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:23, 1.39it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:17, 1.43it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:23, 1.39it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:11, 1.47it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:19, 1.42it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:17, 1.43it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:19, 1.42it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:19, 1.42it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 37%|███▋ | 162/443 [01:28<03:22, 1.39it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:17, 1.43it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:17, 1.43it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<02:59, 1.57it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 37%|███▋ | 162/443 [01:28<03:11, 1.47it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 37%|███▋ | 162/443 [01:28<03:22, 1.39it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:17, 1.43it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<02:59, 1.57it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 37%|███▋ | 162/443 [01:28<03:16, 1.43it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:17, 1.43it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:28, 1.35it/s, Materializing param=model.layers.14.mlp.up_proj.weight] Loading weights: 37%|███▋ | 162/443 [01:28<03:11, 1.47it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<02:59, 1.57it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 37%|███▋ | 162/443 [01:28<03:16, 1.43it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:28, 1.35it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 37%|███▋ | 162/443 [01:28<03:18, 1.42it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 36%|███▋ | 161/443 [01:28<03:28, 1.35it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 37%|███▋ | 162/443 [01:28<02:59, 1.57it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 37%|███▋ | 163/443 [01:28<03:15, 1.43it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 162/443 [01:28<03:18, 1.42it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 37%|███▋ | 163/443 [01:28<03:22, 1.39it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 162/443 [01:28<02:59, 1.57it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 37%|███▋ | 163/443 [01:28<03:15, 1.43it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 163/443 [01:28<03:22, 1.39it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 163/443 [01:28<03:17, 1.42it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 162/443 [01:28<03:27, 1.35it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 37%|███▋ | 162/443 [01:28<03:16, 1.43it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 37%|███▋ | 162/443 [01:28<03:16, 1.43it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 37%|███▋ | 163/443 [01:28<03:17, 1.42it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 162/443 [01:28<03:27, 1.35it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 37%|███▋ | 163/443 [01:28<02:58, 1.57it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 163/443 [01:28<02:58, 1.57it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 163/443 [01:28<03:15, 1.43it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 163/443 [01:28<03:26, 1.35it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 163/443 [01:28<03:26, 1.35it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 163/443 [01:28<03:15, 1.43it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 162/443 [01:28<02:49, 1.65it/s, Materializing param=model.layers.14.post_attention_layernorm.weight] Loading weights: 37%|███▋ | 162/443 [01:28<02:49, 1.65it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 37%|███▋ | 162/443 [01:28<02:49, 1.65it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 37%|███▋ | 163/443 [01:28<02:49, 1.65it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 163/443 [01:28<02:49, 1.65it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:01, 2.29it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:01, 2.29it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:01, 2.31it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<01:59, 2.33it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:03, 2.27it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:01, 2.29it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<01:59, 2.33it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 163/443 [01:28<02:18, 2.02it/s, Materializing param=model.layers.14.self_attn.k_norm.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:01, 2.31it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:03, 2.27it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<01:59, 2.33it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:01, 2.31it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:03, 2.27it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 163/443 [01:28<02:18, 2.02it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 163/443 [01:28<02:18, 2.02it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:17, 2.02it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:05, 2.22it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:01, 2.29it/s, Materializing param=model.layers.14.self_attn.k_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:17, 2.02it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:05, 2.22it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:05, 2.22it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:01, 2.29it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:01, 2.29it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:48, 1.65it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 164/443 [01:28<02:48, 1.65it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:28<02:48, 1.65it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 165/443 [01:28<02:48, 1.65it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 166/443 [01:28<02:47, 1.65it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 166/443 [01:28<02:47, 1.65it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:28<01:46, 2.62it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:28<01:46, 2.62it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 165/443 [01:28<01:46, 2.62it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 166/443 [01:28<01:45, 2.62it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 166/443 [01:28<01:45, 2.62it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:18, 3.53it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:18, 3.53it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:18, 3.53it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:50, 2.51it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:50, 2.51it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:50, 2.51it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 166/443 [01:29<01:50, 2.51it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:44, 2.67it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 166/443 [01:29<01:50, 2.51it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:51, 2.49it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:51, 2.49it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:51, 2.49it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:44, 2.67it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 166/443 [01:29<01:51, 2.49it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 166/443 [01:29<01:51, 2.49it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:44, 2.67it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 166/443 [01:29<01:43, 2.67it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 166/443 [01:29<01:43, 2.67it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:52, 2.48it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:51, 2.50it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:52, 2.48it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:51, 2.50it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:52, 2.48it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 165/443 [01:29<01:51, 2.50it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 166/443 [01:29<01:50, 2.50it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 166/443 [01:29<01:50, 2.50it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 166/443 [01:29<01:51, 2.48it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 166/443 [01:29<01:51, 2.48it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:16, 3.61it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:16, 3.61it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:16, 3.61it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:11, 3.85it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:11, 3.85it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:11, 3.85it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 169/443 [01:29<01:11, 3.85it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 169/443 [01:29<01:11, 3.85it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:16, 3.61it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:16, 3.61it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 169/443 [01:29<00:56, 4.83it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 169/443 [01:29<00:56, 4.83it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 169/443 [01:29<00:56, 4.83it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:23, 3.32it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:23, 3.32it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:23, 3.32it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:29<02:01, 2.29it/s, Materializing param=model.layers.14.self_attn.o_proj.weight] Loading weights: 37%|███▋ | 165/443 [01:29<02:01, 2.29it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 165/443 [01:29<02:01, 2.29it/s, Materializing param=model.layers.14.self_attn.q_norm.weight] Loading weights: 37%|███▋ | 166/443 [01:29<02:00, 2.29it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 37%|███▋ | 166/443 [01:29<02:00, 2.29it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<02:00, 2.29it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<02:00, 2.29it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:22, 3.35it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:22, 3.35it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:22, 3.35it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:22, 3.35it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:22, 3.35it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:22, 3.35it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:18, 3.52it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:22, 3.36it/s, Materializing param=model.layers.14.self_attn.q_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:22, 3.36it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:18, 3.52it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:22, 3.36it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 167/443 [01:29<01:18, 3.52it/s, Materializing param=model.layers.14.self_attn.v_proj.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:22, 3.32it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:22, 3.32it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 169/443 [01:29<01:22, 3.32it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 169/443 [01:29<01:22, 3.32it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:59, 2.29it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:59, 2.29it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 169/443 [01:29<01:59, 2.29it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 169/443 [01:29<01:59, 2.29it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:18, 3.52it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:22, 3.35it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:22, 3.35it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 169/443 [01:29<01:21, 3.35it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 169/443 [01:29<01:21, 3.35it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:18, 3.52it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 169/443 [01:29<01:17, 3.52it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 169/443 [01:29<01:17, 3.52it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:22, 3.35it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:21, 3.36it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:22, 3.35it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 168/443 [01:29<01:21, 3.36it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 169/443 [01:29<01:21, 3.35it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 169/443 [01:29<01:21, 3.35it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 169/443 [01:30<02:10, 2.10it/s, Materializing param=model.layers.15.input_layernorm.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:52, 2.43it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:49, 2.49it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:52, 2.42it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:52, 2.42it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<00:56, 4.83it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<02:00, 2.26it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:46, 2.57it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:52, 2.43it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:52, 2.42it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:52, 2.43it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:52, 2.42it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:46, 2.57it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<00:56, 4.83it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:52, 2.42it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<02:00, 2.26it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 169/443 [01:30<02:10, 2.10it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:52, 2.42it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:49, 2.49it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:46, 2.57it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 169/443 [01:30<02:10, 2.10it/s, Materializing param=model.layers.15.mlp.down_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<02:00, 2.26it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<01:49, 2.49it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<02:10, 2.10it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 38%|███▊ | 170/443 [01:30<02:10, 2.10it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<02:59, 1.52it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<03:01, 1.50it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<02:53, 1.57it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<02:58, 1.53it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<02:44, 1.65it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<03:17, 1.37it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<03:12, 1.41it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<03:02, 1.49it/s, Materializing param=model.layers.15.mlp.gate_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<02:59, 1.52it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<03:01, 1.50it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<02:53, 1.57it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<02:59, 1.52it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<02:44, 1.65it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<03:17, 1.37it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<03:01, 1.50it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<03:12, 1.41it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<03:02, 1.49it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<02:53, 1.57it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<02:44, 1.65it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<03:17, 1.37it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<03:12, 1.41it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<03:02, 1.49it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<02:58, 1.53it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▊ | 171/443 [01:32<02:58, 1.53it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:51, 1.17it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▉ | 172/443 [01:34<04:07, 1.10it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:56, 1.14it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▉ | 172/443 [01:34<04:07, 1.10it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:51, 1.17it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:56, 1.14it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<04:07, 1.10it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:56, 1.14it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:51, 1.17it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<04:06, 1.10it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▉ | 172/443 [01:34<04:06, 1.10it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 173/443 [01:34<03:50, 1.17it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<04:06, 1.10it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 173/443 [01:34<03:50, 1.17it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 173/443 [01:34<04:05, 1.10it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 173/443 [01:34<04:06, 1.10it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 173/443 [01:34<04:05, 1.10it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:47, 1.19it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▉ | 173/443 [01:34<03:55, 1.14it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:47, 1.19it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 173/443 [01:34<04:06, 1.10it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:51, 1.17it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:55, 1.15it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:47, 1.19it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 173/443 [01:34<03:55, 1.14it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:33, 1.27it/s, Materializing param=model.layers.15.mlp.up_proj.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:55, 1.15it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:51, 1.17it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:55, 1.15it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 174/443 [01:34<03:55, 1.14it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 173/443 [01:34<03:46, 1.19it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:33, 1.27it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 174/443 [01:34<03:55, 1.14it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 173/443 [01:34<03:46, 1.19it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:51, 1.17it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 172/443 [01:34<03:33, 1.27it/s, Materializing param=model.layers.15.post_attention_layernorm.weight] Loading weights: 39%|███▉ | 174/443 [01:34<04:05, 1.10it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 174/443 [01:34<03:45, 1.19it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 173/443 [01:34<03:54, 1.15it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 174/443 [01:34<04:05, 1.10it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 174/443 [01:34<03:45, 1.19it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 173/443 [01:34<03:50, 1.17it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 173/443 [01:34<03:54, 1.15it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 173/443 [01:34<03:50, 1.17it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 174/443 [01:34<03:54, 1.15it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 174/443 [01:34<03:50, 1.17it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 174/443 [01:34<03:54, 1.15it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 174/443 [01:34<03:50, 1.17it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 173/443 [01:34<03:33, 1.27it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 173/443 [01:34<03:33, 1.27it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 174/443 [01:34<03:32, 1.27it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 174/443 [01:34<03:32, 1.27it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 174/443 [01:34<02:46, 1.61it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 39%|███▉ | 174/443 [01:34<02:46, 1.61it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 174/443 [01:34<02:46, 1.61it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:45, 1.61it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:45, 1.61it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:13, 2.01it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:17, 1.95it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 174/443 [01:34<02:35, 1.73it/s, Materializing param=model.layers.15.self_attn.k_norm.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:13, 2.01it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:09, 2.08it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:13, 2.01it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:13, 2.01it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:13, 2.01it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:13, 2.01it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:17, 1.95it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:09, 2.08it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:17, 1.95it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:09, 2.08it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 39%|███▉ | 174/443 [01:34<02:35, 1.73it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 39%|███▉ | 174/443 [01:34<02:35, 1.73it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:35, 1.73it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:35, 1.73it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:10, 2.05it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:13, 2.00it/s, Materializing param=model.layers.15.self_attn.k_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:10, 2.05it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:13, 2.00it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:10, 2.05it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 175/443 [01:34<02:13, 2.00it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:12, 2.01it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:16, 1.95it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:12, 2.01it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:16, 1.95it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:12, 2.01it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:12, 2.01it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:12, 2.01it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:13, 2.01it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:05, 2.13it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:10, 2.05it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:13, 2.00it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:52, 2.36it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:13, 2.01it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:54, 2.32it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:10, 2.05it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:09, 2.07it/s, Materializing param=model.layers.15.self_attn.o_proj.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:13, 2.00it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:52, 2.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:54, 2.32it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:52, 2.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:52, 2.36it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:09, 2.07it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:50, 2.40it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:52, 2.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:09, 2.07it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:50, 2.40it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:52, 2.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:52, 2.36it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:54, 2.32it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:50, 2.40it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:05, 2.13it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:52, 2.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 177/443 [01:35<02:12, 2.01it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 177/443 [01:35<02:08, 2.07it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 176/443 [01:35<02:05, 2.13it/s, Materializing param=model.layers.15.self_attn.q_norm.weight] Loading weights: 40%|███▉ | 177/443 [01:35<01:52, 2.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 177/443 [01:35<02:08, 2.07it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 177/443 [01:35<02:12, 2.01it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 177/443 [01:35<02:04, 2.13it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|███▉ | 177/443 [01:35<02:04, 2.13it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:55, 2.30it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:47, 2.46it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:45, 2.52it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:56, 2.27it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:53, 2.33it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:54, 2.30it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:47, 2.46it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:55, 2.30it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:50, 2.40it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:55, 2.30it/s, Materializing param=model.layers.15.self_attn.q_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:56, 2.27it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:55, 2.30it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:54, 2.30it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:53, 2.33it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:47, 2.46it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:50, 2.40it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:54, 2.30it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:55, 2.30it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:56, 2.27it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:55, 2.30it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:50, 2.40it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:45, 2.52it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:45, 2.52it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 178/443 [01:35<01:53, 2.33it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 179/443 [01:35<01:56, 2.27it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:49, 2.40it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:44, 2.52it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:54, 2.30it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:47, 2.46it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:47, 2.46it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:54, 2.30it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:54, 2.30it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:54, 2.30it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:44, 2.52it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:35, 2.75it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 40%|████ | 179/443 [01:35<01:49, 2.40it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:56, 2.27it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 41%|████ | 180/443 [01:35<01:20, 3.27it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:35, 2.75it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 40%|████ | 179/443 [01:35<01:35, 2.75it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 41%|████ | 180/443 [01:35<01:20, 3.27it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 180/443 [01:35<01:20, 3.27it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 40%|████ | 179/443 [01:35<01:37, 2.72it/s, Materializing param=model.layers.15.self_attn.v_proj.weight] Loading weights: 41%|████ | 180/443 [01:35<01:19, 3.31it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 41%|████ | 180/443 [01:35<01:21, 3.23it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 41%|████ | 180/443 [01:35<01:35, 2.75it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 180/443 [01:35<01:19, 3.31it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 180/443 [01:35<01:35, 2.75it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 40%|████ | 179/443 [01:35<01:37, 2.72it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 41%|████ | 180/443 [01:35<01:21, 3.23it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 180/443 [01:35<01:19, 3.31it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 180/443 [01:35<01:21, 3.23it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 40%|████ | 179/443 [01:35<01:37, 2.72it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 41%|████ | 180/443 [01:35<01:36, 2.72it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 180/443 [01:35<01:36, 2.72it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<01:20, 3.27it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<01:19, 3.31it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<01:21, 3.23it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<01:20, 3.27it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<01:19, 3.31it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<02:32, 1.72it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 180/443 [01:37<02:35, 1.69it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 41%|████ | 181/443 [01:37<02:32, 1.71it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<01:21, 3.23it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<02:32, 1.72it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<02:32, 1.71it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<02:32, 1.72it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<02:32, 1.71it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 180/443 [01:37<02:36, 1.68it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 41%|████ | 180/443 [01:37<02:36, 1.68it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 180/443 [01:37<02:36, 1.68it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 180/443 [01:37<02:35, 1.69it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<02:36, 1.68it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 180/443 [01:37<02:35, 1.69it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<02:36, 1.68it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<02:34, 1.69it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 181/443 [01:37<02:34, 1.69it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 180/443 [01:38<02:56, 1.49it/s, Materializing param=model.layers.16.input_layernorm.weight] Loading weights: 41%|████ | 180/443 [01:38<02:56, 1.49it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 180/443 [01:38<02:56, 1.49it/s, Materializing param=model.layers.16.mlp.down_proj.weight] Loading weights: 41%|████ | 181/443 [01:38<02:55, 1.49it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 181/443 [01:38<02:55, 1.49it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:53, 1.12it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:20, 1.30it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<04:04, 1.07it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:45, 1.16it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<04:05, 1.06it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:25, 1.27it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:54, 1.11it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:20, 1.30it/s, Materializing param=model.layers.16.mlp.gate_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:53, 1.12it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<04:04, 1.07it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<04:05, 1.06it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:53, 1.12it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<04:04, 1.07it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<04:05, 1.06it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:45, 1.16it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:20, 1.30it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:54, 1.11it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:25, 1.27it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:45, 1.16it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:25, 1.27it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:54, 1.11it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:20, 1.30it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:20, 1.30it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████ | 182/443 [01:40<03:20, 1.30it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:24, 1.27it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:24, 1.27it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:24, 1.27it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:24, 1.27it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:23, 1.27it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:23, 1.27it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:40<03:22, 1.27it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:40<03:22, 1.27it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:40<01:46, 2.41it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:40<01:46, 2.41it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 187/443 [01:40<01:46, 2.41it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 188/443 [01:40<01:45, 2.41it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 42%|████▏ | 188/443 [01:40<01:45, 2.41it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:39, 1.19it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:39, 1.19it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:39, 1.19it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:38, 1.19it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:38, 1.19it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:35, 1.20it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:35, 1.20it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:35, 1.20it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:35, 1.20it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:35, 1.20it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:34, 1.20it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:34, 1.20it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:37, 1.19it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:37, 1.19it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:40<03:36, 1.19it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:40<03:36, 1.19it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:31, 1.23it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:31, 1.23it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:31, 1.23it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:30, 1.23it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:30, 1.23it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 186/443 [01:40<03:33, 1.20it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:40<03:33, 1.20it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:29, 1.23it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:29, 1.23it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:40<03:29, 1.23it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:40<03:29, 1.23it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:40<01:41, 2.52it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:40<01:41, 2.52it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 187/443 [01:40<01:41, 2.52it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 188/443 [01:40<01:41, 2.52it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 42%|████▏ | 188/443 [01:40<01:41, 2.52it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:40<01:29, 2.84it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:40<01:29, 2.84it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:40<01:29, 2.84it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:40<01:50, 2.33it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:40<01:50, 2.33it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 187/443 [01:40<01:50, 2.33it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 187/443 [01:40<01:49, 2.35it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:40<01:49, 2.35it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 187/443 [01:40<01:49, 2.35it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 188/443 [01:40<01:48, 2.35it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 42%|████▏ | 188/443 [01:40<01:48, 2.35it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 190/443 [01:40<01:28, 2.84it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 190/443 [01:40<01:28, 2.84it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:58, 1.09it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:58, 1.09it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:58, 1.09it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:57, 1.09it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:57, 1.09it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:56, 1.09it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:56, 1.09it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:49, 1.13it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:23, 1.28it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:23, 1.28it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:23, 1.28it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:49, 1.13it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:22, 1.28it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:49, 1.13it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:22, 1.28it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:48, 1.13it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:48, 1.13it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:22, 1.28it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:22, 1.28it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:48, 1.13it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:48, 1.13it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:23, 1.28it/s, Materializing param=model.layers.16.mlp.up_proj.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:23, 1.28it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 41%|████▏ | 183/443 [01:40<03:23, 1.28it/s, Materializing param=model.layers.16.post_attention_layernorm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:22, 1.28it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 184/443 [01:40<03:22, 1.28it/s, Materializing param=model.layers.16.self_attn.k_norm.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:22, 1.28it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 185/443 [01:40<03:22, 1.28it/s, Materializing param=model.layers.16.self_attn.k_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:40<03:56, 1.09it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:40<03:56, 1.09it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:41<03:47, 1.13it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:41<03:47, 1.13it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:41<03:21, 1.28it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:41<03:21, 1.28it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:37, 2.61it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:37, 2.61it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:37, 2.61it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:37, 2.61it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:37, 2.61it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:41<03:21, 1.28it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 186/443 [01:41<03:21, 1.28it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:23, 3.04it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:23, 3.04it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:23, 3.04it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:36, 2.61it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:36, 2.61it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:23, 3.04it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:23, 3.04it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:36, 2.61it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:36, 2.61it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:22, 3.04it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:22, 3.04it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:20, 3.17it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:20, 3.17it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:20, 3.17it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:19, 3.17it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:19, 3.17it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:19, 3.17it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:19, 3.17it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:41<01:49, 2.34it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:41<01:49, 2.34it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 187/443 [01:41<01:49, 2.34it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 187/443 [01:41<01:45, 2.44it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:41<01:45, 2.44it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 187/443 [01:41<01:45, 2.44it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 187/443 [01:41<01:55, 2.21it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:41<01:55, 2.21it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 187/443 [01:41<01:55, 2.21it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:55, 2.21it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:55, 2.21it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:41<01:44, 2.45it/s, Materializing param=model.layers.16.self_attn.o_proj.weight] Loading weights: 42%|████▏ | 187/443 [01:41<01:44, 2.45it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 187/443 [01:41<01:44, 2.45it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:44, 2.45it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:44, 2.45it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:43, 2.45it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:54, 2.21it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:54, 2.21it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:43, 2.45it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:32, 2.75it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:32, 2.75it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:32, 2.75it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:32, 2.75it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:32, 2.75it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:16, 3.29it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:16, 3.29it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:16, 3.29it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:16, 3.29it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:16, 3.29it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:10, 3.61it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:10, 3.61it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:10, 3.61it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:09, 3.61it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:09, 3.61it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:35, 2.67it/s, Materializing param=model.layers.16.self_attn.q_norm.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:35, 2.67it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 42%|████▏ | 188/443 [01:41<01:35, 2.67it/s, Materializing param=model.layers.16.self_attn.q_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:35, 2.67it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 189/443 [01:41<01:35, 2.67it/s, Materializing param=model.layers.16.self_attn.v_proj.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:34, 2.67it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:34, 2.67it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:34, 2.67it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:34, 2.67it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:31, 2.75it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 190/443 [01:41<01:31, 2.75it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:31, 2.75it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 191/443 [01:41<01:31, 2.75it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 191/443 [01:42<01:54, 2.19it/s, Materializing param=model.layers.17.input_layernorm.weight] Loading weights: 43%|████▎ | 191/443 [01:42<01:54, 2.19it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 191/443 [01:42<01:54, 2.19it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:27, 2.86it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:34, 2.66it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:22, 3.04it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:31, 2.75it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:31, 2.74it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:21, 3.06it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:34, 2.66it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:22, 3.04it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:31, 2.75it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:22, 3.04it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:31, 2.74it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:31, 2.75it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:21, 3.06it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:34, 2.66it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:31, 2.74it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:21, 3.06it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:27, 2.86it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:54, 2.19it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:54, 2.19it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:27, 2.86it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:40, 2.51it/s, Materializing param=model.layers.17.mlp.down_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:40, 2.51it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 43%|████▎ | 192/443 [01:42<01:40, 2.51it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:35, 1.61it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:31, 1.65it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:35, 1.61it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<01:27, 2.86it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:26, 1.71it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:31, 1.65it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:23, 1.74it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:32, 1.64it/s, Materializing param=model.layers.17.mlp.gate_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:35, 1.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:35, 1.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:26, 1.71it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:31, 1.65it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:35, 1.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:26, 1.71it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:35, 1.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:32, 1.64it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:23, 1.74it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:23, 1.74it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<01:27, 2.86it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<02:32, 1.64it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<01:39, 2.51it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▎ | 193/443 [01:44<01:39, 2.51it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:44<02:13, 1.86it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:44<02:13, 1.86it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 194/443 [01:44<02:13, 1.86it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 195/443 [01:44<02:12, 1.86it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 195/443 [01:44<02:12, 1.86it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 196/443 [01:44<02:12, 1.86it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 196/443 [01:44<02:12, 1.86it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:44<01:28, 2.78it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:44<01:28, 2.78it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:44<01:28, 2.78it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:31, 1.65it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:31, 1.65it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:31, 1.65it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:30, 1.65it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:30, 1.65it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:29, 1.65it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:29, 1.65it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:34, 1.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:34, 1.61it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:34, 1.61it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:33, 1.61it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:33, 1.61it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:33, 1.61it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:33, 1.61it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:24, 1.72it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:24, 1.72it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:24, 1.72it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:24, 1.72it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:24, 1.72it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:23, 1.72it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:23, 1.72it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:28, 1.68it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:28, 1.68it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:28, 1.68it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:27, 1.68it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:27, 1.68it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:27, 1.68it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:27, 1.68it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:32, 1.61it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:32, 1.61it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:29, 1.65it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:29, 1.65it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:33, 1.63it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:33, 1.63it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:33, 1.63it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:32, 1.63it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:32, 1.63it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:31, 1.63it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:31, 1.63it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:23, 1.72it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:23, 1.72it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:36, 1.59it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:36, 1.59it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:36, 1.59it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:36, 1.59it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:36, 1.59it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:26, 1.68it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:26, 1.68it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:34, 1.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:34, 1.61it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 194/443 [01:45<02:34, 1.61it/s, Materializing param=model.layers.17.post_attention_layernorm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:33, 1.61it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 195/443 [01:45<02:33, 1.61it/s, Materializing param=model.layers.17.self_attn.k_norm.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:32, 1.61it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:32, 1.61it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:31, 1.63it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:31, 1.63it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:35, 1.59it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 196/443 [01:45<02:35, 1.59it/s, Materializing param=model.layers.17.self_attn.k_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:35, 1.59it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:35, 1.59it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:32, 1.61it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 44%|████▍ | 197/443 [01:45<02:32, 1.61it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:32, 2.65it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:31, 2.67it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:32, 2.65it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:39, 2.47it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:31, 2.67it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:32, 2.65it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:31, 2.67it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:28, 2.76it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:31, 2.68it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:28, 2.76it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:33, 2.61it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:31, 2.67it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:31, 2.68it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:33, 2.61it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:28, 2.77it/s, Materializing param=model.layers.17.self_attn.o_proj.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:31, 2.67it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:31, 2.68it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:33, 2.61it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:28, 2.76it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:32, 2.65it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:28, 2.77it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:39, 2.47it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:28, 2.77it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:32, 2.65it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:33, 2.61it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:31, 2.68it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:33, 2.61it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:28, 2.76it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:31, 2.68it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:39, 2.47it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:28, 2.76it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:28, 2.77it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:28, 2.77it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:38, 2.47it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:38, 2.47it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:28, 2.78it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 198/443 [01:45<01:28, 2.78it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:36, 2.52it/s, Materializing param=model.layers.17.self_attn.q_norm.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:36, 2.52it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▍ | 199/443 [01:45<01:36, 2.52it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:36, 2.52it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:36, 2.52it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:14, 3.27it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:14, 3.27it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:14, 3.27it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 46%|████▌ | 202/443 [01:45<01:13, 3.27it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 202/443 [01:45<01:13, 3.27it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:15, 3.24it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:13, 3.31it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:13, 3.31it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:15, 3.24it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:13, 3.31it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:15, 3.24it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:10, 3.43it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:10, 3.43it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:10, 3.43it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:20, 3.01it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:20, 3.01it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:20, 3.01it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:13, 3.32it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:13, 3.32it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:13, 3.32it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:13, 3.32it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:13, 3.32it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:13, 3.32it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:11, 3.40it/s, Materializing param=model.layers.17.self_attn.q_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:11, 3.40it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 200/443 [01:45<01:11, 3.40it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:14, 3.24it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:14, 3.24it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 46%|████▌ | 202/443 [01:45<01:14, 3.24it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 202/443 [01:45<01:14, 3.24it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:12, 3.32it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:12, 3.32it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 46%|████▌ | 202/443 [01:45<01:12, 3.32it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 202/443 [01:45<01:12, 3.32it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:10, 3.43it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:10, 3.43it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:20, 3.01it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 46%|████▌ | 202/443 [01:45<01:10, 3.43it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 202/443 [01:45<01:10, 3.43it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:20, 3.01it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:12, 3.32it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 45%|████▌ | 201/443 [01:45<01:12, 3.32it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 46%|████▌ | 202/443 [01:45<01:12, 3.32it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 202/443 [01:45<01:12, 3.32it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 202/443 [01:47<01:51, 2.16it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:52, 2.14it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:34, 2.55it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:35, 2.51it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:35, 2.52it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:52, 2.14it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 202/443 [01:47<01:51, 2.16it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:36, 2.49it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:35, 2.51it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:35, 2.52it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:52, 2.14it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:34, 2.55it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:35, 2.51it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:35, 2.52it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:34, 2.55it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:36, 2.49it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 202/443 [01:47<01:51, 2.16it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:36, 2.49it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:51, 2.16it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<01:51, 2.16it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 45%|████▌ | 201/443 [01:47<02:07, 1.90it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 201/443 [01:47<02:07, 1.90it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 45%|████▌ | 201/443 [01:47<02:04, 1.94it/s, Materializing param=model.layers.17.self_attn.v_proj.weight] Loading weights: 45%|████▌ | 201/443 [01:47<02:07, 1.90it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 45%|████▌ | 201/443 [01:47<02:04, 1.94it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 45%|████▌ | 201/443 [01:47<02:04, 1.94it/s, Materializing param=model.layers.18.input_layernorm.weight] Loading weights: 46%|████▌ | 202/443 [01:47<02:07, 1.90it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 202/443 [01:47<02:07, 1.90it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 202/443 [01:47<02:03, 1.94it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<02:06, 1.90it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 202/443 [01:47<02:03, 1.94it/s, Materializing param=model.layers.18.mlp.down_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<02:06, 1.90it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<02:03, 1.94it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 203/443 [01:47<02:03, 1.94it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:32, 1.56it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:32, 1.57it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:32, 1.56it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:20, 1.70it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:32, 1.56it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:20, 1.70it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:25, 1.64it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:32, 1.57it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:31, 1.58it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:32, 1.57it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:20, 1.70it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:18, 1.73it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:32, 1.57it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:32, 1.57it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:31, 1.58it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:18, 1.73it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:32, 1.57it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:31, 1.58it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:18, 1.73it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:25, 1.64it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:25, 1.64it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:53, 1.37it/s, Materializing param=model.layers.18.mlp.gate_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:53, 1.37it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▌ | 204/443 [01:49<02:53, 1.37it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:34, 1.11it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:24, 1.17it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:18, 1.20it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:35, 1.10it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:35, 1.10it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:34, 1.11it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:18, 1.20it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:35, 1.10it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:34, 1.11it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:18, 1.20it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:24, 1.17it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:24, 1.17it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 47%|████▋ | 206/443 [01:51<03:18, 1.20it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:20, 1.19it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 47%|████▋ | 206/443 [01:51<03:18, 1.20it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:58, 1.00s/it, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:20, 1.19it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:58, 1.00s/it, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:20, 1.19it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:58, 1.00s/it, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 47%|████▋ | 206/443 [01:51<03:19, 1.19it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 206/443 [01:51<03:19, 1.19it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 207/443 [01:51<03:17, 1.20it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 207/443 [01:51<03:17, 1.20it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 206/443 [01:51<03:57, 1.00s/it, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:33, 1.11it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 47%|████▋ | 206/443 [01:51<03:57, 1.00s/it, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:34, 1.11it/s, Materializing param=model.layers.18.mlp.up_proj.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:33, 1.11it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 47%|████▋ | 207/443 [01:51<03:56, 1.00s/it, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:34, 1.11it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 47%|████▋ | 207/443 [01:51<03:56, 1.00s/it, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:33, 1.11it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 46%|████▋ | 205/443 [01:51<03:34, 1.11it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 47%|████▋ | 206/443 [01:51<03:33, 1.11it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 206/443 [01:51<03:33, 1.11it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 206/443 [01:51<03:33, 1.11it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 206/443 [01:51<03:33, 1.11it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 207/443 [01:51<03:32, 1.11it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 207/443 [01:51<03:32, 1.11it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 207/443 [01:51<03:32, 1.11it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 207/443 [01:51<03:32, 1.11it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:13, 1.76it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<01:59, 1.96it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:05, 1.87it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:05, 1.87it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 206/443 [01:51<02:50, 1.39it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:13, 1.76it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 206/443 [01:51<02:56, 1.35it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 47%|████▋ | 206/443 [01:51<02:57, 1.34it/s, Materializing param=model.layers.18.post_attention_layernorm.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:13, 1.76it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:05, 1.87it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<01:59, 1.96it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:05, 1.87it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 206/443 [01:51<02:56, 1.35it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:05, 1.87it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 206/443 [01:51<02:57, 1.34it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 206/443 [01:51<02:56, 1.35it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 208/443 [01:51<01:59, 1.96it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 206/443 [01:51<02:57, 1.34it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:05, 1.87it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 206/443 [01:51<02:50, 1.39it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 207/443 [01:51<02:56, 1.34it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 207/443 [01:51<02:56, 1.34it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 207/443 [01:51<02:55, 1.35it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 207/443 [01:51<02:55, 1.35it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:55, 1.34it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:55, 1.34it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 206/443 [01:51<02:50, 1.39it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:54, 1.35it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:54, 1.35it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 207/443 [01:51<02:49, 1.39it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 207/443 [01:51<02:49, 1.39it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:49, 1.39it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:49, 1.39it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 207/443 [01:51<02:20, 1.68it/s, Materializing param=model.layers.18.self_attn.k_norm.weight] Loading weights: 47%|████▋ | 207/443 [01:51<02:20, 1.68it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 207/443 [01:51<02:20, 1.68it/s, Materializing param=model.layers.18.self_attn.k_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:19, 1.68it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 208/443 [01:51<02:19, 1.68it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 209/443 [01:52<02:05, 1.87it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<02:12, 1.76it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:56, 2.01it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 209/443 [01:52<02:12, 1.76it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<02:04, 1.87it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:56, 2.01it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<02:04, 1.87it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:56, 2.01it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<02:05, 1.87it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:58, 1.97it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:55, 2.02it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:56, 2.00it/s, Materializing param=model.layers.18.self_attn.o_proj.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:58, 1.97it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:58, 1.97it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:49, 2.12it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:56, 2.00it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:59, 1.96it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:54, 2.03it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:56, 2.00it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:49, 2.12it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:54, 2.03it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:49, 2.12it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:49, 2.13it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:59, 1.96it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:58, 1.97it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:54, 2.03it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:58, 1.97it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:56, 2.00it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:49, 2.13it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:56, 2.00it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:49, 2.13it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:55, 2.02it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:46, 2.20it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:46, 2.20it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:46, 2.20it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 209/443 [01:52<01:55, 2.02it/s, Materializing param=model.layers.18.self_attn.q_norm.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:55, 2.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:55, 2.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:55, 2.02it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 47%|████▋ | 210/443 [01:52<01:55, 2.02it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:42, 2.27it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:59, 1.94it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:42, 2.26it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:42, 2.26it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:52, 2.07it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:59, 1.94it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:55, 2.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:42, 2.26it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:59, 1.94it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:42, 2.26it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:55, 2.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:42, 2.26it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:42, 2.26it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:52, 2.07it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:55, 2.01it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:55, 2.01it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:55, 2.01it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:52, 2.07it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:42, 2.27it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:55, 2.01it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:42, 2.27it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:43, 2.23it/s, Materializing param=model.layers.18.self_attn.q_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:43, 2.23it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 211/443 [01:53<01:43, 2.23it/s, Materializing param=model.layers.18.self_attn.v_proj.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:43, 2.23it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:43, 2.23it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 213/443 [01:53<01:43, 2.23it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:53<01:43, 2.23it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:41, 2.27it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:41, 2.27it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:59, 1.94it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:54, 2.01it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:54, 2.01it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:59, 1.94it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 213/443 [01:53<01:58, 1.94it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:53<01:58, 1.94it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:51, 2.07it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:51, 2.07it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:42, 2.26it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:42, 2.26it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:42, 2.26it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:42, 2.26it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 213/443 [01:53<01:41, 2.26it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:53<01:41, 2.26it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:55, 2.01it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 212/443 [01:53<01:55, 2.01it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 213/443 [01:53<01:54, 2.01it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:53<01:54, 2.01it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<01:59, 1.91it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<01:59, 1.91it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:09, 1.76it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<01:58, 1.93it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:54<02:28, 1.55it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 213/443 [01:54<02:17, 1.67it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:07, 1.80it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<01:59, 1.91it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:09, 1.76it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<01:58, 1.93it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:07, 1.80it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:54<02:28, 1.55it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:54<02:17, 1.67it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<01:58, 1.93it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:07, 1.80it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:54<02:28, 1.55it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:54<02:17, 1.67it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:09, 1.76it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:54<01:58, 1.93it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:54<01:58, 1.93it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:28, 1.55it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:16, 1.67it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:16, 1.67it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:28, 1.55it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:54<02:16, 1.68it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 213/443 [01:54<02:26, 1.57it/s, Materializing param=model.layers.19.input_layernorm.weight] Loading weights: 48%|████▊ | 213/443 [01:54<02:26, 1.57it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:54<02:26, 1.57it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:54<02:16, 1.68it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:25, 1.57it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:25, 1.57it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 213/443 [01:54<02:16, 1.68it/s, Materializing param=model.layers.19.mlp.down_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:15, 1.68it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 48%|████▊ | 214/443 [01:54<02:15, 1.68it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<02:04, 1.83it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<02:04, 1.83it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<02:04, 1.83it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<01:58, 1.92it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<01:58, 1.92it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<01:58, 1.92it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<01:56, 1.96it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<02:03, 1.85it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<02:03, 1.85it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<02:03, 1.85it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<01:56, 1.96it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<01:56, 1.96it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<01:57, 1.95it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<01:57, 1.95it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<01:57, 1.95it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<02:02, 1.87it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<02:02, 1.87it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<02:02, 1.87it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<02:12, 1.72it/s, Materializing param=model.layers.19.mlp.gate_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<02:12, 1.72it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▊ | 215/443 [01:55<02:12, 1.72it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:40, 1.41it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:40, 1.41it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:40, 1.41it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:38, 1.43it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:38, 1.43it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:28, 1.53it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:38, 1.43it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:39, 1.41it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:37, 1.44it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:32, 1.49it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:47, 1.36it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:39, 1.41it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:28, 1.53it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:32, 1.49it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:37, 1.43it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:47, 1.36it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:28, 1.53it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:32, 1.49it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:31, 1.50it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:37, 1.44it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:47, 1.36it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:47, 1.36it/s, Materializing param=model.layers.19.mlp.up_proj.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:39, 1.41it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:37, 1.44it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:37, 1.43it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:31, 1.49it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:47, 1.36it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:27, 1.53it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:46, 1.36it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:31, 1.49it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:47, 1.36it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:39, 1.41it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:46, 1.36it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:27, 1.53it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:31, 1.50it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:30, 1.49it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:36, 1.44it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:30, 1.49it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:46, 1.36it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:45, 1.36it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:36, 1.44it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:46, 1.36it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:45, 1.36it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:37, 1.43it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:37, 1.43it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:35, 1.44it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:45, 1.36it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 216/443 [01:57<02:31, 1.50it/s, Materializing param=model.layers.19.post_attention_layernorm.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:45, 1.36it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:35, 1.44it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:30, 1.50it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 217/443 [01:57<02:30, 1.50it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:29, 1.50it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 218/443 [01:57<02:29, 1.50it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:28, 2.52it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 218/443 [01:57<01:49, 2.06it/s, Materializing param=model.layers.19.self_attn.k_norm.weight] Loading weights: 49%|████▉ | 218/443 [01:57<01:49, 2.06it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:28, 2.52it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 218/443 [01:57<01:49, 2.06it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:28, 2.52it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:32, 2.43it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:30, 2.47it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:34, 2.38it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:28, 2.52it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:31, 2.46it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:48, 2.06it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:33, 2.39it/s, Materializing param=model.layers.19.self_attn.k_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:48, 2.06it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:28, 2.52it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:34, 2.38it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:31, 2.46it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:33, 2.39it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:30, 2.47it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:32, 2.43it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:28, 2.52it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:34, 2.38it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:31, 2.46it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:33, 2.39it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:30, 2.47it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 49%|████▉ | 219/443 [01:57<01:32, 2.43it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:37, 2.29it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:33, 2.39it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:33, 2.38it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:37, 2.29it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:33, 2.39it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:35, 2.33it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:37, 2.29it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:31, 2.43it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:33, 2.38it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:30, 2.47it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:24, 2.63it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:37, 2.29it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:24, 2.62it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:24, 2.63it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:31, 2.43it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:30, 2.47it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:37, 2.29it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:35, 2.33it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:24, 2.63it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:24, 2.62it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:24, 2.62it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:22, 2.69it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:22, 2.69it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:23, 2.66it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:22, 2.69it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:35, 2.33it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:23, 2.66it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:23, 2.66it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:35, 2.33it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:35, 2.33it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:36, 2.32it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:35, 2.33it/s, Materializing param=model.layers.19.self_attn.o_proj.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:36, 2.32it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:36, 2.32it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:35, 2.33it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 220/443 [01:57<01:35, 2.33it/s, Materializing param=model.layers.19.self_attn.q_norm.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:35, 2.32it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:35, 2.32it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:35, 2.33it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|████▉ | 221/443 [01:57<01:35, 2.33it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:34, 2.33it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:34, 2.33it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 223/443 [01:57<00:57, 3.79it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 223/443 [01:57<00:57, 3.79it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 50%|█████ | 223/443 [01:57<00:57, 3.79it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 51%|█████ | 224/443 [01:57<00:57, 3.79it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 224/443 [01:57<00:57, 3.79it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:09, 3.16it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:09, 3.16it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:09, 3.16it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:15, 2.94it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:15, 2.94it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:15, 2.94it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 223/443 [01:57<01:09, 3.16it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 50%|█████ | 223/443 [01:57<01:09, 3.16it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 51%|█████ | 224/443 [01:57<01:09, 3.16it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 224/443 [01:57<01:09, 3.16it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:15, 2.93it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:16, 2.90it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:15, 2.93it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:16, 2.90it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:15, 2.93it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:16, 2.90it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 223/443 [01:57<01:14, 2.94it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 50%|█████ | 223/443 [01:57<01:14, 2.94it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 51%|█████ | 224/443 [01:57<01:14, 2.94it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 224/443 [01:57<01:14, 2.94it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:13, 2.99it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:13, 2.99it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:13, 2.99it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:15, 2.93it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:15, 2.93it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:15, 2.93it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:10, 3.12it/s, Materializing param=model.layers.19.self_attn.q_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:10, 3.12it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 222/443 [01:57<01:10, 3.12it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 223/443 [01:57<01:15, 2.90it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 50%|█████ | 223/443 [01:57<01:15, 2.90it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 51%|█████ | 224/443 [01:57<01:15, 2.90it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 224/443 [01:57<01:15, 2.90it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 50%|█████ | 223/443 [01:57<01:13, 2.99it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 50%|█████ | 223/443 [01:57<01:15, 2.93it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 50%|█████ | 223/443 [01:57<01:13, 2.99it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 50%|█████ | 223/443 [01:57<01:15, 2.93it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 50%|█████ | 223/443 [01:57<01:10, 3.12it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 51%|█████ | 224/443 [01:57<01:13, 2.99it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 224/443 [01:57<01:13, 2.99it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 50%|█████ | 223/443 [01:57<01:10, 3.12it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 51%|█████ | 224/443 [01:57<01:10, 3.12it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 224/443 [01:57<01:10, 3.12it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:38, 2.21it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:40, 2.18it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:43, 2.10it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:44, 2.09it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:38, 2.21it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:43, 2.10it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:43, 2.10it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:44, 2.09it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:43, 2.10it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:38, 2.21it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:43, 2.10it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:40, 2.18it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:44, 2.09it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:38, 2.21it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:43, 2.10it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:40, 2.18it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 224/443 [01:59<02:00, 1.82it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 51%|█████ | 224/443 [01:59<02:00, 1.82it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:38, 2.21it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 224/443 [01:59<02:00, 1.82it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 226/443 [01:59<01:38, 2.21it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [01:59<01:38, 2.21it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:59, 1.82it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:59, 1.82it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<01:38, 2.21it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 50%|█████ | 223/443 [01:59<02:26, 1.51it/s, Materializing param=model.layers.19.self_attn.v_proj.weight] Loading weights: 50%|█████ | 223/443 [01:59<02:26, 1.51it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 50%|█████ | 223/443 [01:59<02:26, 1.51it/s, Materializing param=model.layers.20.input_layernorm.weight] Loading weights: 51%|█████ | 224/443 [01:59<02:25, 1.51it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 224/443 [01:59<02:25, 1.51it/s, Materializing param=model.layers.20.mlp.down_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<02:24, 1.51it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 225/443 [01:59<02:24, 1.51it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:35, 2.28it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:35, 2.28it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:35, 2.28it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:32, 2.36it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:32, 2.36it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:32, 2.36it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:51, 1.95it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:51, 1.95it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:51, 1.95it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:48, 2.00it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:48, 2.00it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:48, 2.00it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:41, 2.14it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:41, 2.14it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:41, 2.14it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:46, 2.04it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:46, 2.04it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<01:46, 2.04it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:41, 2.13it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:38, 2.18it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:45, 2.04it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:38, 2.18it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:45, 2.04it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:38, 2.18it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:45, 2.04it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:38, 2.18it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:45, 2.04it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:38, 2.18it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:41, 2.13it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:45, 2.04it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:41, 2.13it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 52%|█████▏ | 229/443 [02:00<01:44, 2.04it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 229/443 [02:00<01:44, 2.04it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:41, 2.13it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:41, 2.13it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 52%|█████▏ | 229/443 [02:00<01:40, 2.13it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 229/443 [02:00<01:40, 2.13it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:35, 2.26it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:35, 2.26it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:35, 2.26it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:34, 2.26it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:34, 2.26it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 52%|█████▏ | 229/443 [02:00<01:34, 2.26it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 229/443 [02:00<01:34, 2.26it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:43, 2.10it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:43, 2.10it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:43, 2.10it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:42, 2.10it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:42, 2.10it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:37, 2.22it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:37, 2.22it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:46, 2.03it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:37, 2.22it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:46, 2.03it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:36, 2.22it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 51%|█████ | 227/443 [02:00<01:46, 2.03it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:36, 2.22it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:45, 2.03it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<01:45, 2.03it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 52%|█████▏ | 229/443 [02:00<01:45, 2.03it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 229/443 [02:00<01:45, 2.03it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<02:06, 1.71it/s, Materializing param=model.layers.20.mlp.gate_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<02:06, 1.71it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 226/443 [02:00<02:06, 1.71it/s, Materializing param=model.layers.20.mlp.up_proj.weight] Loading weights: 51%|█████ | 227/443 [02:00<02:06, 1.71it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████ | 227/443 [02:00<02:06, 1.71it/s, Materializing param=model.layers.20.post_attention_layernorm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<02:05, 1.71it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 51%|█████▏ | 228/443 [02:00<02:05, 1.71it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 52%|█████▏ | 229/443 [02:00<02:04, 1.71it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 229/443 [02:00<02:04, 1.71it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 229/443 [02:01<01:17, 2.78it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 52%|█████▏ | 229/443 [02:01<01:17, 2.78it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 229/443 [02:01<01:17, 2.78it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 229/443 [02:01<01:36, 2.22it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 229/443 [02:01<01:36, 2.22it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:07, 3.14it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<00:59, 3.60it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:06, 3.18it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:02, 3.42it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:05, 3.26it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<00:59, 3.60it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:02, 3.42it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:07, 3.14it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:06, 3.18it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<00:59, 3.60it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:06, 3.18it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:06, 3.18it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:07, 3.14it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:02, 3.42it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:06, 3.18it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:06, 3.18it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:05, 3.26it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:05, 3.26it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 229/443 [02:01<01:12, 2.94it/s, Materializing param=model.layers.20.self_attn.k_norm.weight] Loading weights: 52%|█████▏ | 229/443 [02:01<01:12, 2.94it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 229/443 [02:01<01:12, 2.94it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:12, 2.94it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:12, 2.94it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:03, 3.33it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:06, 3.18it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:05, 3.24it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:04, 3.27it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:01, 3.47it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:06, 3.18it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<00:58, 3.62it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:01, 3.47it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:05, 3.24it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:04, 3.27it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:05, 3.24it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:01, 3.47it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<00:58, 3.62it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:04, 3.27it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<00:58, 3.62it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<00:57, 3.69it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<00:57, 3.69it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<00:57, 3.69it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 232/443 [02:01<00:57, 3.69it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:03, 3.33it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 232/443 [02:01<00:57, 3.69it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:03, 3.33it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 232/443 [02:01<01:03, 3.33it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:01<01:03, 3.33it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:10, 3.04it/s, Materializing param=model.layers.20.self_attn.k_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:10, 3.04it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 230/443 [02:01<01:10, 3.04it/s, Materializing param=model.layers.20.self_attn.o_proj.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:09, 3.04it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 231/443 [02:01<01:09, 3.04it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 232/443 [02:01<01:09, 3.04it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:01<01:09, 3.04it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:01<00:53, 3.95it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 232/443 [02:01<00:53, 3.95it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:01<00:53, 3.95it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:01<00:58, 3.61it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 232/443 [02:01<00:58, 3.61it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:01<00:58, 3.61it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:08, 3.05it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:00, 3.49it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:04, 3.23it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:00, 3.49it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:02<01:17, 2.74it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:08, 3.05it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:00, 3.49it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:08, 3.05it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:04, 3.23it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:04, 3.23it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:02<01:21, 2.59it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 232/443 [02:02<01:17, 2.74it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:02<01:17, 2.74it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:02<01:21, 2.59it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:02<01:21, 2.59it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:20, 2.59it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:16, 2.74it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:20, 2.59it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:16, 2.74it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:02<01:14, 2.82it/s, Materializing param=model.layers.20.self_attn.q_norm.weight] Loading weights: 52%|█████▏ | 232/443 [02:02<01:14, 2.82it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 52%|█████▏ | 232/443 [02:02<01:14, 2.82it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:14, 2.82it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:14, 2.82it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:05, 3.21it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:05, 3.21it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:01, 3.43it/s, Materializing param=model.layers.20.self_attn.q_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:01, 3.43it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:05, 3.21it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 233/443 [02:02<01:01, 3.43it/s, Materializing param=model.layers.20.self_attn.v_proj.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:01, 3.43it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:01, 3.43it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:05, 3.21it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:00, 3.43it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:00, 3.43it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:05, 3.21it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:04, 3.21it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:04, 3.21it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<00:59, 3.49it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:08, 3.05it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<00:59, 3.49it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:08, 3.05it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<00:59, 3.49it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:08, 3.05it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<00:59, 3.49it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:08, 3.05it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:16, 2.74it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:16, 2.74it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:15, 2.74it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:15, 2.74it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:14, 2.82it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:04, 3.23it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:20, 2.59it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:14, 2.82it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:04, 3.23it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 234/443 [02:02<01:20, 2.59it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:04, 3.23it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:04, 3.23it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:11, 2.92it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:11, 2.92it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 235/443 [02:02<01:11, 2.92it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:26, 2.38it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:33, 2.22it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:33, 2.21it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:36, 2.16it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:29, 2.30it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:22, 2.50it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:26, 2.38it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:25, 2.42it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:33, 2.22it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:36, 2.16it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:26, 2.38it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:22, 2.50it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:33, 2.22it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 235/443 [02:03<01:32, 2.25it/s, Materializing param=model.layers.21.input_layernorm.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:36, 2.16it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:33, 2.21it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:22, 2.50it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:25, 2.42it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 235/443 [02:03<01:32, 2.25it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:33, 2.21it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:25, 2.42it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 235/443 [02:03<01:32, 2.25it/s, Materializing param=model.layers.21.mlp.down_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:03<01:32, 2.22it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:03<01:32, 2.22it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:29, 2.30it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:31, 2.25it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:31, 2.25it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 236/443 [02:03<01:29, 2.30it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:28, 2.33it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:28, 2.33it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:28, 2.33it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:26, 2.40it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:26, 2.40it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:26, 2.40it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:38, 2.09it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:38, 2.09it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:38, 2.09it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:27, 2.36it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:27, 2.36it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:27, 2.36it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:39, 2.07it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:35, 2.16it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:43, 1.98it/s, Materializing param=model.layers.21.mlp.gate_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:35, 2.16it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:35, 2.16it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:43, 1.98it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:43, 1.98it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:39, 2.07it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 53%|█████▎ | 237/443 [02:04<01:39, 2.07it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:58, 1.72it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<02:01, 1.69it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:58, 1.72it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<02:01, 1.69it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<02:01, 1.69it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<02:05, 1.63it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:50, 1.85it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:52, 1.82it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:42, 2.00it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:56, 1.76it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:58, 1.72it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<02:05, 1.63it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:50, 1.85it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:56, 1.76it/s, Materializing param=model.layers.21.mlp.up_proj.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:52, 1.82it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:42, 2.00it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:56, 1.76it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<02:00, 1.69it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<02:05, 1.63it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:50, 1.85it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:42, 2.00it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:52, 1.82it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<02:00, 1.69it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:56, 1.76it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<01:50, 1.85it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<01:58, 1.72it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<01:56, 1.76it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<01:50, 1.85it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<01:56, 1.76it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<01:58, 1.72it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<01:52, 1.82it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:56, 1.76it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<01:49, 1.85it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<01:52, 1.82it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<01:42, 2.00it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<01:55, 1.76it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<01:57, 1.72it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<01:42, 2.00it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<01:55, 1.76it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<01:49, 1.85it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<02:05, 1.63it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<02:00, 1.69it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<01:57, 1.72it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<02:05, 1.63it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<02:00, 1.69it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<01:51, 1.82it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▎ | 238/443 [02:05<01:56, 1.76it/s, Materializing param=model.layers.21.post_attention_layernorm.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<01:41, 2.00it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<01:51, 1.82it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<01:41, 2.00it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<02:04, 1.63it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<02:04, 1.63it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<01:55, 1.76it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 239/443 [02:05<01:55, 1.76it/s, Materializing param=model.layers.21.self_attn.k_norm.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<01:55, 1.76it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 240/443 [02:05<01:55, 1.76it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:06, 3.06it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:06, 3.02it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:06, 3.06it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:06, 3.04it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:06, 3.02it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:06, 3.06it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:06, 3.02it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:06, 3.04it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:05, 3.11it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:06, 3.04it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:05, 3.11it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:05, 3.11it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:04, 3.15it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:08, 2.95it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:04, 3.15it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:04, 3.15it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:08, 2.95it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:08, 2.95it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:10, 2.86it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:06, 3.04it/s, Materializing param=model.layers.21.self_attn.k_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:10, 2.86it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:10, 2.86it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:06, 3.04it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 54%|█████▍ | 241/443 [02:05<01:06, 3.04it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:12, 2.77it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:03, 3.15it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:13, 2.75it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:03, 3.15it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:13, 2.75it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:12, 2.79it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:13, 2.75it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:12, 2.75it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:00, 3.33it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:12, 2.79it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:12, 2.75it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:00, 3.33it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:12, 2.79it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:12, 2.75it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:00, 3.33it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:12, 2.75it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:12, 2.77it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:12, 2.75it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:11, 2.79it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:12, 2.75it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:11, 2.79it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:12, 2.77it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:12, 2.75it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:12, 2.77it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:12, 2.77it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:15, 2.65it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:14, 2.69it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:11, 2.81it/s, Materializing param=model.layers.21.self_attn.o_proj.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:15, 2.65it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:14, 2.69it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:11, 2.81it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:15, 2.65it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:14, 2.69it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 242/443 [02:06<01:11, 2.81it/s, Materializing param=model.layers.21.self_attn.q_norm.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:14, 2.69it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:15, 2.65it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:11, 2.81it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:15, 2.65it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:11, 2.81it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▍ | 243/443 [02:06<01:14, 2.69it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<01:15, 2.65it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<01:10, 2.81it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<01:15, 2.65it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<01:13, 2.69it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<01:10, 2.81it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<01:13, 2.69it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:51, 3.83it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:51, 3.83it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:51, 3.83it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:51, 3.83it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:51, 3.83it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:51, 3.83it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:53, 3.70it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:53, 3.70it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:53, 3.70it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:43, 4.52it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:46, 4.23it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:43, 4.52it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:46, 4.23it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:43, 4.52it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:46, 4.23it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:51, 3.85it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:44, 4.43it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:51, 3.85it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:44, 4.43it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:51, 3.85it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:44, 4.43it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:52, 3.82it/s, Materializing param=model.layers.21.self_attn.q_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:06<00:44, 4.43it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:06<00:44, 4.43it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:52, 3.82it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 244/443 [02:06<00:52, 3.82it/s, Materializing param=model.layers.21.self_attn.v_proj.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:51, 3.83it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:51, 3.83it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:51, 3.83it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:51, 3.83it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 56%|█████▌ | 246/443 [02:06<00:51, 3.83it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:06<00:51, 3.83it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:53, 3.70it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:53, 3.70it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 56%|█████▌ | 246/443 [02:06<00:53, 3.70it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:06<00:53, 3.70it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:51, 3.82it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:51, 3.85it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:51, 3.85it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 55%|█████▌ | 245/443 [02:06<00:51, 3.82it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 56%|█████▌ | 246/443 [02:06<00:51, 3.82it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:06<00:51, 3.82it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:31, 2.14it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:08<01:46, 1.86it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:35, 2.04it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:30, 2.17it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:30, 2.16it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:08<00:46, 4.23it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:31, 2.14it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:08<00:43, 4.52it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:30, 2.17it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:08<01:46, 1.86it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:08<00:46, 4.23it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:35, 2.04it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:31, 2.14it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:30, 2.17it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:35, 2.04it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:08<01:46, 1.86it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:35, 2.05it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:35, 2.05it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:08<00:43, 4.52it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:30, 2.16it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:35, 2.05it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:45, 1.86it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:45, 1.86it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:34, 2.08it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:30, 2.16it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:08<01:46, 1.84it/s, Materializing param=model.layers.22.input_layernorm.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:34, 2.08it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:34, 2.08it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:08<01:46, 1.84it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 246/443 [02:08<01:46, 1.84it/s, Materializing param=model.layers.22.mlp.down_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:46, 1.84it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 247/443 [02:08<01:46, 1.84it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:10, 1.49it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:17, 1.42it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:15, 1.44it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:23, 1.36it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:10, 1.49it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:15, 1.44it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:10, 1.49it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:23, 1.36it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:15, 1.44it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:21, 1.38it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:23, 1.36it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:17, 1.42it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:11, 1.48it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:21, 1.37it/s, Materializing param=model.layers.22.mlp.gate_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:17, 1.42it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:21, 1.38it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:11, 1.48it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:17, 1.42it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:21, 1.37it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:21, 1.38it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:11, 1.48it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:17, 1.42it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:21, 1.37it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 248/443 [02:10<02:17, 1.42it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<03:00, 1.08it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<03:00, 1.08it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:54, 1.11it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:55, 1.10it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<03:00, 1.08it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:54, 1.11it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:50, 1.14it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:54, 1.11it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<03:02, 1.06it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:50, 1.14it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:59, 1.08it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:50, 1.14it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:59, 1.08it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:55, 1.10it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<03:02, 1.06it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<03:02, 1.06it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:55, 1.10it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:58, 1.08it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:55, 1.10it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:49, 1.14it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:58, 1.08it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:49, 1.14it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<03:01, 1.06it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:53, 1.11it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:49, 1.15it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<03:01, 1.06it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:53, 1.11it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:54, 1.10it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:55, 1.10it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:54, 1.10it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:49, 1.15it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<03:00, 1.06it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:52, 1.11it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:49, 1.15it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<03:00, 1.06it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:52, 1.11it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:53, 1.10it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:48, 1.14it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<03:00, 1.07it/s, Materializing param=model.layers.22.mlp.up_proj.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:48, 1.14it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:48, 1.15it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:53, 1.10it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:48, 1.15it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<02:55, 1.10it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<03:00, 1.07it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 56%|█████▌ | 249/443 [02:12<03:00, 1.07it/s, Materializing param=model.layers.22.post_attention_layernorm.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:47, 1.15it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:47, 1.15it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:55, 1.10it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:59, 1.07it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:59, 1.07it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 56%|█████▋ | 250/443 [02:12<02:55, 1.10it/s, Materializing param=model.layers.22.self_attn.k_norm.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:58, 1.07it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:58, 1.07it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:54, 1.10it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 251/443 [02:12<02:54, 1.10it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<02:53, 1.10it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<03:00, 1.06it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<03:00, 1.06it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<02:47, 1.14it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:39, 1.92it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<02:47, 1.14it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:39, 1.92it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:39, 1.92it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<02:53, 1.10it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:39, 1.91it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:37, 1.95it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:38, 1.94it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:37, 1.95it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:39, 1.91it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:35, 1.99it/s, Materializing param=model.layers.22.self_attn.k_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:37, 1.95it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:39, 1.91it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:35, 1.99it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:38, 1.94it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:35, 1.99it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 252/443 [02:12<01:38, 1.94it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:40, 1.89it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:37, 1.95it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:40, 1.89it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:38, 1.92it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:37, 1.95it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:40, 1.89it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:37, 1.95it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:38, 1.92it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:39, 1.92it/s, Materializing param=model.layers.22.self_attn.o_proj.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:37, 1.94it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:35, 1.99it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:36, 1.95it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:37, 1.95it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:39, 1.91it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:36, 1.95it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:37, 1.94it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:35, 1.99it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:37, 1.95it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:40, 1.89it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:39, 1.91it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:40, 1.89it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:22, 2.28it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:24, 2.24it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:24, 2.25it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:22, 2.28it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:24, 2.24it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:24, 2.25it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:22, 2.28it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:39, 1.92it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:24, 2.24it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:24, 2.25it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:25, 2.21it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:25, 2.21it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:25, 2.21it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 253/443 [02:12<01:39, 1.92it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:38, 1.92it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 254/443 [02:12<01:38, 1.92it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:25, 2.19it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:25, 2.19it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:25, 2.20it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:25, 2.19it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:23, 2.26it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:25, 2.20it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:23, 2.26it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 57%|█████▋ | 254/443 [02:13<01:39, 1.90it/s, Materializing param=model.layers.22.self_attn.q_norm.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:25, 2.20it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:25, 2.19it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:23, 2.26it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:26, 2.17it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:24, 2.23it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 57%|█████▋ | 254/443 [02:13<01:39, 1.90it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:25, 2.19it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 57%|█████▋ | 254/443 [02:13<01:39, 1.90it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:25, 2.19it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:39, 1.90it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:26, 2.17it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:24, 2.22it/s, Materializing param=model.layers.22.self_attn.q_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:39, 1.90it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:26, 2.17it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:24, 2.22it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:24, 2.23it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:24, 2.22it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 255/443 [02:13<01:24, 2.23it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:14, 2.50it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:13, 2.53it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:14, 2.51it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:13, 2.53it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:13, 2.53it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:14, 2.51it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:14, 2.51it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:14, 2.50it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:13, 2.51it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:14, 2.50it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:13, 2.54it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:13, 2.51it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:13, 2.56it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:13, 2.56it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:13, 2.53it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:13, 2.56it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:15, 2.48it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:13, 2.53it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:13, 2.54it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:15, 2.48it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:12, 2.56it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:15, 2.48it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:12, 2.56it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:12, 2.56it/s, Materializing param=model.layers.22.self_attn.v_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:13, 2.54it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:13, 2.53it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:13, 2.53it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:15, 2.48it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:15, 2.48it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:13, 2.53it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:12, 2.56it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:13, 2.54it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 256/443 [02:13<01:12, 2.56it/s, Materializing param=model.layers.23.input_layernorm.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:13, 2.54it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:14, 2.50it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:13, 2.53it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:12, 2.56it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:13, 2.53it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:12, 2.56it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 257/443 [02:13<01:14, 2.50it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:52, 1.64it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:52, 1.64it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:52, 1.64it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:47, 1.72it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:47, 1.72it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:47, 1.72it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:48, 1.71it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:53, 1.64it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:49, 1.69it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:45, 1.75it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:52, 1.65it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:49, 1.69it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:53, 1.63it/s, Materializing param=model.layers.23.mlp.down_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:53, 1.64it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:45, 1.75it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:49, 1.69it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:52, 1.65it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:53, 1.64it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:45, 1.75it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:53, 1.63it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:52, 1.65it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:48, 1.71it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:53, 1.63it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:15<01:44, 1.75it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 258/443 [02:15<01:48, 1.71it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:15<01:44, 1.75it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:15<01:44, 1.76it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:15<01:44, 1.76it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:15<01:44, 1.76it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:15<01:48, 1.69it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:15<01:48, 1.69it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:15<01:48, 1.69it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<01:47, 1.72it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<01:47, 1.72it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<01:47, 1.72it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<02:02, 1.50it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<02:02, 1.50it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<02:02, 1.50it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<02:03, 1.49it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<02:03, 1.49it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<02:03, 1.49it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<01:57, 1.57it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<01:57, 1.57it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<01:57, 1.57it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<02:02, 1.50it/s, Materializing param=model.layers.23.mlp.gate_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<02:02, 1.50it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 58%|█████▊ | 259/443 [02:16<02:02, 1.50it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:47, 1.70it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:51, 1.65it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:54, 1.60it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:51, 1.65it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:51, 1.64it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:47, 1.70it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:51, 1.65it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:47, 1.70it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:54, 1.60it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:54, 1.60it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:46, 1.70it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:46, 1.70it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:53, 1.60it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:51, 1.64it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:53, 1.60it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:50, 1.65it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:50, 1.65it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:52, 1.60it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:51, 1.64it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:52, 1.60it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:46, 1.70it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:49, 1.65it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:46, 1.70it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:49, 1.65it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:50, 1.64it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:50, 1.64it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:50, 1.64it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:50, 1.64it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:52, 1.63it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:52, 1.63it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:52, 1.63it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:51, 1.63it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:51, 1.63it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:48, 1.69it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:50, 1.63it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:50, 1.63it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:48, 1.69it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:48, 1.69it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:47, 1.69it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:47, 1.69it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:47, 1.69it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:47, 1.69it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:51, 1.64it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:52, 1.63it/s, Materializing param=model.layers.23.mlp.up_proj.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:51, 1.64it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:52, 1.63it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:51, 1.64it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▊ | 260/443 [02:16<01:52, 1.63it/s, Materializing param=model.layers.23.post_attention_layernorm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:51, 1.63it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:51, 1.64it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:51, 1.63it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 261/443 [02:16<01:51, 1.64it/s, Materializing param=model.layers.23.self_attn.k_norm.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:51, 1.63it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:50, 1.64it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:51, 1.63it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 262/443 [02:16<01:50, 1.64it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:46, 1.69it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:00, 2.96it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:07, 2.67it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:50, 1.63it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:49, 1.64it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:50, 1.63it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:49, 1.64it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:07, 2.67it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:00, 2.96it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:07, 2.67it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:00, 2.96it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:50, 1.63it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:46, 1.69it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:01, 2.91it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:50, 1.63it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:01, 2.91it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:01, 2.91it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:02, 2.86it/s, Materializing param=model.layers.23.self_attn.k_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:02, 2.86it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 59%|█████▉ | 263/443 [02:16<01:02, 2.86it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:16<01:02, 2.86it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 264/443 [02:16<01:02, 2.86it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 265/443 [02:16<01:02, 2.86it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 265/443 [02:16<01:02, 2.86it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:16<00:53, 3.35it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:16<00:53, 3.35it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 264/443 [02:16<00:53, 3.35it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 265/443 [02:16<00:53, 3.35it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 265/443 [02:16<00:53, 3.35it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:54, 3.28it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:53, 3.35it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:53, 3.35it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:53, 3.35it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:54, 3.28it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:54, 3.28it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:53, 3.35it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:53, 3.35it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:54, 3.28it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:53, 3.33it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:54, 3.28it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:53, 3.33it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:53, 3.33it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:53, 3.33it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:53, 3.33it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:53, 3.32it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:53, 3.32it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:53, 3.32it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:53, 3.32it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:53, 3.32it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<01:07, 2.67it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<01:07, 2.67it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:52, 3.36it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:52, 3.36it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:52, 3.36it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:55, 3.20it/s, Materializing param=model.layers.23.self_attn.o_proj.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:55, 3.20it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 264/443 [02:17<00:55, 3.20it/s, Materializing param=model.layers.23.self_attn.q_norm.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:55, 3.20it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|█████▉ | 265/443 [02:17<00:55, 3.20it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:43, 4.03it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:52, 3.36it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:41, 4.27it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:44, 3.98it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:52, 3.36it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:42, 4.15it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:43, 4.03it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:41, 4.27it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:42, 4.14it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:43, 4.03it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:41, 4.31it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:41, 4.27it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:42, 4.15it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:42, 4.14it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:42, 4.15it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:42, 4.14it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:41, 4.31it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:41, 4.31it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:44, 3.98it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:44, 3.98it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:42, 4.13it/s, Materializing param=model.layers.23.self_attn.q_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:42, 4.13it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 266/443 [02:17<00:42, 4.13it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:41, 4.27it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:42, 4.14it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:42, 4.15it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:43, 4.03it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:41, 4.27it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:44, 3.97it/s, Materializing param=model.layers.23.self_attn.v_proj.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:42, 4.14it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:43, 4.03it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:42, 4.15it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:44, 3.97it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:44, 3.97it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:42, 4.14it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:41, 4.27it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:42, 4.14it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:41, 4.27it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:43, 4.03it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:42, 4.15it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:44, 3.97it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:43, 4.03it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:42, 4.13it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:40, 4.31it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:42, 4.15it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:44, 3.98it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:44, 3.97it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:42, 4.13it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:40, 4.31it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:42, 4.13it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:40, 4.31it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:42, 4.13it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:40, 4.31it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 267/443 [02:17<00:44, 3.98it/s, Materializing param=model.layers.24.input_layernorm.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:43, 3.98it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 60%|██████ | 268/443 [02:17<00:43, 3.98it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.41it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:15, 2.31it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:20, 2.15it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:15, 2.32it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.41it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.41it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.40it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.41it/s, Materializing param=model.layers.24.mlp.down_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:15, 2.31it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:15, 2.32it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.40it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:20, 2.15it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:15, 2.31it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:15, 2.32it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.40it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:20, 2.15it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 269/443 [02:19<01:12, 2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:57, 1.48it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:46, 1.63it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:52, 1.54it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:45, 1.65it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:46, 1.62it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:46, 1.63it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:46, 1.62it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:52, 1.54it/s, Materializing param=model.layers.24.mlp.gate_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:46, 1.62it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:52, 1.54it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:46, 1.63it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:57, 1.48it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:46, 1.62it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:46, 1.63it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:52, 1.54it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:46, 1.62it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:52, 1.54it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:46, 1.63it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:57, 1.48it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:46, 1.62it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:46, 1.63it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:52, 1.54it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:45, 1.65it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 270/443 [02:21<01:45, 1.65it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:19, 1.24it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:23, 1.20it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:19, 1.24it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:19, 1.24it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:23, 1.20it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:10, 1.32it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:19, 1.23it/s, Materializing param=model.layers.24.mlp.up_proj.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:23, 1.20it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:11, 1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:19, 1.23it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:22, 1.20it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:18, 1.24it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:19, 1.23it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:22, 1.20it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:11, 1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:18, 1.24it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:11, 1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:19, 1.23it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:21, 1.20it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:11, 1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:17, 1.24it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:19, 1.23it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:21, 1.20it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:11, 1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:12, 1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:11, 1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:11, 1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:10, 1.32it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:17, 1.24it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:11, 1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:18, 1.23it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:11, 1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:18, 1.23it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:11, 1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:11, 1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 61%|██████ | 271/443 [02:22<02:10, 1.32it/s, Materializing param=model.layers.24.post_attention_layernorm.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:11, 1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:10, 1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:10, 1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:10, 1.32it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 61%|██████▏ | 272/443 [02:22<02:10, 1.32it/s, Materializing param=model.layers.24.self_attn.k_norm.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:09, 1.32it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 62%|██████▏ | 273/443 [02:22<02:09, 1.32it/s, Materializing param=model.layers.24.self_attn.k_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:09, 1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:08, 1.32it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:20, 1.20it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:10, 1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:20, 1.20it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:10, 1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:08, 1.32it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:09, 1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:10, 1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:16, 1.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:10, 1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:16, 1.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:17, 1.23it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:10, 1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:10, 1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 274/443 [02:22<02:17, 1.23it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:14, 2.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:14, 2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:16, 2.21it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:15, 2.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:15, 2.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:16, 2.21it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:15, 2.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:15, 2.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:14, 2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:16, 2.21it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:15, 2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:15, 2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:15, 2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:15, 2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:15, 2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:15, 2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:15, 2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:15, 2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:15, 2.21it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:15, 2.21it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:14, 2.24it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:14, 2.24it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:14, 2.24it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:14, 2.24it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:14, 2.24it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:14, 2.24it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:16, 2.19it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:16, 2.19it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:16, 2.19it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:16, 2.19it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:16, 2.19it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<01:15, 2.19it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<01:15, 2.19it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:18, 2.14it/s, Materializing param=model.layers.24.self_attn.o_proj.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:18, 2.14it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 275/443 [02:23<01:18, 2.14it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:18, 2.14it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:18, 2.14it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<01:17, 2.14it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<01:17, 2.14it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:06, 2.51it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:06, 2.51it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<00:57, 2.89it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:06, 2.51it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<00:57, 2.89it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<01:06, 2.51it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<00:57, 2.89it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<01:06, 2.51it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:51, 3.21it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:51, 3.21it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:51, 3.21it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<00:51, 3.21it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<00:51, 3.21it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:50, 3.27it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:50, 3.27it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:50, 3.27it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<00:50, 3.27it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<00:50, 3.27it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<00:57, 2.88it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<00:57, 2.88it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<00:57, 2.87it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<00:57, 2.88it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<00:57, 2.87it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<00:57, 2.87it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<01:05, 2.51it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<01:05, 2.51it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<00:58, 2.85it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<00:58, 2.85it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<00:58, 2.85it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:57, 2.89it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:57, 2.89it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<00:56, 2.89it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<00:56, 2.89it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:57, 2.87it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:57, 2.87it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<00:57, 2.87it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<00:57, 2.87it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:57, 2.88it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:57, 2.88it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:07, 2.47it/s, Materializing param=model.layers.24.self_attn.q_norm.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<00:57, 2.88it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<00:57, 2.88it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:57, 2.85it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:07, 2.47it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<00:57, 2.85it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<00:57, 2.85it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 62%|██████▏ | 276/443 [02:23<01:07, 2.47it/s, Materializing param=model.layers.24.self_attn.q_proj.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<00:57, 2.85it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<01:07, 2.47it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 277/443 [02:23<01:07, 2.47it/s, Materializing param=model.layers.24.self_attn.v_proj.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<01:06, 2.47it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 278/443 [02:23<01:06, 2.47it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<01:06, 2.47it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 279/443 [02:23<01:06, 2.47it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:14, 2.19it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:11, 2.27it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:14, 2.18it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:17, 2.09it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:18, 2.07it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:14, 2.17it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:14, 2.19it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 279/443 [02:25<01:22, 2.00it/s, Materializing param=model.layers.25.input_layernorm.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:14, 2.19it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:14, 2.19it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:14, 2.18it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:17, 2.09it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 279/443 [02:25<01:22, 2.00it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:14, 2.17it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:14, 2.19it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:14, 2.18it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:14, 2.19it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:18, 2.07it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:17, 2.09it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 279/443 [02:25<01:22, 2.00it/s, Materializing param=model.layers.25.mlp.down_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:14, 2.17it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:18, 2.07it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:21, 2.00it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:11, 2.27it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:21, 2.00it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 280/443 [02:25<01:11, 2.27it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:46, 1.51it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:42, 1.59it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:46, 1.51it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:42, 1.59it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:46, 1.51it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:35, 1.70it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:38, 1.64it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:42, 1.59it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:46, 1.52it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:42, 1.59it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:42, 1.59it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:35, 1.70it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:46, 1.52it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:42, 1.59it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:35, 1.70it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:38, 1.64it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:38, 1.64it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:46, 1.52it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:42, 1.58it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:43, 1.57it/s, Materializing param=model.layers.25.mlp.gate_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:43, 1.57it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:42, 1.58it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:43, 1.57it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:26<01:42, 1.57it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▎ | 282/443 [02:26<01:42, 1.57it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 63%|██████▎ | 281/443 [02:26<01:42, 1.58it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 64%|██████▍ | 283/443 [02:26<01:41, 1.57it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 283/443 [02:26<01:41, 1.57it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▎ | 282/443 [02:26<01:41, 1.58it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▍ | 284/443 [02:26<01:41, 1.57it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 284/443 [02:26<01:41, 1.57it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:26<01:41, 1.58it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▍ | 283/443 [02:26<01:40, 1.58it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 283/443 [02:26<01:40, 1.58it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 284/443 [02:26<01:40, 1.58it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 284/443 [02:26<01:40, 1.58it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:26<00:56, 2.80it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:26<00:56, 2.80it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:26<00:56, 2.80it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:26<00:56, 2.80it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:26<00:56, 2.80it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:26<00:56, 2.80it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:36, 1.66it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:36, 1.66it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:36, 1.66it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▍ | 283/443 [02:27<01:36, 1.66it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 283/443 [02:27<01:36, 1.66it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 284/443 [02:27<01:35, 1.66it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 284/443 [02:27<01:35, 1.66it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:35, 1.66it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:35, 1.66it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:41, 1.59it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:41, 1.59it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:41, 1.59it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▍ | 283/443 [02:27<01:40, 1.59it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 283/443 [02:27<01:40, 1.59it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 284/443 [02:27<01:39, 1.59it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 284/443 [02:27<01:39, 1.59it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:41, 1.58it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:41, 1.58it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:41, 1.58it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▍ | 283/443 [02:27<01:41, 1.58it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 283/443 [02:27<01:41, 1.58it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 284/443 [02:27<01:40, 1.58it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 284/443 [02:27<01:40, 1.58it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:39, 1.58it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:39, 1.58it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:40, 1.60it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:40, 1.60it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:40, 1.60it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▍ | 283/443 [02:27<01:39, 1.60it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 283/443 [02:27<01:39, 1.60it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:37, 1.65it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:37, 1.65it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:37, 1.65it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▍ | 283/443 [02:27<01:37, 1.65it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 283/443 [02:27<01:37, 1.65it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 284/443 [02:27<01:36, 1.65it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 284/443 [02:27<01:36, 1.65it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:34, 1.70it/s, Materializing param=model.layers.25.mlp.up_proj.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:34, 1.70it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▎ | 282/443 [02:27<01:34, 1.70it/s, Materializing param=model.layers.25.post_attention_layernorm.weight] Loading weights: 64%|██████▍ | 283/443 [02:27<01:34, 1.70it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 283/443 [02:27<01:34, 1.70it/s, Materializing param=model.layers.25.self_attn.k_norm.weight] Loading weights: 64%|██████▍ | 284/443 [02:27<01:33, 1.70it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 284/443 [02:27<01:33, 1.70it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:35, 1.65it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▍ | 284/443 [02:27<01:39, 1.60it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 284/443 [02:27<01:39, 1.60it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:35, 1.65it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:38, 1.60it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:38, 1.60it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:32, 1.70it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:32, 1.70it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<00:52, 3.02it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<01:35, 1.65it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<00:52, 3.02it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<00:52, 3.02it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<01:35, 1.65it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<00:56, 2.80it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<01:37, 1.60it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<00:55, 2.80it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<00:51, 3.06it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<01:37, 1.60it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<00:56, 2.80it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<00:51, 3.02it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<00:55, 2.80it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<00:51, 3.06it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<00:51, 3.02it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<00:51, 3.06it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<00:52, 2.98it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<01:37, 1.60it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<00:52, 2.98it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<00:52, 2.98it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<01:37, 1.60it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<00:52, 2.98it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<00:52, 2.98it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<00:52, 2.98it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<00:50, 3.06it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<00:50, 3.06it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:00, 2.61it/s, Materializing param=model.layers.25.self_attn.k_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:00, 2.61it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 64%|██████▍ | 285/443 [02:27<01:00, 2.61it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<01:00, 2.61it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 286/443 [02:27<01:00, 2.61it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<00:59, 2.61it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 287/443 [02:27<00:59, 2.61it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 287/443 [02:28<00:54, 2.88it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:52, 2.98it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:49, 3.15it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:47, 3.27it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:48, 3.17it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:49, 3.12it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:52, 2.98it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:52, 2.98it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:48, 3.17it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:49, 3.15it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:47, 3.27it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:49, 3.12it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:52, 2.98it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:48, 3.17it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:49, 3.15it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:47, 3.27it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:49, 3.12it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▍ | 287/443 [02:28<00:54, 2.88it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 287/443 [02:28<00:54, 2.88it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:53, 2.88it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<00:53, 2.88it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▍ | 286/443 [02:28<01:00, 2.58it/s, Materializing param=model.layers.25.self_attn.o_proj.weight] Loading weights: 65%|██████▍ | 286/443 [02:28<01:00, 2.58it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 286/443 [02:28<01:00, 2.58it/s, Materializing param=model.layers.25.self_attn.q_norm.weight] Loading weights: 65%|██████▍ | 287/443 [02:28<01:00, 2.58it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▍ | 287/443 [02:28<01:00, 2.58it/s, Materializing param=model.layers.25.self_attn.q_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<01:00, 2.58it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 288/443 [02:28<01:00, 2.58it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:42, 3.62it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:42, 3.60it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:59, 2.58it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:42, 3.62it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:44, 3.47it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:59, 2.58it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:52, 2.95it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:44, 3.44it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:42, 3.62it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:44, 3.47it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:44, 3.44it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:52, 2.95it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:44, 3.44it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:52, 2.95it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:44, 3.47it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:42, 3.60it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 290/443 [02:28<00:51, 2.95it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:28<00:51, 2.95it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:28<00:44, 3.47it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:44, 3.50it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:28<00:44, 3.47it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:44, 3.50it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 290/443 [02:28<00:59, 2.58it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:44, 3.50it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:52, 2.95it/s, Materializing param=model.layers.25.self_attn.v_proj.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:42, 3.60it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 290/443 [02:28<00:43, 3.50it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:28<00:59, 2.58it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:28<00:43, 3.50it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:52, 2.95it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 289/443 [02:28<00:52, 2.95it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 65%|██████▌ | 290/443 [02:28<00:51, 2.95it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:28<00:51, 2.95it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:28<00:42, 3.60it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:28<00:42, 3.60it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:10, 2.15it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:19, 1.91it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:17, 1.95it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:18, 1.94it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:19, 1.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:01, 2.48it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:19, 1.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:19, 1.91it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:18, 1.94it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:30<01:30, 1.68it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:17, 1.95it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:01, 2.48it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:30<01:24, 1.80it/s, Materializing param=model.layers.26.input_layernorm.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:19, 1.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:18, 1.94it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:17, 1.95it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:01, 2.48it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:30<01:30, 1.68it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:19, 1.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:10, 2.15it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:30<01:30, 1.68it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:30<01:24, 1.80it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:10, 2.15it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 65%|██████▌ | 290/443 [02:30<01:24, 1.80it/s, Materializing param=model.layers.26.mlp.down_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:24, 1.80it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:30, 1.68it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:24, 1.80it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 291/443 [02:30<01:30, 1.68it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:10, 2.15it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:10, 2.15it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:02, 2.42it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:02, 2.42it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:02, 2.42it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:16, 1.97it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:16, 1.97it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:16, 1.97it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:16, 1.98it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:16, 1.98it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:16, 1.98it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:23, 1.81it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:23, 1.81it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:23, 1.81it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:14, 2.03it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:14, 2.03it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:14, 2.03it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:23, 1.81it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:23, 1.81it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:30<01:23, 1.81it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:14, 2.01it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:21, 1.83it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:21, 1.83it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:07, 2.21it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:21, 1.83it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:07, 2.21it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:21, 1.83it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:07, 2.21it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:14, 2.01it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:21, 1.83it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:07, 2.21it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:20, 1.83it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:20, 1.83it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:07, 2.21it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:14, 2.01it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:06, 2.21it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:06, 2.21it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:14, 2.01it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:14, 2.01it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:13, 2.01it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:13, 2.01it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:16, 1.97it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:16, 1.97it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:16, 1.97it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:15, 1.97it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:15, 1.97it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:14, 1.97it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:14, 1.97it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:20, 1.86it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:20, 1.86it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:20, 1.87it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:20, 1.86it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:20, 1.87it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:20, 1.87it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:19, 1.87it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:19, 1.87it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:20, 1.86it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:20, 1.86it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:19, 1.87it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:19, 1.87it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:19, 1.86it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:19, 1.86it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:13, 2.03it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:13, 2.03it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:13, 2.03it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:13, 2.03it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:13, 2.03it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:12, 2.03it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:12, 2.03it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<01:18, 1.87it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<01:19, 1.86it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<01:14, 1.97it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<00:44, 3.27it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<00:48, 3.03it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<00:43, 3.41it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<01:12, 2.03it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<01:18, 1.87it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<01:14, 1.97it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<01:19, 1.86it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<01:12, 2.03it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<00:44, 3.27it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<00:44, 3.27it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<00:43, 3.41it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<00:43, 3.41it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<00:48, 3.03it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<00:48, 3.03it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:31<01:39, 1.52it/s, Materializing param=model.layers.26.mlp.gate_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:31<01:39, 1.52it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 292/443 [02:31<01:39, 1.52it/s, Materializing param=model.layers.26.mlp.up_proj.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:38, 1.52it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▌ | 293/443 [02:31<01:38, 1.52it/s, Materializing param=model.layers.26.post_attention_layernorm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:37, 1.52it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 66%|██████▋ | 294/443 [02:31<01:37, 1.52it/s, Materializing param=model.layers.26.self_attn.k_norm.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:37, 1.52it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 295/443 [02:31<01:37, 1.52it/s, Materializing param=model.layers.26.self_attn.k_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<01:36, 1.52it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 296/443 [02:31<01:36, 1.52it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:44, 3.27it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:47, 3.06it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:44, 3.27it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:47, 3.06it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:47, 3.06it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:47, 3.06it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:48, 3.03it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:42, 3.41it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:52, 2.80it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:42, 3.41it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:47, 3.06it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:48, 2.99it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:47, 3.06it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:48, 2.99it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:52, 2.80it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:42, 3.44it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:52, 2.80it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:47, 3.06it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:48, 2.99it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:47, 3.06it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:42, 3.44it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:48, 3.03it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:42, 3.44it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:51, 2.80it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:48, 2.99it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:51, 2.80it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:48, 2.99it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:46, 3.15it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:43, 3.35it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:43, 3.35it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:43, 3.35it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:46, 3.15it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:46, 3.15it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:50, 2.90it/s, Materializing param=model.layers.26.self_attn.o_proj.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:50, 2.90it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 297/443 [02:32<00:50, 2.90it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:50, 2.90it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:50, 2.90it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:49, 2.90it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:49, 2.90it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:49, 2.90it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:49, 2.90it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 301/443 [02:32<00:49, 2.90it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 301/443 [02:32<00:49, 2.90it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:40, 3.59it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:40, 3.59it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:40, 3.59it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:37, 3.80it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:37, 3.80it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:37, 3.80it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:39, 3.65it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:39, 3.65it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:39, 3.65it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:42, 3.38it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:42, 3.38it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:42, 3.38it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:38, 3.73it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:38, 3.73it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:42, 3.36it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:38, 3.73it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:42, 3.36it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:42, 3.36it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:44, 3.27it/s, Materializing param=model.layers.26.self_attn.q_norm.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:44, 3.27it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 298/443 [02:32<00:44, 3.27it/s, Materializing param=model.layers.26.self_attn.q_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:44, 3.27it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 67%|██████▋ | 299/443 [02:32<00:44, 3.27it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:38, 3.73it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:43, 3.27it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:43, 3.27it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:38, 3.73it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 301/443 [02:32<00:43, 3.27it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 301/443 [02:32<00:43, 3.27it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 301/443 [02:32<00:38, 3.73it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 301/443 [02:32<00:38, 3.73it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:37, 3.80it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:37, 3.80it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:39, 3.65it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:39, 3.59it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:39, 3.59it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:39, 3.65it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 301/443 [02:32<00:38, 3.65it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 301/443 [02:32<00:38, 3.65it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 301/443 [02:32<00:39, 3.59it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 301/443 [02:32<00:39, 3.59it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:42, 3.36it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 300/443 [02:32<00:42, 3.36it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 301/443 [02:33<00:52, 2.69it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 301/443 [02:33<00:52, 2.69it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:44, 3.19it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:46, 3.04it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:47, 2.96it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 301/443 [02:33<00:52, 2.69it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:44, 3.19it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:46, 3.04it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:47, 2.96it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 301/443 [02:33<00:57, 2.47it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:44, 3.19it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:47, 2.96it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:46, 3.04it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:52, 2.69it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:52, 2.69it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 301/443 [02:33<00:57, 2.47it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 301/443 [02:33<00:57, 2.47it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:57, 2.47it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:57, 2.47it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:49, 2.85it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:49, 2.85it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:49, 2.85it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:33<00:49, 2.85it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:33<00:49, 2.85it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 300/443 [02:33<01:03, 2.25it/s, Materializing param=model.layers.26.self_attn.v_proj.weight] Loading weights: 68%|██████▊ | 300/443 [02:33<01:03, 2.25it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 300/443 [02:33<01:03, 2.25it/s, Materializing param=model.layers.27.input_layernorm.weight] Loading weights: 68%|██████▊ | 301/443 [02:33<01:03, 2.25it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 301/443 [02:33<01:03, 2.25it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<01:02, 2.25it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<01:02, 2.25it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:47, 3.00it/s, Materializing param=model.layers.27.mlp.down_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:47, 3.00it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 302/443 [02:33<00:47, 3.00it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:33<00:46, 3.00it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:33<00:46, 3.00it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:50, 2.79it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:50, 2.79it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:50, 2.79it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:50, 2.76it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:50, 2.76it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:50, 2.76it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:48, 2.91it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:48, 2.91it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:48, 2.91it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:52, 2.69it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:52, 2.69it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:52, 2.69it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:49, 2.84it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:49, 2.84it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:49, 2.84it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:48, 2.90it/s, Materializing param=model.layers.27.mlp.gate_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:48, 2.90it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 68%|██████▊ | 303/443 [02:34<00:48, 2.90it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:10, 1.97it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:12, 1.91it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:13, 1.90it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:09, 2.00it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:01, 2.26it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:09, 2.00it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:12, 1.91it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:12, 1.91it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:14, 1.86it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:10, 1.97it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:06, 2.09it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:01, 2.26it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:10, 1.97it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:09, 2.01it/s, Materializing param=model.layers.27.mlp.up_proj.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:09, 2.00it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:12, 1.91it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:06, 2.09it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:09, 2.01it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:06, 2.09it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:01, 2.26it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:09, 2.01it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:12, 1.91it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:10, 1.97it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:14, 1.86it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:10, 1.97it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:14, 1.86it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:06, 2.09it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:08, 2.00it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:13, 1.90it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:01, 2.26it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:06, 2.09it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:08, 2.00it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:09, 1.97it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:08, 2.01it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:01, 2.26it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:14, 1.86it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:08, 2.01it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:05, 2.09it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:09, 1.97it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▊ | 304/443 [02:35<01:13, 1.90it/s, Materializing param=model.layers.27.post_attention_layernorm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:14, 1.86it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:05, 2.09it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:08, 2.00it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:00, 2.26it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:11, 1.91it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:08, 2.00it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:00, 2.26it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:11, 1.91it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:13, 1.86it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:08, 2.01it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:13, 1.86it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:08, 2.01it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:12, 1.90it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 305/443 [02:35<01:12, 1.90it/s, Materializing param=model.layers.27.self_attn.k_norm.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:11, 1.90it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 306/443 [02:35<01:11, 1.90it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:43, 3.12it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:45, 3.00it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:42, 3.21it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:43, 3.12it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:42, 3.22it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:45, 3.00it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:43, 3.12it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:43, 3.12it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:43, 3.11it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:45, 3.00it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:43, 3.12it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:42, 3.22it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:43, 3.12it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:42, 3.22it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:43, 3.10it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:43, 3.10it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:43, 3.10it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:43, 3.11it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:42, 3.21it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:42, 3.21it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:43, 3.11it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:42, 3.19it/s, Materializing param=model.layers.27.self_attn.k_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:42, 3.19it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 69%|██████▉ | 307/443 [02:35<00:42, 3.19it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:42, 3.19it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:42, 3.19it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:41, 3.19it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:41, 3.19it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:40, 3.37it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:40, 3.37it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:39, 3.45it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:40, 3.37it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:39, 3.45it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:39, 3.45it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:39, 3.37it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:39, 3.37it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:38, 3.45it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:38, 3.45it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:40, 3.36it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:41, 3.23it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:41, 3.23it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:41, 3.23it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:40, 3.36it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:41, 3.23it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:41, 3.23it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:40, 3.36it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:39, 3.36it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:39, 3.36it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:39, 3.44it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:39, 3.44it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:39, 3.44it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:40, 3.30it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:40, 3.30it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:38, 3.44it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:40, 3.30it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:38, 3.44it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:30, 4.35it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:30, 4.35it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:30, 4.35it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:30, 4.33it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:30, 4.33it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:30, 4.33it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:29, 4.54it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:29, 4.54it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:29, 4.54it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:29, 4.54it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:29, 4.54it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:29, 4.54it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:36, 3.64it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:36, 3.64it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:36, 3.64it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:36, 3.64it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:36, 3.64it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:28, 4.61it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:28, 4.61it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:28, 4.61it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:30, 4.35it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:30, 4.35it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:30, 4.35it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:30, 4.35it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:30, 4.33it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:30, 4.33it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:30, 4.33it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:30, 4.33it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:29, 4.54it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:29, 4.54it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:28, 4.54it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:36, 3.64it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:28, 4.54it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:36, 3.64it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:28, 4.61it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:28, 4.61it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:35, 3.64it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:35, 3.64it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:28, 4.61it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:28, 4.61it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:30, 4.42it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:30, 4.42it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:30, 4.42it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:29, 4.42it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:29, 4.42it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:29, 4.42it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:29, 4.42it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:44, 3.02it/s, Materializing param=model.layers.27.self_attn.o_proj.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:44, 3.02it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 308/443 [02:35<00:44, 3.02it/s, Materializing param=model.layers.27.self_attn.q_norm.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:44, 3.02it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 309/443 [02:35<00:44, 3.02it/s, Materializing param=model.layers.27.self_attn.q_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:44, 3.02it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|██████▉ | 310/443 [02:35<00:44, 3.02it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:43, 3.02it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 311/443 [02:35<00:43, 3.02it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:43, 3.02it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|███████ | 312/443 [02:35<00:43, 3.02it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:52, 2.47it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:56, 2.31it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:52, 2.47it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:56, 2.31it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:55, 2.34it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:50, 2.60it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:53, 2.44it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:52, 2.47it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:56, 2.31it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:55, 2.36it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:56, 2.31it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:55, 2.34it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:53, 2.44it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:50, 2.60it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:53, 2.44it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:50, 2.60it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:55, 2.34it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 314/443 [02:37<00:52, 2.47it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:55, 2.36it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 314/443 [02:37<00:52, 2.47it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:56, 2.31it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:55, 2.36it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 313/443 [02:37<00:56, 2.31it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 70%|███████ | 311/443 [02:38<01:19, 1.66it/s, Materializing param=model.layers.27.self_attn.v_proj.weight] Loading weights: 70%|███████ | 311/443 [02:38<01:19, 1.66it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 311/443 [02:38<01:19, 1.66it/s, Materializing param=model.layers.28.input_layernorm.weight] Loading weights: 70%|███████ | 312/443 [02:38<01:19, 1.66it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 70%|███████ | 312/443 [02:38<01:19, 1.66it/s, Materializing param=model.layers.28.mlp.down_proj.weight] Loading weights: 71%|███████ | 313/443 [02:38<01:18, 1.66it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 313/443 [02:38<01:18, 1.66it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:54, 2.36it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:54, 2.36it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:54, 2.36it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:54, 2.38it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:54, 2.38it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:54, 2.38it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<01:00, 2.14it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<01:00, 2.14it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<01:00, 2.14it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:56, 2.26it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<01:03, 2.05it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<01:03, 2.05it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:56, 2.26it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<01:03, 2.05it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:56, 2.26it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:59, 2.15it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:59, 2.15it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:59, 2.15it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:58, 2.22it/s, Materializing param=model.layers.28.mlp.gate_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:58, 2.22it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 314/443 [02:38<00:58, 2.22it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 315/443 [02:38<00:57, 2.22it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 315/443 [02:38<00:57, 2.22it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████ | 315/443 [02:38<00:57, 2.22it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████▏ | 316/443 [02:38<00:57, 2.22it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 71%|███████▏ | 316/443 [02:38<00:57, 2.22it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 72%|███████▏ | 317/443 [02:38<00:56, 2.22it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 317/443 [02:38<00:56, 2.22it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 71%|███████ | 315/443 [02:38<00:54, 2.34it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 315/443 [02:38<00:54, 2.34it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████ | 315/443 [02:38<00:54, 2.34it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<00:54, 2.34it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<00:54, 2.34it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<00:33, 3.71it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<00:53, 2.34it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<00:33, 3.71it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<00:53, 2.34it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<00:33, 3.71it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<00:53, 2.34it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<00:53, 2.34it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 71%|███████ | 315/443 [02:39<01:01, 2.08it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 315/443 [02:39<01:01, 2.08it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████ | 315/443 [02:39<01:01, 2.08it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<01:00, 2.08it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<01:00, 2.08it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<01:00, 2.08it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<01:00, 2.08it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<01:00, 2.08it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<01:00, 2.08it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 71%|███████ | 315/443 [02:39<01:03, 2.02it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 315/443 [02:39<01:03, 2.02it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████ | 315/443 [02:39<01:03, 2.02it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<01:02, 2.02it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<01:02, 2.02it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<01:02, 2.02it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<01:02, 2.02it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 71%|███████ | 315/443 [02:39<01:02, 2.05it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 315/443 [02:39<01:02, 2.05it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████ | 315/443 [02:39<01:02, 2.05it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<01:01, 2.05it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<01:01, 2.05it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<01:01, 2.05it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<01:01, 2.05it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<01:02, 2.02it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<01:02, 2.02it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 71%|███████ | 315/443 [02:39<00:57, 2.23it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 315/443 [02:39<00:57, 2.23it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████ | 315/443 [02:39<00:57, 2.23it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<00:57, 2.23it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<00:57, 2.23it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<00:56, 2.23it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<00:56, 2.23it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 71%|███████ | 315/443 [02:39<00:58, 2.20it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 315/443 [02:39<00:58, 2.20it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████ | 315/443 [02:39<00:58, 2.20it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<01:00, 2.05it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<01:00, 2.05it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<00:57, 2.20it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<00:57, 2.20it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<00:57, 2.20it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<00:57, 2.20it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<00:56, 2.23it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<00:56, 2.23it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<00:56, 2.20it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<00:56, 2.20it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 71%|███████ | 315/443 [02:39<01:01, 2.10it/s, Materializing param=model.layers.28.mlp.up_proj.weight] Loading weights: 71%|███████ | 315/443 [02:39<01:01, 2.10it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████ | 315/443 [02:39<01:01, 2.10it/s, Materializing param=model.layers.28.post_attention_layernorm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<01:00, 2.10it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 71%|███████▏ | 316/443 [02:39<01:00, 2.10it/s, Materializing param=model.layers.28.self_attn.k_norm.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<01:00, 2.10it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 317/443 [02:39<01:00, 2.10it/s, Materializing param=model.layers.28.self_attn.k_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<00:59, 2.10it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 318/443 [02:39<00:59, 2.10it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:35, 3.51it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:36, 3.42it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:34, 3.58it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:34, 3.64it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:37, 3.32it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:39, 3.11it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:35, 3.51it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:34, 3.58it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:34, 3.64it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:35, 3.51it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:34, 3.58it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:37, 3.32it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:33, 3.71it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:39, 3.11it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:37, 3.32it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:34, 3.64it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:33, 3.71it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:39, 3.11it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:35, 3.51it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:35, 3.51it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:34, 3.56it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:33, 3.64it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:36, 3.42it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:34, 3.56it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:33, 3.64it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:34, 3.56it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:37, 3.32it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:37, 3.32it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:36, 3.42it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:35, 3.42it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:35, 3.42it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:35, 3.49it/s, Materializing param=model.layers.28.self_attn.o_proj.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:35, 3.49it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 319/443 [02:39<00:35, 3.49it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:35, 3.49it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:35, 3.49it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:34, 3.49it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:34, 3.49it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:39, 3.11it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:39, 3.11it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:34, 3.49it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:34, 3.49it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 323/443 [02:39<00:34, 3.49it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 323/443 [02:39<00:34, 3.49it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:32, 3.79it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:32, 3.79it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:32, 3.79it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:31, 3.79it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:31, 3.79it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 323/443 [02:39<00:31, 3.79it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 323/443 [02:39<00:31, 3.79it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:28, 4.29it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:29, 4.16it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:29, 4.16it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:28, 4.29it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:31, 3.89it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:29, 4.16it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:28, 4.35it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:31, 3.89it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:28, 4.35it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:31, 3.89it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:28, 4.35it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:28, 4.29it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:27, 4.49it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:27, 4.49it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:32, 3.82it/s, Materializing param=model.layers.28.self_attn.q_norm.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:27, 4.49it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:32, 3.82it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 320/443 [02:39<00:32, 3.82it/s, Materializing param=model.layers.28.self_attn.q_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:31, 3.82it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 72%|███████▏ | 321/443 [02:39<00:31, 3.82it/s, Materializing param=model.layers.28.self_attn.v_proj.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:29, 4.16it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:29, 4.16it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 323/443 [02:39<00:28, 4.16it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 323/443 [02:39<00:28, 4.16it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:31, 3.89it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:31, 3.89it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 323/443 [02:39<00:30, 3.89it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:28, 4.29it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 323/443 [02:39<00:30, 3.89it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:28, 4.29it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:27, 4.35it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:27, 4.35it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 323/443 [02:39<00:27, 4.35it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 323/443 [02:39<00:27, 4.35it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:26, 4.49it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:26, 4.49it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 323/443 [02:39<00:26, 4.49it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 323/443 [02:39<00:26, 4.49it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:31, 3.82it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 322/443 [02:39<00:31, 3.82it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:46, 2.58it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:43, 2.76it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:42, 2.80it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:40, 2.91it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:41, 2.87it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 323/443 [02:41<00:48, 2.48it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:38, 3.08it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:42, 2.80it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:46, 2.58it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:40, 2.91it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:41, 2.87it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:42, 2.80it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:46, 2.58it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:40, 2.91it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:41, 2.87it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 323/443 [02:41<00:45, 2.66it/s, Materializing param=model.layers.29.input_layernorm.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:38, 3.08it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:38, 3.08it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 323/443 [02:41<00:48, 2.48it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 323/443 [02:41<00:45, 2.66it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 323/443 [02:41<00:45, 2.66it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:44, 2.66it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:44, 2.66it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 323/443 [02:41<00:48, 2.48it/s, Materializing param=model.layers.29.mlp.down_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:47, 2.48it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:47, 2.48it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:43, 2.76it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 324/443 [02:41<00:43, 2.76it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:42, 2.76it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:42, 2.76it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:40, 2.95it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:40, 2.95it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:40, 2.95it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:41, 2.82it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:44, 2.68it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:42, 2.77it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:39, 2.96it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:44, 2.68it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:47, 2.50it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:43, 2.74it/s, Materializing param=model.layers.29.mlp.gate_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:39, 2.96it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:44, 2.68it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:42, 2.77it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:39, 2.96it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:42, 2.77it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:43, 2.74it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:43, 2.74it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:41, 2.82it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:41, 2.82it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:47, 2.50it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 73%|███████▎ | 325/443 [02:41<00:47, 2.50it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:45, 2.56it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:45, 2.56it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:45, 2.56it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:45, 2.56it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:45, 2.56it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:44, 2.56it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:44, 2.56it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:44, 2.56it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:44, 2.56it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:44, 2.63it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:44, 2.63it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:44, 2.63it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:44, 2.63it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:44, 2.63it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:43, 2.63it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:43, 2.63it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:43, 2.63it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:43, 2.63it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:52, 2.23it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:52, 2.23it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:52, 2.23it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:51, 2.23it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:51, 2.23it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:51, 2.23it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:51, 2.23it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:49, 2.37it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:49, 2.37it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:49, 2.37it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:48, 2.37it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:48, 2.37it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:48, 2.37it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:48, 2.37it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:48, 2.37it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:48, 2.37it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:48, 2.42it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:48, 2.42it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:48, 2.42it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:47, 2.42it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:47, 2.42it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:47, 2.42it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:47, 2.42it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:47, 2.42it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:47, 2.42it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:51, 2.28it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:51, 2.28it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:51, 2.28it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:50, 2.28it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:50, 2.28it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:51, 2.26it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:51, 2.26it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:51, 2.26it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:51, 2.26it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:51, 2.26it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:48, 2.41it/s, Materializing param=model.layers.29.mlp.up_proj.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:48, 2.41it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▎ | 326/443 [02:42<00:48, 2.41it/s, Materializing param=model.layers.29.post_attention_layernorm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:48, 2.41it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 327/443 [02:42<00:48, 2.41it/s, Materializing param=model.layers.29.self_attn.k_norm.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:47, 2.41it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:47, 2.41it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:50, 2.28it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:50, 2.28it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:50, 2.28it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:50, 2.28it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:50, 2.26it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 328/443 [02:42<00:50, 2.26it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:50, 2.26it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:50, 2.26it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:47, 2.41it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:47, 2.41it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:26, 4.21it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:30, 3.71it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:26, 4.21it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:30, 3.71it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:30, 3.71it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:26, 4.21it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:26, 4.22it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:42<00:26, 4.21it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:26, 4.22it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:26, 4.22it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 75%|███████▍ | 331/443 [02:42<00:26, 4.21it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:32, 3.55it/s, Materializing param=model.layers.29.self_attn.k_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:32, 3.55it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 329/443 [02:42<00:32, 3.55it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:31, 3.55it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:31, 3.55it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:26, 4.24it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:26, 4.24it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:42<00:26, 4.24it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 75%|███████▍ | 331/443 [02:42<00:26, 4.24it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:42<00:26, 4.24it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:42<00:24, 4.55it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 75%|███████▍ | 331/443 [02:42<00:25, 4.37it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 75%|███████▍ | 331/443 [02:42<00:25, 4.37it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:42<00:24, 4.55it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:42<00:24, 4.55it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:42<00:25, 4.37it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:42<00:24, 4.55it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:42<00:25, 4.37it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:42<00:24, 4.55it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:42<00:25, 4.37it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:25, 4.37it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:25, 4.37it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 334/443 [02:43<00:24, 4.37it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:24, 4.55it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 334/443 [02:43<00:24, 4.37it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:24, 4.55it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 334/443 [02:43<00:23, 4.55it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▌ | 334/443 [02:43<00:23, 4.55it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 74%|███████▍ | 330/443 [02:43<00:28, 3.93it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 330/443 [02:43<00:28, 3.93it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:43<00:28, 3.93it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:43<00:28, 3.90it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 330/443 [02:43<00:27, 4.04it/s, Materializing param=model.layers.29.self_attn.o_proj.weight] Loading weights: 74%|███████▍ | 330/443 [02:43<00:27, 4.04it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:43<00:27, 4.04it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:43<00:28, 3.90it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 74%|███████▍ | 330/443 [02:43<00:28, 3.90it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 75%|███████▍ | 331/443 [02:43<00:27, 4.04it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:43<00:27, 4.04it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:43<00:28, 3.90it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:43<00:28, 3.90it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:22, 4.89it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:43<00:29, 3.80it/s, Materializing param=model.layers.29.self_attn.q_norm.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:22, 4.89it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:43<00:29, 3.80it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:22, 4.89it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:43<00:29, 3.80it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:29, 3.80it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:29, 3.80it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:28, 3.80it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:28, 3.80it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 334/443 [02:43<00:28, 3.80it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▌ | 334/443 [02:43<00:28, 3.80it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:22, 4.89it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:22, 4.89it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 334/443 [02:43<00:22, 4.89it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▌ | 334/443 [02:43<00:22, 4.89it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:22, 4.93it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:22, 4.93it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:22, 4.93it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:22, 4.93it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:22, 4.93it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:28, 3.90it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:28, 3.90it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:27, 4.04it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:27, 4.04it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:43<00:28, 3.93it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 331/443 [02:43<00:28, 3.93it/s, Materializing param=model.layers.29.self_attn.q_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:28, 3.93it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▍ | 332/443 [02:43<00:28, 3.93it/s, Materializing param=model.layers.29.self_attn.v_proj.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:28, 3.90it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:28, 3.90it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 334/443 [02:43<00:27, 3.90it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▌ | 334/443 [02:43<00:27, 3.90it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:27, 3.93it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:27, 3.93it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:27, 4.04it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 333/443 [02:43<00:27, 4.04it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 75%|███████▌ | 334/443 [02:43<00:27, 3.93it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▌ | 334/443 [02:43<00:27, 3.93it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:35, 3.00it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:37, 2.86it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:37, 2.91it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▌ | 334/443 [02:44<00:42, 2.58it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:33, 3.26it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▌ | 334/443 [02:44<00:36, 3.01it/s, Materializing param=model.layers.30.input_layernorm.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:33, 3.25it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:35, 3.00it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:36, 3.00it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:37, 2.86it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:33, 3.26it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:35, 3.00it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:37, 2.91it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:37, 2.86it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:33, 3.25it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:33, 3.26it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:36, 3.00it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:33, 3.25it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:37, 2.91it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 75%|███████▌ | 334/443 [02:44<00:36, 3.01it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:36, 3.00it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 75%|███████▌ | 334/443 [02:44<00:36, 3.01it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 75%|███████▌ | 334/443 [02:44<00:42, 2.58it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:35, 3.01it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:35, 3.01it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 75%|███████▌ | 334/443 [02:44<00:42, 2.58it/s, Materializing param=model.layers.30.mlp.down_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:41, 2.58it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 335/443 [02:44<00:41, 2.58it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<01:00, 1.77it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:59, 1.81it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:59, 1.79it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<01:02, 1.71it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:51, 2.09it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:53, 1.99it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:53, 1.99it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:59, 1.80it/s, Materializing param=model.layers.30.mlp.gate_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<01:00, 1.77it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<01:02, 1.71it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:59, 1.79it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<01:00, 1.77it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:53, 1.99it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:51, 2.09it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:53, 1.99it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<01:02, 1.71it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:59, 1.79it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:59, 1.80it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:53, 1.99it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:51, 2.09it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:53, 1.99it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:59, 1.80it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:59, 1.81it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 336/443 [02:46<00:59, 1.81it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:31, 1.16it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:27, 1.21it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:27, 1.21it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:27, 1.21it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:31, 1.16it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:28, 1.19it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:31, 1.16it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:19, 1.34it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:27, 1.21it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:28, 1.19it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:15, 1.40it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:19, 1.34it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:28, 1.19it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:15, 1.40it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:19, 1.34it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:26, 1.21it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:15, 1.40it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:30, 1.16it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:19, 1.34it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:26, 1.21it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:27, 1.19it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:30, 1.16it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:25, 1.23it/s, Materializing param=model.layers.30.mlp.up_proj.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:19, 1.34it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:27, 1.19it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:14, 1.40it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:19, 1.34it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:18, 1.34it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:25, 1.23it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:14, 1.40it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:25, 1.21it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:29, 1.16it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:25, 1.23it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:29, 1.16it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:25, 1.21it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:18, 1.34it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:18, 1.34it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:25, 1.23it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:27, 1.21it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:18, 1.34it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:25, 1.23it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:17, 1.34it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:14, 1.40it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:17, 1.34it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:24, 1.23it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:17, 1.34it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:14, 1.40it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 76%|███████▌ | 337/443 [02:48<01:27, 1.21it/s, Materializing param=model.layers.30.post_attention_layernorm.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:24, 1.23it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:17, 1.34it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:26, 1.21it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 76%|███████▋ | 338/443 [02:48<01:26, 1.21it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:25, 1.21it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 339/443 [02:48<01:25, 1.21it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 339/443 [02:49<01:00, 1.71it/s, Materializing param=model.layers.30.self_attn.k_norm.weight] Loading weights: 77%|███████▋ | 339/443 [02:49<01:00, 1.71it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 339/443 [02:49<01:00, 1.71it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:51, 2.00it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<01:00, 1.71it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:51, 2.00it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:48, 2.10it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<01:00, 1.71it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:51, 2.00it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:52, 1.95it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:51, 2.00it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:48, 2.10it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:51, 2.00it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:51, 2.00it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:48, 2.10it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:51, 2.01it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:47, 2.16it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:47, 2.16it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:47, 2.16it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:51, 2.01it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:51, 2.01it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:52, 1.95it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:52, 1.95it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:49, 2.10it/s, Materializing param=model.layers.30.self_attn.k_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:49, 2.10it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 340/443 [02:49<00:49, 2.10it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:50, 2.01it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:48, 2.10it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:50, 2.00it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:48, 2.09it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:48, 2.09it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:50, 2.00it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:48, 2.10it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:50, 2.00it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:48, 2.09it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:50, 2.00it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:48, 2.10it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:47, 2.14it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:50, 2.00it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:47, 2.14it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:50, 2.00it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:47, 2.14it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:50, 2.00it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:50, 2.00it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:47, 2.14it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:48, 2.10it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:47, 2.14it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:48, 2.10it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:50, 2.01it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:50, 2.00it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:50, 2.00it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:43, 2.34it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:43, 2.34it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:43, 2.34it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:48, 2.09it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:48, 2.09it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:48, 2.09it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:48, 2.09it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:51, 1.97it/s, Materializing param=model.layers.30.self_attn.o_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:48, 2.09it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:48, 2.09it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:48, 2.09it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:47, 2.09it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:47, 2.09it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:51, 1.97it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 341/443 [02:49<00:51, 1.97it/s, Materializing param=model.layers.30.self_attn.q_norm.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:51, 1.97it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 342/443 [02:49<00:51, 1.97it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:50, 1.97it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:50, 1.97it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:30, 3.28it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:31, 3.19it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:31, 3.19it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:30, 3.28it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:31, 3.19it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:30, 3.28it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:30, 3.19it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:29, 3.28it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:30, 3.19it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:29, 3.28it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:35, 2.85it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:35, 2.85it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:35, 2.85it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:38, 2.63it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:38, 2.63it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:38, 2.63it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:35, 2.81it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:35, 2.81it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:35, 2.81it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:34, 2.85it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:34, 2.85it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:34, 2.85it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:34, 2.85it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:36, 2.70it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:36, 2.70it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:36, 2.70it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:35, 2.83it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:36, 2.72it/s, Materializing param=model.layers.30.self_attn.q_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:35, 2.83it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:36, 2.72it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:35, 2.83it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 77%|███████▋ | 343/443 [02:49<00:36, 2.72it/s, Materializing param=model.layers.30.self_attn.v_proj.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:37, 2.63it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:37, 2.63it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:35, 2.81it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:35, 2.81it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:34, 2.81it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:34, 2.81it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:36, 2.70it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:36, 2.70it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:36, 2.70it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:36, 2.70it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:34, 2.83it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:34, 2.83it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:36, 2.72it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 344/443 [02:49<00:36, 2.72it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:34, 2.83it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:36, 2.72it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:34, 2.83it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 345/443 [02:49<00:36, 2.72it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:47, 2.03it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:51, 1.90it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:48, 2.00it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:47, 2.03it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:48, 2.00it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:49, 1.95it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:47, 2.06it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:47, 2.03it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:48, 2.00it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:51, 1.90it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:47, 2.03it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:48, 2.00it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:47, 2.03it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:49, 1.95it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:47, 2.06it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:48, 2.00it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:51, 1.90it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:47, 2.03it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:49, 1.95it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:47, 2.06it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:51<00:48, 2.00it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:13, 1.31it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:16, 1.25it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:11, 1.34it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:12, 1.32it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:14, 1.28it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:13, 1.31it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 345/443 [02:53<01:32, 1.06it/s, Materializing param=model.layers.31.input_layernorm.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:13, 1.31it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:11, 1.34it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:16, 1.25it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:13, 1.31it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:12, 1.32it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:14, 1.28it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:16, 1.25it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:13, 1.31it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:13, 1.31it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:11, 1.34it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:12, 1.32it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:14, 1.28it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:12, 1.33it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:12, 1.33it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 345/443 [02:53<01:32, 1.06it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:12, 1.33it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 345/443 [02:53<01:32, 1.06it/s, Materializing param=model.layers.31.mlp.down_proj.weight] Loading weights: 79%|███████▊ | 348/443 [02:53<01:11, 1.33it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▊ | 348/443 [02:53<01:11, 1.33it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▉ | 349/443 [02:53<01:10, 1.33it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 78%|███████▊ | 346/443 [02:53<01:31, 1.06it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 79%|███████▉ | 349/443 [02:53<01:10, 1.33it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▉ | 350/443 [02:53<01:09, 1.33it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▉ | 350/443 [02:53<01:09, 1.33it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 78%|███████▊ | 346/443 [02:53<01:31, 1.06it/s, Materializing param=model.layers.31.mlp.gate_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:30, 1.06it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 78%|███████▊ | 347/443 [02:53<01:30, 1.06it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:09, 1.33it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:09, 1.33it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:35, 2.55it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:35, 2.55it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:35, 2.55it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 80%|███████▉ | 353/443 [02:54<00:35, 2.55it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 353/443 [02:54<00:35, 2.55it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:00, 1.58it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:00, 1.58it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:00, 1.58it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<00:59, 1.58it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<00:59, 1.58it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:09, 1.37it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:09, 1.37it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:09, 1.37it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<01:08, 1.37it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<01:08, 1.37it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<00:58, 1.58it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<00:58, 1.58it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<00:58, 1.58it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<00:58, 1.58it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<01:07, 1.37it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<01:07, 1.37it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:06, 1.37it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:06, 1.37it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:54<00:29, 2.98it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:54<00:29, 2.98it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:54<00:29, 2.98it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:33, 2.70it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:33, 2.70it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:33, 2.70it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 80%|███████▉ | 353/443 [02:54<00:33, 2.70it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 353/443 [02:54<00:33, 2.70it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:12, 1.31it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:12, 1.31it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:11, 1.33it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:12, 1.31it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<01:11, 1.31it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:11, 1.33it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<01:11, 1.31it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:11, 1.33it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<01:11, 1.31it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<01:10, 1.33it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<01:11, 1.31it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<01:10, 1.33it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<01:09, 1.33it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<01:09, 1.33it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:10, 1.34it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:10, 1.34it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:10, 1.34it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<01:10, 1.34it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<01:10, 1.34it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:10, 1.35it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<01:09, 1.34it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:14, 1.28it/s, Materializing param=model.layers.31.mlp.up_proj.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<01:09, 1.34it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:14, 1.28it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:10, 1.35it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:14, 1.28it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▊ | 348/443 [02:54<01:10, 1.35it/s, Materializing param=model.layers.31.post_attention_layernorm.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<01:13, 1.28it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<01:13, 1.28it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<01:09, 1.35it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<01:12, 1.28it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▉ | 349/443 [02:54<01:09, 1.35it/s, Materializing param=model.layers.31.self_attn.k_norm.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<01:12, 1.28it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<01:08, 1.35it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 79%|███████▉ | 350/443 [02:54<01:08, 1.35it/s, Materializing param=model.layers.31.self_attn.k_proj.weight] Loading weights: 80%|████████ | 355/443 [02:54<00:29, 2.98it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 355/443 [02:54<00:29, 2.98it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:08, 1.34it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:08, 1.34it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:11, 1.28it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:11, 1.28it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:09, 1.33it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:10, 1.31it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:09, 1.33it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:10, 1.31it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:07, 1.35it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 351/443 [02:54<01:07, 1.35it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:34, 2.66it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:34, 2.66it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:34, 2.66it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 80%|███████▉ | 353/443 [02:54<00:33, 2.66it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 353/443 [02:54<00:33, 2.66it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:54<00:26, 3.30it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:54<00:26, 3.30it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:54<00:26, 3.30it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|████████ | 355/443 [02:54<00:26, 3.30it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 355/443 [02:54<00:26, 3.30it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 356/443 [02:54<00:26, 3.30it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|████████ | 356/443 [02:54<00:26, 3.30it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:35, 2.58it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:35, 2.58it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:35, 2.58it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 80%|███████▉ | 353/443 [02:54<00:34, 2.58it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 353/443 [02:54<00:34, 2.58it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:35, 2.57it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:35, 2.57it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:35, 2.57it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:36, 2.50it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:36, 2.50it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:36, 2.50it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 80%|███████▉ | 354/443 [02:54<00:26, 3.36it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:54<00:26, 3.36it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:54<00:26, 3.36it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:36, 2.51it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:36, 2.51it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:36, 2.51it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:35, 2.53it/s, Materializing param=model.layers.31.self_attn.o_proj.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:35, 2.53it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 79%|███████▉ | 352/443 [02:54<00:35, 2.53it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 80%|███████▉ | 353/443 [02:54<00:35, 2.53it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 353/443 [02:54<00:35, 2.53it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|████████ | 355/443 [02:54<00:26, 3.36it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 355/443 [02:54<00:26, 3.36it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 356/443 [02:54<00:25, 3.36it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|████████ | 356/443 [02:54<00:25, 3.36it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:55<00:27, 3.27it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:55<00:27, 3.27it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:55<00:27, 3.27it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|███████▉ | 353/443 [02:55<00:32, 2.77it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 80%|███████▉ | 353/443 [02:55<00:32, 2.77it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 353/443 [02:55<00:32, 2.77it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:55<00:32, 2.77it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:55<00:32, 2.77it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:55<00:27, 3.23it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:55<00:27, 3.23it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:55<00:27, 3.23it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|███████▉ | 353/443 [02:55<00:32, 2.77it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 80%|███████▉ | 353/443 [02:55<00:32, 2.77it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 353/443 [02:55<00:32, 2.77it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:55<00:32, 2.77it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:55<00:32, 2.77it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|████████ | 355/443 [02:55<00:26, 3.27it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 355/443 [02:55<00:26, 3.27it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 356/443 [02:55<00:26, 3.27it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|████████ | 356/443 [02:55<00:26, 3.27it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|████████ | 355/443 [02:55<00:31, 2.77it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 355/443 [02:55<00:31, 2.77it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 356/443 [02:55<00:31, 2.77it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|████████ | 356/443 [02:55<00:31, 2.77it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|███████▉ | 353/443 [02:55<00:32, 2.79it/s, Materializing param=model.layers.31.self_attn.q_norm.weight] Loading weights: 80%|███████▉ | 353/443 [02:55<00:32, 2.79it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 353/443 [02:55<00:32, 2.79it/s, Materializing param=model.layers.31.self_attn.q_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:55<00:31, 2.79it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|███████▉ | 354/443 [02:55<00:31, 2.79it/s, Materializing param=model.layers.31.self_attn.v_proj.weight] Loading weights: 80%|████████ | 355/443 [02:55<00:31, 2.79it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 355/443 [02:55<00:31, 2.79it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 356/443 [02:55<00:31, 2.79it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|████████ | 356/443 [02:55<00:31, 2.79it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|████████ | 355/443 [02:55<00:31, 2.77it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 355/443 [02:55<00:27, 3.23it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 355/443 [02:55<00:27, 3.23it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 355/443 [02:55<00:31, 2.77it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 80%|████████ | 356/443 [02:55<00:26, 3.23it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|████████ | 356/443 [02:55<00:26, 3.23it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|████████ | 356/443 [02:55<00:31, 2.77it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|████████ | 356/443 [02:55<00:31, 2.77it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:35, 2.42it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:34, 2.51it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:33, 2.58it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:34, 2.51it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:35, 2.43it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:33, 2.60it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 80%|████████ | 356/443 [02:56<00:44, 1.95it/s, Materializing param=model.layers.32.input_layernorm.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:33, 2.58it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:32, 2.62it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:35, 2.43it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:34, 2.51it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:34, 2.51it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:33, 2.58it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:33, 2.60it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:35, 2.43it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:34, 2.51it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 80%|████████ | 356/443 [02:56<00:44, 1.95it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:33, 2.60it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:34, 2.51it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:32, 2.62it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 80%|████████ | 356/443 [02:56<00:44, 1.95it/s, Materializing param=model.layers.32.mlp.down_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:35, 2.42it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 358/443 [02:56<00:32, 2.60it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:32, 2.62it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 358/443 [02:56<00:32, 2.60it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:35, 2.42it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:44, 1.95it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 357/443 [02:56<00:44, 1.95it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:35, 2.41it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:35, 2.41it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:35, 2.41it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:35, 2.39it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:35, 2.39it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:35, 2.39it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:38, 2.22it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:38, 2.22it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:38, 2.22it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:40, 2.12it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:40, 2.12it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:40, 2.12it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:38, 2.19it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:38, 2.19it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 358/443 [02:57<00:38, 2.19it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:38, 2.16it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:38, 2.16it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:38, 2.16it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████▏ | 360/443 [02:57<00:38, 2.16it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:36, 2.29it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:38, 2.16it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:40, 2.08it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████▏ | 360/443 [02:57<00:38, 2.16it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:36, 2.27it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:38, 2.16it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:40, 2.08it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:36, 2.29it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:38, 2.16it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:40, 2.08it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████▏ | 361/443 [02:57<00:37, 2.16it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:36, 2.29it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████▏ | 361/443 [02:57<00:37, 2.16it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:36, 2.27it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:36, 2.27it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████▏ | 360/443 [02:57<00:38, 2.16it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 360/443 [02:57<00:39, 2.08it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 360/443 [02:57<00:38, 2.16it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 360/443 [02:57<00:36, 2.29it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 360/443 [02:57<00:39, 2.08it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 360/443 [02:57<00:36, 2.29it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 360/443 [02:57<00:36, 2.27it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 361/443 [02:57<00:37, 2.16it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████▏ | 360/443 [02:57<00:36, 2.27it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 361/443 [02:57<00:39, 2.08it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████▏ | 361/443 [02:57<00:37, 2.16it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████▏ | 361/443 [02:57<00:39, 2.08it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████▏ | 361/443 [02:57<00:35, 2.29it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████▏ | 361/443 [02:57<00:35, 2.29it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████▏ | 361/443 [02:57<00:36, 2.27it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████▏ | 361/443 [02:57<00:36, 2.27it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:38, 2.20it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:38, 2.20it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████ | 359/443 [02:57<00:38, 2.20it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████▏ | 360/443 [02:57<00:37, 2.20it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 360/443 [02:57<00:37, 2.20it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 361/443 [02:57<00:37, 2.20it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████▏ | 361/443 [02:57<00:37, 2.20it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:22, 3.55it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:21, 3.73it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:22, 3.55it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:22, 3.56it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:22, 3.55it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:21, 3.73it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:24, 3.34it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:22, 3.55it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:22, 3.56it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:21, 3.73it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:22, 3.56it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:24, 3.34it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:24, 3.34it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:22, 3.55it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:22, 3.55it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:24, 3.30it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:24, 3.30it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:24, 3.30it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 81%|████████ | 358/443 [02:58<00:47, 1.80it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 358/443 [02:58<00:47, 1.80it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 358/443 [02:58<00:47, 1.80it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 359/443 [02:58<00:46, 1.80it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████ | 359/443 [02:58<00:46, 1.80it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████▏ | 360/443 [02:58<00:46, 1.80it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 360/443 [02:58<00:46, 1.80it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 361/443 [02:58<00:45, 1.80it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████▏ | 361/443 [02:58<00:45, 1.80it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████ | 358/443 [02:58<00:44, 1.90it/s, Materializing param=model.layers.32.mlp.gate_proj.weight] Loading weights: 81%|████████ | 358/443 [02:58<00:44, 1.90it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 358/443 [02:58<00:44, 1.90it/s, Materializing param=model.layers.32.mlp.up_proj.weight] Loading weights: 81%|████████ | 359/443 [02:58<00:44, 1.90it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████ | 359/443 [02:58<00:44, 1.90it/s, Materializing param=model.layers.32.post_attention_layernorm.weight] Loading weights: 81%|████████▏ | 360/443 [02:58<00:43, 1.90it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 360/443 [02:58<00:43, 1.90it/s, Materializing param=model.layers.32.self_attn.k_norm.weight] Loading weights: 81%|████████▏ | 361/443 [02:58<00:43, 1.90it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 81%|████████▏ | 361/443 [02:58<00:43, 1.90it/s, Materializing param=model.layers.32.self_attn.k_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:45, 1.80it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:45, 1.80it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:42, 1.90it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 362/443 [02:58<00:42, 1.90it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:22, 3.55it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:22, 3.55it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:21, 3.73it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:22, 3.56it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:25, 3.20it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:25, 3.09it/s, Materializing param=model.layers.32.self_attn.o_proj.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:23, 3.34it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:22, 3.55it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:21, 3.73it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:22, 3.56it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:22, 3.55it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:25, 3.20it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:23, 3.34it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:25, 3.09it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:25, 3.20it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:25, 3.09it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:24, 3.30it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:22, 3.51it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 363/443 [02:58<00:24, 3.30it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:22, 3.51it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:22, 3.51it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:23, 3.32it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:23, 3.32it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:22, 3.50it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:23, 3.32it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:22, 3.50it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:22, 3.50it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:22, 3.50it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:22, 3.50it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:22, 3.50it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:24, 3.20it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:25, 3.09it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:24, 3.20it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:58<00:25, 3.09it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:59<00:28, 2.81it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:25, 3.01it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:59<00:28, 2.81it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:59<00:28, 2.81it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:25, 3.02it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:25, 3.01it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:24, 3.19it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:25, 3.01it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:23, 3.28it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:27, 2.81it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:24, 3.19it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:27, 2.81it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:24, 3.19it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:25, 3.01it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:26, 2.91it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:23, 3.28it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:25, 3.01it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:23, 3.28it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:26, 2.91it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:25, 3.02it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:25, 3.01it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:26, 2.91it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:25, 3.02it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:21, 3.59it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:21, 3.59it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:21, 3.59it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:21, 3.50it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:23, 3.34it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:21, 3.50it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:23, 3.34it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:21, 3.50it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:23, 3.34it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 367/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:20, 3.71it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 83%|████████▎ | 367/443 [02:59<00:22, 3.34it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 367/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 83%|████████▎ | 367/443 [02:59<00:22, 3.34it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:20, 3.71it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:20, 3.71it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 82%|████████▏ | 364/443 [02:59<00:30, 2.61it/s, Materializing param=model.layers.32.self_attn.q_norm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 367/443 [02:59<00:20, 3.71it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:59<00:30, 2.61it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 82%|████████▏ | 364/443 [02:59<00:30, 2.61it/s, Materializing param=model.layers.32.self_attn.q_proj.weight] Loading weights: 83%|████████▎ | 367/443 [02:59<00:20, 3.71it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 367/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 367/443 [02:59<00:22, 3.45it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:29, 2.61it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 82%|████████▏ | 365/443 [02:59<00:29, 2.61it/s, Materializing param=model.layers.32.self_attn.v_proj.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:29, 2.61it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 366/443 [02:59<00:29, 2.61it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 367/443 [02:59<00:29, 2.61it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 367/443 [02:59<00:29, 2.61it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:32, 2.28it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:41, 1.80it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:41, 1.79it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:41, 1.80it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 367/443 [03:01<00:44, 1.72it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:36, 2.03it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:41, 1.80it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:32, 2.28it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:41, 1.79it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:41, 1.80it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 367/443 [03:01<00:44, 1.72it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:41, 1.80it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:32, 2.28it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:41, 1.79it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 367/443 [03:01<00:44, 1.72it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:41, 1.80it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:36, 2.03it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 367/443 [03:01<00:51, 1.48it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:36, 2.03it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:43, 1.72it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 367/443 [03:01<00:51, 1.48it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:01<00:36, 2.03it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:43, 1.72it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:01<00:36, 2.03it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 367/443 [03:01<00:51, 1.48it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:50, 1.48it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:50, 1.48it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:01<00:32, 2.28it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:01<00:32, 2.28it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 367/443 [03:01<00:45, 1.68it/s, Materializing param=model.layers.33.input_layernorm.weight] Loading weights: 83%|████████▎ | 367/443 [03:01<00:45, 1.68it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 367/443 [03:01<00:45, 1.68it/s, Materializing param=model.layers.33.mlp.down_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:44, 1.68it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 368/443 [03:01<00:44, 1.68it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:42, 1.74it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:42, 1.74it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:42, 1.74it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:42, 1.74it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:36, 2.01it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:42, 1.74it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:36, 2.01it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:42, 1.74it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:36, 2.01it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:35, 2.06it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:35, 2.06it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:35, 2.06it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:42, 1.74it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:42, 1.74it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:42, 1.74it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:40, 1.85it/s, Materializing param=model.layers.33.mlp.gate_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:40, 1.85it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 83%|████████▎ | 369/443 [03:02<00:40, 1.85it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:32, 2.22it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:35, 2.03it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:32, 2.22it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:32, 2.22it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:35, 2.03it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:35, 2.03it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:35, 2.03it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:35, 2.03it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:32, 2.22it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:32, 2.22it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:34, 2.03it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:31, 2.22it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:34, 2.03it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:31, 2.22it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:21, 3.18it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:21, 3.18it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:21, 3.18it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:22, 3.16it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:22, 3.16it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:22, 3.16it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:40, 1.80it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:40, 1.81it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:40, 1.80it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:40, 1.80it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:40, 1.81it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:40, 1.81it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:39, 1.80it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:39, 1.80it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:39, 1.81it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:39, 1.81it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:39, 1.81it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:39, 1.81it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:39, 1.80it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:39, 1.80it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:40, 1.80it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:40, 1.80it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:40, 1.80it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:39, 1.80it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:39, 1.80it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:39, 1.80it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:39, 1.80it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:38, 1.80it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:38, 1.80it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:38, 1.80it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:38, 1.80it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:38, 1.81it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:38, 1.81it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:36, 1.97it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:36, 1.97it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:36, 1.97it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:36, 2.01it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:36, 2.01it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:36, 2.01it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:35, 2.01it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:35, 2.01it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:35, 2.01it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:35, 2.01it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:39, 1.85it/s, Materializing param=model.layers.33.mlp.up_proj.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:39, 1.85it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 370/443 [03:02<00:39, 1.85it/s, Materializing param=model.layers.33.post_attention_layernorm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:39, 1.85it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:39, 1.85it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:38, 1.85it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:38, 1.85it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:36, 1.97it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▎ | 371/443 [03:02<00:36, 1.97it/s, Materializing param=model.layers.33.self_attn.k_norm.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:35, 1.97it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 372/443 [03:02<00:35, 1.97it/s, Materializing param=model.layers.33.self_attn.k_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:35, 1.97it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:35, 1.97it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:34, 2.01it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:34, 2.01it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:37, 1.85it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 373/443 [03:02<00:37, 1.85it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:22, 3.03it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:22, 3.03it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:22, 3.02it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:21, 3.15it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:22, 3.12it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:21, 3.18it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:22, 3.03it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:22, 3.03it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:21, 3.16it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:22, 3.02it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:22, 3.03it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:22, 3.03it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:21, 3.18it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:22, 3.02it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:21, 3.16it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:21, 3.15it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:22, 3.03it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:22, 3.03it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.13it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:22, 3.12it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:21, 3.15it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.13it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.13it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.15it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.15it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:22, 3.12it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.12it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.14it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.14it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.14it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.12it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:21, 3.19it/s, Materializing param=model.layers.33.self_attn.o_proj.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:21, 3.19it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 84%|████████▍ | 374/443 [03:03<00:21, 3.19it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.19it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.19it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:21, 3.19it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:21, 3.19it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:20, 3.19it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:20, 3.19it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:20, 3.19it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:20, 3.19it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:17, 3.88it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:17, 3.88it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:17, 3.88it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.23it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:20, 3.31it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:20, 3.31it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.23it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:20, 3.31it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.23it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:20, 3.23it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:20, 3.23it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:20, 3.28it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:20, 3.28it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:20, 3.28it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:17, 3.74it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:17, 3.74it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:17, 3.74it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:17, 3.88it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:17, 3.88it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.15it/s, Materializing param=model.layers.33.self_attn.q_norm.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.15it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 375/443 [03:03<00:21, 3.15it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:21, 3.15it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:21, 3.15it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:20, 3.15it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:20, 3.15it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:20, 3.15it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:20, 3.15it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:20, 3.23it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:19, 3.31it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:20, 3.23it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:19, 3.31it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:20, 3.23it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:20, 3.23it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:19, 3.31it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:19, 3.31it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:17, 3.76it/s, Materializing param=model.layers.33.self_attn.q_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:17, 3.76it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▍ | 376/443 [03:03<00:17, 3.76it/s, Materializing param=model.layers.33.self_attn.v_proj.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:17, 3.76it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:20, 3.28it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:17, 3.76it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:20, 3.28it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:19, 3.28it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:17, 3.76it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:19, 3.28it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:17, 3.76it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:17, 3.74it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 377/443 [03:03<00:17, 3.74it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:17, 3.74it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 378/443 [03:03<00:17, 3.74it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:24, 2.61it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:21, 2.93it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:22, 2.82it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:24, 2.61it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:21, 2.93it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:24, 2.61it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:21, 2.93it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 85%|████████▌ | 378/443 [03:04<00:25, 2.51it/s, Materializing param=model.layers.34.input_layernorm.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:22, 2.82it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:22, 2.82it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 85%|████████▌ | 378/443 [03:04<00:25, 2.51it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:24, 2.64it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:22, 2.86it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 85%|████████▌ | 378/443 [03:04<00:25, 2.51it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:24, 2.64it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:20, 3.05it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:25, 2.51it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:24, 2.64it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:25, 2.51it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:20, 3.05it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:20, 3.05it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:22, 2.86it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:22, 2.86it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:21, 2.92it/s, Materializing param=model.layers.34.mlp.down_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:21, 2.92it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 379/443 [03:04<00:21, 2.92it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:04<00:21, 2.92it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:04<00:21, 2.92it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:22, 2.74it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:22, 2.74it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:22, 2.74it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:24, 2.52it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:24, 2.52it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:24, 2.52it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:24, 2.56it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:24, 2.56it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:24, 2.56it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:22, 2.74it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:22, 2.74it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:22, 2.74it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:24, 2.62it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:24, 2.62it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:24, 2.62it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:22, 2.78it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:22, 2.78it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:05<00:22, 2.78it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:33, 1.87it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:37, 1.67it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:31, 1.99it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:33, 1.87it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:37, 1.67it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:31, 1.99it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:33, 1.87it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:37, 1.67it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:31, 1.99it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:35, 1.75it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:36, 1.67it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:32, 1.87it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:36, 1.67it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:30, 1.99it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:32, 1.87it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:35, 1.75it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:34, 1.79it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:31, 1.99it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:30, 1.99it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:35, 1.75it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:35, 1.67it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:35, 1.67it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:31, 1.99it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:30, 1.99it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:31, 1.99it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:32, 1.87it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:30, 1.99it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:32, 1.87it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:30, 1.99it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:34, 1.75it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:34, 1.79it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:30, 1.99it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:34, 1.75it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:30, 1.99it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:34, 1.79it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:30, 1.99it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:34, 1.75it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:34, 1.75it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:33, 1.79it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:33, 1.79it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:33, 1.79it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:33, 1.79it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:36, 1.70it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:36, 1.70it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:36, 1.70it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:35, 1.70it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:35, 1.70it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:35, 1.70it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:35, 1.70it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:34, 1.70it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:34, 1.70it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:35, 1.67it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:35, 1.67it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:33, 1.75it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:33, 1.75it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:29, 1.99it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:29, 1.99it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:31, 1.87it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:31, 1.87it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:32, 1.79it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:29, 1.99it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:32, 1.79it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:29, 1.99it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:06<00:37, 1.67it/s, Materializing param=model.layers.34.mlp.gate_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:06<00:37, 1.67it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 380/443 [03:06<00:37, 1.67it/s, Materializing param=model.layers.34.mlp.up_proj.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:37, 1.67it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 381/443 [03:06<00:37, 1.67it/s, Materializing param=model.layers.34.post_attention_layernorm.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:36, 1.67it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▌ | 382/443 [03:06<00:36, 1.67it/s, Materializing param=model.layers.34.self_attn.k_norm.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:35, 1.67it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 86%|████████▋ | 383/443 [03:06<00:35, 1.67it/s, Materializing param=model.layers.34.self_attn.k_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:35, 1.67it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 384/443 [03:06<00:35, 1.67it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:19, 2.98it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:20, 2.87it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:19, 3.05it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:20, 2.87it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:19, 3.05it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:20, 2.87it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:19, 2.94it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:19, 3.05it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:19, 2.94it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:19, 2.98it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:18, 3.08it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:19, 2.94it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:19, 2.98it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:18, 3.08it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:18, 3.08it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:18, 3.08it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:18, 3.08it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:20, 2.87it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:20, 2.87it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:20, 2.87it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:19, 2.87it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:19, 2.87it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:19, 2.87it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:20, 2.81it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:19, 2.87it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:20, 2.81it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:20, 2.81it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:20, 2.81it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:20, 2.81it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:19, 2.81it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:19, 2.81it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:19, 2.97it/s, Materializing param=model.layers.34.self_attn.o_proj.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:19, 2.97it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 385/443 [03:07<00:19, 2.97it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:19, 2.97it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:19, 2.97it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:18, 2.97it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:18, 2.97it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:17, 3.32it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:17, 3.32it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:17, 3.32it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:16, 3.32it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:16, 3.32it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:17, 3.19it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:17, 3.19it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:17, 3.19it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:17, 3.19it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:17, 3.19it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:14, 3.74it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:14, 3.74it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:14, 3.74it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:17, 3.26it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:18, 3.15it/s, Materializing param=model.layers.34.self_attn.q_norm.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:18, 3.15it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:18, 3.15it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:17, 3.26it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:17, 3.15it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:17, 3.15it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 386/443 [03:07<00:17, 3.26it/s, Materializing param=model.layers.34.self_attn.q_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:17, 3.26it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 87%|████████▋ | 387/443 [03:07<00:17, 3.26it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:16, 3.32it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:17, 3.19it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:17, 3.19it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:14, 3.74it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:13, 4.18it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:13, 4.12it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:12, 4.28it/s, Materializing param=model.layers.34.self_attn.v_proj.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:13, 4.18it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:13, 4.18it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:12, 4.28it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:13, 4.12it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:12, 4.28it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:13, 4.12it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:14, 3.74it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:16, 3.32it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:16, 3.26it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:14, 3.74it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:16, 3.19it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:13, 4.12it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:14, 3.74it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:16, 3.19it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:12, 4.28it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:13, 4.12it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:17, 3.15it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:16, 3.32it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:12, 4.28it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:16, 3.26it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:16, 3.32it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 388/443 [03:07<00:17, 3.15it/s, Materializing param=model.layers.35.input_layernorm.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:12, 4.18it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:12, 4.18it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:17, 3.15it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:17, 3.15it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:16, 3.26it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 389/443 [03:07<00:16, 3.26it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.46it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.51it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.43it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.48it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:22, 2.32it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:23, 2.29it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.46it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:23, 2.26it/s, Materializing param=model.layers.35.mlp.down_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.51it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.43it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.48it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:22, 2.32it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:23, 2.29it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.46it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.51it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.43it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:23, 2.26it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:22, 2.32it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.48it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:23, 2.29it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.46it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:23, 2.26it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:09<00:20, 2.51it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.46it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:09<00:20, 2.51it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 390/443 [03:09<00:21, 2.46it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:09<00:23, 2.22it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:09<00:23, 2.22it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:09<00:23, 2.22it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:22, 2.26it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:22, 2.26it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:22, 2.26it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:22, 2.35it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:22, 2.35it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:22, 2.35it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:24, 2.13it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:24, 2.13it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:24, 2.13it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:26, 2.00it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:26, 2.00it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:26, 2.00it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:24, 2.16it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:24, 2.16it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:24, 2.16it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:24, 2.10it/s, Materializing param=model.layers.35.mlp.gate_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:24, 2.10it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 391/443 [03:10<00:24, 2.10it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:24, 2.04it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:23, 2.19it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:24, 2.09it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:24, 2.09it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:24, 2.04it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:24, 2.09it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:24, 2.04it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:22, 2.26it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:23, 2.19it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:24, 2.04it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:23, 2.09it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:23, 2.19it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:24, 2.04it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:23, 2.09it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:22, 2.26it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:23, 2.04it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:22, 2.19it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:22, 2.26it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:23, 2.09it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:23, 2.04it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:22, 2.19it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:23, 2.09it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:22, 2.26it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:22, 2.19it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:22, 2.26it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:22, 2.19it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:21, 2.26it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:21, 2.26it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:23, 2.18it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:23, 2.18it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:23, 2.18it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:22, 2.18it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:22, 2.18it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:22, 2.18it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:22, 2.18it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:14, 3.30it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:13, 3.44it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:13, 3.48it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:14, 3.30it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:13, 3.44it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:13, 3.53it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:14, 3.30it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:13, 3.48it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:13, 3.44it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:13, 3.53it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:13, 3.48it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:13, 3.53it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:23, 2.16it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:23, 2.20it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:23, 2.16it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:24, 2.06it/s, Materializing param=model.layers.35.mlp.up_proj.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:23, 2.20it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:24, 2.06it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:23, 2.16it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:23, 2.20it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 88%|████████▊ | 392/443 [03:10<00:24, 2.06it/s, Materializing param=model.layers.35.post_attention_layernorm.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:23, 2.16it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:24, 2.06it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:23, 2.16it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:24, 2.06it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:22, 2.20it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▊ | 393/443 [03:10<00:22, 2.20it/s, Materializing param=model.layers.35.self_attn.k_norm.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:22, 2.16it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:23, 2.06it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:23, 2.06it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:22, 2.16it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:22, 2.20it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 394/443 [03:10<00:22, 2.20it/s, Materializing param=model.layers.35.self_attn.k_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:22, 2.18it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:22, 2.18it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:22, 2.16it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:22, 2.16it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:21, 2.20it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:23, 2.06it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:21, 2.20it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 395/443 [03:10<00:23, 2.06it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.39it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.39it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:14, 3.30it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.39it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.39it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.44it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.53it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.39it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.39it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:14, 3.30it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.53it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.44it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:12, 3.68it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:12, 3.61it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:13, 3.39it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:13, 3.39it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:13, 3.46it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:12, 3.61it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:12, 3.68it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:12, 3.61it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:13, 3.46it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:12, 3.68it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:13, 3.46it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:13, 3.39it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:13, 3.39it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:14, 3.29it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:14, 3.29it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:14, 3.29it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:13, 3.29it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:13, 3.29it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:13, 3.29it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:13, 3.29it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.48it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.48it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:12, 3.58it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:12, 3.58it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:12, 3.58it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:12, 3.58it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:12, 3.58it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.36it/s, Materializing param=model.layers.35.self_attn.o_proj.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.36it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 89%|████████▉ | 396/443 [03:11<00:13, 3.36it/s, Materializing param=model.layers.35.self_attn.q_norm.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:13, 3.36it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 397/443 [03:11<00:13, 3.36it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:13, 3.36it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:13, 3.36it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:12, 3.61it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:13, 3.39it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:12, 3.68it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:12, 3.61it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:13, 3.39it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:12, 3.68it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:11, 3.82it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:11, 3.82it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:11, 3.82it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:09, 4.70it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:09, 4.74it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:12, 3.58it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:09, 4.79it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:09, 4.70it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:13, 3.29it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:09, 4.74it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:12, 3.58it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:09, 4.70it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:09, 4.74it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:09, 4.79it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:13, 3.29it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:09, 4.79it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:11, 3.82it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:11, 3.82it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 400/443 [03:11<00:13, 3.29it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:11<00:08, 4.79it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:11<00:08, 4.79it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:11<00:13, 3.29it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:13, 3.36it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 400/443 [03:11<00:09, 4.70it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:11<00:11, 3.82it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:11<00:09, 4.70it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:11<00:11, 3.82it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:13, 3.36it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 400/443 [03:11<00:12, 3.36it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:11<00:12, 3.36it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:10, 4.20it/s, Materializing param=model.layers.35.self_attn.q_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:10, 4.20it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|████████▉ | 398/443 [03:11<00:10, 4.20it/s, Materializing param=model.layers.35.self_attn.v_proj.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:10, 4.20it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 90%|█████████ | 399/443 [03:11<00:10, 4.20it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:14, 2.89it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:16, 2.55it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:18, 2.30it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:14, 2.85it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:17, 2.43it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:16, 2.55it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:14, 2.85it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:18, 2.30it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:13<00:09, 4.74it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:17, 2.43it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:16, 2.55it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:14, 2.85it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:18, 2.30it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:13<00:09, 4.74it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:17, 2.43it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:14, 2.89it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.input_layernorm.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:14, 2.89it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 90%|█████████ | 400/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.mlp.down_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 401/443 [03:13<00:18, 2.33it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:24, 1.65it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:21, 1.89it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:22, 1.79it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:27, 1.49it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:21, 1.92it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:23, 1.77it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:25, 1.58it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:27, 1.47it/s, Materializing param=model.layers.36.mlp.gate_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:21, 1.89it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:22, 1.79it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:25, 1.58it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:21, 1.89it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:23, 1.77it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:27, 1.47it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:22, 1.79it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:27, 1.49it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:25, 1.58it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:23, 1.77it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:27, 1.47it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:27, 1.49it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:21, 1.92it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:24, 1.65it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:24, 1.65it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 402/443 [03:15<00:21, 1.92it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:36, 1.09it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:39, 1.02it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:30, 1.30it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:39, 1.02it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:36, 1.09it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:30, 1.30it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:39, 1.02it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:36, 1.09it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:30, 1.30it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:33, 1.20it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:33, 1.20it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:33, 1.20it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:30, 1.30it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:38, 1.02it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:38, 1.02it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:30, 1.30it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:30, 1.31it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:32, 1.20it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:32, 1.20it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:38, 1.03it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:35, 1.13it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:37, 1.02it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:38, 1.03it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:32, 1.22it/s, Materializing param=model.layers.36.mlp.up_proj.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:31, 1.20it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:37, 1.02it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:35, 1.13it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:38, 1.03it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:31, 1.20it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:35, 1.13it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:32, 1.22it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:30, 1.31it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:32, 1.22it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:37, 1.03it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:34, 1.13it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:32, 1.22it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:37, 1.03it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:34, 1.13it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:32, 1.22it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:36, 1.03it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████ | 403/443 [03:17<00:30, 1.31it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:36, 1.03it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:31, 1.22it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:33, 1.13it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:31, 1.22it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:33, 1.13it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:29, 1.31it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:29, 1.31it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:28, 1.31it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:28, 1.31it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:21, 1.78it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:29, 1.33it/s, Materializing param=model.layers.36.post_attention_layernorm.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:21, 1.78it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:29, 1.33it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:18, 2.05it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████ | 404/443 [03:17<00:29, 1.33it/s, Materializing param=model.layers.36.self_attn.k_norm.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:20, 1.83it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:18, 1.98it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:19, 1.91it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:18, 1.99it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:21, 1.78it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:18, 1.98it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:20, 1.83it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:19, 1.91it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:18, 1.99it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:28, 1.33it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:20, 1.83it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:19, 1.91it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:18, 1.99it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:18, 1.98it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 91%|█████████▏| 405/443 [03:17<00:28, 1.33it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:20, 1.78it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:18, 2.05it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:20, 1.78it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:27, 1.33it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:27, 1.33it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:18, 2.05it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:20, 1.83it/s, Materializing param=model.layers.36.self_attn.k_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:20, 1.83it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 406/443 [03:17<00:20, 1.83it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:18, 1.95it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:18, 1.99it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:18, 1.99it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:18, 1.99it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:18, 1.95it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:19, 1.89it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:18, 1.91it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:19, 1.83it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:19, 1.89it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:19, 1.88it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:19, 1.83it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:18, 1.95it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:19, 1.89it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:17, 2.03it/s, Materializing param=model.layers.36.self_attn.o_proj.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:19, 1.88it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:18, 1.91it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:19, 1.88it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:17, 2.03it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:17, 1.99it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:17, 2.03it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:17, 1.99it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:16, 2.14it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:18, 1.88it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:17, 2.03it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:16, 2.14it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:18, 1.88it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:17, 2.03it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:16, 2.14it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:16, 2.08it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:16, 2.08it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:16, 2.08it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:18, 1.89it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:18, 1.89it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:19, 1.83it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 407/443 [03:18<00:19, 1.83it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:16, 2.08it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:16, 2.08it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:16, 2.08it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:16, 2.08it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:16, 2.08it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:15, 2.08it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:15, 2.08it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:15, 2.08it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:15, 2.08it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:15, 2.19it/s, Materializing param=model.layers.36.self_attn.q_norm.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:12, 2.65it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:12, 2.65it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:12, 2.65it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:15, 2.19it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 408/443 [03:18<00:15, 2.19it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:15, 2.19it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:15, 2.19it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:14, 2.38it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:14, 2.38it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:14, 2.38it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:13, 2.53it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:14, 2.31it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:13, 2.53it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:14, 2.31it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:13, 2.53it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:14, 2.31it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:13, 2.38it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:13, 2.38it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:13, 2.38it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:13, 2.38it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:13, 2.52it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:12, 2.63it/s, Materializing param=model.layers.36.self_attn.q_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:13, 2.52it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:12, 2.63it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:13, 2.52it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 92%|█████████▏| 409/443 [03:18<00:12, 2.63it/s, Materializing param=model.layers.36.self_attn.v_proj.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:15, 2.19it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:12, 2.65it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:12, 2.65it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:15, 2.19it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:14, 2.19it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:12, 2.65it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:12, 2.65it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:14, 2.19it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:14, 2.31it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:14, 2.31it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:13, 2.31it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:13, 2.31it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:13, 2.52it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:13, 2.52it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:13, 2.53it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:13, 2.53it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:12, 2.53it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:12, 2.53it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:12, 2.63it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 410/443 [03:18<00:12, 2.63it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:12, 2.63it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 411/443 [03:18<00:12, 2.63it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 411/443 [03:19<00:17, 1.88it/s, Materializing param=model.layers.37.input_layernorm.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:14, 2.21it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 411/443 [03:19<00:17, 1.88it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 411/443 [03:19<00:17, 1.88it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:14, 2.08it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:13, 2.23it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:15, 2.05it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:14, 2.10it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:13, 2.28it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:14, 2.17it/s, Materializing param=model.layers.37.mlp.down_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:14, 2.21it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:14, 2.21it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:13, 2.23it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:15, 2.05it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:14, 2.08it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:14, 2.17it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:14, 2.10it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:15, 2.05it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:14, 2.10it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:13, 2.23it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:14, 2.17it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:14, 2.08it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:16, 1.88it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:16, 1.88it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:13, 2.28it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 412/443 [03:19<00:13, 2.28it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:22, 1.34it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:22, 1.35it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:20, 1.44it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:20, 1.47it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:21, 1.43it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:21, 1.40it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:20, 1.47it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:20, 1.44it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:20, 1.44it/s, Materializing param=model.layers.37.mlp.gate_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:21, 1.40it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:22, 1.34it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:21, 1.43it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:22, 1.35it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:20, 1.47it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:20, 1.44it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:22, 1.34it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:21, 1.43it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:21, 1.40it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:20, 1.47it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:22, 1.35it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:20, 1.44it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:20, 1.44it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:21<00:20, 1.43it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 93%|█████████▎| 414/443 [03:21<00:20, 1.43it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 94%|█████████▎| 415/443 [03:21<00:19, 1.43it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▎| 415/443 [03:21<00:19, 1.43it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:20, 1.47it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 94%|█████████▍| 416/443 [03:21<00:18, 1.43it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 416/443 [03:21<00:18, 1.43it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 93%|█████████▎| 413/443 [03:21<00:20, 1.47it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:09, 2.62it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:09, 2.62it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:09, 2.62it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:19, 1.49it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:19, 1.49it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:19, 1.49it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:18, 1.49it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:18, 1.49it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:18, 1.49it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:18, 1.49it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:19, 1.50it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:19, 1.50it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:19, 1.50it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:18, 1.50it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:18, 1.50it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:18, 1.50it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:18, 1.50it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:17, 1.49it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:17, 1.49it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:17, 1.50it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:17, 1.50it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:20, 1.45it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:20, 1.45it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:20, 1.45it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:19, 1.45it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:19, 1.45it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:18, 1.45it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:18, 1.45it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:17, 1.45it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:17, 1.45it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.62it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.62it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.85it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.85it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.85it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:21, 1.36it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:21, 1.36it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:21, 1.36it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:20, 1.36it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:20, 1.36it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:19, 1.45it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:19, 1.45it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:19, 1.45it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:19, 1.45it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:19, 1.45it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.76it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.76it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.76it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.76it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.76it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:08, 2.87it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:08, 2.87it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:08, 2.87it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.87it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.87it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:21, 1.34it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:21, 1.34it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:21, 1.34it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:20, 1.34it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:20, 1.34it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:20, 1.34it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:20, 1.34it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:18, 1.45it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:18, 1.45it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:17, 1.45it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:17, 1.45it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:20, 1.44it/s, Materializing param=model.layers.37.mlp.up_proj.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:20, 1.44it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 93%|█████████▎| 414/443 [03:22<00:20, 1.44it/s, Materializing param=model.layers.37.post_attention_layernorm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:19, 1.44it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▎| 415/443 [03:22<00:19, 1.44it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:19, 1.36it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:19, 1.36it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:19, 1.36it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:19, 1.36it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:08, 2.81it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:08, 2.81it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:08, 2.81it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:19, 1.34it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:19, 1.34it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:13, 2.07it/s, Materializing param=model.layers.37.self_attn.k_norm.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:13, 2.07it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 416/443 [03:22<00:13, 2.07it/s, Materializing param=model.layers.37.self_attn.k_proj.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.68it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.68it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.68it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.68it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.68it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.68it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.68it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.77it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.68it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.77it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:09, 2.77it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.68it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.77it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.68it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:08, 2.77it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:12, 2.07it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 417/443 [03:22<00:12, 2.07it/s, Materializing param=model.layers.37.self_attn.o_proj.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:12, 2.07it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 94%|█████████▍| 418/443 [03:22<00:12, 2.07it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:11, 2.07it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 419/443 [03:22<00:11, 2.07it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:07, 3.19it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.32it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.75it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:08, 2.76it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.32it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:07, 3.19it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:08, 2.76it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.32it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:07, 3.19it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:08, 2.76it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.75it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 419/443 [03:23<00:08, 2.88it/s, Materializing param=model.layers.37.self_attn.q_norm.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.75it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 419/443 [03:23<00:08, 2.88it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 419/443 [03:23<00:08, 2.88it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:07, 2.88it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:07, 2.88it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.48it/s, Materializing param=model.layers.37.self_attn.q_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.48it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▍| 420/443 [03:23<00:06, 3.48it/s, Materializing param=model.layers.37.self_attn.v_proj.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:06, 3.48it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:06, 3.48it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:06, 3.32it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:06, 3.32it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:06, 3.32it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:06, 3.32it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:06, 3.19it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:06, 3.19it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:07, 2.76it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:07, 2.76it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:06, 3.19it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:07, 2.76it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:06, 3.19it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:07, 2.88it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:07, 2.76it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:05, 3.75it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:07, 2.88it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:07, 2.88it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:07, 2.88it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:05, 3.75it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:05, 3.75it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:05, 3.75it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 421/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 422/443 [03:23<00:06, 3.40it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.42it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.49it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.40it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.42it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 422/443 [03:24<00:09, 2.16it/s, Materializing param=model.layers.38.input_layernorm.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:07, 2.58it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.37it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.49it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.42it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.42it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.49it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:09, 2.16it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 422/443 [03:24<00:09, 2.16it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.40it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.37it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 422/443 [03:24<00:09, 2.16it/s, Materializing param=model.layers.38.mlp.down_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.40it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.42it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:09, 2.16it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.37it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:08, 2.42it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:09, 2.16it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:09, 2.16it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:09, 2.16it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:07, 2.58it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 95%|█████████▌| 423/443 [03:24<00:07, 2.58it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:11, 1.59it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:13, 1.38it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.50it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.55it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.50it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:11, 1.59it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.55it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:13, 1.38it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.50it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.50it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:11, 1.59it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.55it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:13, 1.38it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:12, 1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:11, 1.59it/s, Materializing param=model.layers.38.mlp.gate_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:11, 1.59it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 424/443 [03:27<00:11, 1.59it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.59it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.59it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.59it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.59it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:10, 1.59it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:10, 1.59it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:09, 1.59it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:09, 1.59it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 2.92it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 2.92it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 2.92it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:04, 2.92it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:04, 2.92it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:10, 1.64it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:10, 1.64it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:10, 1.64it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.64it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.64it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:09, 1.64it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:09, 1.64it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:09, 1.64it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:09, 1.64it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:03, 3.49it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:03, 3.49it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:03, 3.49it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.58it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.58it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.58it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.62it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.62it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.58it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.58it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.62it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.58it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.58it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.58it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.58it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.62it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.58it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.62it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:11, 1.58it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:12, 1.48it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.58it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.58it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:12, 1.48it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.58it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:12, 1.48it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.58it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:10, 1.58it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:10, 1.58it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:10, 1.58it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:11, 1.48it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:10, 1.58it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:10, 1.58it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:11, 1.48it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:10, 1.58it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:10, 1.48it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:10, 1.48it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 3.13it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 3.13it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 3.13it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:10, 1.66it/s, Materializing param=model.layers.38.mlp.up_proj.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:10, 1.66it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 425/443 [03:27<00:10, 1.66it/s, Materializing param=model.layers.38.post_attention_layernorm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.66it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▌| 426/443 [03:27<00:10, 1.66it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:09, 1.66it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:09, 1.66it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:10, 1.48it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:10, 1.48it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:03, 3.49it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:03, 3.49it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:09, 1.66it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:09, 1.66it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:09, 1.58it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:09, 1.58it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:09, 1.58it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:09, 1.58it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:04, 3.13it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:04, 3.13it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:03, 3.91it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:03, 3.91it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:03, 3.91it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 3.03it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 3.03it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 3.03it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:02, 3.91it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:02, 3.91it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:02, 3.91it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:02, 3.91it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:07, 2.22it/s, Materializing param=model.layers.38.self_attn.k_norm.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:07, 2.22it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 96%|█████████▋| 427/443 [03:27<00:07, 2.22it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:06, 2.22it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:06, 2.22it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:06, 2.22it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:06, 2.22it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:05, 2.22it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:05, 2.22it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 2.88it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:05, 2.79it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:05, 2.79it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 2.88it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:05, 2.79it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 2.88it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:04, 2.88it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:04, 2.88it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:04, 2.79it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:04, 2.79it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:05, 2.55it/s, Materializing param=model.layers.38.self_attn.k_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:05, 2.55it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 428/443 [03:27<00:05, 2.55it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:05, 2.55it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:05, 2.55it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:05, 2.55it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:05, 2.55it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 2.90it/s, Materializing param=model.layers.38.self_attn.o_proj.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 2.90it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 429/443 [03:27<00:04, 2.90it/s, Materializing param=model.layers.38.self_attn.q_norm.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:04, 3.03it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:04, 3.03it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:03, 3.88it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:03, 3.88it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:03, 3.88it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:05, 2.22it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:05, 2.22it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:04, 2.88it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:04, 2.88it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:02, 3.88it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:02, 3.88it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:02, 3.88it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:02, 3.88it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:04, 2.55it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:04, 2.55it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:04, 2.79it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:04, 2.79it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:04, 2.90it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 430/443 [03:27<00:04, 2.90it/s, Materializing param=model.layers.38.self_attn.q_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:04, 2.90it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 97%|█████████▋| 431/443 [03:27<00:04, 2.90it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:04, 2.22it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:04, 2.22it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:04, 2.22it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:04, 2.22it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:03, 2.88it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:03, 2.88it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:03, 2.88it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:03, 2.88it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:04, 2.55it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:04, 2.55it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:03, 2.55it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:03, 2.55it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:02, 4.12it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:02, 4.12it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:02, 4.12it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:02, 4.12it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:02, 4.12it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:02, 4.23it/s, Materializing param=model.layers.38.self_attn.v_proj.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:02, 4.23it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 432/443 [03:27<00:02, 4.23it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:02, 4.23it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 433/443 [03:27<00:02, 4.23it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.45it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.81it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.45it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.90it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.55it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.40it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.45it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.45it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.81it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.72it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.90it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.55it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.81it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.45it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.40it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.45it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.90it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.55it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.40it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.72it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:03, 2.72it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 433/443 [03:29<00:04, 2.08it/s, Materializing param=model.layers.39.input_layernorm.weight] Loading weights: 98%|█████████▊| 435/443 [03:29<00:03, 2.55it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:29<00:03, 2.55it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 433/443 [03:29<00:04, 2.08it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 433/443 [03:29<00:04, 2.08it/s, Materializing param=model.layers.39.mlp.down_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:04, 2.08it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 434/443 [03:29<00:04, 2.08it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.57it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.57it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.57it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.33it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.33it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.33it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:02, 2.71it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:02, 2.71it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:02, 2.71it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.63it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.63it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.63it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.45it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.45it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.31it/s, Materializing param=model.layers.39.mlp.gate_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.31it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.31it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:02, 2.74it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:02, 2.74it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:02, 2.74it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.74it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.74it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:01, 2.74it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:01, 2.74it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.40it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 435/443 [03:30<00:03, 2.40it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 3.97it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 3.97it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 3.97it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:02, 2.55it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:02, 2.55it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:02, 2.55it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.55it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.55it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:01, 2.55it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:01, 2.55it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:03, 2.24it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:03, 2.24it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:03, 2.24it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.24it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.24it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:02, 2.24it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:02, 2.24it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.55it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.55it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.24it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.24it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:02, 2.37it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:02, 2.37it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:03, 2.19it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:02, 2.37it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.37it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.37it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:03, 2.19it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:02, 2.37it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:02, 2.37it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:03, 2.19it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.19it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.19it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:02, 2.19it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:02, 2.19it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:02, 2.42it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:02, 2.42it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:02, 2.42it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.42it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.42it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:02, 2.42it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:02, 2.42it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.19it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:03, 2.19it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:03, 2.19it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:03, 2.19it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.19it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.19it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.19it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:02, 2.19it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:02, 2.19it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:03, 2.16it/s, Materializing param=model.layers.39.mlp.up_proj.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:03, 2.16it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 98%|█████████▊| 436/443 [03:30<00:03, 2.16it/s, Materializing param=model.layers.39.post_attention_layernorm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.16it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▊| 437/443 [03:30<00:02, 2.16it/s, Materializing param=model.layers.39.self_attn.k_norm.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:02, 2.16it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 438/443 [03:30<00:02, 2.16it/s, Materializing param=model.layers.39.self_attn.k_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.37it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.37it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.42it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.42it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.19it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.19it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.16it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 439/443 [03:30<00:01, 2.16it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 3.97it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 4.09it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 3.97it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 4.23it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 4.23it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 4.09it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 3.89it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 4.23it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 4.09it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 3.89it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 3.89it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 4.09it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 4.09it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 3.92it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 4.11it/s, Materializing param=model.layers.39.self_attn.o_proj.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 4.11it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 4.11it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 3.92it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 4.11it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 4.11it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:00, 3.92it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 3.92it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 3.92it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:01, 2.42it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:01, 2.42it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 4.65it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 4.65it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 4.65it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:01, 2.19it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:01, 2.19it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:01, 2.16it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 99%|█████████▉| 440/443 [03:30<00:01, 2.16it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 2.16it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 441/443 [03:30<00:00, 2.16it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.83it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 441/443 [03:31<00:00, 3.54it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 100%|█████████▉| 441/443 [03:31<00:00, 3.54it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.27it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 441/443 [03:31<00:00, 3.54it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.91it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 4.65it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.27it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 441/443 [03:31<00:00, 3.36it/s, Materializing param=model.layers.39.self_attn.q_norm.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.97it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.66it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.27it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.91it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 4.65it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 441/443 [03:31<00:00, 3.36it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.54it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.66it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.91it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.97it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.83it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.54it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 441/443 [03:31<00:00, 3.36it/s, Materializing param=model.layers.39.self_attn.q_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.66it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.97it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.83it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.36it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|█████████▉| 442/443 [03:31<00:00, 3.36it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 3.83it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 4.04it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 4.51it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 4.11it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 3.97it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 4.04it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 3.64it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 4.51it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 4.24it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 3.88it/s, Materializing param=model.layers.39.self_attn.v_proj.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 4.04it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 4.11it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 3.97it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 4.51it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 3.64it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 4.11it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 3.88it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 3.83it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 3.64it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 4.24it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 3.88it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 4.24it/s, Materializing param=model.norm.weight] Loading weights: 100%|██████████| 443/443 [03:31<00:00, 2.09it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00, 2.09it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00, 2.09it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00, 2.09it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00, 2.09it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00, 2.09it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00, 2.09it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00, 2.09it/s, Materializing param=model.norm.weight]
Downloading builder script: 0.00B [00:00, ?B/s] Downloading builder script: 9.18kB [00:00, 27.9MB/s]
Downloading extra modules: 0.00B [00:00, ?B/s] Downloading extra modules: 6.10kB [00:00, 31.6MB/s]
Generating test split: 0%| | 0/164 [00:00<?, ? examples/s] Generating test split: 100%|██████████| 164/164 [00:00<00:00, 36447.09 examples/s]
Generating test split: 0%| | 0/164 [00:00<?, ? examples/s] Generating test split: 100%|██████████| 164/164 [00:00<00:00, 3259.33 examples/s]
Generating train split: 0%| | 0/374 [00:00<?, ? examples/s] Generating train split: 100%|██████████| 374/374 [00:00<00:00, 55912.09 examples/s]
Generating test split: 0%| | 0/500 [00:00<?, ? examples/s] Generating test split: 100%|██████████| 500/500 [00:00<00:00, 170264.84 examples/s]
Generating validation split: 0%| | 0/90 [00:00<?, ? examples/s] Generating validation split: 100%|██████████| 90/90 [00:00<00:00, 47554.47 examples/s]
Generating prompt split: 0%| | 0/10 [00:00<?, ? examples/s] Generating prompt split: 100%|██████████| 10/10 [00:00<00:00, 5596.14 examples/s]
Generating test split: 0%| | 0/378 [00:00<?, ? examples/s] Generating test split: 100%|██████████| 378/378 [00:00<00:00, 18723.24 examples/s]
2026-04-30:10:58:47 INFO [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:47 INFO [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:47 INFO [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:47 INFO [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:47 INFO [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:47 INFO [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:47 INFO [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:47 INFO [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:47 INFO [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:47 INFO [api.task:312] Building contexts for humaneval on rank 5...
0%| | 0/20 [00:00<?, ?it/s] 100%|██████████| 20/20 [00:00<00:00, 1323.69it/s]
2026-04-30:10:58:50 INFO [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:50 INFO [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:50 INFO [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:50 INFO [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:50 INFO [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:50 INFO [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:50 INFO [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:50 INFO [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:50 INFO [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:50 INFO [api.task:312] Building contexts for humaneval on rank 0...
0%| | 0/21 [00:00<?, ?it/s] 100%|██████████| 21/21 [00:00<00:00, 1198.91it/s]
ywang29-p4d-debug-2-worker-0:39120:39120 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39120:39120 [0] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39120:39120 [0] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39125:39125 [5] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39120:39120 [0] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39125:39125 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39125:39125 [5] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39125:39125 [5] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39120:39120 [0] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-2-worker-0:39125:39125 [5] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)
[2026-04-30 10:58:50] ywang29-p4d-debug-2-worker-0:39120:40089 [0] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol
[2026-04-30 10:58:50] ywang29-p4d-debug-2-worker-0:39120:40089 [0] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed
[2026-04-30 10:58:50] ywang29-p4d-debug-2-worker-0:39120:40089 [0] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
[2026-04-30 10:58:50] ywang29-p4d-debug-2-worker-0:39125:40091 [5] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol
[2026-04-30 10:58:50] ywang29-p4d-debug-2-worker-0:39125:40091 [5] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed
[2026-04-30 10:58:50] ywang29-p4d-debug-2-worker-0:39125:40091 [5] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO ncclCommInitRankConfig comm 0x55cf3325aa10 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO ncclCommInitRankConfig comm 0x5584194aa180 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 commId 0xdd4996e97ba7d4db - Init START
2026-04-30:10:58:51 INFO [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:51 INFO [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO [api.task:312] Building contexts for humaneval on rank 7...
0%| | 0/20 [00:00<?, ?it/s] 100%|██████████| 20/20 [00:00<00:00, 1294.30it/s]
ywang29-p4d-debug-2-worker-0:39127:39127 [7] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39127:39127 [7] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39127:39127 [7] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39127:39127 [7] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39127:39127 [7] NCCL INFO Comm config Blocking set to 1
2026-04-30:10:58:51 INFO [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:51 INFO [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO [api.task:312] Building contexts for humaneval on rank 3...
0%| | 0/21 [00:00<?, ?it/s] 100%|██████████| 21/21 [00:00<00:00, 1287.93it/s]
ywang29-p4d-debug-2-worker-0:39123:39123 [3] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39123:39123 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39123:39123 [3] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39123:39123 [3] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39123:39123 [3] NCCL INFO Comm config Blocking set to 1
2026-04-30:10:58:51 INFO [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:51 INFO [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO [api.task:312] Building contexts for humaneval on rank 4...
0%| | 0/20 [00:00<?, ?it/s] 100%|██████████| 20/20 [00:00<00:00, 1297.48it/s]
ywang29-p4d-debug-2-worker-0:39124:39124 [4] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39124:39124 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39124:39124 [4] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39124:39124 [4] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39124:39124 [4] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39127:40099 [7] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39127:40099 [7] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39127:40099 [7] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO ncclCommInitRankConfig comm 0x5570936d9f40 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39123:40101 [3] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39123:40101 [3] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39123:40101 [3] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO ncclCommInitRankConfig comm 0x55ea64d13050 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39124:40105 [4] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39124:40105 [4] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39124:40105 [4] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO ncclCommInitRankConfig comm 0x557c784ec2c0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO RAS client listening socket at ::1<28028>
2026-04-30:10:58:51 INFO [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:51 INFO [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO [api.task:312] Building contexts for humaneval on rank 2...
0%| | 0/21 [00:00<?, ?it/s] 100%|██████████| 21/21 [00:00<00:00, 1314.48it/s]
ywang29-p4d-debug-2-worker-0:39122:39122 [2] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39122:39122 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39122:39122 [2] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39122:39122 [2] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39122:39122 [2] NCCL INFO Comm config Blocking set to 1
2026-04-30:10:58:51 INFO [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:51 INFO [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO [api.task:312] Building contexts for humaneval on rank 1...
0%| | 0/21 [00:00<?, ?it/s] 100%|██████████| 21/21 [00:00<00:00, 1289.42it/s]
ywang29-p4d-debug-2-worker-0:39121:39121 [1] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39121:39121 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39121:39121 [1] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39121:39121 [1] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39121:39121 [1] NCCL INFO Comm config Blocking set to 1
2026-04-30:10:58:51 INFO [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:51 INFO [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:51 INFO [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO [api.task:312] Building contexts for humaneval on rank 6...
0%| | 0/20 [00:00<?, ?it/s]ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)
100%|██████████| 20/20 [00:00<00:00, 1191.07it/s]
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39122:40113 [2] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39122:40113 [2] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39122:40113 [2] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39126:39126 [6] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39126:39126 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39126:39126 [6] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39126:39126 [6] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39126:39126 [6] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO ncclCommInitRankConfig comm 0x556db8b8dd00 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39121:40117 [1] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39121:40117 [1] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39121:40117 [1] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO ncclCommInitRankConfig comm 0x556f2a64bf80 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39126:40121 [6] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39126:40121 [6] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed
[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39126:40121 [6] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO ncclCommInitRankConfig comm 0x559361b695d0 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Bootstrap timings total 0.000963 (create 0.000041, send 0.000076, recv 0.000220, ring 0.000207, delay 0.000002)
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Bootstrap timings total 0.596544 (create 0.000045, send 0.000095, recv 0.000129, ring 0.000120, delay 0.000001)
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Bootstrap timings total 0.821776 (create 0.000040, send 0.000156, recv 0.820908, ring 0.000117, delay 0.000001)
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Bootstrap timings total 0.181152 (create 0.000046, send 0.000079, recv 0.000191, ring 0.050217, delay 0.000001)
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Bootstrap timings total 0.459399 (create 0.000043, send 0.000077, recv 0.000260, ring 0.458590, delay 0.000001)
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Bootstrap timings total 0.821808 (create 0.000040, send 0.000179, recv 0.770821, ring 0.050200, delay 0.000002)
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Bootstrap timings total 0.570657 (create 0.000038, send 0.000069, recv 0.111426, ring 0.180242, delay 0.000001)
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Bootstrap timings total 0.051126 (create 0.000047, send 0.000088, recv 0.000246, ring 0.050217, delay 0.000002)
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Setting affinity for GPU 3 to 0-23,48-71
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NVLS multicast support is not available on dev 3 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Setting affinity for GPU 2 to 0-23,48-71
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NVLS multicast support is not available on dev 2 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Setting affinity for GPU 4 to 24-47,72-95
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NVLS multicast support is not available on dev 4 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Setting affinity for GPU 1 to 0-23,48-71
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NVLS multicast support is not available on dev 1 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Setting affinity for GPU 5 to 24-47,72-95
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NVLS multicast support is not available on dev 5 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Setting affinity for GPU 0 to 0-23,48-71
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NVLS multicast support is not available on dev 0 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Setting affinity for GPU 6 to 24-47,72-95
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NVLS multicast support is not available on dev 6 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Setting affinity for GPU 7 to 24-47,72-95
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NVLS multicast support is not available on dev 7 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO comm 0x556db8b8dd00 rank 2 nRanks 8 nNodes 1 localRanks 8 localRank 2 MNNVL 0
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO comm 0x559361b695d0 rank 6 nRanks 8 nNodes 1 localRanks 8 localRank 6 MNNVL 0
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO comm 0x556f2a64bf80 rank 1 nRanks 8 nNodes 1 localRanks 8 localRank 1 MNNVL 0
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO comm 0x5570936d9f40 rank 7 nRanks 8 nNodes 1 localRanks 8 localRank 7 MNNVL 0
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO comm 0x5584194aa180 rank 5 nRanks 8 nNodes 1 localRanks 8 localRank 5 MNNVL 0
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO comm 0x55cf3325aa10 rank 0 nRanks 8 nNodes 1 localRanks 8 localRank 0 MNNVL 0
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO comm 0x55ea64d13050 rank 3 nRanks 8 nNodes 1 localRanks 8 localRank 3 MNNVL 0
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO comm 0x557c784ec2c0 rank 4 nRanks 8 nNodes 1 localRanks 8 localRank 4 MNNVL 0
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 00/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 01/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 02/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 03/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 04/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 05/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 06/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 07/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 08/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 09/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 10/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 11/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 12/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 13/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 14/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 15/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 16/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 17/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 18/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 19/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 20/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 21/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 22/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 23/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so.
ywang29-p4d-debug-2-worker-0:39126:40130 [6] NCCL INFO [Proxy Service UDS] Device 6 CPU core 26
ywang29-p4d-debug-2-worker-0:39126:40129 [6] NCCL INFO [Proxy Service] Device 6 CPU core 90
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so.
ywang29-p4d-debug-2-worker-0:39124:40132 [4] NCCL INFO [Proxy Service UDS] Device 4 CPU core 95
ywang29-p4d-debug-2-worker-0:39124:40131 [4] NCCL INFO [Proxy Service] Device 4 CPU core 37
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0
ywang29-p4d-debug-2-worker-0:39125:40135 [5] NCCL INFO [Proxy Service UDS] Device 5 CPU core 88
ywang29-p4d-debug-2-worker-0:39125:40133 [5] NCCL INFO [Proxy Service] Device 5 CPU core 91
ywang29-p4d-debug-2-worker-0:39122:40134 [2] NCCL INFO [Proxy Service] Device 2 CPU core 6
ywang29-p4d-debug-2-worker-0:39122:40136 [2] NCCL INFO [Proxy Service UDS] Device 2 CPU core 4
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Proxy Service] Device 0 CPU core 68
ywang29-p4d-debug-2-worker-0:39120:40138 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 12
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so.
ywang29-p4d-debug-2-worker-0:39127:40142 [7] NCCL INFO [Proxy Service UDS] Device 7 CPU core 36
ywang29-p4d-debug-2-worker-0:39121:40141 [1] NCCL INFO [Proxy Service UDS] Device 1 CPU core 10
ywang29-p4d-debug-2-worker-0:39121:40140 [1] NCCL INFO [Proxy Service] Device 1 CPU core 18
ywang29-p4d-debug-2-worker-0:39127:40139 [7] NCCL INFO [Proxy Service] Device 7 CPU core 25
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so.
ywang29-p4d-debug-2-worker-0:39123:40143 [3] NCCL INFO [Proxy Service] Device 3 CPU core 5
ywang29-p4d-debug-2-worker-0:39123:40144 [3] NCCL INFO [Proxy Service UDS] Device 3 CPU core 3
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO CC Off, workFifoBytes 1048576
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO ncclCommInitRankConfig comm 0x5584194aa180 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO ncclCommInitRankConfig comm 0x559361b695d0 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO ncclCommInitRankConfig comm 0x557c784ec2c0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Init timings - ncclCommInitRankConfig: rank 6 nranks 8 total 0.35 (kernels 0.17, alloc 0.02, bootstrap 0.00, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.06, rest 0.02)
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Init timings - ncclCommInitRankConfig: rank 4 nranks 8 total 0.81 (kernels 0.17, alloc 0.02, bootstrap 0.46, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.07, rest 0.02)
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Init timings - ncclCommInitRankConfig: rank 5 nranks 8 total 1.20 (kernels 0.18, alloc 0.04, bootstrap 0.82, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.06, rest 0.03)
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO ncclCommInitRankConfig comm 0x556f2a64bf80 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Init timings - ncclCommInitRankConfig: rank 1 nranks 8 total 0.40 (kernels 0.17, alloc 0.02, bootstrap 0.05, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.06, rest 0.02)
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO ncclCommInitRankConfig comm 0x5570936d9f40 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO ncclCommInitRankConfig comm 0x55ea64d13050 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Init timings - ncclCommInitRankConfig: rank 7 nranks 8 total 0.94 (kernels 0.17, alloc 0.02, bootstrap 0.60, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.06, rest 0.03)
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Init timings - ncclCommInitRankConfig: rank 3 nranks 8 total 0.92 (kernels 0.17, alloc 0.02, bootstrap 0.57, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.05, rest 0.03)
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO ncclCommInitRankConfig comm 0x556db8b8dd00 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Init timings - ncclCommInitRankConfig: rank 2 nranks 8 total 0.54 (kernels 0.17, alloc 0.03, bootstrap 0.18, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.06, rest 0.03)
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO ncclCommInitRankConfig comm 0x55cf3325aa10 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Init timings - ncclCommInitRankConfig: rank 0 nranks 8 total 1.20 (kernels 0.18, alloc 0.05, bootstrap 0.82, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.06, rest 0.03)
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 00/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 01/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 02/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 03/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 04/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 05/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 00/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 06/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 07/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 08/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 01/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 02/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 09/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 03/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 10/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 04/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 11/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 05/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 12/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 13/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 06/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 07/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 14/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 15/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 16/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 08/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 17/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 16/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 18/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 09/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 17/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 10/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 18/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 16/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 19/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 19/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 11/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 12/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 20/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 20/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 17/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 16/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 21/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 18/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 21/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 13/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 14/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 22/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 22/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 19/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 17/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 15/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 20/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 23/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 23/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 21/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 16/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 17/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 18/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 22/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 19/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 18/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 23/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 16/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 19/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 20/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 17/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 20/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 21/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 21/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 18/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 22/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 16/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 22/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 17/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 23/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 16/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 19/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 23/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 18/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 17/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 20/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 19/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 18/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 21/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 20/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 19/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 22/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 21/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 20/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 22/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 23/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 21/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 22/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 23/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 23/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for humaneval_plus on rank 0...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for humaneval_plus on rank 5...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for humaneval_plus on rank 3...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for humaneval_plus on rank 6...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for humaneval_plus on rank 2...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for humaneval_plus on rank 1...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for humaneval_plus on rank 4...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for humaneval_plus on rank 7...
0%| | 0/21 [00:00<?, ?it/s] 0%| | 0/20 [00:00<?, ?it/s] 0%| | 0/20 [00:00<?, ?it/s] 0%| | 0/20 [00:00<?, ?it/s] 0%| | 0/21 [00:00<?, ?it/s] 0%| | 0/21 [00:00<?, ?it/s] 0%| | 0/20 [00:00<?, ?it/s] 0%| | 0/21 [00:00<?, ?it/s] 100%|██████████| 20/20 [00:00<00:00, 1295.60it/s]
100%|██████████| 20/20 [00:00<00:00, 1282.50it/s]
100%|██████████| 20/20 [00:00<00:00, 1278.17it/s]
100%|██████████| 20/20 [00:00<00:00, 1301.55it/s]
100%|██████████| 21/21 [00:00<00:00, 1291.46it/s]
100%|██████████| 21/21 [00:00<00:00, 1267.62it/s]
100%|██████████| 21/21 [00:00<00:00, 1240.32it/s] 100%|██████████| 21/21 [00:00<00:00, 1264.45it/s]
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for mbpp on rank 4...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for mbpp on rank 5...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for mbpp on rank 3...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for mbpp on rank 7...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for mbpp on rank 1...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for mbpp on rank 2...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for mbpp on rank 6...
2026-04-30:10:58:52 INFO [api.task:312] Building contexts for mbpp on rank 0...
0%| | 0/63 [00:00<?, ?it/s] 0%| | 0/63 [00:00<?, ?it/s] 0%| | 0/63 [00:00<?, ?it/s] 0%| | 0/63 [00:00<?, ?it/s] 0%| | 0/62 [00:00<?, ?it/s] 0%| | 0/62 [00:00<?, ?it/s] 0%| | 0/62 [00:00<?, ?it/s] 0%| | 0/62 [00:00<?, ?it/s] 23%|██▎ | 14/62 [00:00<00:00, 139.42it/s] 22%|██▏ | 14/63 [00:00<00:00, 138.47it/s] 22%|██▏ | 14/63 [00:00<00:00, 136.64it/s] 23%|██▎ | 14/62 [00:00<00:00, 136.75it/s] 23%|██▎ | 14/62 [00:00<00:00, 136.39it/s] 22%|██▏ | 14/63 [00:00<00:00, 135.45it/s] 22%|██▏ | 14/63 [00:00<00:00, 135.64it/s] 23%|██▎ | 14/62 [00:00<00:00, 134.46it/s] 44%|████▍ | 28/63 [00:00<00:00, 137.70it/s] 44%|████▍ | 28/63 [00:00<00:00, 137.84it/s] 45%|████▌ | 28/62 [00:00<00:00, 137.75it/s] 45%|████▌ | 28/62 [00:00<00:00, 137.02it/s] 45%|████▌ | 28/62 [00:00<00:00, 136.82it/s] 44%|████▍ | 28/63 [00:00<00:00, 136.19it/s] 47%|████▋ | 29/62 [00:00<00:00, 141.63it/s] 46%|████▌ | 29/63 [00:00<00:00, 140.40it/s] 67%|██████▋ | 42/63 [00:00<00:00, 138.58it/s] 68%|██████▊ | 42/62 [00:00<00:00, 138.44it/s] 67%|██████▋ | 42/63 [00:00<00:00, 137.98it/s] 68%|██████▊ | 42/62 [00:00<00:00, 137.68it/s] 68%|██████▊ | 42/62 [00:00<00:00, 137.67it/s] 67%|██████▋ | 42/63 [00:00<00:00, 136.69it/s] 71%|███████ | 44/62 [00:00<00:00, 141.75it/s] 70%|██████▉ | 44/63 [00:00<00:00, 140.41it/s] 89%|████████▉ | 56/63 [00:00<00:00, 138.85it/s] 90%|█████████ | 56/62 [00:00<00:00, 138.66it/s] 89%|████████▉ | 56/63 [00:00<00:00, 137.96it/s] 90%|█████████ | 56/62 [00:00<00:00, 137.92it/s] 90%|█████████ | 56/62 [00:00<00:00, 137.75it/s] 89%|████████▉ | 56/63 [00:00<00:00, 136.68it/s] 95%|█████████▌| 59/62 [00:00<00:00, 142.23it/s] 94%|█████████▎| 59/63 [00:00<00:00, 140.92it/s] 100%|██████████| 62/62 [00:00<00:00, 141.84it/s]
100%|██████████| 63/63 [00:00<00:00, 140.57it/s]
100%|██████████| 62/62 [00:00<00:00, 138.37it/s]
100%|██████████| 62/62 [00:00<00:00, 137.66it/s]
100%|██████████| 62/62 [00:00<00:00, 137.51it/s]
100%|██████████| 63/63 [00:00<00:00, 138.48it/s]
100%|██████████| 63/63 [00:00<00:00, 137.79it/s]
100%|██████████| 63/63 [00:00<00:00, 136.47it/s]
2026-04-30:10:58:53 INFO [api.task:312] Building contexts for mbpp_plus on rank 4...
2026-04-30:10:58:53 INFO [api.task:312] Building contexts for mbpp_plus on rank 7...
2026-04-30:10:58:53 INFO [api.task:312] Building contexts for mbpp_plus on rank 5...
2026-04-30:10:58:53 INFO [api.task:312] Building contexts for mbpp_plus on rank 3...
2026-04-30:10:58:53 INFO [api.task:312] Building contexts for mbpp_plus on rank 1...
2026-04-30:10:58:53 INFO [api.task:312] Building contexts for mbpp_plus on rank 2...
2026-04-30:10:58:53 INFO [api.task:312] Building contexts for mbpp_plus on rank 6...
2026-04-30:10:58:53 INFO [api.task:312] Building contexts for mbpp_plus on rank 0...
0%| | 0/48 [00:00<?, ?it/s] 0%| | 0/47 [00:00<?, ?it/s] 0%| | 0/48 [00:00<?, ?it/s] 0%| | 0/47 [00:00<?, ?it/s] 0%| | 0/47 [00:00<?, ?it/s] 0%| | 0/47 [00:00<?, ?it/s] 0%| | 0/47 [00:00<?, ?it/s] 0%| | 0/47 [00:00<?, ?it/s] 28%|██▊ | 13/47 [00:00<00:00, 125.27it/s] 27%|██▋ | 13/48 [00:00<00:00, 124.07it/s] 28%|██▊ | 13/47 [00:00<00:00, 121.29it/s] 28%|██▊ | 13/47 [00:00<00:00, 121.41it/s] 28%|██▊ | 13/47 [00:00<00:00, 120.55it/s] 28%|██▊ | 13/47 [00:00<00:00, 120.35it/s] 28%|██▊ | 13/47 [00:00<00:00, 120.84it/s] 27%|██▋ | 13/48 [00:00<00:00, 120.22it/s] 55%|█████▌ | 26/47 [00:00<00:00, 124.44it/s] 54%|█████▍ | 26/48 [00:00<00:00, 123.36it/s] 55%|█████▌ | 26/47 [00:00<00:00, 121.89it/s] 55%|█████▌ | 26/47 [00:00<00:00, 121.67it/s] 55%|█████▌ | 26/47 [00:00<00:00, 121.25it/s] 55%|█████▌ | 26/47 [00:00<00:00, 120.65it/s] 54%|█████▍ | 26/48 [00:00<00:00, 120.51it/s] 55%|█████▌ | 26/47 [00:00<00:00, 119.51it/s] 83%|████████▎ | 39/47 [00:00<00:00, 123.77it/s] 81%|████████▏ | 39/48 [00:00<00:00, 122.78it/s] 83%|████████▎ | 39/47 [00:00<00:00, 121.97it/s] 83%|████████▎ | 39/47 [00:00<00:00, 121.77it/s] 83%|████████▎ | 39/47 [00:00<00:00, 121.41it/s] 83%|████████▎ | 39/47 [00:00<00:00, 120.67it/s] 81%|████████▏ | 39/48 [00:00<00:00, 120.44it/s] 83%|████████▎ | 39/47 [00:00<00:00, 119.90it/s] 100%|██████████| 47/47 [00:00<00:00, 123.88it/s]
100%|██████████| 47/47 [00:00<00:00, 121.89it/s]
100%|██████████| 47/47 [00:00<00:00, 121.70it/s]
100%|██████████| 47/47 [00:00<00:00, 121.34it/s]
100%|██████████| 47/47 [00:00<00:00, 120.68it/s]
100%|██████████| 48/48 [00:00<00:00, 122.76it/s]
100%|██████████| 47/47 [00:00<00:00, 119.92it/s]
100%|██████████| 48/48 [00:00<00:00, 120.33it/s]
2026-04-30:10:58:53 INFO [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO [evaluator:575] Running generate_until requests
Passed argument batch_size = auto. Detecting largest batch size
Passed argument batch_size = auto. Detecting largest batch sizePassed argument batch_size = auto. Detecting largest batch size
Passed argument batch_size = auto. Detecting largest batch size
Passed argument batch_size = auto. Detecting largest batch sizePassed argument batch_size = auto. Detecting largest batch size
Passed argument batch_size = auto. Detecting largest batch size
Running generate_until requests: 0%| | 0/153 [00:00<?, ?it/s]Passed argument batch_size = auto. Detecting largest batch size
Determined Largest batch size: 11
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
Determined Largest batch size: 11
Determined Largest batch size: 11
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
Determined Largest batch size: 11
Determined Largest batch size: 11
Determined Largest batch size: 11
Determined Largest batch size: 11
Determined Largest batch size: 11
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
Running generate_until requests: 1%| | 1/153 [06:43<17:02:16, 403.53s/it] Running generate_until requests: 8%|▊ | 12/153 [07:43<1:08:50, 29.29s/it] Running generate_until requests: 15%|█▌ | 23/153 [08:43<33:49, 15.61s/it] Running generate_until requests: 22%|██▏ | 34/153 [08:54<17:52, 9.01s/it] Running generate_until requests: 28%|██▊ | 43/153 [09:05<11:36, 6.33s/it] Running generate_until requests: 35%|███▌ | 54/153 [09:11<06:51, 4.16s/it] Running generate_until requests: 42%|████▏ | 65/153 [09:16<04:13, 2.88s/it] Running generate_until requests: 50%|████▉ | 76/153 [09:23<02:45, 2.14s/it] Running generate_until requests: 57%|█████▋ | 87/153 [09:31<01:49, 1.67s/it] Running generate_until requests: 64%|██████▍ | 98/153 [09:35<01:09, 1.27s/it] Running generate_until requests: 71%|███████ | 109/153 [09:42<00:46, 1.06s/it]2026-04-30:11:08:37 WARNING [models.huggingface:1038] Left truncation applied. Original sequence length was 4271, truncating to last 3840 tokens. Some content will be lost.
Running generate_until requests: 78%|███████▊ | 120/153 [09:48<00:29, 1.11it/s] Running generate_until requests: 86%|████████▌ | 131/153 [09:55<00:18, 1.20it/s] Running generate_until requests: 93%|█████████▎| 142/153 [10:00<00:07, 1.41it/s] Running generate_until requests: 100%|██████████| 153/153 [10:01<00:00, 1.94it/s] Running generate_until requests: 100%|██████████| 153/153 [10:01<00:00, 3.93s/it]
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 00/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 00/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 00/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 00/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 00/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 00/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 00/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 01/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 01/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 01/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 01/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 01/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 01/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 01/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 02/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 02/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 02/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 02/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 02/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 02/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 03/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 03/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 03/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 02/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 03/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 03/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 04/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 03/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 04/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 04/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 04/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 03/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 04/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 05/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 04/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 05/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 05/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 04/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 05/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 05/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 05/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 06/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 06/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 05/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 06/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 07/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 07/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 06/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 06/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 07/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 06/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 08/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 07/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 06/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 07/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 08/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 07/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 08/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 09/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 10/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 08/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 08/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 07/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 08/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 09/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 08/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 10/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 11/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 09/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 09/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 09/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 10/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 10/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 09/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 11/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 12/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 10/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 12/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 13/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 09/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 11/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 12/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 10/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 14/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 10/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 11/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 12/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 11/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 11/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 11/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 15/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 16/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 13/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 13/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 12/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 12/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 14/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 13/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 12/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 17/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 13/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 13/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 14/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 14/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 15/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 14/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 18/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 15/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 14/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 15/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 19/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 13/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 15/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 16/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 16/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 14/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 16/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 17/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 15/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 16/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 17/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 15/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 18/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 16/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 17/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 17/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 17/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 19/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 20/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 16/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 18/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 18/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 17/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 21/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 20/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 18/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 19/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 18/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 22/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 18/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 21/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 22/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 20/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 23/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 19/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 19/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 19/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 20/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 23/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 20/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 19/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 20/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 24/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 21/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 20/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 25/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 21/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 21/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 24/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 25/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 21/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 26/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 22/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 22/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 21/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 22/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 26/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 27/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 23/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 23/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 24/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 22/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 22/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 23/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 24/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 28/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 25/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 24/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 25/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 29/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 23/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 27/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 25/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 26/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 24/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 23/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 30/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 31/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 26/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 28/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 25/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 24/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 27/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 26/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 25/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 28/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 26/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 27/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 27/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 29/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 26/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 29/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 28/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 27/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 30/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 31/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 27/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 30/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 29/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 28/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 28/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 30/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 31/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 29/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 28/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 29/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 29/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 30/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 31/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 30/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 30/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 31/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 31/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 31/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 4
ywang29-p4d-debug-2-worker-0:39123:40143 [3] NCCL INFO [Service thread] Connection closed by localRank 4
ywang29-p4d-debug-2-worker-0:39125:40133 [0] NCCL INFO [Service thread] Connection closed by localRank 4
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 2
ywang29-p4d-debug-2-worker-0:39123:40143 [3] NCCL INFO [Service thread] Connection closed by localRank 2
ywang29-p4d-debug-2-worker-0:39121:40140 [1] NCCL INFO [Service thread] Connection closed by localRank 2
ywang29-p4d-debug-2-worker-0:39125:40133 [0] NCCL INFO [Service thread] Connection closed by localRank 6
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 6
ywang29-p4d-debug-2-worker-0:39127:40139 [7] NCCL INFO [Service thread] Connection closed by localRank 6
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 5
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 1
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 3
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 7
2026-04-30:11:12:27 INFO [_cli.run:449] Logging to W&B failed: 'def pass_at_k(references: list[str], predictions: list[list[str]], k: list[int] = None):\n global compute_\n assert k is not None\n if isinstance(k, int):\n k = [k]\n res = compute_.compute(\n references=references,\n predictions=predictions,\n k=k,\n )\n return res[0]\n'
2026-04-30:11:12:27 INFO [loggers.evaluation_tracker:247] Saving results aggregated
2026-04-30:11:12:27 INFO [loggers.evaluation_tracker:119] Saving per-task samples to saves/qwen3_14b/coding/fft/eval_coding/fft/saves__qwen3_14b__coding__fft/*.jsonl
hf ({'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'max_length': 4096}), gen_kwargs: ({}), limit: None, num_fewshot: None, batch_size: auto
| Tasks |Version| Filter |n-shot| Metric | |Value | |Stderr|
|--------------|------:|-----------|-----:|---------|---|-----:|---|-----:|
|humaneval | 1|create_test| 0|pass@1 |↑ |0.7439|± |0.0342|
|humaneval_plus| 1|create_test| 0|pass@1 |↑ |0.6890|± |0.0363|
|mbpp | 1|none | 3|pass_at_1|↑ |0.7060|± |0.0204|
|mbpp_plus | 1|none | 3|pass_at_1|↑ |0.8333|± |0.0192|
wandb: updating run metadata; uploading artifact run-jm4o6xvg-evaluationeval_results-1S5ClQ; uploading artifact results
wandb: uploading artifact run-jm4o6xvg-evaluationeval_results-1S5ClQ; uploading artifact results
wandb: uploading artifact results
wandb:
wandb: Run history:
wandb: humaneval/pass@1,create_test ▁
wandb: humaneval/pass@1_stderr,create_test ▁
wandb: humaneval/sample_len ▁
wandb: humaneval_plus/pass@1,create_test ▁
wandb: humaneval_plus/pass@1_stderr,create_test ▁
wandb: humaneval_plus/sample_len ▁
wandb: mbpp/pass_at_1 ▁
wandb: mbpp/pass_at_1_stderr ▁
wandb: mbpp/sample_len ▁
wandb: mbpp_plus/pass_at_1 ▁
wandb: +2 ...
wandb:
wandb: Run summary:
wandb: humaneval/alias humaneval
wandb: humaneval/name humaneval
wandb: humaneval/pass@1,create_test 0.7439
wandb: humaneval/pass@1_stderr,create_test 0.03419
wandb: humaneval/sample_len 164
wandb: humaneval_plus/alias humaneval_plus
wandb: humaneval_plus/name humaneval_plus
wandb: humaneval_plus/pass@1,create_test 0.68902
wandb: humaneval_plus/pass@1_stderr,create_test 0.03626
wandb: humaneval_plus/sample_len 164
wandb: +10 ...
wandb:
wandb: 🚀 View run fft_coding at: https://wandb.ai/maskmoe/MFT-LM-eval/runs/jm4o6xvg
wandb: ⭐️ View project at: https://wandb.ai/maskmoe/MFT-LM-eval
wandb: Synced 5 W&B file(s), 1 media file(s), 4 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./wandb/run-20260430_105344-jm4o6xvg/logs
Reference in New Issue View Git Blame Copy Permalink